---
title: "Discussion on \"The NLP Pipeline Behind Insight Engine: Extracting 10.7M Entities from 229K Research Papers (Part 2)\" | Hashnode"
description: "Discussion on \"The NLP Pipeline Behind Insight Engine: Extracting 10.7M Entities from 229K Research Papers (Part 2)\". How I used spaCy, SciSpacy, and a rule-based relation extractor to extract \n10,779,699 entities and 2,294,895 relationships from 229,498 arXiv papers — \nfully tracked with DVC and MLflow on a single RTX 4060 GPU."
url: "https://hashnode.com/posts/the-nlp-pipeline-behind-insight-engine-extracting-10-7m-entities-from-229k-research-papers-part-2/69c3b4578a263e79cb21fa63"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [The NLP Pipeline Behind Insight Engine: Extracting 10.7M Entities from 229K Research Papers (Part 2)](https://sumukha87.hashnode.dev/the-nlp-pipeline-behind-insight-engine-extracting-10-7m-entities-from-229k-research-papers-part-2?utm_source=hashnode&utm_medium=feed)

How I used spaCy, SciSpacy, and a rule-based relation extractor to extract 10,779,699 entities and 2,294,895 relationships from 229,498 arXiv papers — fully tracked with DVC and MLflow on a single RTX 4060 GPU.

Mar 25 · 7 min read · sumukha87.hashnode.dev  
Read the article

-   [#nlp](https://hashnode.com/tag/nlp)
-   [#knowledge-graph](https://hashnode.com/tag/knowledge-graph)
-   [#dvcs](https://hashnode.com/tag/dvcs)
-   [#mlops](https://hashnode.com/tag/mlops)
-   [#build-in-public](https://hashnode.com/tag/build-in-public)

## Written by

[SriSumukha S](https://hashnode.com/@sumukha)

@sumukha · Tokyo, Japan

  
[View profile](https://hashnode.com/@sumukha)

ML Engineer · GraphRAG · MLOps · AWS

Aspiring AWS ML Engineer building Insight Engine — a GraphRAG-powered cross-domain research discovery platform. I write about MLOps, knowledge graphs, LLMs, and SageMaker. Currently documenting the full build process in public, one post at a time. AWS Certified: Cloud Practitioner · AI Practitioner

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
