[Question]: what knowledge graph database is ragflow using?

[eluyutao]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

What kowledge graph database is ragflow using? Are we able to see which chunk the entity was extracted from?

[eluyutao]

@dosu

[dosubot]

RAGFlow doesn't use a traditional graph database like Neo4j. Instead, it stores knowledge graph data in document stores such as Elasticsearch, OpenSearch, or Infinity, depending on your setup. The graph structure itself is managed in memory using NetworkX, and then serialized and stored as chunks in the backend database.

Entities and relationships extracted from your documents are tagged with a source_id that corresponds to the chunk they came from, so you can trace which chunk an entity was extracted from. This info is preserved throughout the extraction and storage process, and you can retrieve it via the chunk's metadata in the database or through the API [source] [source].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[eluyutao]

In our pipeline, we managed to chunk documents beforehand, and then we use the Python API to upload an empty document and then add chunks to the documents. By doing this, I think the knowledge graph is not constructed even thought I've set knowledge graph settings in knowledge base configuration. How can I make ragflow construct knowledge graph in my settings? @dosu

[dosubot]

Knowledge graph construction in RAGFlow is only triggered by the document parsing pipeline—not by uploading chunks directly, even if your knowledge base config enables the knowledge graph. To build the knowledge graph, you need to either upload documents with parsing enabled or explicitly call the parse/async_parse_documents API after uploading your chunks. Manual chunk upload via the Python API does not invoke the entity/relation extraction or graph-building steps, so the knowledge graph won't be created in your current workflow. For your use case, after uploading your document and chunks, make sure to trigger parsing using the appropriate API call (like async_parse_documents) to start knowledge graph extraction and construction. This is by design and consistent across the backend and SDK layers.
Reference

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[eluyutao]

Okay. I tried the following process:

1. I created a knowledge base from UI, with Document parse = Naive, Chunk method = One (because I've chunked the documents myself), and enabled knowledge graph.
2. add an empty document --> This worked
3. add multiple chunks to that document --> This worked. And I'm able to see the correct Chunk Number in Knowledge Base tab in the UI. Although Parsing Status is PENDING, I'm able to retrieve related chunks in chat.
4. call DataSet.async_parse_documents --> this failed and the error message is pasted below

2025-06-18 18:18:22,755 INFO     17 From minio(0.17411763966083527) CTD模块4，5和2_主要内容介绍.txt/CTD模块4，5和2_主要内容介绍.txt
2025-06-18 18:18:22,755 INFO     17 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2025-06-18T18:18:22.755+08:00", "boot_at": "2025-06-10T17:55:24.280+08:00", "pending": 1, "lag": 0, "done": 8, "failed": 1, "current": {"9025f98a4c2d11f099f1f2de547c644c": {"id": "9025f98a4c2d11f099f1f2de547c644c", "doc_id": "eb7d10f24c1911f0b8f0f2de547c644c", "from_page": 0, "to_page": 100000000, "retry_count": 0, "kb_id": "26721e304c1211f0ad62f2de547c644c", "parser_id": "one", "parser_config": {"layout_recognize": "Plain Text", "auto_keywords": 0, "auto_questions": 0, "graphrag": {"use_graphrag": true, "entity_types": ["organization", "person", "geo", "event", "category", "drug", "disease", "regulation", "guideline", "procedure", "date"], "method": "general", "resolution": true, "community": true}}, "name": "CTD\u6a21\u57574\uff0c5\u548c2_\u4e3b\u8981\u5185\u5bb9\u4ecb\u7ecd.txt", "type": "doc", "location": "CTD\u6a21\u57574\uff0c5\u548c2_\u4e3b\u8981\u5185\u5bb9\u4ecb\u7ecd.txt", "size": 0, "tenant_id": "b42a36561ff111f0b485621449caa4ba", "language": "English", "embd_id": "qwen3-embedding-0.6b___OpenAI-API@OpenAI-API-Compatible", "pagerank": 0, "kb_parser_config": {"layout_recognize": "Plain Text", "auto_keywords": 0, "auto_questions": 0, "graphrag": {"use_graphrag": true, "entity_types": ["organization", "person", "geo", "event", "category", "drug", "disease", "regulation", "guideline", "procedure", "date"], "method": "general", "resolution": true, "community": true}}, "img2txt_id": "qwen-vl-max@Tongyi-Qianwen", "asr_id": "paraformer-realtime-8k-v1@Tongyi-Qianwen", "llm_id": "deepseek-v3___OpenAI-API@OpenAI-API-Compatible", "update_time": 1750241902418, "task_type": ""}}}
2025-06-18 18:18:22,861 INFO     17 set_progress(9025f98a4c2d11f099f1f2de547c644c), progress: 0.1, progress_msg: 18:18:22 Page(1~100000001): Start to parse.
2025-06-18 18:18:22,962 INFO     17 set_progress(9025f98a4c2d11f099f1f2de547c644c), progress: -1, progress_msg: 18:18:22 Page(1~100000001): [ERROR]Internal server error while chunking: [Errno 2] No such file or directory: CTD模块4，5和2_主要内容介绍.txt
2025-06-18 18:18:22,962 ERROR    17 Chunking CTD模块4，5和2_主要内容介绍.txt/CTD模块4，5和2_主要内容介绍.txt got exception
Traceback (most recent call last):
  File "/ragflow/rag/svr/task_executor.py", line 241, in build_chunks
    cks = await trio.to_thread.run_sync(lambda: chunker.chunk(task["name"], binary=binary, from_page=task["from_page"],
  File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 447, in to_thread_run_sync
    return msg_from_thread.unwrap()
  File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    raise captured_error
  File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 373, in do_release_then_return_result
    return result.unwrap()
  File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    raise captured_error
  File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 392, in worker_fn
    ret = context.run(sync_fn, *args)
  File "/ragflow/rag/svr/task_executor.py", line 241, in <lambda>
    cks = await trio.to_thread.run_sync(lambda: chunker.chunk(task["name"], binary=binary, from_page=task["from_page"],
  File "/ragflow/rag/app/one.py", line 101, in chunk
    txt = get_text(filename, binary)
  File "<@beartype(deepdoc.parser.utils.get_text) at 0x729eaf2a0700>", line 31, in get_text
  File "/ragflow/deepdoc/parser/utils.py", line 26, in get_text
    with open(fnm, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'CTD模块4，5和2_主要内容介绍.txt'

Could you help me debug this? What have I missed? @dosu

[dosubot]

The FileNotFoundError happens because RAGFlow’s parsing pipeline expects the actual document file to be present in storage (MinIO) when you trigger async_parse_documents. Uploading only an empty document and chunks via the API does not provide the physical file, so the backend can’t find or parse it—hence the error.

To fix this, you need to upload the original document file itself (not just chunks) to the knowledge base, either through the UI or using the Python SDK’s upload_documents method. This ensures the file is stored in MinIO under the correct path (kb_id/filename) and is accessible for parsing and knowledge graph construction. After uploading the file, you can then trigger parsing (either from the UI or with async_parse_documents) and the knowledge graph will be built as expected [docs: file management] [docs: knowledge base config].

If you want to use your own chunking logic, you still need to upload the file first, then let RAGFlow parse it (you can intervene in chunking results after parsing if needed). Manual chunk upload alone won’t trigger the knowledge graph pipeline [source].

Let me know if you need details on the upload steps or how to structure your pipeline!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[eluyutao]

if I let async_parse_documents do parsing for me, will it override my chunks? @dosu

[dosubot]

No, calling async_parse_documents will not explicitly override your manually uploaded chunks. It processes the original document file and creates new chunks based on the parsing pipeline. Your existing chunks will remain unless the new parsing produces chunks with the same content and document ID hash as your manual ones, in which case they could be overwritten. In general, manual and parser-generated chunks can coexist, and there’s no automatic cleanup or deletion of your manual chunks when parsing runs. Just be aware that chunk updates are asynchronous due to Elasticsearch, so changes might not show up immediately in the UI or API responses reference.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[eluyutao]

How to use metadata to filter down chunks when retrieving chunks? @dosu

[dosubot]

Currently, RAGFlow does not support filtering chunks by metadata at retrieval time in the main API or SDK—metadata is attached to results for informational purposes, but cannot be used as a filter condition in queries. This applies to both the REST API and Dify integration; the metadata_condition parameter is ignored and all matching documents are returned regardless of metadata values. Metadata filtering is planned for future releases, but as of v0.19.0, it is not available reference.

If you need property-based filtering, the recommended workaround is to use the "Tag" knowledge base feature: create a tag KB, bulk tag your chunks, and then apply that tag KB to your main KB for downstream filtering reference. For now, direct metadata-based filtering is not possible, but you can prioritize chunks (e.g., by pagerank or tags) and metadata will be shown to the LLM when relevant chunks are retrieved reference.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[eluyutao]

I found this: You could use Tag functionality. A. create a knowledge base with Tag chunking method. B. Uploading samples of tags. C. Apply the tag knowledge base to the questioned knowledg base.

My question is, when a user (with only certain permissions, already set by the tags) ask a query, how does retrieve process know to only retrieve the chunks set by those tags? @dosu

[dosubot]

RAGFlow does not automatically restrict chunk retrieval based on tags or user permissions set via tags. Permissions are enforced at the knowledge base (KB) or dataset level—if a user has access to a KB, they can retrieve all its chunks, regardless of tags. Tags are mainly used for organizing, ranking, or expanding queries, not for access control. To filter retrieval by tags, you would need to explicitly add tag-based filters (like {'tag_feas': [...]}) to your retrieval queries in the API or SDK. Out of the box, RAGFlow does not enforce tag-based access restrictions during retrieval—this would require custom logic in your application layer or pipeline reference reference.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[eluyutao]

A few questions.

1. do I have to upload the document twice, first time to the tag knowledge base A, second time to a real knowledge base B that will be used for retrieval?
2. looks like I have to split the chunks and set tags beforehand, when creating the tag knowledge base. So that means the chunking in knowledge base A should be identical to chunking in knowledge base B, so that the tags can be mapped?
3. Help me locate the function/method used in the code that is responsible for processing filtering based on tags.
   @dosu

[dosubot]

1. You do not need to upload the document twice. The tag knowledge base (with the "Tag" chunking method) acts as a tag set or library. You create it once, upload your tag samples there, and then apply this tag KB to your main knowledge base for tagging and retrieval. The actual document for retrieval only needs to be uploaded to the real knowledge base, not to the tag KB as well reference.

2. Yes, the chunking in the tag KB and the real KB should be consistent. The tag KB is used to auto-tag chunks in your main KB, so the chunk boundaries and content should match for the tags to map correctly. If you split chunks differently, the tags may not align as expected reference.

3. Tag-based filtering is handled generically in the code. The main logic is in the Dealer class (rag/nlp/search.py), specifically the get_filters method, which builds filter conditions—including tags—from request parameters. The search method then applies these filters when querying chunks. There is no dedicated "tag filter" function; any field (including tags like tag_kwd) included in the filter dictionary will be processed and passed down to the data store connector (like ESConnection.search), which translates it into an Elasticsearch filter reference.

So, to filter by tags, make sure your retrieval/filter request includes the relevant tag field(s); the backend will handle it as part of the generic filtering mechanism.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}