Introduction

Graph-based Retrieval Augmented Generation (GraphRAG) represents a cutting-edge approach in the realm of AI, where graph structures are utilized to enhance the traditional RAG models. The goal is to improve the contextual understanding and organization of data, but the journey is not without its challenges. In this article, we delve into the practicalities of implementing GraphRAG using Langchain's framework and Microsoft's approach, examining their strengths and limitations.

Evaluation Goals

We try to answer the following questions:

• What are the key challenges faced when implementing GraphRAG?
• What is the difference between Langchain's and Microsoft's GraphRAG implementations?

Setup & Key Challenges

Langchain offers an open-source base for implementing GraphRAG. As GraphRAG can incur significant cost for larger documents, we wanted to try using it with Llama 3.1 8B, with gpt-4o-mini as a fallback. Using a basic setup with Llama 3.1, we aimed to evaluate the performance and feasibility of GraphRAG in a real-world scenario. For input data, we used unstructured text, PDFs, and CSVs to assess the system's adaptability and efficiency. However, adapting this framework to different models like Llama 3.1 reveals some significant hurdles:

• Model Schema Incompatibility: Llama 3.1 with 8M parameters often deviates from expected schemas, necessitating adjustments to Langchain’s experimental library. In order to achieve compatibility, we needed to modify langchain to handle the discrepancies between the model’s schema and the expected schema.
• Bottleneck in Graph Inserts: The primary bottleneck in this implementation is the conversion of content into graph inserts via LLMs. While text processing is slow, handling PDFs is more manageable, but CSVs pose significant delays. This slow translation process limits the efficiency of the overall system.

Performance Insights

• Query Generation: While Langchain’s GraphRAG framework generally generates queries effectively, the responses are not always reliable. The process of inserting structured data could be expedited with traditional code, potentially generated by LLMs, rather than relying on the slower, less accurate graph insertion methods.
• Utility vs. Performance: Given the growing context sizes of modern LLMs, the necessity and advantage of GraphRAG diminish in many cases. The benefits of using a graph structure become debatable when weighed against the performance drawbacks.

Differences between Microsoft's GraphRAG (MGR) and Langchain's GraphRAG

Microsoft’s implementation of GraphRAG provides a more structured approach, particularly in how it handles text segmentation and the subsequent extraction of entities, relations, and labels. MGR segments texts into smaller parts, from which it extracts entities, relations, and labels based on the information (“claims”) within each segment. These entities are then clustered, with each cluster receiving a bottom-up generated description.

Query Types

• Global Queries: Based on cluster descriptions, these queries offer a broad overview of the data.
• Local Queries: Users can specify entities and their neighbors for a more targeted query, focusing on specific aspects of the graph.

Advantages and Drawbacks

• No Need for a Graph Database: MGR manages data within the context until it opens a vector store at the end, which is then populated with the graph and text. This reduces the initial overhead of managing a graph database.
• Inserting Documents: Despite its structured approach, document insertion in MGR remains slow and expensive, particularly for structured data. The process is more efficient when a translation script is written or generated, bypassing the need for real-time graph inserts.
• Accuracy Issues: Smaller LLMs in MGR tend to mislabel nodes or establish incorrect relations, affecting the accuracy of the generated graph.

Conclusion

GraphRAG presents an innovative way to enhance RAG models, but it is not without significant challenges. Langchain’s implementation struggles with schema compatibility and performance bottlenecks, particularly when handling structured data. On the other hand, Microsoft’s approach offers a more structured process but still faces challenges in document insertion speed and accuracy.

As LLMs continue to evolve, the utility of GraphRAG remains under scrutiny. While it holds promise for certain applications, especially those involving unstructured data, its performance limitations must be carefully considered before widespread adoption.

In many cases, traditional methods or even direct code generation by LLMs may offer a more efficient solution, especially as the context-handling capabilities of LLMs improve.