16 September 2026

 

Imagine preparing to restart a project that has been dormant for six months. Before your first meeting, you ask the organisation’s AI assistant: “Why did we stop working on this?”

The reply is confident. Technical problems made the project difficult to continue. It links to a progress report describing those difficulties. Then your (human) colleague interrupts: “No. Our funding was withdrawn.”

This example exposes a gap in the promise of workplace AI: access to an archive does not automatically provide a reliable understanding of its contents.

 

How workplace AI searches your documents

 

A general-purpose language model does not automatically know your employer’s internal policies, unpublished research or project history. An assistant needs a way to obtain that information when answering. One common approach is retrieval-augmented generation, usually shortened to RAG.

A retrieval system searches available material and supplies selected information to the language model, which uses it to compose a response. Think of someone handing a colleague a small bundle of extracts from a much larger archive.

Selection is necessary because models have limits on how much information they can process at once. Searching also needs to balance thoroughness with the expectation of a quick answer. 

In many systems, documents are divided into passages called chunks. These can be converted into embeddings: numerical representations that capture patterns associated with their meaning.

Vector search compares those representations with one generated from the question. It can find related concepts despite differences in wording: a question about “parental leave” might retrieve a passage discussing “time off after childbirth”. That helps overcome the limitations of exact keyword matching. But resemblance to a question does not guarantee that a passage contains the answer. Even a capable model can struggle if the decisive evidence never reaches it.

 

Why an AI answer can miss decisive evidence

 

In our imagined project, the proposal and progress report repeatedly mention the project name. The cancellation appears under “Budget adjustments”, identified only by an accounting code. A retrieval system could rank the descriptive reports above the brief paragraph explaining why the work ended. The model might then mistake a documented difficulty for the cause of cancellation. Not every failure starts with ranking. Perhaps the minutes sit in an unconnected archive, or the revised budget has not yet been indexed for search.

Academic readers face a similar challenge. Imagine asking an AI assistant whether research supports a hypothesis. It retrieves three papers reporting encouraging results and produces an upbeat summary. A later correction or a contradictory study could change that assessment. Without them, the response might faithfully describe the selected papers while giving a distorted impression of the wider evidence.

A literature review requires more than a collection of passages discussing the same subject. It needs attention to study quality, conflicting results and what was published subsequently.

 

How knowledge graphs and GraphRAG connect information

 

Knowledge graphs offer one way to make connections between records explicit.

A knowledge graph represents entities (people, projects, publications and departments) together with relationships between them. A researcher belongs to a department. A paper reports findings from a study. A committee reviews a funded project.

Now let's return to the cancelled project example. A graph could link its name to its accounting code, the code to a funding allocation, and that allocation to the decision recorded in the minutes. The system would have additional routes to the relevant material, even when the final document barely mentions the project itself.

GraphRAG brings graph-based information into retrieval-augmented generation. A prominent open-source implementation, released by Microsoft Research in July 2024, uses a language model to extract entities and relationships from text, identify connected groups and generate summaries. 

Those summaries can help with broader questions, too. “What problems recur across our research portfolio?” calls for evidence spanning many records. Searching only for passages similar to that question may produce a narrow selection.

The graph’s usefulness depends on which relationships it captures. A link between a project and a committee decision helps locate evidence; it does not, by itself, prove why the committee acted.

 

What GraphRAG cannot fix on its own

 

Building these connections introduces its own difficulties.

An extraction model could confuse similarly named projects, misidentify who approved a decision or treat a proposed arrangement as an established fact. An error introduced during graph construction can influence later answers.

Dates and authority also need attention. A draft policy and its approved replacement may discuss almost identical subjects while giving different instructions. The system needs to preserve those differences.

Permissions create another boundary. Workplace retrieval should respect the user’s access rights, which may legitimately exclude relevant records. Connecting information must not become a route around confidentiality.

Graph construction also requires computation, and changing records require updates. Its benefits must justify the cost and complexity. For some organisations, simpler improvements may be sufficient: connecting an overlooked archive, preserving document status or combining keyword and vector search. Others use models to break complex questions into several targeted searches. GraphRAG is one approach within a wider effort to improve retrieval.

 

How to check workplace AI answers

 

You do not need to become a database engineer to examine a workplace AI response. Three checks can help.

Does the source support the claim? Open the cited passage. Is the assistant reporting what it says, or adding an explanation the record does not establish?

Is it the current authoritative version? Check dates, amendments and approval status, particularly for policies and procedures.

What could be missing? Establish which repositories the assistant can search. An answer drawn from project reports may need checking against financial or decision records.

A useful follow-up prompt is: “Which records support this conclusion, do any contradict it, and what remains uncertain?” It can encourage closer examination, although it cannot guarantee that overlooked evidence will be found.

For a consequential decision, verify the response against an authoritative record.

And when the records leave a gap, the colleague who attended the budget meeting may be your best next step.

 

 

Further reading on MoveTheNeedle.news:

The memory economy, part II: from preserving knowledge to operationalising it

Explore the broader challenge of turning an organisation’s accumulated knowledge into information people and AI systems can use.

Can mechanistic interpretability reveal what AI is thinking?

Retrieving evidence is part of the problem. Understanding how a model reaches its decisions opens another window onto AI reliability.