Knowledge Graphs as Agent Memory

Vector databases have become the default memory substrate for AI agents, offering efficient similarity search via embeddings. However, vector memory suffers from a fundamental limitation: it encodes semantic proximity, not structural truth. An agent relying solely on vectors struggles with multi-hop reasoning, precise temporal ordering, and entity disambiguation. Knowledge graphs (KGs) provide a structured, relational alternative where memory is explicit, queryable, and temporally aware. For ML engineers building agents that require deterministic recall and logical consistency, KGs offer a mechanism to ground language in structured data.

As Part 4 of the Agent Memory Systems series, this article examines the architectural shift from unstructured vector stores to graph-based memory, detailing the mechanisms of entity resolution and temporal tracking.

The Structural Gap in Vector Memory

When an agent stores information in a vector database, it converts text into high-dimensional vectors. Retrieval is based on cosine similarity. This works well for finding documents "about" a topic, but it fails when the agent needs to reason about relationships between specific entities.

Consider an agent managing a project. If stored in a vector store, a query for "Alice's tasks" might retrieve documents containing "Alice" and "task," but it cannot guarantee that the task is assigned to Alice rather than merely mentioned by her. The vector space collapses the distinction between "Alice assigned the task" and "Alice discussed the task."

Knowledge graphs solve this by explicitly modeling triples: (Subject, Predicate, Object). This structure allows the agent to traverse relationships deterministically. Instead of hoping the right context appears in the top-k results, the agent can query:

MATCH (a:Person {name: 'Alice'})-[r:ASSIGNED]->(t:Task) RETURN t

This eliminates hallucination risks associated with ambiguous retrieval, providing a clear path for multi-hop reasoning that vector spaces cannot support natively. The reliability of knowledge graph memory lies in this explicit structural integrity, ensuring that the agent's understanding of the world is grounded in verifiable connections rather than probabilistic associations.

Entity Resolution: From Mentions to Nodes

A critical challenge in agent memory is entity resolution: mapping natural language references to unique, persistent identities in the graph. In conversation, users refer to entities ambiguously: "He finished it," "The previous project," or "Sarah."

The mechanism for handling this involves two steps:

Coreference Resolution: The agent identifies that "He" refers to "John" based on prior context.
Node Matching: The system checks if "John" already exists in the graph. If so, new facts are linked to the existing node. If not, a new node is created.

For example, if an agent processes the input: "John likes Python. Later, he switched to Go," the memory system must recognize that "he" and "John" are the same entity. In a KG, this results in a single node John with two edges: LIKES(Python) and SWITCHED_TO(Go). Without entity resolution, the agent might create two separate nodes, leading to fragmented and contradictory memory.

Libraries like Graphiti automate this process. Graphiti uses LLMs to extract entities and relationships from text, then performs entity resolution by comparing embeddings of extracted entities against existing nodes. If the similarity exceeds a threshold, it merges the nodes; otherwise, it creates a new one. This automation is crucial for maintaining graph integrity without manual schema engineering. Furthermore, in a temporal graph context, resolving entities correctly ensures that historical facts are attached to the correct persistent identity, preserving the timeline of events accurately.

Temporal Graphs: Adding Time to Structure

Agents operate in time-sensitive environments. Knowing what happened is insufficient; the agent must know when. Traditional KGs often lack native temporal support, requiring workarounds like adding time stamps as properties. However, this complicates queries.

Temporal knowledge graphs (TKGs) model time as a first-class citizen. Edges can carry valid-from and valid-to timestamps, or be associated with temporal events. This allows for queries like: "What was Alice's role in January 2023?" even if her role changed in March 2023.

Consider a scenario where an agent monitors a system. If a server S1 was owned by Team A in Q1 2023 and transferred to Team B in Q2 2023, a TKG captures this transition. A query for current ownership returns Team B, while a historical query returns Team A. This temporal integrity prevents the agent from making decisions based on outdated facts.

Implementing TKGs requires careful handling of interval arithmetic. Libraries like Zep provide abstractions for storing conversational memory with timestamps, allowing agents to retrieve context within specific time windows. Zep's architecture supports both vector and graph-based memory, enabling hybrid queries that combine semantic relevance with temporal precision. The concept of a temporal graph is central here, as it allows the agent to navigate through time-aware relationships, ensuring that the state of memory reflects the exact moment of inquiry.

Practical Implementation: Graphiti and Zep

Building a KG-based memory system from scratch involves significant engineering overhead: schema design, entity resolution algorithms, and temporal query optimization. Libraries like Graphiti and Zep abstract these complexities.

Graphiti focuses on the "memory" aspect of agents. It provides a Python SDK that handles:

Extraction: Using LLMs to pull entities and relations from text.
Resolution: Merging duplicate entities based on semantic similarity.
Storage: Persisting the graph in a backend (e.g., Neo4j, FalkorDB).

Graphiti’s approach is schema-agnostic, allowing dynamic addition of node types and edge labels. This flexibility is useful for agents operating in unpredictable domains. However, the lack of a fixed schema can lead to graph pollution if entity resolution is not accurate. Engineers must implement validation layers, such as confidence scoring for graphiti operations, to maintain knowledge graph memory quality.

Zep offers a more integrated approach, combining vector memory, conversational memory, and graph-like structures. Zep’s memory engine automatically extracts entities and relationships, storing them in a way that supports both semantic search and structured queries. It is designed for long-term memory, preserving context across multiple sessions. When using zep, developers benefit from built-in temporal features that simplify the management of historical data, reducing the need for custom temporal logic.

Both libraries highlight a key trade-off: automation vs. control. Fully automated KG construction can introduce noise if the LLM makes extraction errors. Engineers must implement validation layers, such as confidence scoring for entity resolution, to maintain graph quality.

Conclusion

Knowledge graphs provide a robust foundation for agent memory by offering structural clarity, entity resolution, and temporal awareness. While vector databases excel at semantic retrieval, they lack the precision required for complex reasoning. By integrating KGs into their memory architecture, agents can achieve higher accuracy in multi-hop queries, historical context retrieval, and entity disambiguation. Tools like Graphiti and Zep lower the barrier to entry, but engineers must still address challenges related to schema design, entity resolution accuracy, and temporal query optimization. As agents become more autonomous, structured memory will be essential for reliable operation.