~2m34:13
Wanderloots

Multi-Agent Memory? 🧠 Full Hindsight Guide (Hermes + Codex)

Aug 18, 2026

Read: ~2m · You save: 32 min

Multi-Agent Memory: A Full Hindsight Guide with Hermes and Codex

Unlock seamless AI collaboration! Learn to connect multiple agents like Codex & Hermes with Hindsight's shared memory system. Full setup guide!

The integration of multiple AI agents into a single workflow often necessitates a shared context or memory system to facilitate seamless communication and avoid redundant information exchange. This article explores the setup and functionality of Hindsight, a memory tool designed to act as a bridge between disparate AI agents, enabling them to access and contribute to a common knowledge base. The demonstration utilizes Hindsight with AI agents Codex and Hermes, but the principles discussed are applicable to any agent-based system.

Understanding the Memory System Architecture

The core concept behind this multi-agent memory system is analogous to a shared notebook accessible across multiple workstations. Each workstation represents an AI agent, comprising a model for intelligence and a harness for tools and environment. Hindsight serves as the central repository, extracting key information from agent interactions, organizing it, and making it retrievable by any connected agent. This shared memory prevents the need for constant re-explanation of context as users switch between agents.

Setting Up Hindsight

The setup process involves several key stages:

1. Environment and Authentication

  • Environment File (.env): A .env file is created to store sensitive information such as API keys and control plane access keys. This ensures that these secrets are maintained across Hindsight restarts.
  • Docker Compose Configuration: The docker-compose.yml file is configured to manage the Hindsight instance. This includes specifying the Large Language Model (LLM) for Hindsight's internal processing, such as OpenAI's Codex. Authentication for Codex is set up separately to leverage existing quotas without interfering with other Codex applications.
  • Docker Volume for Authentication: A dedicated Docker volume is created to manage Codex authentication specifically for Hindsight. This volume stores the authentication credentials, allowing Hindsight to sign in using a

Introduction to Multi-Agent Memory

Introduction to the concept of multi-agent memory systems, highlighting the problem of context sharing between agents and introducing Hindsight as a solution. The video aims to demonstrate how to connect a single memory tool to multiple agents using Hindsight, Codex, and Hermes.

  • Many users employ multiple AI agents but lack a way to share context between them.
  • Hindsight acts as a bridge, functioning like a shared notebook for agents.
  • The video will cover Hindsight setup, connection to Hermes and Codex, memory banks, tags, and automatic memory transfer.
  • The system works for any agent, not just the ones demonstrated.

System Architecture: Hindsight as a Shared Notebook

Explains the core architecture of the memory system, comparing it to a private studio with multiple workstations (agents) and Hindsight as the shared notebook connecting them. It details how Hindsight extracts and organizes information for readability by different agents.

  • Agents (like Codex for code, Hermes for personal assistance) use a model and a harness.
  • Agents lack a direct way to pass information and context.
  • Hindsight bridges this gap by acting as a shared notebook.
  • It extracts key information from agent chats and organizes it for recall.

Initial Setup and Environment Configuration

Details the setup process, including prerequisites like Docker and Python, and the creation of an environment file for API keys and control plane access. It emphasizes securing the Hindsight instance with API keys.

  • Prerequisites include Docker, Python, Codex, and Hermes.
  • An environment file (.env) is created to store API keys and control plane access keys.
  • These secrets limit access to the Hindsight Docker container and control plane.
  • Hindsight instance runs on localhost:999.

Docker Compose Configuration for Hindsight

Focuses on configuring the Docker Compose file for Hindsight, including setting up LLMs for memory processing (using OpenAI's Codex authentication), defining ports for the container and control plane, and ensuring automatic restarts.

  • A Docker Compose file is used to manage Hindsight's services.
  • An LLM (e.g., OpenAI's Codex) is required for Hindsight's memory processing.
  • Separate authentication for Hindsight to connect to Codex is necessary.
  • Ports are defined for the Hindsight container and its control plane.
  • The restart unless stopped setting ensures automatic restarts.

Codex Authentication Setup for Hindsight

Describes the process of creating a dedicated Docker volume for Codex authentication to allow Hindsight to use a separate Codex login, ensuring it doesn't interfere with existing Codex applications. It covers authenticating Codex within the volume and verifying the login.

  • A Docker volume is created for dedicated Codex authentication for Hindsight.
  • This volume provides a separate storage area for authentication purposes.
  • The process involves starting a node container, downloading the Codex CLI, and authenticating.
  • Device code authentication might need to be enabled in ChatGPT settings.
  • Successful login is verified via a provided link and code.

Launching Hindsight and Accessing the Control Plane

Explains how to launch Hindsight, transitioning from manual Docker commands to a simplified single-button launch script. It details the script's function in starting the Docker Compose stack and opening the Hindsight control plane, including inputting the control plane access key.

  • Hindsight can be launched via a script that runs Docker Compose.
  • This script automates starting the container, waiting for Hindsight, and opening the control plane.
  • The control plane access key is required to access Hindsight after setup.
  • The setup confirms Docker installation and the presence of the .env file.

Creating and Configuring Memory Banks

Covers the creation and configuration of a new memory bank within Hindsight, specifically a global user bank named 'Wanderlust'. It discusses the strategy of using a single global bank versus multiple project-specific banks and the concept of banks as hard recall boundaries.

  • A new memory bank named 'Wanderlust' is created for global user memory.
  • Banks serve as hard limits for memory recall.
  • A global bank is suitable for a single user or solo developer.
  • Per-project or per-repo banks might be better for distinct projects.
  • Hindsight can be served on networks like Tailscale for multi-user access with separate banks.

Hindsight Bank Configuration: Retain, Recall, Reflect, and Tags

Details the configuration options within a Hindsight memory bank, including extraction mode (concise), the roles of 'retain', 'recall', and 'reflect' functions, and the setup of free-form entities and tags for better memory organization and filtering.

  • Extraction mode can be set to 'concise' to reduce token usage.
  • 'Retain' extracts information from conversations into the memory bank.
  • 'Recall' pulls information from the memory bank into the agent's context.
  • 'Reflect' uses a complex system for autonomous evidence gathering and reasoning.
  • Free-form entities (people, places, concepts) can be extracted automatically.
  • Tags can be used for soft partitioning within a bank (e.g., by project or agent).

Advanced Hindsight Configuration: Observations, Reflect, and Memory Defense

Explains the 'Observations' and 'Reflect' sections in Hindsight configuration, focusing on knowledge consolidation and deeper reasoning. It also covers 'Memory Defense' for redacting sensitive content.

  • Observations consolidate facts into durable knowledge, identifying patterns.
  • Reflect provides a deeper reasoning layer, connecting relevant memories.
  • Memory Defense allows redacting or quarantining sensitive content.
  • Configuration settings are one-time and can be adjusted based on use cases.

Testing Hindsight and Implementing Backups

Demonstrates testing the Hindsight setup by adding a document, recalling information, and verifying the connection to the LLM. It also covers setting up automated backups for the Hindsight memory system.

  • A test document is added to Hindsight, and recall functionality is verified.
  • Background operations confirm the connection to the LLM.
  • Automated backups are crucial for data safety.
  • A backup script creates a host backup directory and performs an immediate backup.
  • Backups can be run manually or scheduled.

Upgrading Hindsight to Version 0.9.0

Details the process of upgrading Hindsight to a new version (0.9.0), including using the backup system, pulling the latest version, and relaunching the application. It highlights changes in the user interface and new features like knowledge pages.

  • Hindsight version 0.9.0 introduced new features.
  • Upgrading involves backing up existing data, pulling the latest version, and relaunching.
  • The user interface has been updated with features like a home button, knowledge pages, and a knowledge base.
  • Existing banks and memories are retained after the upgrade.

Connecting Hermes to Hindsight

Guides on connecting Hermes to Hindsight, switching from the default memory provider to Hindsight, and configuring the API key and bank ID. It also shows how to verify the connection using Hermes' memory status command.

  • Hermes settings are accessed via Command + Comma.
  • Switch from default memory provider (e.g., Mnemosyne) to Hindsight.
  • Configure Hindsight settings: local external mode, API key, and bank ID ('Wanderer's').
  • Restart the gateway for changes to take effect.
  • Verify connection using Hermes memory status command.

Connecting Codex to Hindsight via Coding Agent Plugin

Explains the process of connecting Codex to the Hindsight memory bank using a new coding agent plugin. This connection is distinct from using Codex to power Hindsight's internal processing and allows Codex to utilize the shared memory.

  • A new coding agent plugin simplifies connecting multiple agents to Hindsight.
  • This connection allows repos, history, and conversations to flow into the memory bank.
  • Previous Codex connections with hooks might need uninstallation.
  • An npx command installs the Hindsight Coding Agents plugin.
  • The configuration file (.hindsight/codingagent.json) allows customization.

Configuring the Hindsight Coding Agent Plugin

Details the configuration of the Hindsight Coding Agent plugin, including setting the server mode, API URL, bank ID, and private key. It discusses the option of dynamic bank IDs and the implications for project-related memory organization.

  • The Coding Agent configuration file (.hindsight/codingagent.json) is used.
  • Settings include server mode (self-hosted), API URL, bank ID, and private key.
  • Dynamic bank ID can be set to false for related projects sharing a single memory bank.
  • This allows for soft partitioning using tags within the bank.

Enabling and Trusting Codex Hooks for Hindsight

Describes enabling and trusting hooks within Codex for Hindsight integration. These hooks initialize memory, recall context, and retain completed sessions, enhancing Codex's ability to interact with the shared memory system.

  • Hooks in Codex automatically initialize memory, recall context, and retain sessions.
  • Hooks need to be reviewed and trusted due to their ability to run outside the sandbox.
  • Re-trusting changed files is necessary after updates.
  • The MCP (Multi-Agent Communication Protocol) server for Hindsight is installed as a plugin.
  • The MCP server provides tools for bank selection and specific actions.

Testing Cross-Agent Memory Transfer (Codex to Hermes)

Demonstrates a practical test case where Codex remembers a specific detail ('purple' for 'Project Memory Test'), which is then successfully recalled by Hermes, proving the cross-agent memory transfer functionality.

  • Codex is prompted to remember 'Project Memory Test uses purple as its temporary test color'.
  • Hindsight retains this memory, adding it to the memory bank.
  • Hermes is then asked 'What temporary color did I choose for my Project Memory Test?'.
  • Hermes successfully recalls the information ('purple') via Hindsight.
  • This confirms the seamless memory transfer between Codex and Hermes.

Testing Cross-Agent Memory Transfer (Hermes to Codex)

Tests memory transfer in the reverse direction, from Hermes to Codex. Hermes is asked to remember a favorite animal ('lemur'), which is then recalled by Codex, further validating the shared memory system's effectiveness.

  • Hermes is prompted to remember 'my favorite animal is a lemur'.
  • Hindsight retains this memory, increasing the total memory count.
  • Codex is asked 'What is my favorite animal?'.
  • Codex successfully recalls the information ('lemur') via Hindsight.
  • This demonstrates bidirectional memory transfer between agents.

Filtering Memories by Tags (Hermes and Codex)

Explores filtering memories by tags within Hindsight, showing how to view memories created specifically by Hermes or Codex. This highlights the utility of tags for organizing and accessing information based on its origin.

  • Memories can be filtered by tags within Hindsight.
  • Filtering by 'Harness Hermes' shows memories created by Hermes.
  • Filtering by 'Harness Codex' shows memories created by Codex.
  • Tags allow for structured data organization and future filtering.
  • This capability stems from the configuration of entity labels and tags.

Using the Reflect Function for Workflow Analysis

Demonstrates the 'Reflect' function, where an agent (Hermes) can be asked to reflect on the user's workflow. Hindsight processes this request, generating a concise observation about the durable memory-first workflow.

  • The 'Reflect' function can be initiated from Hermes.
  • Hermes is asked to 'reflect on my workflow concisely'.
  • Hindsight runs the reflection process, gathering evidence and reasoning.
  • The output is a concise observation: 'Your workflow is durable memory first. Hindsight holds cross-session facts, decisions, and all this information.'
  • Reflections are stored as observations, allowing agents to learn and improve.

Introducing Knowledge Pages in Hindsight v0.9.0

Introduces 'Knowledge Pages', a feature in Hindsight version 0.9.0, which automatically generates and updates summaries from agent memories. These pages serve as projections of data, offering an evolving overview of concepts and initiatives.

  • Knowledge Pages are a new feature in Hindsight version 0.9.0.
  • They are projections of data from agent memories that update automatically.
  • These pages provide an evolving summary of components, concepts, conventions, and decisions.
  • They help produce mental models based on real files and agent interactions.
  • Knowledge Pages are distinct from human-reviewed documentation like Obsidian wikis.

Creating a Knowledge Page for Agent Memory Philosophy

Demonstrates creating a Knowledge Page to summarize the agent memory philosophy for Hermes and Codex. The process involves querying Hindsight based on existing memories, generating content, and configuring automatic refreshes.

  • A Knowledge Page is created to summarize the agent memory philosophy.
  • The query focuses on the method of operating with the Hindsight memory bank for Hermes and Codex.
  • The generated page outlines the philosophy: using Hindsight as a shared external memory provider.
  • It details bank boundaries and normal session flow, extracted from existing memories.
  • The page includes default concepts, initiatives, and enhancements that can be modified.

Conclusion: Unified Memory and Future Steps

Concludes by emphasizing the power of a unified memory system connecting multiple agents. It reiterates that this is one layer of the memory stack and highlights the need for a trusted human-in-the-loop knowledge system for combating AI errors, setting up the topic for the next video.

  • A single memory system connects multiple AI agents.
  • Hindsight acts as the engine for consolidating knowledge and patterns.
  • This system helps agents understand the user and their work better.
  • It can be applied to various agents like Claude Code, Codex, Gemini, and Hermes.
  • A trusted human-in-the-loop knowledge system is necessary to combat AI errors and hallucinations, which will be covered in the next video.