Retrieval Context Should Be Portable Too#

Most teams can build retrieval into one agent app. They gather documents, chunk them, generate embeddings, point a retriever at a vector index, and get something working. The trouble starts when the same prepared context needs to move.

A second agent needs the same corpus. A template starter wants to ship with it. A Python app and a Node app both depend on it. Another team wants to reuse the same retrieval layer without inheriting the original app's preprocessing scripts, directory conventions, and hidden assumptions.

At that point, a lot of RAG systems stop behaving like shared infrastructure and start behaving like app-local glue.

This matters because retrieval is no longer niche. The RAG market is growing at 38–42% CAGR, and enterprise teams now use RAG for roughly 30–60% of their generative AI workloads.[1][2][3] Retrieval is not an experiment bolted onto one assistant anymore. It is becoming a standard part of agent systems.

Production numbers bear this out. Retrospectives on 2024–2025 production RAG deployments regularly point to retrieval failures at scale, especially when teams rely on naive chunking, weak provenance, or loosely managed corpora.[4][7] A 2025 clinical decision support study with 80,502 human annotations found that only 22% of the top-16 retrieved passages were actually relevant to the query, and that standard retrieval degraded factuality and completeness versus no retrieval at all.[5] The model was not the whole problem. The corpus was.

A corpus that only works inside one app is not portable context. It is app-local preprocessing.

The RAG rebuild problem#

When a corpus has no package boundary, every app that needs it tends to rebuild it differently.

One app downloads the source files during startup. Another checks in a snapshot. A third app chunks the same source material with slightly different window sizes. One team uses one embedding model, another upgrades to a different one, and nobody records the difference in a shared contract. Chunk IDs drift. Metadata schemas drift. Retrieval settings move into application config. Indexes get rebuilt locally with no clear provenance. The install story is usually just "copy this folder and hope the paths still line up."

Then somebody says two agents depend on "the same" corpus.

They usually do not.

The VersionRAG paper makes this problem explicit: small changes in source material can materially change the validity of answers, yet most retrieval setups do not make versioned corpus state visible to dependent systems.[6] If the corpus changes and the agent cannot tell which version it is querying, retrieval correctness becomes guesswork.

The useful unit is not just the retriever. It is the prepared corpus. Most systems still treat that unit as disposable build output.

Prepared retrieval context is part of the agent system#

Agent systems need more than executable actions.

They need actions, procedures, context, composition, and a template to bootstrap a working system from those pieces. AgentPM's artifact model has been moving in that direction:

• Tool = executable capability
• Skill = procedural know-how
• Knowledge = prepared retrieval context
• Agent = composition and orchestration
• Template = starter system

The industry has already spent years learning that code worth reusing deserves a package boundary. The same lesson applies here. If an agent depends on a corpus to answer correctly, that corpus is part of the system. It should have a name, a version, a defined shape, install semantics, and a way to inspect what was actually built.

This is not a claim that the whole ecosystem already agrees on a standard package model for retrieval corpora. Most of it does not. The claim is narrower and more practical:

If Skills package know-how, Knowledge packages the context that know-how depends on.

The portability argument is getting harder to ignore. Enterprise AI buyers are increasingly concerned about proprietary dependencies in orchestration, memory, and context handling because those dependencies make systems hard to move and govern.[8][11] Recent industry analysis projects that large enterprises could run well over 100,000 agents by 2028.[9] At that scale, prepared context stops being a private implementation detail. It becomes shared infrastructure.

Knowledge is a package, not a vector database#

Treating retrieval context as a package does not mean turning a package manager into a hosted search platform. A Knowledge package is not a replacement for Pinecone, pgvector, Weaviate, Chroma, or any local ANN index you already use. It is not a complete RAG framework. It is not an opinionated query engine.

There are two distinct concerns in retrieval systems.

The first is how search is executed:

• which database or local index is used
• which similarity metric runs at query time
• whether the app uses reranking, filtering, or hybrid retrieval
• which framework owns the retriever logic

The second is what prepared corpus is being searched:

• which source documents were used
• how they were chunked
• what metadata schema the chunks follow
• what embedding model and dimensions the vectors expect
• whether the vectors are normalized
• what build outputs exist
• what retrieval defaults were declared when the corpus was prepared

AgentPM's Knowledge model is aimed squarely at the second problem.

That boundary is deliberate. The first job is not to own retrieval compute. It is to make the corpus understandable outside the app that built it.

Real retrieval gains disappear when teams cannot share build decisions cleanly. A 2026 benchmark found that semantic chunking can improve retrieval accuracy dramatically compared with fixed-size chunking.[12] That improvement lives in the prepared artifact. If the chunking choice vanishes into a one-off preprocessing script, the downstream app can benefit from it, but it cannot reproduce it, version it, or inspect it.

Retrieval context as a first-class agent dependency#

The package boundary becomes more useful when retrieval context participates in the same dependency graph as the rest of the agent system.

Agents already depend on tools when they need to take actions. They depend on Skills when they need procedural guidance. Knowledge extends that graph to prepared context.

Once the corpus is treated as a dependency, basic but important questions become answerable from the artifact:

• What corpus is this?
• What version is installed?
• Which source documents were used?
• How was the corpus chunked?
• What embedding metadata does it assume?
• What retrieval settings were declared at build time?
• What agents or templates depend on it?

Lockfiles matter here. If a lockfile records tools, Skills, agents, and templates, it can record Knowledge too. The corpus stops being invisible build residue and starts being part of the install story. A downstream app can inspect what it got instead of trusting that someone ran the right script last week.

This becomes even more critical in multi-agent systems. Research on federated RAG and multi-agent retrieval converges on the same point: once multiple agents share retrieval, they need a stable, reproducible, inspectable unit of context.[10][13] Without that, agents can disagree not just about conclusions, but about what knowledge base they are even consulting.

If a corpus affects what the agent knows, it should be visible in the artifact graph.

What an AgentPM knowledge package looks like in practice#

In AgentPM, Knowledge ships in two modes.

Context mode is for direct-attachment content: policy bundles, playbooks, support guidance, curated docs. The package declares the document set, content types, roles, and hashes. There are no embeddings and no vector index. The point is to make prepared context portable in a lightweight way.

Vector mode is for retrieval-ready corpora. A vector package can carry:

• source documents
• chunks.jsonl
• sources.jsonl
• embedding payloads
• build metadata
• local exact-search outputs
• manifest-declared dimensions, metric, normalization, and retrieval settings

What the manifest captures is the point. For a vector package, that includes the chunking strategy, corpus paths, embedding model and dimensions, normalization, index location, and retrieval defaults. For a context package, it includes each document's path, content type, role, and hash. Either way, the manifest is what makes the corpus legible to the app that installed it — not just to the app that built it.

Without a package contract, a downstream team has to reverse-engineer what they got: which source files, which chunking parameters, which embedding model, which retrieval settings. With a package contract, those questions have answers that travel with the artifact through agentpm publish and agentpm install.

The same applies once a corpus is installed transitively through an agent or template. The Node SDK exposes loadKnowledge() and the Python SDK exposes load_knowledge() — so the corpus does not disappear into a hidden directory layout once installed. It is loadable and inspectable from wherever the agent runs.

Why the corpus needs its own package boundary#

There is a temptation, whenever retrieval comes up, to jump straight to infrastructure ambition: hosted indexing, shared query services, framework integration layers, hybrid ranking engines, and cross-provider orchestration.

Those things may matter later. They are not the right first move.

The right first move is to package the corpus.

That boundary is durable:

• it survives framework changes
• it survives host-language changes
• it survives app rewrites
• it makes retrieval decisions inspectable
• it makes prepared context portable

Agent systems need boring, explicit infrastructure around the pieces they depend on. Tools have it. Skills have it. Agents have it. Templates have it.

Prepared retrieval context needs it too.

Knowledge adds a package boundary around the corpus itself, so context can move with the same discipline as the rest of the agent system.

Sources#

1. MarketsandMarkets, RAG Market Report 2025–2030. https://www.marketsandmarkets.com/Market-Reports/retrieval-augmented-generation-rag-market-135976317.html

2. Next MSC, RAG Market Outlook 2035. https://www.nextmsc.com/report/retrieval-augmented-generation-rag-market-ic3918

3. Vectara, Enterprise RAG Predictions 2025. https://www.vectara.com/blog/top-enterprise-rag-predictions

4. Ragie, Architect's Guide to Production RAG. https://www.ragie.ai/blog/the-architects-guide-to-production-rag-navigating-challenges-and-building-scalable-ai

5. MDPI, Evaluating Retrieval-Augmented Generation Variants for Clinical Decision Support. https://www.mdpi.com/2079-9292/14/21/4227

6. VersionRAG: Version-Aware RAG for Evolving Documents. https://arxiv.org/pdf/2510.08109

7. Data Quality Challenges in RAG. https://arxiv.org/pdf/2510.00552

8. Zylos Research, AI Agent Ecosystem Fragmentation: Platform Lock-In and Portability. https://zylos.ai/research/2026-04-05-ai-agent-ecosystem-fragmentation-platform-lock-in-portability

9. Agentic AI Infrastructure Landscape 2025–2026. https://medium.com/@vinniesmandava/the-agentic-ai-infrastructure-landscape-in-2025-2026-a-strategic-analysis-for-tool-builders-b0da8368aee2

10. Multi-Agent RAG Framework for Entity Resolution. https://www.mdpi.com/2073-431X/14/12/525

11. Atlan, Context Portability: Keeping AI Agent Context Consistent. https://atlan.com/know/ai-agent/context-portability/

12. RAG in Production 2026: GraphRAG, Hybrid Retrieval, and Evals. https://ailearningguides.com/rag-production-patterns-2026/

13. A Systematic Review of Key RAG Systems: Progress, Gaps, and Future Directions. https://arxiv.org/pdf/2507.18910