Jenny Morris is a technical seller at Vespa. This video reflects my technical perspective and links to official Vespa resources.
RAG · Chunking · Ranking
All your chunks. One document. Ranked to the answer.
I help customers harness the power of AI search, AI agent and AI assistant by building real-world solutions on Vespa.ai.
▶ One Document. Every Chunk. Ranked to the Answer. (~9 min)
Multi-vector since 2014ㅤㅤㅤㅤㅤㅤㅤㅤ Powers Perplexity ㅤㅤㅤㅤㅤㅤㅤㅤBillions of documents ㅤㅤㅤㅤㅤㅤㅤㅤ ~100 ms at thousands of queries/secㅤㅤㅤㅤㅤㅤㅤㅤ Open source
Prefer to read?
The whole story — with the visuals from the video
A user query rarely needs a whole document. It needs the right paragraph, section, table, or answer span. Here’s why that makes chunking hard everywhere else — and simple in Vespa.
1. The chunking design crossroads
Every RAG system starts the same way: split documents into chunks, turn each chunk into a vector. Simple — until you decide how to store them.
The wrong decision often doesn’t show up until you’re at scale — when it’s expensive to undo.
2. How Vespa stores chunks
Vespa removes the hardest part of that decision. You keep one document. All of its chunk embeddings are stored as an array of vectors inside that document, and all of those vectors share one set of metadata.
No parent-child modeling. No duplicated titles. Keyword scoring stays clean because every metadata field appears exactly once. Update the document, and you update one document — not hundreds of chunk records.
You still choose your chunk size and method — that depends on your content. But everything after that (storage, metadata, updates, and ranking) becomes dramatically simpler.
3. Two meanings of “multi-vector”
Chunk-level vectors — the core model above. One document, split into chunks, one vector per chunk. The document stays one document, but each section gets its own retrieval signal while metadata stays clean.
Token-level vectors — the ColBERT-style approach. A passage is represented by many small vectors, often one per token. At query time each query token finds its best match inside the passage, and those best matches are combined. That’s late interaction, or MaxSim.
Two levels: chunk-level vectors help you find the right part of the document; token-level vectors help you score that text more precisely. Vespa supports both.
4. The competitive landscape
Vespa removes the hardest part of that decision. You keep one document. All of its chunk embeddings are stored as an array of vectors inside that document, and all of those vectors share one set of metadata.
No parent-child modeling. No duplicated titles. Keyword scoring stays clean because every metadata field appears exactly once. Update the document, and you update one document — not hundreds of chunk records.
You still choose your chunk size and method — that depends on your content. But everything after that (storage, metadata, updates, and ranking) becomes dramatically simpler.
Storing the vectors was never the hard part. The hard part is what you do at query time
The question is not whether you can store multiple vectors — it’s whether you can rank and retrieve the right chunks.
5. Where Vespa shines: ranking down to the chunk
RAG needs the right passages for the LLM’s context window. Most multi-vector search gives you one score per document — that tells you which document is relevant, not which chunks inside it are the best to send.
Vespa ranks down to the chunk. First it finds the best candidate documents. Then, inside each document, it scores the individual chunks — using vector similarity, keyword relevance, metadata, freshness, business logic, or a blend. Because the chunks live inside the document, Vespa ranks them with the full document context available. That’s layered ranking — read the deep dive on the Vespa engineering blog: Introducing layered ranking for RAG applications.
Only the most relevant passages reach your LLM. No bloated context. No wasted tokens. No filler. And no accuracy sacrificed just to make the architecture simpler.
6. What disappears from your codebase
In a typical vector-database setup, a lot of chunking complexity gets pushed into your application code. In Vespa, much of it simply goes away.
The real payoff: less code · fewer failure modes · cleaner, better retrieval.
7. Don’t just take our word for it — look at Perplexity
Perplexity published what great RAG actually requires: completeness, freshness, and speed; fine-grained content understanding; and hybrid retrieval and ranking.
“Great AI search has to treat the individual sections and spans of documents as first-class units — not just whole documents.” — Perplexity · which is exactly the idea behind ranking down to the chunk
They found that many vector databases lacked support for even simple boolean filters, while traditional search engines offered only primitive semantic retrieval and ranking. Vespa stood out because it eliminates these trade-offs — powerful filtering, retrieval, and ranking alongside state-of-the-art vector search, in one system.
Proof: this is what powers Perplexity
Their answer quality traces straight back to their search quality — built on Vespa
“Treat the individual sections and spans of documents as first-class units in their own right.”
— Perplexity · i.e. ranking to the chunk, in production, at web scale
Great AI answers start with great search. The speed and quality behind Perplexity’s search experience are powered by Vespa.
Questions engineers ask
Do I still have to choose chunk size and method?
Yes — those decisions depend on your content and are inherent to RAG. Vespa makes everything after them (storage, metadata, updates, ranking) dramatically simpler.
Can Vespa do ColBERT / token-level multi-vector?
Yes. The same model holds token-level vectors, and Vespa supports late-interaction (MaxSim) scoring as well as chunk-level vectors. You can use either or both.
How is this different from Qdrant / Weaviate / Milvus multi-vector?
They can store multiple vectors too — that part of the industry has converged. The difference is layered ranking: selecting the best chunks within the best documents, with full document context, in parallel, at scale.
Is there a free way to try it?
Yes — start with the self-paced Vespa 101 workshop, or the free Vespa Cloud trial and the open-source Docker image.
One document. Every chunk ranked. The right answer returned.
Ready to simplify your RAG architecture and improve retrieval quality?
Start with the self-paced Vespa 101 workshop and see how easy it is to put this together — or speak with a Vespa engineer to discuss your use case and get a fast start with a simpler, better chunking architecture.
Free to start · No credit card · Open source · Proven in speed, quality, and scale since 2014