Head-to-head · updated 13 September 2026
DataShield vs Vespa: the retrieval is fast, but who says this agent may read that corpus?
Vespa is the real thing. It came out of Yahoo, spun out as its own company in October 2023, and it runs the search behind Perplexity, Spotify, Thomson Reuters and DuckDuckGo. Their pitch is a distributed serving engine that unifies retrieval, ranking and model inference: billions of changing items, thousands of queries a second, under 100 milliseconds. If you are building a public answer engine, buy Vespa and stop reading comparison pages.
We are not a serving engine and we will not pretend otherwise. DataShield governs the data an agent reads. Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. Our RAG runs on pgvector with local embeddings, so nothing leaves your machines to be vectorized. Every governed tool call is checked against the agent's authority before dispatch, and the decision is sealed into a hash chain you can verify yourself. Below is the honest split, including the rows Vespa wins outright.
The short version
Pick DataShield when
- The corpus has PHI, claims files or HR records in it, and someone has to say which agent may read which corpus. We check that on every governed tool call, not at index time. How Auth does it.
- You need to pull an agent's authority mid-session and have the very next call fail. Not the next token refresh.
- An examiner will ask you to prove the retrieval log was not edited. Our chain answers with math. Run the verifier.
- Embeddings must not leave your machines. We run
nomic-embed-textlocally, so no text is shipped to a vendor to be vectorized.
Pick Vespa when
- Scale is the problem. Billions of items, thousands of queries a second, under 100 milliseconds. We are not in that conversation and we are not going to bluff about it.
- You need learned ranking in the serving path: multi-phase ranking, tensors, ColBERT and ColPali style late interaction, model inference at query time.
- You want an Apache 2.0 core you can fork, plus a managed cloud if you change your mind later. That is a genuinely good deal.
- Your reference class is public search and recommendation, where the hard part is relevance, not who is allowed to see the document.
Bottom line: Vespa serves the retrieval. DataShield decides what an agent may retrieve and keeps proof of the decision. Running both is a normal answer: Vespa for the big public index, us for the corpora with regulated content in them.
Feature by feature
Competitor cells describe what Vespa's public site and blog say as of the date above. If we have mischaracterised something, email support@myorg.ai and we will correct it, credited.
| What matters | DataShield | Vespa | Edge |
|---|---|---|---|
| Retrieval scale and latency | pgvector with 768-dimension local embeddings, hybrid keyword plus vector retrieval. Sized for governed datasets and document corpora, not for a web-scale index. We publish no RAG accuracy number, because we have no benchmark document to point at. | The core of the product and the best part of it. Their homepage claims billions of changing items, thousands of queries a second, latency under 100 milliseconds, with Perplexity and Spotify as references. | ◇ |
| Ranking and relevance | Hybrid retrieval with a tsvector keyword leg and a vector leg. No learned ranking, no tensor framework, no query-time model inference. | Multi-phase ranking, tensor expressions, in-process ML inference, ColBERT and ColPali embedders shipped in the engine. Years of relevance engineering. | ◇ |
| Data handling before the model | Fields are classified against 129 built-in PII, PHI, financial and secret classes, using regex, checksum validation, column-name lexicons and anti-pattern suppressors. Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. | A serving engine. Classification and redaction of indexed content are your job upstream. We found nothing about PII or PHI handling in their public material. | ◆ |
| Embeddings | nomic-embed-text runs locally, 768 dimensions. No text is sent to a third party to be embedded. Chunk views apply read-time masking, and the keyword index is rebuilt from the masked text so search cannot be used to probe for a hidden value. | Embedders run inside Vespa too, which is a point in their favour, and they support external models where you want them. Content masking is not part of the story. | ◆ |
| Prompt egress | We do not proxy your LLM traffic. We do gate every value that leaves a governed dataset for a prompt, and a PHI dataset refuses an endpoint without a BAA. Columns classified as PII with no configured treatment are redacted rather than passed through. | Not in scope. Vespa returns what you index and rank. What your app then puts in a prompt is up to your app. | ◆ |
| Agent authorization | Every governed tool call passes a scope ceiling, a consented-tool allowlist, an authority tier, and a mid-session revocation re-check before dispatch. The call fails closed. | Vespa Cloud has mutual TLS and role-based access control at the application level. Their July 2026 essay on agent search argues agents deserve expert-grade retrieval, and never mentions who is allowed to run it. | ◆ |
| Audit evidence | SHA-256 hash chain with Ed25519-signed checkpoints that are themselves chained. Verification returns clean, attested damage, or tampered, and names the failure. Try the verifier. | Logs and a metrics dashboard, which are operational rather than evidential. We found no cryptographic tamper evidence in their docs. | ◆ |
| Break-glass | Scoped, time-boxed emergency access for agents. It auto-revokes and cannot be quietly removed from the log. | Not described in their public material. | ◆ |
| GDPR erasure | Crypto-shred of per-subject key material plus ISO 27560 consent receipts. The audit chain still verifies afterwards. | Delete the document and reindex. That works for the index. It says nothing about derived copies or about the log. | ◆ |
| MCP and agents | More than 200 MCP tools across Ontology, Auth, Corpus and Lighthouse. Auth issues MCP tool tokens with scope ceilings, and meters each call with the agent attributed. | A public Vespa MCP server exists as of 13 September 2026, built by interns over the summer. Control plane auth is Auth0. Data plane access uses pre-generated tokens and is single-tenant test only, and the team says the server is under security review before any release. | ◆ |
| Deployment | Self-hosted in your own cloud or data center, or a dedicated single-tenant server we operate. Docker images for Auth, Ontology, Corpus and Lighthouse. Ed25519 audit-signing keys can live in your KMS or HSM. HMAC tokenization keys sit in your environment today, not in a KMS, and we would rather say that than let you find out later. | Apache 2.0 core you can run anywhere, plus Vespa Cloud. Cloud Enclave, launched January 2024, runs managed Vespa inside an AWS account or GCP project you control. That is a strong answer on residency. | ◈ |
| Maturity signals | Live in production (Guardian and Lighthouse since April 2026). Field classification cut over from shadow mode very recently, so treat it as new rather than battle-worn. SOC 2 not yet certified, and we say so. | Roots in Yahoo search from the early 2000s, open source since 2017, independent company since October 2023, GigaOm Sonar Leader for vector databases in February 2024. Reference customers most vendors would trade a round of funding for. | ◇ |
| Pricing | Published model and a scoped instant quote. Self-hosting means you pay for your own hardware. | The open-source core is free, which is hard to beat. Vespa Cloud is consumption-based, and their pricing page publishes a free trial and a contact form rather than a rate card. | ◈ |
◆ DataShield leads◇ Vespa leads◈ comparable
Vespa claims are drawn from vespa.ai and the Vespa blog, last checked 13 September 2026. We link the sources below rather than work from memory.
Three things you get here that you won't get from a search engine
A corpus an agent has to ask permission to read
An index is a flat thing. It answers whoever queries it. We put a tool token with a scope ceiling in front of the corpus, re-check the agent's authority on each call, and fail closed when the answer is no. Corpus promote and demote between personal and org-visible are audited events, not config edits. How Auth does it.
Proof that survives an audit
A retrieval log that can be silently edited proves nothing. Ours is a hash chain with signed checkpoints, and the verifier names what broke: tampering, insertion, deletion, or truncation. That is the property EU AI Act Article 12 and HIPAA §164.312(b) reviewers care about. Try it in your browser, no signup.
Sensitive fields handled before they are indexed
129 classes cover PII, PHI, financial data and secrets, including all 18 HIPAA Safe Harbor identifiers and eight non-US national ID formats. The method is regex plus checksums plus column-name evidence, so a verdict is reproducible from a config digest. No model, no drift. See the catalog.
Where Vespa is genuinely stronger
Let me be blunt about the gap. Vespa has been serving search since the early 2000s inside Yahoo, has been open source since 2017, and became its own company in October 2023 with Yahoo staying on as an owner and a customer. GigaOm called it a Leader in vector databases in February 2024. Perplexity runs on it. That is a production record no RAG-era startup can buy, and we cannot either. The engineering is deep in ways that matter: multi-phase ranking, tensors, ColBERT and ColPali late interaction, streaming search they claim is 20x cheaper than indexing, and autoscaling with cost boundaries. On pure retrieval quality and throughput they will beat us, and we would tell a prospect so on the call.
Here is the push-back. Read Jon Bratseth's July 2026 essay arguing that your agent should search like a 2010 quant. It is a good essay. It is also a perfect X-ray of the blind spot: thousands of words about giving agents richer retrieval, not one about which agent is allowed to run the query, what happens when that person's access is revoked halfway through a job, or what you hand an examiner afterwards. Their own MCP server, published this month, makes the same shape visible. The control plane uses Auth0 and the data plane runs on pre-generated tokens in single-tenant testing, with a security review pending. They are being honest about that, and it tells you where retrieval engines put their effort. Authorization and evidence are a different discipline. That is the one we ship.
Questions worth asking both of us
These are the questions we would want answered if we were the ones buying. Ask them on every call, ours included.
Can you cryptographically prove an audit log entry wasn't deleted?
DataShield: yes. Each record commits to the one before it, checkpoints are Ed25519-signed and chained, and verification tells deletion apart from truncation and from tampering. Try it on a sample chain at /verify. Vespa: they ship logs and a metrics dashboard for operating a cluster. We found no tamper-evidence mechanism in their public docs. Ask them what proof survives a root-level edit.
What happens to a revoked agent mid-session?
DataShield re-checks authority on every governed tool call, so revocation lands on the very next call and the context drops to anonymous. Vespa Cloud offers mutual TLS and role-based access control on the application. We could not find a per-query authority re-check or a mid-session revocation path. Ask how long a revoked client keeps querying your index.
How does GDPR erasure interact with the audit trail?
DataShield crypto-shreds per-subject key material and issues an ISO 27560 consent receipt. Actor identities in the chain are HMAC-committed, so the evidence still verifies once the subject is gone. With a search engine you delete the document and reindex. Ask what happens to derived copies, to the embeddings, and to the query log.
Is DataShield a vector database? Do we drop Vespa?
No, and probably not. Our RAG is real and GA: pgvector, local embeddings, hybrid keyword and vector retrieval, ingest from files, URLs and S3-compatible storage, with PDF, DOCX and EML extraction. It is sized for governed corpora, not for billions of items at sub-100ms. If your problem is web-scale relevance, Vespa wins. Run us where the content is regulated and the question is who may read it.
We're building a RAG app on regulated data. What does Vespa leave to us?
Finding the PII in the documents before they are indexed. Deciding which agent may query which corpus. Stopping a sensitive value from reaching a prompt. Keeping evidence of both. Those are all your problem with a serving engine, which is fair enough, because that is not what a serving engine sells. It is what we sell. Ask Vespa the same question and see whether the answer matches.
Does DataShield have SOC 2?
Not yet, and we will not imply otherwise. Auth is live with a published threat model and a verifier anyone can run. Guardian and Lighthouse have been in production since April 2026. Design-partner terms include source escrow, so a small vendor is not a single point of failure. Details on the security page. Vespa publishes no certification list on the pages we fetched either, so ask both of us.
- Vespa's hero: "We Make AI Work", describing "a distributed serving engine that unifies retrieval, ranking, machine learning inference, and real-time serving", with scale claims of billions of items, thousands of queries per second and latency below 100 milliseconds. — vespa.ai, 13 Sep 2026
- Vespa spun out of Yahoo as an independent company; Yahoo retained an ownership stake and remained a major customer. — Vespa Blog, 3 Oct 2023
- Vespa Cloud Enclave runs managed Vespa "in an AWS account or GCP project controlled by you". — Vespa Blog, 2 Jan 2024
- GigaOm Sonar for Vector Databases positions Vespa as a Leader. — Vespa Blog, 12 Feb 2024
- Perplexity's AI search runs on Vespa, Vespa's marquee RAG-at-scale reference. — Vespa Blog, 6 Oct 2025
- Jon Bratseth argues agents should get expert-grade retrieval rather than search built for casual humans. The essay covers query richness and ranking, and does not address authorization, permissions or PII. — Vespa Blog, 7 Jul 2026
- The public Vespa MCP server is an intern project: Auth0 on the control plane, pre-generated tokens on the data plane limited to single-tenant testing, and "the whole server is undergoing a security review to ensure it is up to our standard before a possible public release." — Vespa Blog, 13 Sep 2026
- Vespa's pricing page publishes a free trial and a contact form, with no rate card. — vespa.ai/pricing, 13 Sep 2026
Other head-to-heads
DataShield vs Pinecone
Managed vector search, and the governance it leaves to you.
SearchDataShield vs Elastic
The incumbent search stack, versus governed corpora for agents.
Vector DBDataShield vs Weaviate
Open-source vector search, and who is allowed to query it.
AllEvery comparison
One honest scorecard per vendor.
See both mechanisms run in your browser: break a live audit chain, revoke an agent mid-session, then decide what your retrieval engine still owes you. Demo Center access is free with a work email.
Get free Demo Center accessYou've seen the proof
Ready for a number? Scope your deployment and we'll price it against your own economics.
Get your quote →