Get Demo Center access

Head-to-head · updated 13 September 2026

DataShield vs Databricks Mosaic AI: what does the model see before it answers?

Databricks AI Search, the product that used to be called Mosaic AI Vector Search, is a very good vector database. It does semantic, keyword and hybrid retrieval behind one API, with reranking built in. Sync pipelines keep the index fresh on their own. It scales to billions of records. Agent Bricks sits on top and builds the agent for you: it writes synthetic evaluation data, tunes the thing, and ships a Knowledge Assistant or a Supervisor Agent without much hand work. If your data already lives on Databricks, this is the shortest path to a working RAG agent, and we will say so on a sales call.

DataShield is not a vector database vendor. We run RAG on pgvector inside a governed data plane you host yourself. Embeddings are computed locally by nomic-embed-text, so no text leaves your walls to become a vector. Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. Every governed tool call lands in a hash chain you can verify without asking us. Below is the honest split, and Databricks takes several rows.

Get free Demo Center access Get an instant quote
DataShield vs Databricks Mosaic AI at a glanceEight questions regulated buyers ask us. Scored from each vendor's public documentation. DataShield vs Databricks Mosaic AI at a glance Eight questions regulated buyers ask us. Scored from each vendor's public documentation. DataShield Mosaic AI Tamper-evident audit chain you can verify Values tokenized before retrieval Embeddings computed on your own hardware Authority re-checked on every tool call Runs with no vendor control plane Billion-scale serverless retrieval Agent evaluation, judges and auto-tuning Document parsing at enterprise scale shipped partial / roadmap not offered Sources at the bottom of this page.

The short version

Pick DataShield when

  • The text must not leave your building. We embed locally with nomic-embed-text. Databricks computes managed embeddings for you, or asks you to bring your own vectors. Either way the boundary moves.
  • You need row and column control on the index itself, not just on the source table. The Databricks docs say row and column level permissions are not supported on a vector search index, and point you at the filter API to write your own.
  • An examiner will ask you to prove a log was not edited. Ours is a SHA-256 chain with signed checkpoints. Break one in your browser and watch the verifier name the failure. Try it.
  • Your agents also read Postgres, an S3 drop and a vendor API. One policy plane and one log covers all of it. See the architecture.

Pick Mosaic AI when

  • Your corpus is large and your team is small. Serverless retrieval at billions of records with no shard balancing is worth real money, and we do not offer it.
  • You want the agent quality loop handed to you. Agent Bricks writes synthetic evaluation data, runs LLM judges, and tunes the agent. We have no evaluation harness at all.
  • You have a mountain of PDFs. Document Intelligence parses, extracts and classifies at scale. Our extraction covers PDF, DOCX and EML, and that is the whole list.
  • You want one bill. AI Search, Agent Bricks, Model Serving and governance ride on the platform you already bought, and the retrieval rates are published per hour.

Bottom line: Mosaic AI is the fastest way to build a good agent on data that already lives in Databricks. DataShield is how you prove what that agent was allowed to read, on any platform, with the sensitive values already replaced. In Databricks shops we expect to run alongside.

Feature by feature

Competitor cells describe what Databricks' public site and docs say as of the date above. If we've mischaracterised something, email support@myorg.ai and we'll correct it, credited.

What mattersDataShieldMosaic AIEdge
Retrieval engine and scalePostgres pgvector, a 768-dimension chunk table per corpus, and hybrid retrieval that unions a tsvector keyword leg with the vector leg. Good for a governed corpus. Not built for billions of vectors, and we have no reranker.Serverless hybrid search over semantic, BM25 keyword and combined modes in one API, with built-in reranking and quality evaluation. The product page claims it "scales to billions of records and thousands of queries per second" with automated sync pipelines. This is their strongest row and it is not close.◇
Inside the index: embeddings and permissionsEmbeddings are computed locally by nomic-embed-text, so document text never leaves your infrastructure to become a vector. For column-masked datasets, chunks and embeddings derive from the masked view at index time, and the keyword index is rebuilt from masked text so search cannot be used to probe for a redacted value.Databricks-managed embeddings are computed by Databricks from a source Delta table, or you supply pre-calculated vectors. On permissions the docs are blunt: "Row and column level permissions are not supported." You are told to "implement your own application level ACLs using the filter API."◆
Tokenization and data handlingDatasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. Tokens are deterministic, join-preserving and vault-reversible. Quasi-identifier generalization covers dates to year, decade or age band, ZIPs to 3 or 4 digits, and partial phones, SSNs and emails, with a measured cardinality-reduction score per column. Masking and generalization are switches you turn on.Access control and lineage on governed tables, with guardrails on model traffic. We found no vault-reversible token architecture and no governed detokenization path in their public docs.◆
What reaches the modelWe do not proxy your LLM traffic. We do gate every value that leaves a governed dataset for a prompt. A PII or PHI column with no configured treatment is redacted rather than passed through, and a PHI dataset refuses an AI endpoint that is not marked BAA or ZDR approved.Unity Gateway guardrails let you "Block or Mask PII data such as names, addresses, credit card numbers" in requests and responses, across models hosted outside Databricks too. The docs mark guardrails Public Preview and note that "Output guardrails are not supported for embeddings models or for streaming."◈
Agent quality: evaluation and tuningNothing. We do not generate synthetic evaluation sets, we do not run LLM judges, and we publish no retrieval accuracy number because we have no benchmark document to stand behind.Agent Bricks generates task-specific synthetic data, builds evaluation criteria, and auto-optimizes agent quality, with LLM-as-judge scoring and human feedback loops. Document Intelligence parses and classifies enterprise documents at scale. Real engineering, honestly earned.◇
Audit evidenceSHA-256 hash chain with Ed25519-signed checkpoints that are themselves chained. Verification returns a named verdict and tells tampering from insertion, deletion and truncation. Signing keys can sit in your KMS or HSM. Public demo.Usage tracking in system tables and request and response logging into Unity Catalog Delta tables. Detailed and queryable. We found no claim anywhere that these records are hash-chained, signed, or independently verifiable.◆
Agent authorizationEvery governed tool call runs a fixed pipeline before dispatch: auth gate, consented-tool narrowing, token scope ceiling, declared authority tier, revocation re-check, then metering. Delegation is RFC 8693 token exchange with an enforced scope ceiling.Unity Catalog grants on tables, functions and MCP services, plus Unity Gateway rate limits and service policies. Managed MCP servers support on-behalf-of user authentication with per-server OAuth scopes, so a call runs with the requesting user's permissions.◆
Break-glassScoped, time-boxed emergency access for agents. Admin plus IP allowlist plus step-up, auto-revoking, and it cannot be quietly deleted from the log.Not described in the docs we read. Ask them.◆
GDPR erasureCrypto-shred of the subject's key material plus ISO 27560 consent receipts. Actor identities in the chain are HMAC-committed, so the evidence still verifies after the subject is gone.Delete the rows and resync the index. What erasure does to inference tables, to audit history, or to embeddings already computed from the deleted text is not documented.◆
MCP and agentsMore than 200 MCP tools across Ontology, Auth, Corpus and Lighthouse. Auth issues MCP tool tokens with scope ceilings, and per-call metering is attributed to the agent that made the call.Five managed MCP servers covering Genie, AI Search, Databricks SQL, Unity Catalog functions and a code interpreter, plus custom servers hosted as Databricks Apps and external servers registered as MCP Services. The docs mark managed MCP Public Preview.◈
DeploymentSelf-hosted in your own cloud or data center, or a dedicated single-tenant server we operate. Docker images for Auth, Ontology, Corpus and Lighthouse, with a signed deploy manifest Guardian verifies. No vendor control plane in the loop.A Databricks-managed control plane on AWS, Azure or GCP paired with a customer-side data plane. AI Search and Agent Bricks are not self-hostable, not open source, and not available on-prem or air-gapped.◆
Maturity and pricingLive in production (Guardian and Lighthouse since April 2026). SOC 2 not yet certified, and we say so. Published pricing and an instant quote with no call.A very large private company with a deep enterprise bench and a long compliance record. AI Search rates are published per hour: Standard at 4.00 DBU/hr, Storage Optimized at 18.29 DBU/hr, with worked monthly examples. Agent Bricks launched in beta in June 2025 and parts of the agent surface still carry Beta labels.◇

◆ DataShield leads◇ Mosaic AI leads◈ comparable

Databricks claims are drawn from databricks.com and docs.databricks.com, last checked 13 September 2026. We link them below rather than paraphrase from memory.

Three things you get here that you won't get from a platform RAG stack

The index does not hold the secret

Access control on a vector index is a filter, and a filter is a request. Tokenize the values first and the question changes. There is nothing sensitive in the chunk to leak, so a bad filter costs you a wrong answer instead of a breach notice. Detokenization is a separate, admin-tier, tenant-scoped call, and the audit write has to succeed or the lookup is refused. How the data plane works.

A witness that isn't the defendant

Databricks writes the log, stores the log in its own Delta tables, and runs the agent that wrote it. That is fine telemetry. It is not proof. Our chain sits in a separate control plane with its own signing keys, and verification tells deletion from truncation from tampering. Edit a record in your browser and see what happens. Try it, no signup.

Authority that can change mid-flight

An analyst resigns on a Friday afternoon. Their agent is forty minutes into a retrieval job. With DataShield the next governed tool call re-checks authority and fails closed. A revoked grant does not reach into a query that is already running. How Auth does it.

Where Mosaic AI is genuinely stronger

Start with retrieval, because that is the product. Serverless hybrid search with built-in reranking, automatic sync from the source Delta table, and scale into the billions of records is a hard thing to build and they have built it. No cluster sizing, no shard balancing, no ETL to keep the index warm. Then Agent Bricks, which is the part we admire most and compete with least. Describe the task, and it generates synthetic evaluation data, builds the judges, and tunes the agent for you. Evaluation is the bit most RAG projects skip and then regret. Document Intelligence handles the PDF mountain that every real deployment turns out to have. And the retrieval pricing is published per hour, in dollars, with worked examples, which is more than most vendors in this market manage. We are not going to run the "they hide their pricing" line, because it isn't true here.

Here is the push-back, and it is narrow. All of that quality work happens after the sensitive values are already in the index, already in the embeddings, and already in the inference tables. Unity Catalog governs the source table beautifully. The retrieval index is a copy, and the docs say so plainly: row and column level permissions are not supported there, and you are invited to write your own application-level ACLs with the filter API. That is a reasonable engineering answer and a poor compliance answer. Add guardrails that are still Public Preview and do not cover embeddings or streaming output, and the protection story leans on a lot of parts that are not finished. Ours leans on one that is: the value the model reads is a token, and the record of who read it is signed.

Questions worth asking both of us

These are the questions we'd want answered if we were buying. Ask them on every vendor call, including ours.

Can you cryptographically prove an audit log entry wasn't deleted?

DataShield: yes. Each record commits to the one before it, checkpoints are signed and chained, and verification tells deletion from truncation from tampering. Try it at /verify. Databricks: usage lands in system tables and payloads land in inference tables. Both are queryable and detailed. Neither is claimed to be hash-chained or signed, and we went looking. Ask them to show you the integrity proof, not the query.

What happens to a revoked agent mid-session?

DataShield re-checks authority on every governed tool call, so a revocation bites on the very next call. On the Databricks side, managed MCP servers use on-behalf-of user authentication, so a call carries the user's permissions. We found no description of a running agent session being downgraded in flight. Ask how long a compromised agent keeps working after you pull its access.

How does GDPR erasure interact with the audit trail?

DataShield crypto-shreds the subject's key material and issues an ISO 27560 consent receipt. Actor identities in the chain are HMAC-committed, so the chain still verifies after the subject is erased. Deleting rows from a Delta table is easy. Ask Databricks what erasure does to the vectors already computed from that text, and to the inference tables that logged it.

Does row-level security carry into a Databricks AI Search index?

No, and this is the fact worth writing down. The vector search docs state that row and column level permissions are not supported, and suggest you implement your own application-level ACLs using the filter API. Unity Catalog still governs the source table. The index is a copy with coarser rules. Our answer is different in kind: we tokenize the value before it is chunked, so the index holds a token and detokenization is a separate audited call.

Is DataShield a Mosaic AI alternative for RAG, or does it sit alongside?

Both, depending on your corpus. If you need billions of vectors and a reranker, use AI Search and put DataShield underneath for tokenization, per-call authorization and evidence. If you need a governed corpus that runs on your own hardware with local embeddings and no vendor control plane, we can be the whole retrieval path. Our honest limit: document free-text is indexed and served raw apart from a standing sensitive-pattern chunk filter, so do not load documents whose body text must be masked. The lakehouse governance argument is next door in our Unity Catalog comparison.

Does DataShield have SOC 2?

Not yet, and we will not imply otherwise. Auth is live in production with a public threat model and a verifier anyone can run. Guardian and Lighthouse have been running since April 2026. Design-partner terms include source escrow, so a small vendor is not a single point of failure. Details on the security page.

Other head-to-heads

Same market

DataShield vs Elastic

DataShield vs Elastic: Elasticsearch wins retrieval scale and hybrid search. DataShield tokenizes before the.

 Same market

DataShield vs Cortex Search

DataShield vs Snowflake Cortex Search: Cortex Search is fast managed RAG inside Snowflake. We add tokens.

 Same market

DataShield vs Glean

DataShield vs Glean: Glean owns enterprise search and Work AI context across SaaS apps. DataShield tokenizes.

 All

Every comparison

One honest scorecard per vendor, sources at the bottom.

Keep AI Search. Then see what a governed layer adds underneath it: tokenized values, local embeddings, and an audit chain you can break on purpose. Demo Center access is free with a work email.

Get free Demo Center access

You've seen the proof

Ready for a number? Scope your deployment and we'll price it against your own economics.

Get your quote →