Head-to-head · updated 13 September 2026
DataShield vs Lakehouse Monitoring: who watches the table, and who proves what the agent did with it?
Databricks calls it Data Quality Monitoring now, and the pitch is blunt: "Intelligent quality monitoring for data and AI on the lakehouse," so teams can watch their tables "without added tools or complexity." That is aimed at every data observability vendor with a Databricks connector, and it lands. Anomaly detection flags tables whose freshness or row counts look wrong. Data profiling writes profile and drift metric tables back to Delta, with a dashboard and SQL alerts on top. If your data is on the lakehouse, you already own this.
We do a different job on the same tables. Ontology gives a dataset a 20-section profile, scores a trust KPI you can roll up by domain, and labels every field against 129 PII, PHI, financial and secret classes. Then Auth governs what an agent may do with that data. Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. We have no freshness monitor and no alerting, and we say so in the table below.
The short version
Pick DataShield when
- The data you worry about is not all in Databricks. We scan and profile a live PostgreSQL source in place, with no rows leaving it. Snowflake, BigQuery, Databricks and S3 are on the roadmap, not shipped.
- You need to know which fields are sensitive, not just which table went stale. We label fields against 129 classes, including all 18 HIPAA Safe Harbor identifiers, and the verdict is reproducible from a config digest.
- An examiner will ask you to prove a log was not edited. Ours is a SHA-256 chain with signed checkpoints. Break it in a browser and watch the verifier name the failure. Try it.
- Agents read the data, and someone has to answer for that. Every governed tool call re-checks authority, and you can pull an agent's access mid-session. How Auth works.
Pick Lakehouse Monitoring when
- Your tables are Delta tables in Unity Catalog. Turn it on per schema, or across a whole catalog, and pay only serverless compute. No new contract, no new vendor.
- You want freshness and completeness anomaly detection. Databricks runs a background job that learns what normal looks like and flags tables that drift off it. We have nothing like this.
- You want drift metric tables, a dashboard and alerts. Profile and drift metrics land back in Delta, a SQL alert sits on top, and it pages you in Slack or PagerDuty. Again, not us.
- Root cause matters more than proof. Monitoring reads the lineage the platform already captured, so the hunt starts closer to the answer.
Bottom line: Lakehouse Monitoring tells you a table looks wrong. DataShield tells you what is in the data, who may touch it, and what happened when an agent did. In a Databricks shop we expect to run alongside it, not instead of it.
Feature by feature
Competitor cells describe what Databricks' public site and docs say as of the date above. If we've mischaracterised something, email support@myorg.ai and we'll correct it, credited.
| What matters | DataShield | Lakehouse Monitoring | Edge |
|---|---|---|---|
| Anomaly detection and freshness | None, and we will not fudge this. There is no statistical anomaly detector in the stack, no freshness SLA and no on-call alert. If a table stops loading at 3am, we are not the thing that wakes you. | A background job monitors tables for freshness and completeness, enabled on a schema or a whole catalog. The docs mark it Public Preview, and note it skips views and foreign tables, and that completeness ignores nulls, zeros and NaN. | ◇ |
| Profiling depth | A 20-section analysis profile per dataset: completeness, field statistics, top values, patterns, column semantics, relationship graph, quality metrics, compliance governance, transformation lineage and source fingerprint. A weighted trust KPI rolls up by entity type, domain and estate, and recomputes hourly. | Profile metric tables with summary statistics, plus drift metric tables against a baseline, plus custom metrics you define in SQL. Broad, and it covers model outputs as well as tables, which we do not. | ◈ |
| Alerting, dashboards and incidents | No alerting layer and no incident workflow. We surface change and drift through a diff engine, a change classifier, schema evolution tracking and a CDC log, which you read or query. Nobody gets paged. | A SQL query over the metric table, a Databricks SQL alert on top, email by default, plus webhooks into Slack or PagerDuty. An out-of-the-box dashboard ships with the monitor. | ◇ |
| Sensitive-data classification | 129 field classes covering PII, PHI, financial data and secrets, with checksum validators (Luhn, NPI, Verhoeff, ABA, IBAN, GTIN), column-name lexicons and anti-pattern suppressors. Verdicts are deterministic and stamped with a config digest, so two runs agree. No model, so no drift. | Not part of monitoring. Unity Catalog handles tagging and classification separately. The monitor reports that a column changed, not that it holds a medical record number. | ◆ |
| Audit evidence | SHA-256 hash chain with Ed25519-signed checkpoints that are themselves chained. Verification returns a named verdict and tells tampering from insertion, deletion and truncation. Signing keys can sit in your KMS or HSM. Public demo. | Monitoring writes metric tables, not evidence. Platform audit events land in a Unity Catalog system table. We found no claim that those records are hash-chained, signed or independently verifiable. | ◆ |
| Agent authorization | Every governed tool call runs a fixed pipeline before dispatch: auth gate, consented-tool narrowing, token scope ceiling, declared authority tier, revocation re-check, then metering. Delegation is RFC 8693 token exchange with an enforced scope ceiling. | Out of scope. The docs are explicit that monitoring "does not modify any tables it monitors, nor does it add overhead to any jobs that populate these tables." It is a reader. Access control lives in Unity Catalog. | ◆ |
| Break-glass | Scoped, time-boxed emergency access for agents. Admin plus IP allowlist plus step-up, auto-revoking, and it cannot be quietly deleted from the log. | Not a monitoring feature, and not described in the docs we read. Ask them. | ◆ |
| GDPR erasure | Crypto-shred of the subject's key material plus ISO 27560 consent receipts. Actor identities in the chain are HMAC-committed, so the evidence still verifies after the subject is gone. | Delete the rows. Metric tables hold statistics rather than raw values, which helps, but what erasure does to historical profiles and drift baselines is not documented. | ◆ |
| Tokenization and data handling | Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. Tokens are deterministic, join-preserving and vault-reversible, and every lookup is audited or refused. Masking and generalization are switches you turn on, not defaults. | No token vault and no reversible masking in the monitoring path. Monitoring reads the table as it is and writes statistics beside it. | ◆ |
| MCP and agents | More than 200 MCP tools across Ontology, Auth, Corpus and Lighthouse. An agent can ask for a dataset profile, read the trust KPI, and act on it, under a tool token with a scope ceiling. | The tagline pairs quality with AI, and Databricks has managed MCP servers elsewhere in the platform. We found no MCP surface for the monitor itself, so an agent reads the metric tables with SQL like anyone else. | ◆ |
| Reach and deployment | Self-hosted in your own cloud or data center, or a dedicated single-tenant server we operate. Docker images for Auth, Ontology, Corpus and Lighthouse, with a signed deploy manifest Guardian verifies. Profiling in place is PostgreSQL today, not your whole estate. | Unity Catalog tables on a Databricks workspace, on AWS, Azure or GCP. Serverless compute required for anomaly detection. Data outside the lakehouse is outside the monitor. | ◆ |
| Maturity and pricing | Live in production (Guardian and Lighthouse since April 2026). SOC 2 not yet certified, and we say so. Published pricing and an instant quote with no call. | A very large company with a deep enterprise bench. No licence to buy: it bills as serverless DBUs under the DATA_QUALITY_MONITORING product, split out from serverless jobs in February 2026. Cheap to start, and hard to forecast before you turn it on. | ◇ |
◆ DataShield leads◇ Lakehouse Monitoring leads◈ comparable
Databricks claims are drawn from databricks.com and docs.databricks.com, last checked 13 September 2026. We link them below rather than paraphrase from memory.
Three things you get here that you won't get from a built-in table monitor
A profile that names the risk, not just the shape
A metric table tells you a column's null rate moved. It does not tell you the column is a medical record number. We label fields against 129 classes, including all 18 HIPAA Safe Harbor identifiers and eight non-US national ID formats, with checksum validation behind the label. The catalog stores no column values by default. What Ontology does.
A witness that isn't the defendant
Databricks runs the job, writes the metrics and holds the keys to the storage. That is fine for quality work. It is thin when an examiner asks for the March logs and wants to know nobody edited them. Our chain sits in a separate control plane with its own signing keys. Edit a record in your browser and the verifier names the exact failure. Try it, no signup.
Authority that can change mid-flight
An analyst resigns on a Friday afternoon. Their agent is forty minutes into a job over your customer table. With DataShield the next governed tool call re-checks authority and fails closed. A quality monitor would report that the job ran. How we deploy and secure it.
Where Lakehouse Monitoring is genuinely stronger
Three things, and none of them are close. First, anomaly detection. Databricks learns the normal rhythm of a table and tells you when freshness or row volume slips, on a schema or across a catalog, with one click. We do not have that at all, and building it is not a weekend. Second, the operational loop. Metric tables, a dashboard, a SQL alert, a page to your phone. Data teams live in that loop, and a profile you have to go and read is not the same product. Third, distribution. It is already in the account. Nobody has to win a procurement cycle to try it, which is why standalone quality vendors keep adding features to stay ahead of a free default.
Here is the push-back, and it is narrow. Anomaly detection is marked Public Preview, it skips views and foreign tables, and the docs say completeness ignores nulls, zeros and NaN. So a table can arrive on time, at full row count, stuffed with zeros, and pass. More to the point, the monitor is a reader by design: its own docs say it does not modify the tables it watches or add overhead to the jobs that fill them. That is the right call for a monitor, and it means quality is the only question it answers. Who may read the column, what was tokenized before the model saw it, whether the log of that read can be verified later: none of that is in scope, and in a 2026 agent estate those are the questions that get asked in the room with the lawyers. Our fuller argument about lakehouse governance is on the Unity Catalog page.
Questions worth asking both of us
These are the questions we'd want answered if we were buying. Ask them on every vendor call, including ours.
Can you cryptographically prove an audit log entry wasn't deleted?
DataShield: yes. Each record commits to the one before it, checkpoints are signed and chained, and verification tells deletion from truncation from tampering. Try it at /verify. Lakehouse Monitoring: it is not an audit product. Databricks logs platform activity in a Unity Catalog system table, and we found no claim that those rows are hash-chained or signed. Ask for the integrity proof, not the query.
What happens to a revoked agent mid-session?
DataShield re-checks authority on every governed tool call, so a revocation bites on the very next call. A quality monitor has no answer here, and should not pretend to: access is Unity Catalog's job. Ask Databricks how long a compromised agent keeps reading after you pull its grants, then ask what record of that reading you can hand to an auditor.
How does GDPR erasure interact with the audit trail?
DataShield crypto-shreds the subject's key material and issues an ISO 27560 consent receipt. Actor identities in the chain are HMAC-committed, so the chain still verifies after the subject is erased. On the Databricks side, deleting rows is easy. What we could not find is what erasure does to the profile history and drift baselines already written to metric tables. Ask both of us.
Is DataShield a Lakehouse Monitoring alternative?
Only if what you wanted was profiling plus classification plus proof. If you wanted freshness alerts on Delta tables, no. We have no anomaly detection, no freshness SLA, no monitors on warehouse tables and no incident management, and any vendor who tells you those are the same thing as a profile is selling. Most Databricks shops that talk to us keep the monitor and add a layer for classification and agent evidence.
What does data quality monitoring actually cost on Databricks?
It bills as serverless DBUs under the DATA_QUALITY_MONITORING product, which Databricks split out from serverless jobs in February 2026. So the answer depends on your table count and how often you refresh, and you can only see it after you turn it on. Ask for an estimate at your scale, and ask what happens to the bill when someone enables it across a whole catalog. Our pricing is published and the quote is instant.
Does DataShield have SOC 2?
Not yet, and we will not imply otherwise. Auth is live in production with a public threat model and a verifier anyone can run. Guardian and Lighthouse have been running since April 2026. Design-partner terms include source escrow, so a small vendor is not a single point of failure. Details on the security page.
- Product page, verbatim: "Intelligent quality monitoring for data and AI on the lakehouse," and the promise to "monitor the health of their datasets without added tools or complexity." — databricks.com, 13 Sep 2026
- Anomaly detection: "Databricks creates a background job that monitors tables for freshness and completeness." It is marked Public Preview, "does not support views or foreign tables," and completeness "does not take into account metrics such as the fraction of nulls, zero values, or NaN." — docs.databricks.com, 13 Sep 2026
- Scope statement, verbatim: "Data quality monitoring does not modify any tables it monitors, nor does it add overhead to any jobs that populate these tables." — docs.databricks.com, 13 Sep 2026
- Alerting is assembled, not built in: "You create a Databricks SQL query on the profile metrics table or drift metrics table. You then create a Databricks SQL alert for this query," with email by default and webhooks to Slack or PagerDuty. — docs.databricks.com, 13 Sep 2026
- Billing: anomaly detection "runs on serverless compute and is billed as serverless DBUs under the DATA_QUALITY_MONITORING billing product," a line item split out from serverless jobs usage for results created from February 2026. — docs.databricks.com, 13 Sep 2026
- Launch framing, 12 December 2023: profile metrics "provide summary statistics of your data" and drift metrics "allow you to compare against your baseline tables," with an out-of-the-box customizable dashboard. — Databricks blog, 12 Dec 2023
Other head-to-heads
DataShield vs Qualytics
DataShield vs Qualytics: Qualytics scores and monitors the data an AI system reads. DataShield decides.
Same marketDataShield vs Snowflake DMFs
DataShield vs Snowflake DMFs: DMFs run scheduled data quality checks inside Snowflake. DataShield profiles,.
Same marketDataShield vs Telmai
DataShield vs Telmai: Telmai watches lakehouse data with ML baselines. DataShield profiles and classifies.
AllEvery comparison
One honest scorecard per vendor, sources at the bottom.
Keep the monitor. Then see what a governed data plane adds: classify a dataset, break a live audit chain, revoke an agent mid-session. Demo Center access is free with a work email.
Get free Demo Center accessYou've seen the proof
Ready for a number? Scope your deployment and we'll price it against your own economics.
Get your quote →