Head-to-head · updated 13 September 2026

DataShield vs Apache Polaris: does an open catalog know which of those tables hold PHI?

Apache Polaris is the open catalog for Iceberg tables, and it is very good at that job. Spark, Flink, Trino, Doris, StarRocks and Dremio all speak to it. It hands out short-lived cloud credentials so an engine can read the files. Apache Gravitino sits a layer up and federates many catalogs at once, plus filesets, Kafka topics, ML models and vector data. Both are free, both are Apache top-level projects, and both run on your own hardware. If you are buying your way out of catalog lock-in, buy one of these.

We do a different job. An open catalog records that a table exists and who may open it. It does not say which columns hold a patient number. DataShield classifies the fields, tokenizes them, decides what an agent may do next, and writes proof you can check later. Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. Most teams here run both, and we think that is the right answer.

Get free Demo Center access Get an instant quote

The short version

Pick DataShield when

You need to know which columns hold PHI, from a scan rather than a spreadsheet. We label fields against 129 classes and the verdict is reproducible. See Ontology.
An agent is calling your data and its rights can change. We re-check authority on every governed tool call, so a revoked agent fails on the next one. How Auth does it.
You have to prove a log wasn't edited. Ours is a hash chain with signed checkpoints. Run the verifier.
A GDPR erasure request has to land without breaking the audit trail. Crypto-shred does that.

Pick Polaris or Gravitino when

You want one Iceberg catalog every engine can use. That is the whole point of Polaris, and it works.
Lock-in is the fear. An Apache top-level project with Snowflake and Dremio both contributing is the safest answer there is.
Your estate is wider than tables. Gravitino federates Hive, Glue, Kafka topics, filesets, models and vector data in one place.
You want free software you can read, fork and run. Both are Apache-2.0, and the price is your ops time.

Bottom line: Polaris and Gravitino tell your engines where the tables are. Neither claims to know what is in the columns, and neither writes evidence a regulator would accept. Run one of them, and put us next to it.

Feature by feature

Competitor cells describe what the Polaris and Gravitino docs say as of the date above. If we've mischaracterised something, email support@myorg.ai and we'll correct it, credited.

What matters	DataShield	Polaris / Gravitino	Edge
Iceberg REST catalog	Not us. We don't implement the Iceberg REST spec, and `iceberg_rest` is a declared, not-yet-built provider kind on our side. Your engines should talk to their catalog, not ours.	Polaris is the reference. It implements the Iceberg REST API and serves Spark, Flink, Trino, Doris, StarRocks and Dremio. Gravitino ships its own Iceberg REST service too.	◇
Field-level classification	129 field classes covering PII, PHI, financial data and secrets, including all 18 HIPAA Safe Harbor identifiers. Regex plus checksum validation, column-name lexicons and anti-pattern suppressors. No model, so a verdict is reproducible from a config digest.	None in either project. Gravitino has free-form tags you can attach by hand. A tag is a label someone typed, not a scan.	◆
Access control grain	Column-level. Masking and generalization are features you switch on per dataset, and the masked golden views mask no matter what the posture says.	Object level in both. Polaris privileges stop at catalog, namespace, table, view and policy. Gravitino stops at the securable object. No column rules, no row filters, no masking in either.	◆
Policy engine	Cedar decides admin, config, token and vault actions. On the tool path we use a scope ceiling, an authority tier and a mid-session revocation re-check. We don't claim Cedar on every call.	Polaris can delegate to Open Policy Agent, but the docs call it "a preview feature" that "may undergo breaking changes" and tell you to keep privilege management native. Gravitino pushes authorization down to Ranger or a JDBC plugin.	◈
Audit evidence	SHA-256 hash chain with Ed25519-signed checkpoints that are themselves chained. Verification names the failure: tampering, insertion, deletion or truncation. Try the verifier.	Real, and not tamper-evident. Polaris 1.7.0 added an OpenTelemetry listener and Kafka event publishing. Gravitino's MCP server writes one JSON line per call to `gravitino-mcp-audit.log`. Both are files a holder of root can rewrite.	◆
Agent authorization	Every governed tool call passes a scope ceiling, an authority tier and a revocation re-check before dispatch. Revocation lands mid-session. Delegation is RFC 8693 token exchange with an enforced scope ceiling.	Gravitino's MCP server takes a static token or forwards the caller's `Authorization` header. Its own access-control page says Gravitino "won't check the privileges when Gravitino receives the requests" and leans on a plugin instead.	◆
Break-glass	Scoped, time-boxed emergency access for agents. It auto-revokes and it can't be quietly deleted from the log.	We couldn't find a break-glass concept in either project's docs. In practice the answer is a second admin principal.	◆
GDPR erasure	Crypto-shred of per-subject key material plus ISO 27560 consent receipts. The chain still verifies after the subject is gone.	No notion of a data subject in either. Erasure is a rewrite of the Iceberg table, and the catalog just tracks the new snapshot.	◆
Tokenization and data handling	Deterministic, join-preserving, vault-reversible tokens applied at ingest, plus quasi-identifier generalization (dates to year, decade or age band; ZIPs to 3 or 4 digits; partial phones, SSNs and emails) with a measured cardinality-reduction score per column. Detokenization is a privileged, logged operation.	Neither transforms data. Polaris vends a short-lived cloud credential and the engine reads the raw files. That is by design, and it is also the gap.	◆
Scan and profile a live source	Ontology registers a live PostgreSQL source, scans it in place and profiles its columns with no row egress. Sample values are omitted by default; storing them in the clear needs an acknowledgement and writes an audit row. PostgreSQL today, not your whole estate.	Gravitino federates metadata from Hive, Iceberg, Paimon, Hudi, JDBC, ClickHouse, Glue, Kafka and filesets, which is far more breadth than we have. It reads schemas, not column contents.	◇
MCP and agents	More than 200 MCP tools across Ontology, Auth, Corpus and Lighthouse. MCP tool tokens carry scope ceilings, and every call is metered and attributed to the agent.	Both ship one. Gravitino's exposes 30-plus metadata tools. Polaris ships a Python MCP server that wraps its REST APIs. Each hands an agent the map, and neither polices what the agent does with it.	◈
Deployment	Self-hosted in your own cloud or data center, or a dedicated single-tenant server we operate. Docker images ship for Auth, Ontology, Corpus and Lighthouse, and a signed deploy manifest is verified by Guardian.	Self-hosted by design. Java services, Helm charts, your own Postgres or MySQL. Gravitino is built for multi-region estates and says so.	◈
Maturity signals	Auth, Guardian and Lighthouse are live in production (Guardian and Lighthouse since April 2026). SOC 2 not yet certified, and we say so.	Polaris became an Apache top-level project on 19 February 2026 and shipped 1.7.0 on 2 August 2026. Gravitino graduated in June 2025, runs at Uber and Pinterest, and shipped 1.3.0 on 29 June 2026. Older and busier than us.	◇
Pricing	Published model, scoped instant quote, no sales wall.	Free, Apache-2.0. You pay in engineers. Managed Polaris from Snowflake is closed to first-time sign-ups, and Dremio's version is quote-only.	◇

◆ DataShield leads◇ Polaris / Gravitino leads◈ comparable

Polaris and Gravitino claims are drawn from polaris.apache.org and gravitino.apache.org project docs and release notes, last checked 13 September 2026. We link them below rather than paraphrase from memory.

Three things you get here that you won't get from an open table catalog

Labels that came from a scan

Gravitino lets you tag a table. Someone still has to know it holds patient data, and type that in. We scan the columns and score them against 129 classes, with checksums and name lexicons behind the call. There is no model in the path, so the same data gives the same verdict next quarter. That is the answer when an auditor asks how you knew. See the catalog.

Proof that survives an audit

Polaris can stream audit events to OpenTelemetry. Gravitino writes a JSON line per MCP call. Both are useful. Neither can show that nothing was removed. Ours is a hash chain with signed checkpoints, and the verifier says what broke, not just that something did. EU AI Act Article 12 and HIPAA §164.312(b) reviewers look for that property. Try it in your browser, no signup.

Authority that can change mid-flight

An analyst leaves on a Friday. Their agent is 20 minutes into a job against your lakehouse. A catalog checked the grant when the session opened and has no reason to look again. We re-check on the next governed tool call and fail closed. How Auth does it.

Where Polaris and Gravitino are genuinely stronger

Let's be fair. Polaris won its category on merit. It is the Iceberg REST catalog most engines target, it vends scoped cloud credentials so query engines never hold long-lived keys, and its 1.7.0 release in August added Kafka event publishing and an OpenTelemetry audit listener. Snowflake and Dremio both ship it and both contribute to it, and the Apache board now holds the trademark. Gravitino covers ground we do not: Hive, Glue, ClickHouse, Kafka topics, filesets, ML models and Lance vector data, all federated, across regions, with Uber and Pinterest running it. Their connector breadth is years ahead of our one live provider kind. Neither costs a cent.

Here is the push-back. Both projects use the word governance, and buyers read it as the governance their privacy office means. It is not. Polaris privileges stop at the table. Gravitino's own access-control page says it "won't check the privileges when Gravitino receives the requests" and points you at a plugin. Tags are typed by hand. There is no classifier, no masking, no subject, no erasure story, and no chain over the log. None of that is a flaw, because none of it is what these projects set out to build. It only becomes a problem when a platform team tells a CISO the catalog has governance covered, and nine months later an examiner asks which tables held PHI in March. That is the gap we fill, and we would rather you keep Polaris.

Questions worth asking both of us

These are the questions we'd want answered if we were buying. Ask them on every vendor call, including ours.

Can you cryptographically prove an audit log entry wasn't deleted?

DataShield: yes. Each record commits to the one before it, checkpoints are signed and chained, and verification tells deletion apart from truncation and tampering. Try it at /verify. Polaris emits audit events over OpenTelemetry and Kafka, and Gravitino's MCP server writes a JSON line per call. Both are plain records. Whoever holds the file or the topic can edit history, so the proof has to come from somewhere else.

What happens to a revoked agent mid-session?

DataShield re-checks authority on every governed tool call, so revocation bites on the next call. Gravitino's MCP server takes a static bearer token or forwards the caller's header, and its docs say Gravitino does not check privileges on incoming requests. Polaris checks its own grants per request at table grain. Neither has a way to cut a running agent short of pulling the token.

How does GDPR erasure interact with the audit trail?

DataShield crypto-shreds per-subject key material and issues an ISO 27560 consent receipt. Actor identities in the chain are HMAC-committed, so the evidence still verifies after the subject is gone. Neither open catalog has a subject concept. You rewrite the Iceberg table, expire the old snapshots, and hope nobody kept a copy.

Is DataShield an Iceberg REST catalog? Do we drop Polaris?

No, and no. We don't implement the Iceberg REST spec, and Iceberg is a declared but unbuilt provider kind for us today. Your engines should keep talking to Polaris. We sit beside it and answer a different question: which fields are sensitive, what may an agent do with them, and can you prove it later.

Gravitino ships an MCP server. Isn't that agent governance?

It is agent access, which is the easier half. The server exposes 30-plus metadata tools and logs each call, and that is genuinely useful. What it does not do is issue a scoped token per agent, narrow that scope at dispatch, re-check the grant mid-session, meter the call, or seal the record. Read their access-control page next to ours and the split is clear.

Snowflake closed Open Catalog to new sign-ups. Does that hurt Polaris?

Not the project. The docs say new customers should use Horizon Catalog instead, which reads as Snowflake selling on top of Polaris rather than through it. The code is Apache-2.0 and the board holds it. It does mean that if you want managed Polaris today you are looking at Dremio or running it yourself, and running it yourself is the common answer.

Does DataShield have SOC 2?

Not yet, and we won't imply otherwise. Auth is live with a public threat model and a verifier anyone can run. Guardian and Lighthouse have been in production since April 2026. Design-partner terms include source escrow, so a small vendor isn't a single point of failure. Details on the security page.

Polaris is "an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg's REST API, enabling… multi-engine interoperability" across Doris, Flink, Spark, Dremio, StarRocks and Trino. Hero line: "The open catalog for data and AI assets." — Apache Polaris project site, 13 Sep 2026
Polaris privileges are granted at catalog, namespace, table, view and policy level (TABLE_READ_DATA, NAMESPACE_LIST, CATALOG_MANAGE_ACCESS and so on). No column-level or row-level rules, classification or masking appear in the model. — Polaris 1.7.0 access control docs, 13 Sep 2026
Polaris "policy" means table maintenance: the four predefined system types are system.data-compaction, system.metadata-compaction, system.orphan-file-removal and system.snapshot-expiry. External OPA policy is "currently a preview feature and may undergo breaking changes". — Polaris 1.7.0 policy and external PDP docs, 13 Sep 2026
"Gravitino only supports authorization for securable objects… Gravitino doesn't support metadata authentication. It means that Gravitino won't check the privileges when Gravitino receives the requests." Built-in authorization is a config flag; the design pushes down to Ranger or JDBC plugins. — Gravitino 1.3.0 access control docs, 13 Sep 2026
The Gravitino MCP server "provides the ability to manage Gravitino metadata for LLM", exposes 30-plus tools, authenticates with a static bearer token or a forwarded Authorization header, and records "one structured JSON line in gravitino-mcp-audit.log" per tool call. — Gravitino 1.3.0 MCP server docs, 13 Sep 2026
"Snowflake Open Catalog is a managed service for Apache Polaris™." New customers "can't sign up for their first Open Catalog account" and are pointed at Horizon Catalog; billing on REST API requests "will begin in the first half of 2026". — Snowflake Open Catalog docs, 13 Sep 2026

Other head-to-heads

Same market

Get free Demo Center access

You've seen the proof

Ready for a number? Scope your deployment and we'll price it against your own economics.

Get your quote →