Head-to-head · updated 13 September 2026
DataShield vs Dagster: the pipeline ran, but was the agent allowed?
Dagster earned its following. It flipped orchestration on its head by making the data asset the unit of work instead of the task. You declare what should exist. Dagster works out what to run, then tracks lineage, freshness and failures for you. The asset graph is a better mental model than a task DAG, and plenty of good teams will tell you so. One thing to know before you sign: on 13 July 2026 Prefect announced it was buying Dagster Labs, and from August the combined company trades as Prefect. Dagster and Dagster+ keep their names, and the open-source license is unchanged.
We are not an orchestrator. DataShield does not schedule your jobs and will not pretend to. We govern the data agents read after the pipeline has done its work. Datasets are tokenized at ingest; agents query tokenized data over MCP; detokenization is a privileged, audited operation. Every governed tool call is re-checked against the agent's current authority before it runs, and each decision is sealed into a hash chain you can verify without trusting us. Most of our buyers keep their orchestrator. Here is the honest split, rows Dagster wins included.
The short version
Pick DataShield when
- An examiner will ask you to prove an agent's access log was not edited. Article 12 of the EU AI Act asks it. HIPAA §164.312(b) asks it. Our chain answers with math, not a policy PDF. Run the verifier.
- You need to pull an agent's authority mid-job and have the very next tool call fail. Not the next token refresh. The next call.
- The columns an agent reads must be tokenized or generalized first, and someone has to hold the keys. That someone is you.
- The data plane, the policy engine and the evidence all have to run on your own kit, with no control plane calling home.
Pick Dagster when
- You need an orchestrator. That is the whole job here, and it is not one we do. Asset-oriented scheduling, backfills, retries, partitions, sensors: all theirs.
- You want lineage and freshness across your whole stack, including tools we never touch. Their asset graph spans dbt, Snowflake, Spark and the rest.
- SOC 2 Type II matters on the paper this quarter. Their pricing page lists it on every tier, with HIPAA and SCIM higher up. We have none of the three, and we will not pretend.
- You want a managed control plane with a data plane in your own cluster. Their September 2026 hybrid write-up on Kubernetes and Azure is a real, detailed answer.
Bottom line: Dagster makes the asset. DataShield decides what an agent may do with it and keeps proof of the decision. Run both. If agents are already reading production data, the proof gap is the one that bites first.
Feature by feature
Competitor cells describe what Dagster's public site, docs and blog say as of the date above. If we have mischaracterised something, email support@myorg.ai and we will correct it, credited.
| What matters | DataShield | Dagster | Edge |
|---|---|---|---|
| Pipeline orchestration | We ingest, transform, snapshot and watch storage on a schedule. That is dataset plumbing, not orchestration. No DAG engine, no backfills, no partitions, no sensors. | The core product, and a good one. Software-defined assets, declarative automation, retries, partitions, run history, cross-tool asset graphs. | ◇ |
| Audit evidence | A SHA-256 hash chain with Ed25519-signed checkpoints that are themselves chained. Verification names the failure: tampering, insertion, deletion or truncation. Try the verifier. | Run history, asset materialization history and audit logs on the upper tiers. We found no published tamper-evidence mechanism in their docs. | ◆ |
| Agent authorization | Every governed tool call passes a scope ceiling, an authority tier and a revocation re-check before dispatch, then gets metered and attributed to the agent. It fails closed. | Compass, their text-to-analytics Slack app, is governed by private Slack channels, a table allowlist and pull requests to a config repo. That is a perimeter, not a per-call decision. | ◆ |
| Break-glass | Scoped, time-boxed emergency access for agents, gated by admin plus IP allowlist plus step-up. It auto-revokes and cannot be quietly deleted from the log. | Not described in their public material. | ◆ |
| GDPR erasure | Crypto-shred of per-subject key material, plus ISO 27560 consent receipts. Delete the subject key and every ciphertext for that person goes dark at once. The chain still verifies. | GDPR is listed on every pricing tier, with data portability. The erasure mechanism itself is not described, which is fair: an orchestrator does not own your rows. | ◆ |
| Tokenization and masking | Deterministic, join-preserving, vault-reversible tokens applied at ingest, plus quasi-identifier generalization (dates to year, decade or age band; ZIPs to 3 or 4 digits; partial phones, SSNs and emails) with a measured cardinality-reduction score per column. Detokenization is privileged and logged. | Not a feature. You would write it yourself as a pipeline step, and then you own the keys, the vault and the audit. | ◆ |
| PII and PHI classification | 129 field classes covering PII, PHI, financial data and secrets. Regex plus checksum validators (Luhn, NPI, Verhoeff, ABA, IBAN, GTIN), a column-name lexicon and anti-patterns. No model, so verdicts are reproducible from a config digest. All 18 HIPAA Safe Harbor identifiers are discrete classes. It cut over from shadow mode this month, so it is new, not battle-worn. | Their July 2026 case study classified a million Snowflake columns in nine days with an LLM. Read it closely: that is a pipeline a customer built on Dagster, not a classifier Dagster ships. | ◆ |
| Catalog and lineage | A real catalog over live connections: in-place scan and profiling with no rows leaving the source, a business glossary, typed lineage traversal with access gating at every hop, and a 29-command stewardship workflow. PostgreSQL today, not your whole estate. Other providers are declared and not built. | Asset catalog and lineage across every tool the orchestrator touches, plus freshness and health. Broader reach than ours by a wide margin, and it comes free with the pipelines. | ◇ |
| MCP and agents | More than 200 MCP tools across Ontology, Auth, Corpus and Lighthouse. MCP tool tokens carry scope ceilings, delegation is RFC 8693 token exchange with an enforced ceiling, and every call is metered and attributed. | The marketing says AI-native, and Dagster+ AI reasons over run and lineage context. We could not find an MCP server in their public docs. Worth asking them directly. | ◆ |
| Deployment | Self-hosted in your own cloud or data center, or a dedicated single-tenant server we operate. Docker images for Auth, Ontology, Corpus and Lighthouse, with a signed deploy manifest Guardian verifies. Ed25519 audit-signing keys can live in your KMS or HSM. HMAC tokenization keys sit in your environment or derive from your machine key today, not in a KMS. | Open-source Dagster runs anywhere. Dagster+ Serverless is fully managed. Dagster+ Hybrid puts the agent, code servers and run workers in your cluster while Dagster Labs keeps the control plane. | ◈ |
| Open source | Not open source. Design-partner terms include source escrow, so a small vendor is not a single point of failure. | Dagster Core is open source and self-hostable at no license cost, with a large community. The acquisition page says the license does not change. | ◇ |
| Maturity signals | Auth, Guardian and Lighthouse are live in production (Guardian and Lighthouse since April 2026). SOC 2 not yet certified, and we say so. No SCIM endpoint either. | Founded 2018, big OSS community, enterprise logos, SOC 2 Type II and GDPR on every tier, HIPAA on Pro and Enterprise. Now merging into Prefect, which is the one open question. | ◇ |
| Pricing | Published model, scoped instant quote, no sales wall. | Also published, which is rarer than it should be. Solo and Starter list prices, Pro and Enterprise on request. Billing is per credit, where a credit is one asset materialization or one op executed, plus serverless compute per minute. | ◈ |
◆ DataShield leads◇ Dagster leads◈ comparable
Dagster claims are drawn from dagster.io, its pricing page and its own blog, last checked 13 September 2026. We link them below rather than work from memory.
Three things you get here that you won't get from an orchestrator
Proof that survives an audit
Run history tells you a job succeeded. It does not tell a regulator that nobody edited the record. Ours is a hash chain with signed checkpoints, and the verifier says what broke: tampering, insertion, deletion or truncation. That is the property EU AI Act Article 12 and HIPAA §164.312(b) reviewers care about. Try it in your browser, no signup.
Authority that can change mid-flight
An analyst resigns at 4pm on a Friday. Their agent is 20 minutes into a 40-minute job. With DataShield the next governed tool call re-checks current authority and fails closed. A Slack channel with the wrong people still in it does not. How Auth does it.
Columns that are safe before the model sees them
Tokenize at ingest and the agent joins on TOK_ values it can never reverse. Detokenization is a separate, privileged, audited call, and it is tenant-scoped so an admin cannot reach another tenant's vault. See the data plane.
Where Dagster is genuinely stronger
We would rather you heard this from us. The asset model is good engineering. Declaring what should exist and letting the system work out the runs is a cleaner idea than wiring tasks by hand, and the lineage and freshness you get as a side effect are real. Their September 2026 hybrid post is the kind of writing you only get from people who have actually run this in someone's Azure tenant: node pools, GitOps, workload identity, who owns which boundary. The open-source core is genuine, the community is large, and the pricing is on the web page. And on compliance paper they beat us today. SOC 2 Type II on every tier, HIPAA higher up, SCIM and SSO where you would expect. We have none of that yet and we are not going to fudge it.
Here is the push-back, and it is about Compass. Point a language model at your warehouse tables, let people ask questions in Slack, govern it with channel membership plus a table allowlist plus pull requests to a config repo. Every one of those controls is set at the edge, before anyone asks anything. None of them re-checks at the moment of the query, none of them can revoke someone mid-answer, and none produces an artefact an examiner can verify independently. Gartner expects most unauthorized agent transactions through 2028 to be internal policy violations rather than attacks, which is exactly the case a Slack channel handles worst. That seam is the layer we sell, and it sits under an orchestrator quite happily.
Questions worth asking both of us
These are the questions we would want answered if we were the ones buying. Ask them on every call, ours included.
Can you cryptographically prove an audit log entry wasn't deleted?
DataShield: yes. Each record commits to the one before it, checkpoints are Ed25519-signed and chained, and verification tells deletion apart from truncation and from tampering. Run it against a sample chain at /verify. Dagster: their upper tiers list audit logs, and run history is rich. We found no tamper-evidence mechanism in their docs. Ask them what stops a database admin editing a row.
What happens to a revoked agent mid-session?
DataShield re-checks authority on every governed tool call, so revocation lands on the very next call, not the next token refresh. Dagster: we could not find a mid-session revocation story. For Compass the nearest control is removing someone from a Slack channel. Ask how long a compromised identity keeps getting answers after you pull its access.
How does GDPR erasure interact with the audit trail?
DataShield crypto-shreds the per-subject key, which destroys every ciphertext for that person at once, and issues an ISO 27560 consent receipt. Actor identities in the chain are HMAC-committed, so the evidence still verifies once the subject is gone. Dagster lists GDPR support on every tier. What that means for rows inside your warehouse, and for run history that quotes them, is yours to work out. Ask for the mechanism, not the badge.
Does DataShield replace Dagster?
No. If you need an orchestrator, buy an orchestrator. We have no DAG engine, no backfills, no partitions and no sensors, and nothing on our roadmap says otherwise. The normal shape is Dagster building the assets and DataShield governing the datasets your agents then read: classification, tokenization, per-call authority and the audit chain. Two layers, one handshake.
Dagster is merging into Prefect. Should that change my decision?
Probably not this quarter, but ask about it. The announcement on 13 July 2026 says deployments, contracts, pricing and support are unchanged, that the open-source license stands, and that the combined company runs as Prefect from August 2026. Fine. The question worth asking is the three-year one: when two orchestrators sit under one roof, which architecture gets the roadmap, and what does migration look like if the answer is not yours. Get it in writing. We also wrote up the Prefect side of this.
Does DataShield have SOC 2?
Not yet, and we will not imply otherwise. Dagster does, on every tier, and if that is a hard gate for you this quarter then this is an easy call. What we offer instead is a published threat model, a verifier anyone can run, and design-partner terms with source escrow. Auth is live. Guardian and Lighthouse have been in production since April 2026. Details on the security page.
- Prefect is acquiring Dagster Labs. Announced 13 July 2026, closing August 2026, with the combined company operating under the Prefect name from August 2026. Dagster and Dagster+ stay supported and the open-source license is unchanged. — dagster.io/prefect, 13 Sep 2026
- Current hero positioning: "AI-native DataOps platform" and "Data your team trusts. AI that runs on it." — dagster.io, 13 Sep 2026
- Pricing is published: Solo and Starter list prices with Pro and Enterprise on request. A credit is the sum of asset materializations and ops executed, plus serverless compute per minute. SOC 2 Type II and GDPR on all tiers, HIPAA on Pro and Enterprise. — dagster.io/pricing, 13 Sep 2026
- Dagster+ Hybrid: the customer's cluster runs the agent, code servers and run workers while Dagster Labs operates the control plane. — Dagster blog, 8 Sep 2026
- Compass governance as described by Dagster: private Slack channels, configuration to read only specific warehouse tables, and pull requests to the project repo for context changes. — Dagster blog, 1 Jun 2026
- Column classification at scale was a customer-built pipeline on Dagster, not a shipped Dagster feature: "Classifying a Million Snowflake Columns in 9 Days, Solo, with Dagster." — Dagster blog, 16 Jul 2026
Other head-to-heads
DataShield vs Prefect
The other half of the merger, and the same missing layer.
TransformDataShield vs dbt Labs
Models and tests, versus authority at the tool call.
IngestDataShield vs Fivetran
Pipes that move the data, and the plane that governs it.
AllEvery comparison
One honest scorecard per vendor.
See both mechanisms run in your browser: break a live audit chain, revoke an agent mid-session, then ask what your orchestrator still owes you. Demo Center access is free with a work email.
Get free Demo Center accessYou've seen the proof
Ready for a number? Scope your deployment and we'll price it against your own economics.
Get your quote →