In DataJoint, the schema is the workflow. Not a description of the data after it lands, but a formal, executable specification of the process that produces it: tables are the steps, foreign keys are the dependency graph, and each derived table carries the code that computes it. That specification has a name — the Relational Workflow Model — and it answers the question infrastructure architects put to every system they are asked to trust: what is the formal language underneath? What follows is the case for it.
A new class of scientific harnesses arrived in 2025 and 2026. Systems like Edison Scientific’s Kosmos and Anthropic’s Claude Science give AI agents the tools, memory, and guardrails to run long, auditable research campaigns. Each of them had to build or assume a data substrate to stand on. Each drew its boundary just above the researcher’s own experimental data. That is where DataJoint sits: upstream, in the governed foundation where the lab’s own data and workflows live — where scientists and their agents reason over the same model of the science, and where reproducibility, provenance, and trust are properties of the foundation rather than artifacts assembled after the fact. Which raises the question this piece takes up: what is the formal language for the workflows at the heart of that foundation?
The question
What formal language describes a research workflow — the entities, the dependencies among them, and the chain of transformations that turns raw experimental output into a scientific result?
Twenty-eight minutes into a recent conversation with a senior pharma infrastructure architect, he stopped and asked it directly: “What language do we use to describe the process?”
He listed the comparators he had in mind:
- YAML, for network configuration
- Terraform, for cloud infrastructure
- Helm, for Kubernetes packaging
- Ansible, for config-drift management
- Cedar, Amazon’s policy language, for authorization rules
Each is declarative. Each is formal enough that tools can parse it, reason over it, and exchange it. Each became a small standard in its corner of the stack because someone wrote down the spec, and others adopted it.
“If you can define the language or find one that you like,” he said, “that becomes a way to pull it up. I could feed that into anything. I could feed that into an LLM. Now I have written the control plane for where I’m going to use RAG and where I’m going to use context.”
The instinct underneath the question is correct, and it is the same instinct that produces every generation of durable infrastructure: a formal language is what survives the next platform shift. The tools come and go. The spec is what gets standardized, interchanged, and built upon.
So he asked: what is the formal spec for the scientific work DataJoint is recording?
One natural answer is “it’s codified in Python,” and it is a good one: it points to where the schema actually gets written. His question was aiming a level deeper, though. “Python is not a formal spec,” he added. “That’s just a programming language.” Exactly right, and it sharpens the question to its real form: what is the formal spec that Python is a binding to?
The answer to that sharper question was already written down in the DataJoint 2.0 paper (Yatsenko & Nguyen 2026).
What “schema” has meant
DataJoint’s core innovation is that the schema is the workflow: the proven principles of relational database design, refined and augmented to make computation first-class in the data model. To see what that adds, it helps to recall what a schema has meant until now.
The word schema is a database term of art. In Edgar F. Codd’s 1970 relational model, a schema is the formal description of stored data: typed columns, primary keys, foreign keys, integrity constraints. Peter Chen’s 1976 entity-relationship model added a conceptual model above that — entities, attributes, and the relationships between them, expressed at the modeling tier before the table layout is fixed. Both formalisms specify what the data looks like once it has landed. Neither specifies how it got there.
The transformation that produced the data has classically lived elsewhere — in application code, ETL scripts, workflow files, ingestion services. The schema describes the resting state; the production process is somewhere else, recorded (if at all) in logs and filenames after the fact.
DataJoint’s revision is one declarative move. The transformation is part of the schema. A make() method attached to each derived (Computed or Imported) table specifies how an entity in that table gets produced — invoked once per tuple in the join of the upstream tables referenced by its foreign keys. The transformation carries the same formal weight as the type of the column it produces. The schema is no longer a description of what exists. It is a specification of how a result comes into being.
This is the move that lets a schema serve as a workflow grammar — not just a data grammar. And that is the strength of a grammar: a small set of rules generates an unbounded space of valid workflows, comprehensible to anyone who knows the rule.
The schema is the pipeline
The contrast is easiest to see in a picture: the same liquid-chromatography–mass-spectrometry (LC-MS) pipeline as a traditional relational database would model it, and then as a DataJoint schema.
A traditional database: the same pipeline as a relational schema (crow’s-foot notation). The entities as tables, each relationship named and carrying its crow’s-foot cardinality — the shape only, attributes left off for clarity (from the lcms-demo schema):
A computational database: the same pipeline as a DataJoint schema (equivalent to Figure 1 of Yatsenko & Nguyen 2026):
A traditional database records what happened. A computational database defines how it happens.
The ERD is accurate, and it is static. It shows the entities and how they relate — the shape of the data once it has landed — and nothing about how any of it came to be. It cannot say that ACQUISITION, MASS_ANALYSIS, and PEAK_DETECTION are produced by a computation rather than entered by hand.
The DataJoint schema is the same pipeline. It has the same entities and the same keys. Here, though, the schema specifies both: the data that exists, and the dependencies and code that bring it into existence. Tables are workflow steps. Foreign keys prescribe execution order. Every computed table declares its transformation in its own definition, versioned with your code. So the workflow is self-executing: change an input, and everything downstream recomputes, with full lineage preserved.
One formal system — the Relational Workflow Model — where structure, computation, and integrity are queryable and enforceable together. Reproducible by construction. Reusable by default.
| A traditional database | A computational database (DataJoint) |
|---|---|
| Static — records relationships | Self-executing — manages data and runs computation |
| Describes the resting state | Specifies how each result is produced |
| The diagram is drawn about the data | The diagram is generated from the schema. It cannot drift from the pipeline that runs |
The whole table in four lines
The whole-pipeline picture shows the shape. Zoom in on a single table and the same economy appears in its declaration. Take PeakDetection from the lcms-demo pipeline — the computed table where peaks are found in each spectrum under a chosen parameter set.
In DataJoint, the table definition is the whole spec, in four lines:
Source: src/lcms_demo/pipeline/scan.py; its parents MassAnalysis and PeakDetectionParams live in the same module.
Here is essentially what that compiles to in SQL:
Two declarative lines — -> MassAnalysis and -> PeakDetectionParams — carry what the SQL spends most of its length spelling out: four inherited key columns, a four-part composite primary key, and two foreign-key constraints with their exact column tuples and referential actions. DataJoint says the same thing with far less to read. And it says the one thing the DDL cannot. Because PeakDetection is a computed table, its rows are produced by a make() method attached to this same definition, not entered by hand, and DDL has nowhere to put that. Attaching the transformation to the table itself is the Relational Workflow Model move formalized in DataJoint 2.0 — the subject of the next section.
Where the clarity comes from
The DataJoint form is shorter, but the point is not brevity. Each construct carries meaning the SQL has to rebuild by convention:
- A foreign key carries the key, not just a pointer.
-> MassAnalysisinheritsMassAnalysis‘s entire primary key —subject_id, sample_id, session_idx— in a single line, as both the reference and part of this table’s own identity. You never retype an inherited column. The SQL names all four twice, once in the column list and again inside a constraint. AComputedtable’s key is inherited from its dependencies in full. - The
---divider makes identity explicit. Everything above it is what identifies a detection run: which spectra, under which parameters. Everything below is dependent data (total_peaks). SQL relocates that distinction into a trailingPRIMARY KEY (...)clause read separately from the columns it governs. - A name means one thing, and it is enforced. Because keys propagate by name,
subject_idrefers to the same subject everywhere it appears. DataJoint’s semantic matching rejects any join between namesake attributes that do not share lineage. SQL’s natural join has no such guard: two unrelatedidcolumns match silently. - The dependency is the execution order.
-> MassAnalysisdoes not only add a constraint. It placesPeakDetectiondownstream ofMassAnalysisin the DAG, anddj.Computedtogether with itsmake()method declares that the table is produced from its parents. DDL describes storage. It says nothing about what fills the rows, or when. - You declare intent, not mechanism. The DataJoint definition states what the table is: a computed entity, identified by its two parents, holding a peak count. The SQL states how to store it: columns, types, keys, cascade rules. That gap, between a specification and its compiled form, is the same one this essay draws between the Relational Workflow Model and the Python that binds it.
The answer
The formal definition of scientific workflows — the typed entities, dependencies, and chains of data transformation that produce scientific results, in the same sense that Terraform is the formal specification of desired infrastructure state — is the Relational Workflow Model. It is the central contribution of DataJoint 2.0 (Yatsenko & Nguyen 2026) and it is independent of any particular binding, including the Python one most users see.
A Relational Workflow Model is a declarative description of five things:
- The entities involved in the work: each a typed relation with an explicit primary key.
- The dependencies between them: foreign keys forming a directed acyclic graph.
- The transformation that produces each derived entity: a
make()method on each Computed or Imported table, declaring how each tuple in the join of the upstream foreign-key parents becomes a downstream tuple. - The integrity constraints that must hold: referential, type, and uniqueness constraints, all enforced by the database itself rather than by convention.
- The state of every derivation:
ComputedandImportedtables record, for every entity that should exist, whether it does, when it was produced, from what, and by whom.
That is a formal specification. Machines can parse it, tools can reason over it, and it is complete: from the model alone one can recover not only what the workflow is supposed to produce, but the rules for the transformations that produce it, the constraints they must respect, and the state of having (or not having) produced it.
Python is the binding. The binding is expressive and convenient, but it is not the spec. The spec lives one level below the binding, in the structure of the model itself. The grammar, semantics, query algebra, and machine-readable surface are written up in the new Schema as a Workflow Specification concept page.
In a follow-up conversation a few days later, the same architect saw the model demonstrated in a live workflow — every derivation row carrying the git hash of the code that produced it, the foreign-key DAG visible at a glance, every recomputation kicked off by deleting a row and letting the engine bring the world back into agreement with the schema. His phrase for what he was looking at: “You can now go build a dependency tree of provenance.” That is what the Relational Workflow Model produces, by construction. The model is not a new artifact. It is the formalization of what infrastructure architects already build for every other long-lived system, made explicit for the workflows that produce scientific results.
What “formal” gets us that the comparators do not
The architect’s instinct was that a formal language is what makes ecosystem interoperability possible. The question is whether scientific work fits in any of the existing languages.
It does not — at least, not without losing what makes the work scientific.
YAML, Terraform, Helm. These describe a desired static state: this is the network I want, this is the service I want to deploy, this is the cluster I want to stand up. They do not describe derivation. The derivation is delegated to an engine outside the spec — Terraform’s planner, Helm’s templating, Ansible’s runtime. That works for infrastructure because the desired state is the contract. For scientific work, the derivation is the contract. Outsourcing it to an engine breaks the spec at exactly the point where the spec needs to be authoritative.
Cedar. Cedar describes authorization policy: who may do what under what conditions. It is a fine formal language for one slice of a control plane: the gate-keeping slice. It does not describe the process the gates are gating.
CWL, Nextflow, Snakemake. These come closer. They were designed to describe scientific workflows. They do not, however, attach the transformation to the dependency edge as a single construct, and they do not unify data state with workflow state. The workflow lives in one file. The data lives somewhere else. The integrity of the join between them is the user’s responsibility. The structural side-by-side runs across nine concerns: data structure, types, foreign-key integrity, computation, execution order, provenance, drift detection, query interface, and retry. That comparison is laid out in the Comparison to Workflow Languages docs page.
We have tested this directly. A compiler we built takes a CWL project — using a 13-step GATK whole-genome-sequencing pipeline with four levels of nested sub-workflows as the worked example — and produces a self-contained DataJoint workflow: dj.Computed tables, explicit make() methods, and a foreign-key chain that mirrors the CWL step DAG. The same model also exports to Nextflow DSL2, one process per table, channel wiring mirroring the foreign-key chain. That is the demonstration that no information is locked in. The translation is asymmetric in exactly one place. DataJoint requires what CWL does not specify: entity names, typed primary keys, gather group keys. A short annotation YAML supplies that data structure. In operational terms, the Relational Workflow Model is a strict superset of the workflow-language category: anything CWL describes, the Relational Workflow Model describes — plus the data structure the science actually depends on.
The Relational Workflow Model satisfies a stricter brief than any of these. It is declarative (schema, not script); formal (relational algebra over a typed DAG); executable as its own spec (the spec runs itself — make() lives on each derived table, applied across the join of its upstream parents); code-attached at every dependency (no separate workflow file); integrity-bearing (constraints are part of the spec); and state-aware (the spec includes the current production status of every derivation). No comparator delivers all six.
What the formal language unlocks
A formal language is the precondition for interoperability. If the Relational Workflow Model is the spec for scientific workflows — the entities, dependencies, and chains of data transformation that produce scientific results — then:
- An LLM can read it and answer questions about the work without first inferring what was done. The spec is the description.
- A Unity-Catalog-style lineage view can attach to it and gain the transformation on each edge. The catalog records column-to-column dependencies; the Relational Workflow Model records what happened between them.
- A simulation environment can run forward against it: change one node, propagate the effect, see what breaks before the change is committed. (Network operators have been doing this against network topologies for two decades. Scientific work deserves the same affordance.)
- A policy engine like Cedar can attach to specific edges, gating consequential actions. The Relational Workflow Model says what the work is; Cedar says who is allowed to do it. They compose.
- A new tool entering the ecosystem can speak the same language as the tools already there. That is the position the architect was pointing at — “that’s why Cedar took off — Amazon said, this is a really useful way to get people to come play our policy game.”
The Relational Workflow Model is the version of that play for scientific workflows. It is already specified. What remains is to set the model on the table next to Terraform and Cedar, so that the people who ask the formal-language question recognize the answer when they see it.
What adoption looks like
The architect’s harder question was the second one: how does a formal language actually become a standard? If interoperability is the prize, it is unclaimed until the language is adopted by an ecosystem of tools that did not originally know about it.
His own analogy was Network File System. NFS shipped as a simple, idempotent protocol in 1984. It was, in its first years, dismissed — files over a network, too slow, unreliable. Forty years later it is still in use. NetApp built a multibillion-dollar business on it, and a long tail of storage vendors followed. The protocol won not because it was clever, but because it was small enough to be implementable by anyone who needed it.
The Relational Workflow Model has the same shape. The construction is small: typed entities, an FK-directed acyclic graph, a transformation attached to each edge, a small set of integrity constraints, and the state of every derivation. The DataJoint 2.0 paper (Yatsenko & Nguyen 2026) specifies all five. The grammar is implementable in any host language with a relational substrate. Python is the binding we maintain, but it is not the only possible binding. The path to adoption is the same as NFS: make the spec simple enough that an instrument capture system, a regulatory submission packager, and a literature-evidence engine can each publish into or read from the model without rewriting their own data structures.
That is the work the formal language unlocks. The spec lives one level below the binding. The ecosystem can live one level above.
Closing
It was exactly the right question to ask: the kind worth answering with care, in writing, so the answer can be cited rather than restated each time.
The question was: what formal language describes a research workflow — the process that produces a scientific result?
The answer: the Relational Workflow Model. The schema is the specification. Tables are the typed entities, foreign keys are the dependency graph, make() methods on derived tables are the transformations, and integrity constraints and derivation state are part of the spec. Python is one binding; the spec does not depend on it. The model is defined in the DataJoint 2.0 paper (Yatsenko & Nguyen 2026); its grammar, semantics, and algebra are written up in the Schema as a Workflow Specification concept page; and its position among workflow languages is in the Comparison to Workflow Languages page.
This is also the exact meaning of the verb at the foundation of DataJoint’s capability roadmap: codify. To codify an experiment is to express it as a Relational Workflow Model — a formally defined, self-executing workflow whose integrity, provenance, and reproducibility hold by construction and stay free to evolve, so that people and their AI agents build on work that compounds instead of disappearing.
Nothing had to be invented to give this answer. The model has been built into DataJoint’s foundation from the start. Every DataJoint schema is already this specification. What the 2.0 paper (Yatsenko & Nguyen 2026) contributed was to articulate it succinctly and precisely: define the model, name it, and set it beside the other formal languages, so that the people who think in specifications can cite it, compare it, and build on it.