Purpose: End the confusion. One document. Every tool mapped to exactly one job.
You keep asking this because the tools have overlapping names but non-overlapping jobs. The confusion comes from the fact that several tools touch "data" or "workflows" but at completely different levels. This document maps each tool to ONE primary job, explains what that job IS, and shows how they connect.
Think of running an experiment like running a restaurant kitchen:
| Job | Tool | Kitchen Analogy |
|---|---|---|
| Ingredients storage | DVC | The walk-in fridge: stores raw ingredients (data), tracks what came in, version-stamps batches |
| Recipe execution | redun | The head chef: follows the recipe (workflow), knows which steps are done, skips re-cooking identical dishes |
| Ingredient registry | LaminDB | The inventory system: tracks every ingredient's origin, quality cert, allergen info, what dish it went into |
| Cooking log | MLflow | The cooking journal: records oven temp, timing, taste scores for each dish attempt |
| Health inspection report | RO-Crate | The formal documentation package: bundles recipe + ingredients used + cooking log + photos into one inspectable folder |
| Restaurant portfolio website | FAIRDOM-SEEK | The public-facing website: shows all your menus, recipes, ingredients, certifications in a browsable catalog |
| Recipe sharing platform | WorkflowHub | The recipe-sharing site: publishes your recipes so other chefs can find and reuse them |
| Menu planning | Hydra | The menu designer: defines "tonight we cook dishes A, B, C with these variations" |
~/datasets/umls/. DVC creates a tiny .dvc file (a few bytes) containing the SHA256 hash of that 50GB..dvc file goes into Git. The 50GB goes to GCS (gs://cytognosis-data-hub/dvc-cache/).dvc push uploads data. dvc pull downloads data. dvc checkout restores a specific version.dvc.yaml can also define a pipeline (stages with deps/outs), but that's DVC's secondary function.@task. redun builds a DAG from them.redun run my_workflow.py main, it executes the task graph.ln.Artifact with:cytognosis:// URI (for addressing)
- Feature annotations grounded in ontologies (via bionty): "this h5ad has cell types from CL ontology, genes from Ensembl, tissues from UBERON"
- A ln.Run record: "this artifact was produced by Transform X, using Inputs Y, at time Z"
ln.Artifact.filter(cell_types__contains="T cell").df()ro-crate-metadata.json file that describes:cytoskeleton.crate library emits these. The redun_rocrate.py bridge (124 LOC, already exists in cytos) connects redun execution to crate emission.hub.cytognosis.org)
What it does, concretely:
cytoskeleton publish pushes metadata from LaminDB → SEEK.
workflows.cytognosis.org)
What it does, concretely:
hub.cytognosis.org (general SEEK) and workflows.cytognosis.org (workflow-specific).
experiments/neuro-pilot.yaml (which models, which datasets, which HPO strategy)
- models/gnn.yaml (model-specific hyperparameters and search spaces)
- datasets/nbb.yaml (dataset-specific preprocessing steps)
cytos experiment run --model=gnn,transformer --dataset=nbb,pec``mermaid
flowchart LR
classDef config fill:#5145A8,color:#fff
classDef exec fill:#3B7DD6,color:#fff
classDef data fill:#14A3A3,color:#fff
classDef track fill:#E0309E,color:#fff
classDef pkg fill:#8B3FC7,color:#fff
classDef ui fill:#F26355,color:#fff
H["❶ Hydra\n'What to run'\n(YAML config)"]:::config
R["❷ redun\n'Run it'\n(execute + cache)"]:::exec
DVC["❸ DVC\n'Version data'\n(hash + push)"]:::data
LDB["❹ LaminDB\n'Catalog artifacts'\n(lineage + ontology)"]:::track
MLF["❺ MLflow\n'Log ML metrics'\n(params + scores)"]:::track
RC["❻ RO-Crate\n'Package for FAIR'\n(bundle + attest)"]:::pkg
SEEK["❼ SEEK\n'Browse experiments'\n(web UI + ISA)"]:::ui
WFH["❽ WorkflowHub\n'Share workflows'\n(TRS + federate)"]:::ui
H -->|"config"| R
DVC -->|"versioned data"| R
R -->|"artifacts"| LDB
R -->|"metrics"| MLF
R -->|"run metadata"| RC
LDB -->|"catalog entry"| SEEK
RC -->|"workflow crate"| WFH
RC -->|"DOI"| Z["Zenodo"]
`
1. You write: experiments/neuro-pilot.yaml ← Hydra (Job 8)
2. Data is at: ~/datasets/nbb/ tracked by DVC ← DVC (Job 1)
3. You execute: cytos experiment run neuro-pilot.yaml ← redun (Job 2)
4. During run:
- redun checks AST+data hashes, skips cached steps
- Each output registered in LaminDB with ontology tags ← LaminDB (Job 3)
- ML metrics logged to MLflow ← MLflow (Job 4)
- A Workflow Run Crate is emitted per run ← RO-Crate (Job 5)
5. On publish:
- cytoskeleton publish → pushes metadata to SEEK ← SEEK (Job 6)
- cytoskeleton publish → pushes workflow to WorkflowHub ← WorkflowHub (Job 7)
- Zenodo mints a DOI for the crate
- Software Heritage archives the code (SWHID)
`
The Comparison You Keep Asking About
LaminDB vs Kedro
| Dimension | LaminDB | Kedro |
|---|---|---|
| Primary job | Artifact catalog + lineage graph | Pipeline framework + project structure |
| Data catalog | Ontology-grounded features, hash-based, queryable | YAML-based DataCatalog, no ontologies |
| Lineage | Full graph: Transform → Run → Artifact | None (session-only) |
| Versioning | Hash-based deduplication | Timestamp-based copies |
| Bio awareness |
bionty plugin (CL, UBERON, GO, MONDO) | None |
| Caching | Content-addressed | Session-based |
| Why Kedro was rejected | — | "Triple overlap with redun (pipeline) and LaminDB (catalog)" |
The fundamental difference: Kedro tries to be a pipeline runner AND a data catalog AND a project template. LaminDB is ONLY a catalog (but a much better one, with ontologies and lineage). redun is ONLY a pipeline runner (but a much better one, with AST hashing). Using the two dedicated tools beats using one jack-of-all-trades.
LaminDB vs DVC
| Dimension | LaminDB | DVC |
|---|---|---|
| Primary job | "What IS this artifact? What's in it? Where did it come from?" | "Where IS this file? What version?" |
| Knows contents | Yes (features, cell types, genes, ontologies) | No (just a hash) |
| Lineage | Full graph (code → run → artifact) | Limited (dvc.yaml deps/outs) |
| Stores data | No (references VFS/GCS URIs) | Yes (content-addressed cache on GCS) |
| Git integration | Independent | Tightly coupled (.dvc files in Git) |
They're complementary: DVC versions the bytes. LaminDB catalogs the meaning.
FAIRDOM-SEEK vs LaminDB
| Dimension | FAIRDOM-SEEK | LaminDB |
|---|---|---|
| Primary job | Web-browsable experiment registry | Programmatic artifact catalog |
| Interface | Web UI (Ruby on Rails) + REST API | Python API (
import lamindb as ln) |
| Users | Collaborators, reviewers, funders, the public | Developers, data scientists (you) |
| Data model | ISA (Investigation-Study-Assay) | Artifact + Transform + Run + Feature |
| Federation | Mirrors to fairdomhub.org, Bioschemas | None (local instance) |
| Daily use | Browse, review, share | Query, annotate, track |
They're complementary: LaminDB is the developer-facing catalog (like a database). SEEK is the human-facing portal (like a website). cytoskeleton publish is the bridge: it reads from LaminDB and writes to SEEK.
FAIRDOM-SEEK vs WorkflowHub
| Dimension | FAIRDOM-SEEK (
hub.cytognosis.org) | WorkflowHub (workflows.cytognosis.org) |
|---|---|---|
| Same code? | YES — same Ruby on Rails codebase | YES — same codebase, different config |
| Content | Experiments, data, models, protocols, publications | Workflows only |
| Standard | ISA (Investigation-Study-Assay) | Workflow RO-Crate + GA4GH TRS |
| Federation | fairdomhub.org | workflowhub.eu |
| Why two? | Separation of concerns: experiments vs workflow definitions | — |
WorkflowHub vs MLflow
| Dimension | WorkflowHub | MLflow |
|---|---|---|
| Primary job | Publish and share workflow definitions | Track ML experiment runs |
| Content | "Here's our KG build pipeline: steps, inputs, outputs" | "Run #47: lr=0.001, val_loss=0.23, model.pt" |
| Audience | External community, reproducibility | Internal team, ML iteration |
| Federation | Yes (TRS API) | No |
Summary: One Tool Per Job
`
┌──────────────────────────────────────────────────────────────────┐
│ │
│ ❶ DVC ──────────── "Version the bytes" │
│ ❷ redun ─────────── "Execute the workflow" │
│ ❸ LaminDB ────────── "Catalog the artifacts (programmatic)" │
│ ❹ MLflow ─────────── "Log the ML experiments" │
│ ❺ RO-Crate ──────── "Package for FAIR publication" │
│ ❻ FAIRDOM-SEEK ──── "Browse experiments (web UI)" │
│ ❼ WorkflowHub ───── "Share workflow definitions" │
│ ❽ Hydra ──────────── "Describe what to run (config)" │
│ │
│ ✗ Kedro ──────────── REJECTED (redundant with ❷+❸) │
│ │
└──────────────────────────────────────────────────────────────────┘
``
No overlaps. Each tool does exactly one thing. They connect via metadata cross-references (hashes, URIs, IDs), not by duplicating each other's work.
| Document | What it covers |
|---|---|
| tools-infrastructure-stack.md | The 16-layer architecture with all tools mapped |
| provenance-lineage.md | The DVC+redun+LaminDB+MLflow cooperation model, CytognosisExecutor code |
| seek-workflowhub.md | SEEK + WorkflowHub deployment, ISA model, publish flow |
| master-metaplan.md | Phase 2A tracks infrastructure implementation |