The Cytognosis grant pipeline is an ambitious system that transforms unstructured funding opportunity announcements (PDFs, DOCX, XLSX) into structured, harmonized data, and then renders that data into funder-specific submission documents. The architecture spans six Python modules, a 27-slot canonical template schema (v1.2), 17 funder profile YAML files, 71 mapped funding opportunities, and a Nox/CLI-based orchestration layer.
Key finding: The schema layer (manifest, slots, funders, opportunity mappings) is remarkably mature and well-designed. The pipeline code layer (parser → extractor → harmonizer → generator → renderer) has production-quality bookends (parser and extractor work end-to-end) but a hollow middle (harmonizer uses placeholder logic, generator needs templates, renderer lacks Pandoc/Quarto wiring).The 27-slot system (U01–U22 universal + A01–A05 administrative) is the intellectual backbone of the entire pipeline. Slot IDs are immutable and append-only, which is the correct design choice.
| Metric | Value | Assessment |
|---|---|---|
| Total slots defined | 27 | ✅ Sufficient for all 71 tracked opportunities |
| Slots with authored prose | 4 (U01, U03 partial, U06 partial, U08) | ⚠️ 15% coverage |
| Slots as stubs | 23 | 🔴 Content authoring is the main bottleneck |
| Sub-slots identified but deferred | 30+ | Tracked in CHANGELOG v1.1/v1.2 |
heading: text, not in slot bodies.
Design decision (sound): Length limits are render-time, not author-time. Content is authored at the longest funder limit and trimmed during rendering.
The F-field family (F01–F38) captures 38 metadata dimensions per opportunity. This was extended from 11 (Monday board columns) to 30 in v1.1, then to 38 in v1.2.
| Version | Fields | Driver |
|---|---|---|
| v1.0 | F01–F11 | Existing Monday columns |
| v1.1 | F12–F30 | Research of 71 opportunities |
| v1.2 | F31–F38 | Drive funding-research cross-reference (insider connections, strategic pairings, thematic tags) |
Notable v1.2 additions:
strategic_connection — Tracks insider contacts (Adam Marblestone, Milad Alucozai, Angela Pisco, etc.)strategic_pairing — Links opportunities that move together (NSF Tech Labs ↔ Convergent Research FRO)thematic_tags — Hashtag taxonomy (#Moonshot, #FRO, #Cytoscope, etc.)heilmeier_required — Whether Heilmeier Catechism is mandatory17 funder YAML files exist under schemas/funders/. Coverage:
| Funder File | Kind | Has Template Definition | Has F31–F38 |
|---|---|---|---|
| heilmeier.yaml | universal_baseline | No | Unknown |
| nih_r01.yaml | federal_grant | Likely | Unknown |
| nih_r21.yaml | federal_grant | Likely | Unknown |
| nsf_xlabs.yaml | federal_OT | Yes (via primary_artifacts) | ✅ Yes |
| nsf_tech_labs.yaml | federal_OT | Likely | Unknown |
| nsf_sbir_phase1.yaml | federal_grant | Likely | Unknown |
| nsf_sbir_phase2.yaml | federal_grant | Likely | Unknown |
| nsf_sbir_phase2b.yaml | federal_grant | Likely | Unknown |
| arpah_solution_summary.yaml | federal_BAA | Yes (3-page solution summary) | ✅ Yes |
| arpah_mission_office.yaml | federal_OT | Likely | Unknown |
| arpah_program_iso.yaml | federal_OT | Likely | Unknown |
| doe_genesis.yaml | federal_OT | Likely | Unknown |
| astera_residency.yaml | private_fellowship | Likely | Unknown |
| brains_accelerator.yaml | private_fellowship | Likely | Unknown |
| foresight_nodes.yaml | private_fellowship | Likely | Unknown |
| google_impact_challenge.yaml | private_grant | Likely | Unknown |
| yc_nonprofit.yaml | accelerator | Likely | Unknown |
corporate_credit shared template (~7 funders), nonprofit_discount shared template (~20 funders), vc_investment shared template (~7 funders).
groups.yaml defines 6 U-groups, 4 A-groups, 8 F-groups, and 11 composition presets. This is a powerful inheritance system that allows funder profiles to declare preset: preset_federal_OT instead of listing every slot.
Status: Fully designed but not yet adopted. All 17 existing funder YAMLs still use explicit slot lists. Migration to preset shorthand is deferred to v1.3.
opportunity_mapping.yaml contains 71 detailed opportunity blocks with all 30+ F-fields, required/useful slot lists, and submission notes. opportunity_mapping.csv is the flat tabular equivalent, loadable into Monday via CSV import.
This is the most complete artifact in the entire system. 71 opportunities, each with 30+ metadata fields, slot citations, and sub-slot specifics.
parser.py) — ✅ PRODUCTIONThe parser handles PDF, DOCX, and XLSX ingestion using PyMuPDF as the primary engine. Key capabilities:
{==highlight==}, {--deletion--}, {++insertion++} directly into the markdown output._tables.md files with markdown tables._metadata.json files with document properties.data/raw/papers/ and all grant documents across 4 funder directories. Running via nox -s parse_papers and nox -s parse_grants.
extractor.py) — ✅ PRODUCTION (with caveats)Uses instructor library with Pydantic validation to constrain LLM output to the GrantInfo schema. Successfully extracted structured JSON for NSF X-Labs documents.
GrantInfo schema. It frequently outputs $ref dictionary syntax instead of arrays, triggering Pydantic validation errors and instructor retry loops.use_mmap: False is critical for GPU offloading on Strix Halo hardware.nsf-topic2-fy26-xlabssensingandimaging.json | ✅ High (deadlines, phases, requirements extracted correctly) |
| nsf-otaso-fy26-xlabsinitiative.json | ⚠️ Low (empty arrays for deadlines, contacts, requirements after retry exhaustion) |
| nsf-otaso-fy26-xlabsinitiative_criticmarkup.json | ✅ Good (eligibility, funding, phases extracted) |
| xlabs_initial_submission_template.json | ✅ Good (template structure extracted) |
registry.py) — ✅ PRODUCTIONComprehensive registry system with:
GrantTemplateRegistry — loads manifest, lazily resolves funder profiles and templatesFunderProfile — 16-field dataclass mapping F-fields to typed attributesGrantTemplate — full template definition with sections, fields, validation rulesvalidate_submission() — checks required slots, page limits, useful-slot warningstemplate_definition and primary_artifacts fallback formats, and provides a clean public API.
harmonizer.py) — ⚠️ SCAFFOLDEDThe harmonizer is architecturally sound (it takes extracted GrantInfo and maps it to canonical CanonicalSection objects with slot IDs) but every section is filled with placeholder text ("Placeholder mapped content for {section.id}").
generator.py) — ⚠️ SCAFFOLDEDThe generator uses instructor to create submission scaffolds from funder profiles. It works end-to-end but produces generic outlines rather than real proposal content.
nsf_xlabs.md exists in templates/)slots/ directoryGrantInfo extractionsrender.py) — ⚠️ SCAFFOLDEDThe renderer provides TemplateRenderer with render_template() and render_from_string() methods. It correctly initializes a Jinja2 environment with autoescaping and block trimming.
compile_document nox session exists but lacks wiring to this module| Funder | Directory | Files | Parsed | Extracted | Harmonized |
|---|---|---|---|---|---|
| NSF X-Labs | data/staged/grants/nsf_xlabs/ | 16 | ✅ All | ✅ 4 JSONs | ❌ |
| ARPA-H Delphi | data/staged/grants/arpah/programs/delphi/ | 8 | ✅ All | ❌ | ❌ |
| ARPA-H EVIDENT TA1-3 | data/staged/grants/arpah/programs/evident/ta1_3/ | 6 | ✅ All | ❌ | ❌ |
| ARPA-H EVIDENT TA4 | data/staged/grants/arpah/programs/evident/ta4/ | 16 | ✅ All | ❌ | ❌ |
| ARPA-H PROSPR | data/staged/grants/arpah/programs/prospr/ | 8 | ✅ All | ❌ | ❌ |
| ARPA-H Mission Office ISOs | data/staged/grants/arpah/mission_office_isos/ | 4 | ✅ All | ❌ | ❌ |
| ARPA-H Templates | data/staged/grants/arpah/templates/ | 15 | ✅ All | N/A | N/A |
| NSF Tech Labs | data/staged/grants/nsf_tech_labs/ | 6 | ✅ All | ❌ | ❌ |
| DOE Genesis | data/staged/grants/doe_genesis/ | 6 | ✅ All | ❌ | ❌ |
These are the actual submission template documents from ARPA-H, parsed into markdown:
arpah_solution_summary_template.md (3-page Solution Summary)arpah_solution_summary_content.md (Content guide)arpah_solution_summary_template.yaml (Structured YAML template definition)arpa-h_tdd_task_description_document_-_ots.mdarpa-h_tech_management_proposal_-_ots.mdarpa-h_cost_proposal_ots.mdarpa-h_cost_proposal_workbook_-_ots.md (1.2MB budget workbook)arpa-h_admin-national-policy-document-ots-amend_2.mdxlabs_initial_submission_template.md (8-page Written Proposal)xlabs_initial_submission_content.md (Content guide)NSF_XLabs_Consolidated_Reference.md (Merged reference document from 3 NSF docs)cytos grants ...)| Command | Implementation | Status |
|---|---|---|
| cytos grants parse | GrantDocumentParser | ✅ Working |
| cytos grants extract | GrantInfoExtractor + LLMConfig | ✅ Working |
| cytos grants harmonize | GrantDocumentHarmonizer | ⚠️ Placeholder |
| cytos grants generate | GrantMaterialGenerator | ⚠️ Scaffolded |
| cytos grants registry list | GrantTemplateRegistry.list_funders() | ✅ Working |
| cytos grants registry show | GrantTemplateRegistry.get_funder() | ✅ Working |
| cytos scholarly parse-papers | GROBIDParser | ✅ Working |
| Session | Command | Status |
|---|---|---|
| parse_grants | cytos grants parse on archive input | ✅ Working |
| parse_papers | cytos scholarly parse-papers on data/raw/papers | ✅ Working |
| extract_grants | cytos grants extract on all staged grants | ✅ Working |
| extract_nsf_xlabs | cytos grants extract on NSF X-Labs only | ✅ Working |
| compile_document | cytos grants compile with Pandoc | ⚠️ Exists, not wired |
| harmonize | cytos grants harmonize | ⚠️ Placeholder |
tests/ reference grant, extractor, harmonizer, registry, or any grants module. This is the single biggest risk in the system.
The only validation that exists is:
1. Structural verification of slot files (mentioned in CHANGELOG v1.0: "7 cross-consistency blockers fixed")
2. The validate_submission() method in registry.py (runtime, not tested)
| Gap | Impact | Effort |
|---|---|---|
| Harmonizer semantic mapping | Blocks slot-aware proposal generation | High (LLM integration) |
| Renderer Pandoc/Quarto wiring | Blocks PDF/DOCX output | Medium |
| Funder-specific Jinja2 templates | Only NSF X-Labs exists | Medium per funder |
| Unit tests for all 6 modules | No safety net for refactoring | High (comprehensive) |
| Nemotron-70B model switch | Extraction quality + speed | Low (reboot + config change) |
| Gap | Impact | Effort |
|---|---|---|
| UMLS/SnomedCT semantic tagging | Blocks accurate medical term matching | High |
| Monday.com board sync | Manual board updates required | Medium |
| Funding opportunity scraper | Manual discovery of new opportunities | High |
| Slot prose authoring | 23/27 slots are stubs | High (content work) |
| Preset migration | 17 funder YAMLs use verbose explicit lists | Low |
| Schema validators (lib/validators/) | No automated consistency checking | Medium |
| Gap | Impact | Effort |
|---|---|---|
| Google Docs renderer | Views in Workspace | High |
| XLSX budget auto-fill | Budget template automation | Medium |
| 15 Tier-2 funder profiles | Long-tail coverage | Low per funder |
| Lightweight templates (corporate_credit, nonprofit_discount, vc_investment) | Covers ~34 opportunities | Low |
1. Slot IDs are immutable. Never renumber. New slots get appended (U23, U24...).
2. No funder-specific prose in slot files. Funder framing lives in YAML heading: text.
3. Source-of-truth stays in git. Google Docs renders are views, not sources.
4. Author at longest funder limit; trim at render time.
5. F-fields describe the opportunity, not the proposal. They live on funder YAMLs and Monday boards, not in slot files.
6. Pydantic schemas in extractor.py are the LLM contract. The GrantInfo model defines what the LLM must produce. Any schema change here requires re-extraction of all documents.
7. instructor library provides retry + validation loop. The LLM is retried automatically when output fails Pydantic validation. This is powerful but model-dependent (Llama 3.1 8B struggles; Nemotron-70B should be reliable).