Swasti · mForm V2→V3
Home / Import pipeline

Import pipeline

Six-job pipeline turning Swasti mForm V2 data into Frappe V3 records. A16.0 (Livelihood greenfield) is parallel-track and unblocked.

Key posture
  • Gated per-form, not end-to-end automated — each job verified before the next starts
  • A16.0 (Livelihood) is greenfield: no V2 source needed, can start now
  • A14 closed 2026-05-06 via direct MongoDB Atlas read — masters extracted, A16.1 + A16.4 unblocked

Pipeline jobs

A14
completed
Confirm V2 access method — Mongo Atlas direct read
No dependencies — can start now
A16.0
completed
Greenfield Livelihood (no V2)
No dependencies — can start now
A16.1
in progress
Cross-form (formId, order)→fieldname registry — questions_index.jsonl is the foundation
Depends on: A14
A16.2
pending
Skill run, per-form, gated
Depends on: A16.1
A16.3
pending
Self-link collapse → audit log
Depends on: A16.2
A16.4
in progress
Master-data hydration — JSONL exports ready, awaiting Doctype targets
Depends on: A14A16.2
A16.5
pending
Member + response import
Depends on: A16.3A16.4
A17
pending
Staging validation
Depends on: A16.5

Job detail

A14 Confirm V2 access method — DONE 2026-05-06 completed

Resolved via direct MongoDB Atlas read on mformv2_swastihealthprod. Read-only credentials persisted to ~/Dhwani/keys/swasti-mform-v2-mongo.txt. Export script + venv at ~/Dhwani/swasti-mform-migration/scripts/export_masters.py; ~30s to refresh all masters.

Closes A14, A14b (masters: 185 schemes, 31 docs, full geography), A15 (probe). Unblocks A16.1, A16.4, A18. See note 09 for the architectural surprises.

A16.0 Greenfield Livelihood Doctype pending

No V2 source — Livelihood is a new program. Author the Frappe Doctype directly from the PM sheet (05-pm-design-doc.md, Livelihood tab). This is the only unblocked job that can start immediately. See action item A20.

Owner: Dev team · Related: A20

A16.1 Cross-form (formId, order) → fieldname registry pending

Build the lookup table mapping (formId, order) pairs to V3 Doctype fieldnames. This is the universal join key across all 13 V2 forms. Validated approach: PM already uses this pattern (e.g. Health Screening (1010) || Current Status (order 36)). All cross-form references use order, never field names.

Depends on: A14 · Owner: Dev team

A16.2 Skill run — per-form, gated pending

Run the mform-to-frappe conversion skill on each V2 form JSON, one form at a time. Each run produces 4 files: Doctype JSON + .py + .js + .dart. Verify against PM design (A19 delta report) before proceeding to the next form.

Depends on: A16.1 · See: Conversion skill note · A19

A16.3 Self-link collapse → audit log pending

mForm's follow-up pattern creates new response rows per follow-up (self-link via closedItself on form 1011, terminal codes 2/4/5/12/13/14 on order 22). Collapse these chains into Frappe-native audit log entries on the original record. Decoder is fully known from V2 JSON analysis — no additional mForm team input needed.

Depends on: A16.2 · Decision: D3 · See: V2 JSON — self-link section

A16.4 Master-data hydration pending

Load reference data into Frappe: 107 schemes, document master, donor master, geography hierarchy (district → block → GP → village). Custom Fields catalog (Occupation/Marital/Religion/Caste) is already in the PM sheet and partially resolves A14b. The 107 schemes, doc master, donors, and geography values still need a V2 export endpoint.

Depends on: A14, A16.2 · Related: A14b

A16.5 Member + response import pending

Import all existing members and their historical responses from V2 into Frappe V3. Geography link resolved by A16.4; self-links collapsed by A16.3; field names resolved by A16.1 registry.

Depends on: A16.3, A16.4 · Owner: Dev team

A17 Staging validation pending

Run the full import end-to-end on a staging Frappe instance. Verify member counts, response counts, status transitions, master-data links. Sign off before production import.

Depends on: A16.5 · Owner: Dev team

Last updated 2026-06-05