Swasti · mForm V2→V3

mform_to_frappe skill — reference for Swasti import pipeline

Companion to 01-kickoff-brief.md §10, 02-reliance-frappe-context.md, and 03-mform-json-analysis.md. This file documents the existing Dhwani migration pack that converts mForm form JSONs into Frappe Doctypes (web side) + a Flutter FormEventConfig (mobile side). It is the tooling the team will run for action item A16.

Source extracted to: ~/Dhwani/swasti-mform-migration/raw/mform-to-frappe-skill/mform_to_frappe/ — 82 files, 11 phase folders.


0. TL;DR — how do I run it?

The pack is not a single CLI converter. It is a per-form workflow with three AI-agent skills + one fetch script + 7 phase folders of step-by-step recipes. The flow:

00a frappe setup   →   bench, custom app, modules, geography masters       (once)
00b flutter boot   →   Flutter app skeleton, theme, branding                 (once)
01  fetch forms    →   node scripts/getAllforms.js → form JSONs on disk      (once)
02  convert web    →   per form: invoke `mform_to_frappe` skill              (per form)
                       → produces DocType JSON + .py + .js (3 web artifacts)
03  verify web     →   bench migrate, Desk checklist, API smoke tests        (per form)
04  sync mobile    →   first form: invoke `frappe-mobile-form-engine`        (once)
                       per form: invoke `frappe-form-event-sync`             (per form)
                       → produces .dart `FormEventConfig` (4th artifact)
05  verify mobile  →   on-device + dart tests                                (per form)

Each form converted produces four files in parity that must stay in sync forever:

  1. <app>/<module>/doctype/<doctype>/<doctype>.json — DocType definition
  2. <app>/<module>/doctype/<doctype>/<doctype>.py — Python controller (server validate)
  3. <app>/<module>/doctype/<doctype>/<doctype>.js — JS client script (Desk)
  4. <flutter_app>/lib/form_events/doctypes/<doctype>.dart — Mobile event config

One form at a time. Verify each before moving to the next. Skipping verification breaks the sync engine’s topological ordering. (dos-and-donts.md)


1. Skill structure

mform_to_frappe/
├── README.md, DESIGN.md, PLAN.md            ← onboarding + design spec
├── DESIGN-corrections.md, PLAN-corrections.md  ← 2026-04-22 corrections (the "current" intent)
├── dos-and-donts.md                         ← gotchas list — print this

├── 00a-frappe-setup/                        ← bench setup, modules, geography masters
├── 00b-flutter-bootstrap/                   ← Flutter skeleton + theme + release builds
├── 01-fetch-forms/                          ← scripts/getAllforms.js usage
├── 02-convert-web/                          ← 9-step recipe + 3 templates (DocType/py/js) + recipe selector
├── 03-verify-web/                           ← Desk checklist + python tests + API smoke + regression log
├── 04-sync-mobile/                          ← engine setup (once) + per-form sync + Dart reference
├── 05-verify-mobile/  (new in corrections)  ← Dart tests + on-device checklist

├── reference/                               ← Rosetta-stone tables (consult anytime)
│   ├── mform-to-frappe-mapping.md
│   ├── field-types-table.md                 ← input_type → fieldtype master table
│   ├── validation-layer-matrix.md           ← VAL_* codes → server / JS / Dart layer
│   ├── naming-rules-matrix.md               ← autoname / naming_rule pairings
│   ├── frappe-quirks.md                     ← 14 quirks the POC paid to learn
│   ├── architecture-overview.md             ← SDK API, 22 widgets, lifecycle
│   └── glossary.md

├── recipes/                                 ← 7 recurring conversion patterns
│   ├── simple-form.md
│   ├── form-with-loops.md
│   ├── geography-cascade.md
│   ├── conditional-visibility.md
│   ├── scoring-computation.md
│   ├── did-range-filter.md
│   └── fetch-from-read-only.md

├── scripts/                                 ← Node scripts (only one)
│   ├── getAllforms.js                       ← fetch mForm JSONs via mForm API
│   ├── getAllforms.test.mjs
│   ├── package.json                         ← Node ≥ 20, "type": "module"
│   └── .env.example                         ← MFORM_ORG_ID, MFORM_PROJECT_ID, MFORM_TOKEN

└── skills/                                  ← three custom skills (vendored)
    ├── README.md                            ← by-phase / by-skill lookup
    ├── mform_to_frappe/SKILL.md             ← per-form web converter (Phase 02)
    ├── frappe-mobile-form-engine/SKILL.md   ← first-form-only Dart engine bootstrap (Phase 04)
    └── frappe-form-event-sync/SKILL.md      ← per-form Dart sync (Phase 04)

Origin: Generalised from the SNF POC (Frappe + mForm + Flutter). The pack’s stated goal (DESIGN.md §1): a new dev produces a working Doctype + Python + JS + Dart for one form in under a day, never re-solving POC problems.


2. Phase 01 — fetch the form JSONs

scripts/getAllforms.js is the only script. Node ≥ 20.

cd scripts
npm install   # installs the sole dep, then nothing else

# either prompt-mode
node getAllforms.js

# or env-mode
export MFORM_ORG_ID=...           # Swasti's mForm organisation ID
export MFORM_PROJECT_ID=...       # the project containing the forms
export MFORM_TOKEN=eyJh...        # auth token (long-lived bearer)
node getAllforms.js

Output: one JSON per form into the script’s working dir. For Swasti this has already been run — the result is ~/Dhwani/swasti-mform-migration/raw/mform-forms/1000.json … 1012.json (13 forms, see 03-mform-json-analysis.md §1). When we need to re-pull (e.g., after the mForm team confirms the language[] translations or hands over condition strings), we re-run with the same env vars.


3. Phase 02 — mform_to_frappe skill (web side)

Invoked once per form. Inputs (the skill prompts for these):

  • form JSON path
  • target Frappe module name (e.g., swasti)
  • autoname prefix (e.g., MEM for Member Profile, SCH for Scheme)
  • DocType name (e.g., Member Profile)
  • is_submittable (yes/no)
  • permission roles list

Outputs: 3 web files at the canonical Frappe paths above. The 4th file (.dart) is produced in Phase 04 by frappe-form-event-sync.

The skill internally follows the 9-step recipe (02-convert-web/step-by-step.md):

StepWhat it doesReference
1Note form ID, title, submittable flag. Record description: "form_<id>" for cross-refdos-and-donts.md
2Decide autoname / naming rule / modulereference/naming-rules-matrix.md
3Map every language[0].question[] entry → Frappe fieldtypereference/field-types-table.md
4Identify loops (input_type 20/21) → plan child DocTypes (istable: 1)recipes/form-with-loops.md
5Identify DID cascades (geography) → plan link_filtersrecipes/geography-cascade.md
6Identify restrictions → plan depends_on / mandatory_depends_on + Python guardsrecipes/conditional-visibility.md
7Write DocType JSON02-convert-web/doctype-json-template.md
8Write Python controller02-convert-web/python-controller-template.md
9Write JS client script02-convert-web/js-client-script-template.md

4. Conversion logic — field by field

4.1 input_type → fieldtype (the rosetta stone)

From reference/field-types-table.md:

mForm input_typemForm meaningFrappe fieldtypeNotes
1Short textDataAdd length: <n> to cap chars
1Long text / paragraphSmall TextMulti-line
2Whole numberIntAdd non_negative: 1 if needed
2DecimalFloatprecision: <n> for rounding (validation _id == "14")
3Single-select fixed listSelectOptions as \nA\nB\nC (leading \n allows blank)
3Single-select from masterLinkIf question’s order is in geo_orders OR has DID with large option set
4, 6, 16, 17Multi-selectTable MultiSelectTwo-level structure mandatory — see §4.4
5RadioSelectRenders as dropdown in Desk
6Checkbox yes/noCheck0 or 1
7Image captureAttach Image
10Section headerSection Break (or Tab Break if VAL _id == "150"/"151")
11File uploadAttach
12Audio recordingAttach
13Multi-line textText or Small Text(Sample needed — see gap §10)
14DateDate or Datetime
15AadhaarDatalength: 12 + regex ^[2-9]{1}[0-9]{11}$
18Dropdown w/ parent options hiddenSelect or Link
19GPSGeolocation (or Data coord-string — see frappe-quirks.md)
20, 21Looping groupTable field + child DocType (istable: 1)input_type 21 always referenced in getDynamicOptions[]
22Consent checkboxCheck
23Unit conversionFloat + Select (two fields) + Dart customHandlers
25, 26Geo trace/shapeGeolocation
27Dynamic actionLink (NOT Button) — points to parent DocType per getDynamicOptionThis is the self-link carrier (§7)
28TimeTime
29Content displayHTML(Sample needed — see gap §10)
30Opt-in/out toggleCheck(Sample needed — see gap §10)

4.2 Validation _id codes — validation[] array → Frappe rules

The validation[] array on each question carries codes that translate across three layers (server Python, JS client script, Dart customValidate). Critical codes:

_idActionLayer
"1"Required. If always-visible: reqd: 1. If conditional (has depends_on): mandatory_depends_on: <same expr>. Never both.Server + client
"2"Regex pattern (entry.value is the regex)Server validate() + JS
"3"read_only: 1DocType JSON
"3.1"Override: read_only: 0DocType JSON
"5"Formula (math expr w/ shortKeys) → read_only: 1 + compute in server + clientAll 3 layers
"5.1"Conditional formula — false → frappe.throw()Server validate()
"5.2"Color formulaJS client
"5.3"Conditional read_onlyJS client
"6"OR-visibility (affects parent → depends_on join with `
"6.1"Negate visibilityDocType JSON
"8"All-different checkCustom
"14"Decimal pattern (number is Float not Int)DocType JSON
"26", "26.1-26.4"Date in past N days/weeks/months/yearsServer + JS
"27", "27.1-27.4"Date in future N days/weeks/months/yearsServer + JS
"31"Deselect-all (entry.error_msg is exclusive option ID)JS client
"32"Multi-select max countJS client
"51"Repeater min 1 rowServer validate()
"55.1"Dynamic dropdown — convert SelectLink to master DocType. Use set_query. Never static optionsDocType JSON
"81"Max file size (always 5120 = 5MB)Server
"83"Allowed MIME type per entry — collect ALL 83 for the same fieldServer validation
"92"Auto-ID with cross-form lookup → read_only: 1, treat as LinkDocType JSON
"96"Form loop reference ID → Data or LinkDocType JSON
"99"Auto-ID old-format pattern (tokens: orderXX_YYY, Date(...), fillcounter(NNNN)) → autoname format string or before_insert hookDocType JSON
"99.1"Auto-ID flag (always present on input_type 27) → read_only: 1DocType JSON
"99.2"Auto-ID newer pattern with geo tokens (gramPanchayat, village, state, district, block) → autoname format stringDocType JSON
"150" / "151"Tab Break (rather than Section Break) on input_type 10DocType JSON

Detail: reference/validation-layer-matrix.md.

4.3 Skip logic — parent[] array → depends_on / mandatory_depends_on

IF question.parent is empty → no depends_on

IF question.parent has entries:
    conditions = []
    FOR each parent_entry in question.parent:
        1. Find parent question: parent_q = q_map[parent_entry.order]
        2. Get parent fieldname: parent_fieldname = parent_q.shortKey
        3. Parse parent_entry.value regex to extract option _id(s):
           "^([5])$"         → [5]
           "^([3]|[4])$"     → [3, 4]
           ".*"              → ANY (truthy)
        4. Resolve option _id → option name from parent_q.answer_option
        5. Build condition string

    Check validation _id "6":   YES → join with " || " (OR)
                                NO  → join with " && " (AND)
    Check validation _id "6.1": YES → negate: "!(" + joined + ")"

    field["depends_on"] = 'eval:' + final_expression

CRITICAL: parent[].value references answer_option[]._id, not the option name. You MUST resolve _idname before building the eval string. Frappe Select stores option name, not _id. Get this wrong and depends_on always evaluates false.

If the field is also required (validation._id == "1") AND has a parent: use mandatory_depends_on (same expression) — NOT reqd: 1. Setting reqd on a hidden field blocks form save.

Plus: every mandatory_depends_on needs a server-side guard in _validate_conditional_mandatory() because Frappe enforces mandatory_depends_on client-side only. Mobile or API saves bypass it. (dos-and-donts.md)

4.4 Multi-select two-level rule (Frappe quirk)

For input_types 4, 6, 16, 17 — Table MultiSelect requires:

  1. Option Master DocType (standalone, Data name field — the value list)
  2. Link Child DocType (istable: 1, exactly one Link field → points to the Option Master)

Using Data or Select in the link child fails at import time with:

ValidationError: DocType <X> provided for the field <Y> must have atleast one Link field

This blocks the parent DocType creation silently during git commit. (reference/frappe-quirks.md, also documented in 02-reliance-frappe-context.md.)

4.5 Loops — input_type 20/21

  • input_type 20: simple loop → child DocType (istable: 1).
  • input_type 21: linked loop, ALWAYS in getDynamicOptions[]. Find the matching getDynamicOption where orderToDisplayIn == this question's order. dataOrdersMapping[].toOrder fields are FETCHED → read_only: 1. Non-mapped child fields stay editable.

Child DocType description = "form_<id>_order<n>" (cross-ref).

4.6 Geography cascade

State → district → block → GP → village → hamlet. Pattern handled by DocType JSON link_filters strings + JS cascade-clears + SDK auto-handle on Dart side.

"link_filters": "[[\"<Master>\",\"<parent_fieldname>\",\"=\",\"eval:doc.<parent_fieldname>\"]]"

JS clears children when parent changes:

frappe.ui.form.on("<DocType>", {
  state(frm)    { frm.set_value("district", ""); frm.set_value("block", ""); /* ... */ },
  district(frm) { frm.set_value("block", ""); /* ... */ },
  // ...
});

🔴 Anti-pattern: Adding Dart customHandlers for geography fields. The SDK auto-cascades via link_filters. Hand-written Dart double-clears and overrides the SDK. The skill’s job during Phase 04 is to skip Dart for geography. If it doesn’t, that’s a skill bug. (recipes/geography-cascade.md, dos-and-donts.md)

4.7 Naming rules (reference/naming-rules-matrix.md)

Form typeautonamenaming_rule
Top-level survey formformat:<PREFIX>-{#####}Expression (old style)
Child table (istable: 1)hashRandom
Option masterfield:option_nameBy fieldname
Geography masterpromptSet by user

description field is the cross-ref to mForm:

  • Top-level: "description": "form_<id>" e.g., "form_610"
  • Child tables: "description": "form_<id>_order<n>"

5. Cross-form references & dataOrdersMapping

getDynamicOption (form-level) + dataOrdersMapping (per-link) tell the conversion:

  • input_type 21 (linked loop) → child DocType, toOrder fields are read_only: 1 (auto-fetched from parent on save).
  • input_type 27 (dynamic action / self-link carrier) → Link field that points to the parent DocType (or another DocType per getDynamicOption).

The skill does not auto-cross-link two separately-converted forms. Each form is a Doctype; cross-form fetches are wired manually via fetch_from (matching the dataOrdersMapping).


6. (formId, order) → fieldname registry

The skill does NOT build the cross-form registry. Each form’s conversion produces its own internal q_map (per step-by-step.md STEP 6) keyed by ordershortKey (the latter becomes Frappe fieldname), but this stays inside one form’s conversion and is not persisted across forms.

Per 03-mform-json-analysis.md §finding 2, this registry is Job 1 of the broader Swasti import pipeline — distinct from the skill itself. The pipeline must:

  1. Walk all 13 form JSONs once before any conversion.
  2. Build the (formId, order) → fieldname map (using the same shortKey rule the skill uses internally).
  3. Persist it (JSON or fixture) so downstream can resolve:
    • getDynamicOption.dataOrdersMapping[].fromOrder (parent form) → toOrder (child form) for fetch_from.
    • parent[].order references in skip-logic eval expressions (when the parent question lives in a different form, which 03 doesn’t show but Swasti might have).
    • The self-link collapse rules (§7).

Without the registry, fetch_from and self-link collapse will silently break. Build it before invoking the skill on form 1003 / 1005 / 1011.


The skill does NOT handle the self-link → audit-log transformation. Phase 02 produces a per-form Doctype with input_type 27 carriers represented as Link fields that loop back to the same DocType (getDynamicOption.formId == publishedFormId). That preserves the shape but does not collapse the response chain into one record + N audit entries.

The collapse logic (kickoff D3 / 01-kickoff-brief.md §10) is a separate import-pipeline concern, sitting downstream of the skill:

mForm responses (N rows linked via input_type 27)


 [import pipeline — A16 step we still owe]

        ├─ keep first response → Frappe parent record
        ├─ each subsequent response → audit-log child row
        │   (mapped fields per dataOrdersMapping; status field per closedItself)
        └─ terminate chain when status ∈ closedItself values


 single Frappe record + audit-log child table

Forms that need this: 1003 (Scheme Followup), 1005 (Document Application followup?), 1011 (HS Followup) per 03-mform-json-analysis.md. Form 1011’s closedItself array spells out the terminal status codes (field order: 22, values 2, 4, 5, 12, 13, 14).

For the Frappe side, model after rel-mis’s Submission Approval Log child Doctype + workflow_utils.py:log_workflow_action() on_update hook (02-reliance-frappe-context.md). Rename the Doctype → Followup Log and trigger on field-set of the status field, not workflow_state.


8. Master data handling

The skill does NOT import master data values. It produces:

  • Form definitions (Doctypes).
  • Option Master Doctypes (empty — names but no rows).
  • Geography master Doctypes (empty — to be hydrated separately).

Master data for Swasti — the 107 schemes, ~30 documents, donor list, geography hierarchy — must be loaded by a separate step (Frappe fixtures, bulk insert, or admin import). Per 03-mform-json-analysis.md §gap 2, those values aren’t in the form-JSON dump either; the master data needs a separate pull from mForm V2 (or hand-curation if the mForm team can’t expose it).

Reuse-existing rule: reference/frappe-quirks.md and the rel-mis pattern both warn — check existing masters before creating new ones. rel-mis already has HVRA Training Method, HVRA Training Topic, HVRA Facilitator Type, HVRA Participant Selection, HVRA Volunteer Type. Swasti will share some of these (Geography master) and need its own (Scheme Master, Document Master, Donor Master).


9. Multi-language

Out of scope of the skill. The skill reads language[0] only — and Swasti’s dump only has language[0].lng = "en" anyway (03-mform-json-analysis.md §gap 1).

Frappe’s translation system is the recommended path:

  • View labels (Doctype field labels) → Frappe Translation Doctype (managed in Desk or via .csv files in the app).
  • Data values (free-text, dropdown options) → no auto-translation; remain in entered language. Mixed-script free-text is the unsolved problem flagged in 01-kickoff-brief.md §3 P7.

Swasti needs Kannada + Telugu in addition to English. A14b (mForm team): confirm where the non-English translations live in mForm V2 (separate API endpoint? embedded in language[1]/language[2] of forms not in our dump? or never digitised, only on the mobile UI?). Until answered, the skill produces English-only Doctypes; multi-language is a layer added later.


10. Validation rule emission

The skill emits validations from validation[]._id codes per the table in §4.2. Output layers:

  • DocType JSON: reqd, mandatory_depends_on, read_only, non_negative, precision, length, link_filters, autoname, etc.
  • Python controller (.py): validate() method with regex checks (_id == "2"), conditional formulas (_id == "5.1"), repeater-min-1 (_id == "51"), file size/MIME (_id == "81"/"83"), _validate_conditional_mandatory() for every mandatory_depends_on.
  • JS client script (.js): real-time formula compute (_id == "5"), color indicators (_id == "5.2"), conditional read-only (_id == "5.3"), deselect-all (_id == "31"), max-count (_id == "32").

Critical gap from 03-mform-json-analysis.md §gap 3: the validation rule expressions themselves (validation[].condition) are null everywhere in the Swasti dump. So while the skill knows how to emit each _id, for many Swasti forms the actual condition string is missing. Either:

  • the rules live elsewhere (separate API on mForm V2),
  • they’re input_type-implicit (e.g., input_type 14 always = Date, no extra condition needed),
  • or they were never authored and the form has no validation today.

A14c (mForm team): confirm where the condition strings live. The skill cannot synthesise them.


11. How rel-mis used the pack

The pack is generalised from rel-mis’s predecessor POC (“SNF” — sister project under Catalyst). The most cited rel-mis case study in the docs is form_610 “Library Visit Report” — a 12-question form including Date, Int (with non_negative), Select, Link with link_filters (geography cascade), conditional Data field with depends_on + mandatory_depends_on, and a Table loop into a Library Visit Item child Doctype. (02-convert-web/step-by-step.md worked example.)

For deeper rel-mis Frappe context — naming conventions, hooks, fixtures, permissions — see 02-reliance-frappe-context.md already in this folder. The mform_to_frappe pack is the web-side conversion mechanism; the rel-mis context is what good shape looks like once converted.


12. Known issues / quirks (the POC paid to learn)

From dos-and-donts.md and reference/frappe-quirks.md:

  • 🔴 mandatory_depends_on is client-only. API/mobile saves bypass it. Always pair with a server _validate_conditional_mandatory().
  • 🔴 Don’t add Dart handlers for geography. The SDK auto-cascades via link_filters. Hand-written Dart double-clears.
  • 🔴 Table MultiSelect requires Link child + Option Master. Silent ValidationError at import otherwise.
  • 🔴 fieldname (snake_case) ≠ label. JS frm.set_value("vilage", …) silently no-ops on typo.
  • 🔴 reqd: 1 on a hidden field blocks save. Use mandatory_depends_on exclusively for conditionals.
  • 🔴 link_filters parse error from JSON-in-JSON escaping. Value must be a string of a JSON array literal, not a raw array.
  • 🔴 getDynamicOption[].orderToDisplayIn reference is by order, not by name. Resolve the registry first.
  • 🟡 Every FormEventConfig must be registered in the app’s form-handlers registry (<Org>FormHandlers._registry). Missing entry = handlers silently never run; form looks fine.
  • 🟡 Run bench migrate after every JSON change; bench --site <site> clear-cache for Python/JS-only edits.
  • 🟡 Convert one form at a time. The sync engine’s topological ordering punishes skipped verification.
  • 🟡 GPS as Geolocation vs Data coord-string — see frappe-quirks.md for the case.
  • 🟡 Section Break vs Tab Break depends on validation _id == "150" / "151" on the input_type 10 question.
  • 🟡 Naming-rule mismatches between top-level and child tables (one is “Expression (old style)”, the other is “Random”).

The corrections plan added 3 more in dos-and-donts.md: sync-ordering, status-enum, schema-sync timestamp (per DESIGN-corrections.md §10).


13. Integration path for Swasti — putting it all together

The skill is one stage of the broader import pipeline (kickoff §10). End-to-end:

[mForm V2 instance]
        │  (A14: confirm access method)
        │  (A15: probe script for forms / schemes / members / responses)

1. Form JSONs    →  scripts/getAllforms.js (Phase 01)         ✅ done — 13 forms in raw/mform-forms/
2. Master data   →  separate pull (gap §8) — A14a              ⏳ blocked on mForm team
3. Build (formId, order) → fieldname registry (§6)             ⏳ Job 1 of A16 — NEW
4. Convert each form →  invoke `mform_to_frappe` skill (Phase 02)
                          → DocType JSON + .py + .js per form
                          → option masters + geography masters created empty
5. Verify each form  →  bench migrate, Desk checklist, API smoke (Phase 03)
6. Hydrate masters   →  load 107 schemes, document master, donor master, geography
                          (gap §8 — A16 step)
7. Mobile bootstrap  →  invoke `frappe-mobile-form-engine` ONCE (Phase 04)
                          → engine config, compiler, pattern handlers
8. Mobile per-form   →  invoke `frappe-form-event-sync` per form (Phase 04)
                          → .dart FormEventConfig + registry entry
9. Verify mobile     →  on-device + Dart tests (Phase 05)
10. Import responses →  collapse self-linked chains using §7 logic + registry from step 3
                          → parent records + audit-log children
                          (NEW step beyond the skill — A16 closing step)

What we expect to come out cleanly

  • All 12 production forms (excluding 1012 up-test) → 12 Doctypes + their child tables + option masters (empty).
  • Geography cascade (P2 pain point) handled by link_filters + JS — works once the geography master is hydrated.
  • Conditional visibility (skip logic) → depends_on + Python guards.
  • Member Profile → Scheme Application linkage via input_type 27 (Link to Member).

What needs post-processing

  • Self-link collapse for forms 1003, 1005, 1011 (§7) — pipeline step, not skill output.
  • Master data hydration (§8) — pipeline step.
  • Multi-language translations (§9) — added later; depends on A14b.
  • Validation rule expressions (§10) — many are null in the dump; depends on A14c.
  • input_type 13 / 29 / 30 mappings — rare codes not seen during POC; pipeline must hard-fail on unmapped types and we audit the few cases manually (03-mform-json-analysis.md §finding 3).

Excel form designs from the PM (incoming)

Per kickoff: the PM said designs would be shared soon. Reconciliation:

  • For the 3 existing programs (Scheme / Document / Health) → the JSON-driven Doctype is authoritative. Excel design is a sanity-check / missing-fields-audit document, not the source of truth. If Excel says a field is required but the JSON has validation._id == "1" only as mandatory_depends_on, the JSON wins (it’s what the live mobile users actually fill).
  • For the 4th new program (Livelihood) → no JSON exists yet (the program is greenfield). The Excel design is the only source. We use the same mform_to_frappe 9-step recipe but starting from the Excel directly — i.e., we manually author the form JSON in mForm V2 first (so the skill can run), OR skip mForm V2 entirely and compose the DocType from the Excel using the same templates.
  • Open question to ask the PM: which path for Livelihood? (Does it ever live in mForm V2, or is V3 its only home?)

In order:

  1. Read everything in ~/Dhwani/swasti-mform-migration/raw/mform-to-frappe-skill/mform_to_frappe/, especially dos-and-donts.md, DESIGN.md, and reference/field-types-table.md + reference/validation-layer-matrix.md. ~2 hours. Don’t skip — every “don’t” is a bug the SNF POC paid to find.
  2. Install the three skills locally so they’re invocable:
    cp -r ~/Dhwani/swasti-mform-migration/raw/mform-to-frappe-skill/mform_to_frappe/skills/mform_to_frappe         ~/.claude/skills/
    cp -r ~/Dhwani/swasti-mform-migration/raw/mform-to-frappe-skill/mform_to_frappe/skills/frappe-mobile-form-engine ~/.claude/skills/
    cp -r ~/Dhwani/swasti-mform-migration/raw/mform-to-frappe-skill/mform_to_frappe/skills/frappe-form-event-sync    ~/.claude/skills/
  3. Build the registry (§6). Walk all 13 forms in raw/mform-forms/, extract language[0].question[].orderlanguage[0].question[].shortKey, persist as JSON. One day.
  4. Dry-run mform_to_frappe on form 1000 (Member Profile) — the simplest, the master, no self-link, no loops. Land 3 web files + bench migrate cleanly. Half-day.
  5. Dry-run on form 1003 (Scheme Followup) — has the self-link via input_type 27. Confirm the skill produces a Link carrier; verify with dataOrdersMapping from step 3 that we know which fields will fetch on collapse. Half-day.
  6. Dry-run on form 1011 (HS Followup) — second self-link, plus closedItself terminal-status logic. Same as above; cross-check against closedItself. Half-day.
  7. Land 1 form fully through Phase 03 (Desk + API smoke + python tests) before touching Phase 04. One day.
  8. Bootstrap the mobile engine with frappe-mobile-form-engine against form 1000. Then run frappe-form-event-sync on 1000 to produce its .dart. Phase 05 verify on device. One day.
  9. Stop and audit. Before converting forms 2–12, confirm patterns are clean. Each subsequent form is faster (the corrections plan claims under a day per form once the engine is set up). Half-day audit.
  10. Then bulk-convert the remaining forms using the same loop. Forms 1004 / 1006 / 1007 / 1008 / 1009 / 1010 first (no self-link), then 1005 (the second self-link). 2–4 days.
  11. Then the master-data import + the response/self-link-collapse import. These are NOT covered by the skill. 3–5 days.
  12. Then the Livelihood program from Excel design. Per kickoff: 18-day budget; this should fit in ~5 days post-pipeline if Excel is authoritative.

Total runway: ~3 weeks of dev work for one engineer. Hits the May-25 deadline if started this week and QA automation (A10) is also live by then.


15. Known gaps / open questions

Consolidated from 03-mform-json-analysis.md + this skill review:

#GapOwnerBlocks
G1Access method to live mForm V2 instance for re-pulls / response datamForm team (A14)A15, A16 step 11
G2Master data values (107 schemes, ~30 docs, donors, geography hierarchy) — not in form JSONsmForm team (A14a)Pipeline step 6 (master hydration)
G3Non-English translations — Kannada, Telugu — locationmForm team (A14b)Multi-language layer; pre-launch QA
G4validation[].condition is null everywhere in dump — where do real expressions live?mForm team (A14c)Phase 02 quality; many fields will under-validate
G5input_type 13 / 29 / 30 — rare codes not seen during SNF POCdev teamPhase 02 hard-fail on unmapped — audit when found
G6Livelihood program — Excel-only or also seeded into mForm V2 first?Swasti PMPhase 02 path for the new form
G7(formId, order) → fieldname registry not built by skill — manual stepdev team (A16 Job 1)Cross-form fetch_from, self-link collapse
G8Self-link collapse logic — terminal-status semantics for forms 1003 and 1005dev team (cross-ref forms 1003/1005 vs 1011’s closedItself)Pipeline step 10
G9Whether existing geography master in any rel-mis Frappe install is reusable for Swasti, or Swasti needs its owndev teamPhase 00a

16. Hand-off checklist (for the dev who picks up A16)

  • Read dos-and-donts.md end-to-end.
  • Read DESIGN.md + DESIGN-corrections.md (the corrections are the live spec).
  • Read reference/field-types-table.md and reference/validation-layer-matrix.md.
  • Skim each of the 7 recipes in recipes/.
  • Install 3 skills into ~/.claude/skills/.
  • Build the (formId, order) → fieldname registry (§6).
  • Stand up the Frappe bench + custom app (Phase 00a).
  • Bootstrap the Flutter app skeleton (Phase 00b).
  • Confirm with PM: dump access method + master-data plan + translation plan (G1-G3).
  • Convert form 1000 (Member Profile) end-to-end through Phase 05 before any other form.
  • Audit, then bulk-convert remaining forms.
  • Implement the master-data hydration step (post-skill).
  • Implement the response import + self-link collapse (post-skill, uses §7 + closedItself + registry).
  • Add the Livelihood program (path TBD — see G6).
  • Wire QA regression automation (A10) before final cut.

References

  • Skill folder: ~/Dhwani/swasti-mform-migration/raw/mform-to-frappe-skill/mform_to_frappe/
  • Form JSONs: ~/Dhwani/swasti-mform-migration/raw/mform-forms/ (13 files)
  • Companion notes:
    • 01-kickoff-brief.md — kickoff decisions (D1–D14, A1–A17, §10 import plan)
    • 02-reliance-frappe-context.md — rel-mis Frappe shape, audit-log pattern, anti-patterns
    • 03-mform-json-analysis.md — JSON shape, 13-form inventory, 3 known gaps, self-link decoder

Last updated 2026-05-04