# Farm Data Schema v1.0 **Tamil guidance:** ஒவ்வொரு பண்ணைக்கும் இதுபோன்ற ஒரு JSON — புதிய snapshot = புதிய தேதி கோப்பு, பழையதை அழிக்காதே. --- ## Directory Layout ``` farm-data/ SCHEMA.md ← this file / index.json ← snapshot registry for this farm YYYY-MM-DD_-farm-data.json ← versioned snapshots (never delete old ones) ``` **Rule:** Each snapshot is immutable. To update, create a new file with today's date. The ingester always reads the latest file by date-sort. --- ## Top-level keys | Key | Type | Description | |-----|------|-------------| | `schema_version` | string | Always `"1.0"` for this generation | | `generated` | string | ISO date `YYYY-MM-DD` when snapshot was created | | `sources` | string[] | List of source conversations / documents | | `farm` | object | Static farm identity & location | | `people` | object | All people associated with the farm | | `finances` | object | Historical financial summaries & policies | | `crop_history` | array | Seasons → crops array | | `vayal_map` | object | Current vayal (plot) layout | | `natural_farming_protocols` | object | Inputs schedule, seed treatment, pest control | | `standing_rules` | array | SR-001 … SR-N permanent operational rules | | `infrastructure` | object | CCTV, fencing, water, sheds | | `farm_events_log` | array | Dated event entries from audio reports | | `open_issues` | array | OI-001 … OI-N open action items | | `dispatch_log` | array | Produce dispatch records | | `metadata` | object | LLM / Syntropa usage notes | --- ## `farm` object ```json { "id": "TKP", "name_en": "Tholkappiyar Farm", "name_ta": "தொல்காப்பியர் பண்ணை", "organisation": "Eyarkai Puratchi Niruvanam / Antikva Produce", "org_ta": "இயற்கை புரட்சி நிறுவனம் / ஆண்டிக்வா ப்ரொடியூஸ்", "location": { "village": "string", "taluk": "string", "district": "string", "state": "Tamil Nadu", "lat": number, "lng": number }, "area_acres": number, "vayal_count": number, "lease": { "owner": "string", "amount_annual": number, "start_date": "YYYY-MM-DD" }, "water_sources": [ { "type": "string", "note": "string" } ], "philosophy": "string", "cluster_nearby_farms": ["string"] } ``` --- ## `people` object Four sub-arrays: `supervisors`, `field_staff`, `reporters`, `buyers`, `consultants`. Each person entry: `{ "name": "string", "role": "string", ... }`. --- ## `crop_history` array Each element is a season object: ```json { "season": "May-Jun 2026 (Rain-fed season)", "crops": [ { "crop": "Groundnut (கடலை)", "vayals": [5, 9, 10], "area_acres": 1.14, "planting_date": "YYYY-MM-DD", "status": "string" } ] } ``` --- ## `vayal_map` object ```json { "last_updated": "YYYY-MM-DD", "vayals": [ { "id": 1, "crop": "string", "area_cents": number, "status": "string" } ] } ``` Vayal IDs can be integers or strings (e.g., `"6A"`, `"6B"`). --- ## `standing_rules` array ```json [ { "id": "SR-001", "rule": "string — imperative, one sentence", "applies_to": "string (optional)", "evidence": "string (optional)", "date_set": "YYYY-MM-DD (optional)", "status": "string (optional — OVERDUE, pending, etc.)" } ] ``` --- ## `open_issues` array ```json [ { "id": "OI-001", "issue": "string", "priority": "CRITICAL | HIGH | MEDIUM | LOW", "pending_since": "YYYY-MM-DD (optional)", "date": "YYYY-MM-DD (optional)", "action": "string (optional)", "responsible": "string (optional)" } ] ``` --- ## `farm_events_log` array ```json [ { "date": "YYYY-MM-DD", "day": "string (optional)", "reporter": "string (optional)", "transcribed": "YYYY-MM-DD (optional)", "events": [ { "topic": "string", "detail": "string" } ] } ] ``` Entries with no audio have `"content": "Announced — no audio content"` instead of `events`. --- ## Ingestion targets The `tools/ingest_farm_data.py` script reads the latest snapshot per farm and writes to: | Output | Description | |--------|-------------| | `plans/crop-cycles.json` | Active/planned crop cycles | | `plans/vayal-map.json` | Plot map per farm | | `plans/farm-people.json` | All people across farms | | `plans/standing-rules.json` | SR rules | | `plans/farm-plans.json` | Plans (natural inputs, open issues as tasks) | | `thittam/managers.json` | Manager / coordinator roster | | `plans/data-audit-ingestion.md` | Audit log of what was applied | --- ## Versioning - Schema version bumps on structural changes to top-level keys. - Minor additions (new fields inside existing objects) do not bump version. - Current: **v1.0** (2026-07-04) --- ## Schema v2.0 (Full Farm Snapshot with Layered Finance) **Status: ACCEPTED** — First ingested 2026-07-04 for Maruthu Sagothirargal Farm. v2.0 snapshots use a **nested `schema` object** (vs v1.0 top-level `schema_version`). Detection rule in ingester: | Condition | Version | |-----------|---------| | `schema.version == "2.0"` (nested) | v2.0 | | `schema_version == "1.0"` (top-level) | v1.0 | | `schema.version == "2.1-partial"` (nested) | 2.1-partial | ### v2.0 Top-level keys | Key | Type | Description | |-----|------|-------------| | `schema` | object | Nested schema metadata (`name`, `version`, `data_layers`, caveats) | | `farm` | object | Farm identity, `land` sub-object (total/cultivated/idle acres, fields, start_date), `water`, `personnel_note` | | `bank_verified_maruthu_transactions` | object | Layer 1: KVB bank-verified `revenue[]`, `expenses[]`, `vendors[]` — `is_verified: true` | | `computed_master_pnl` | object | Layer 3: Modelled P&L — ALL figures tagged `is_computed: true`. Includes `monthly_fixed_cost_maruthu_inr`, `per_crop_fully_loaded_pnl[]`, `annual_fixed_burden_maruthu_inr` | | `farm_logs` | array | Dated entries with source type (`bank_verified`, `call_recording_transcript`, `whatsapp_voice_transcript`, `structured_document`). Crop mentions mined for crop-cycles. `data_detail.maruthu_vayals` used for vayal-map. | | `source_chats_index` | object | Coverage provenance index | ### v2.0 Financial Layering Rule > **⚠️ CRITICAL:** Any UI showing computed figures MUST label them **'மதிப்பீடு (computed)'** vs **'வங்கி சரிபார்க்கப்பட்டது (bank verified)'**. > > The distinction is: `bank_verified_maruthu_transactions` = authoritative money (Layer 1); `computed_master_pnl` = modelled apportionment NOT bank-traced (Layer 3). NEVER present Layer 3 figures as bank-verified facts. ### v2.0 Ingestion Mapping | Source field | Output target | Notes | |---|---|---| | `farm` | `farm-logs/farms.json` entry + `farm-facts.json` (area, vayals, water, start_date) | `conf: "high"` | | `bank_verified_maruthu_transactions.revenue[]` / `.expenses[]` | `farm-facts.json` → `bank_verified` layer | `is_verified: true`, `conf: "high"` | | `computed_master_pnl` | `farm-facts.json` → `computed` layer | `is_computed: true` on EVERY figure, `conf: "med"` | | `farm_logs[].data_detail.maruthu_vayals` | `plans/vayal-map.json` | vayals 6-10 with sqft | | `farm_logs` crop mentions | `plans/crop-cycles.json` | `conf: "med"` for log-derived cycles | | `bank_verified_maruthu_transactions.vendors[]` + log people | `plans/farm-people.json` | Nadesan S vendor MUST NOT merge with நடேசன் ஐயா (Bogar manager) | | Draft thittam tasks from Jun logs | `thittam/thittam-data.json` | 🔴 coconut cut OVERDUE, 🔴 sorghum/brinjal/chili harvest, etc. | | `farm` personnel + vendors | `thittam/managers.json` | Govindaraj + Priya marked `confirm: true` | ### v2.1 Guideline (Recommended for Future Snapshots) A future v2.1 full snapshot should combine v2.0's financial layering with v1.0's structured sections: - v2.0 strengths: `bank_verified` vs `computed` separation, `data_layers` disclaimer, `schema.critical_coverage_caveat` - v1.0 strengths: `vayal_map` with per-vayal sqft/status, `standing_rules[]` with SR-IDs, `open_issues[]` with OI-IDs, `natural_farming_protocols` - Recommended v2.1 structure: Keep `schema.data_layers`, add `vayal_map` block, `standing_rules`, `open_issues`, unify `farm_logs` + `farm_events_log` --- ## Partial Snapshots (schema.version: "2.1-partial") சில நேரங்களில் ஒரு பண்ணையின் எல்லா பிரிவுகளும் இல்லாமல் — குறிப்பிட்ட பிரிவுகள் மட்டும் சேகரிக்கப்பட்டு ஒரு snapshot உருவாக்கப்படலாம். இதுவே **partial snapshot**. **Rules for partial snapshots:** - `schema.version` must be `"2.1-partial"` - `schema.sections_present` lists which top-level keys are included - The ingester MUST process ONLY the listed sections - The ingester MUST NEVER blank out or overwrite data from other sections not listed - Partial snapshots are additive — they layer on top of existing data ```json { "schema": { "name": "syntropa_farm_dataset", "version": "2.1-partial", "sections_present": ["farm", "crop_history", "derived_insights", "metadata"] } } ``` **Ingester behaviour for 2.1-partial:** 1. Read `sections_present` from `schema` 2. Process ONLY those sections 3. All other existing data in output files remains untouched 4. Idempotent: running twice produces the same result (use id-based deduplication) பகுதி snapshot — ஏற்கனவே உள்ள தரவை அழிக்காது. புதிய பிரிவுகள் மட்டும் சேர்க்கும்.