sigma_ingest

1# Sigma Ingest
2 
3Sigma ingest turns staged data model specs and workbook summaries into durable ktx wiki knowledge. The deterministic `project()` step has already written semantic-layer YAML for all warehouse-table data model elements before this skill runs — do not re-write those SL sources.
4 
5## Work unit structure
6 
7Sigma produces at minimum two work units per ingest run:
8 
9- `sigma-data-models` or `sigma-data-models-N`
10  - `rawFiles`: `data-models/<id>.json` files (one per data model in this batch)
11  - `peerFileIndex`: `workbooks/<id>.json` files + `sigma-manifest.json` + `sigma-projection-config.json`
12  - When the workspace has more than 50 data models, split into batches: `sigma-data-models-0`, `sigma-data-models-1`, … with `displayLabel` like `"Sigma: data models (1/8)"`. When ≤50 data models, the unitKey is simply `sigma-data-models` with no suffix.
13- `sigma-workbooks` or `sigma-workbooks-N`
14  - `rawFiles`: `workbooks/<id>.json` files (one per workbook in this batch)
15  - `peerFileIndex`: `data-models/<id>.json` files + `sigma-manifest.json` + `sigma-projection-config.json`
16  - When the workspace has more than 2000 workbooks, split into batches: `sigma-workbooks-0`, `sigma-workbooks-1`, … with `displayLabel` like `"Sigma: workbooks (1/4)"`. When ≤2000 workbooks, the unitKey is simply `sigma-workbooks` with no suffix.
17 
18`sigma-manifest.json` and `sigma-projection-config.json` are never in `rawFiles`. They live at the staged dir root and always appear in `peerFileIndex`.
19…