ryan
4e815df327
400 rows / 200 patients, two samples each, so grouped splitting is actually exercised. Signal is modest and noisy on purpose — a separable toy would score 1.0 with a broken model and prove nothing.
81 lines
3.2 KiB
Markdown
81 lines
3.2 KiB
Markdown
# hypertower-sample
|
||
|
||
A minimal **job repo** for HyperTower — the smallest thing that exercises the
|
||
repo-backed job flow end to end.
|
||
|
||
A job repo carries *configs and data*, not the framework. When a job names this
|
||
repo and a ref, the provider clones it, checks out that exact commit, and runs
|
||
with the checkout as the working directory. HyperTower itself comes from the
|
||
provider's own install via `PYTHONPATH`. That is why `configs/tabular_smoke.json`
|
||
can reference `data/smoke/labels.csv` as a plain relative path.
|
||
|
||
```
|
||
job: { repo: <this repo>, ref: main,
|
||
args: ["--config", "configs/tabular_smoke.json"] }
|
||
|
||
provider: git clone/fetch → checkout <ref> --detach → reset --hard → clean -fd
|
||
cd <checkout> && python -m hypertower_provider.orchestrator --config …
|
||
tar results/ → POST /results/<job_id>
|
||
```
|
||
|
||
## Contents
|
||
|
||
| Path | What |
|
||
|---|---|
|
||
| `configs/tabular_smoke.json` | Tabular binary classification, 3 folds, small epoch counts |
|
||
| `data/smoke/labels.csv` | 400 rows, 200 patients, 24% positive |
|
||
| `scripts/make_smoke_data.py` | Regenerates the CSV deterministically |
|
||
|
||
## The dataset
|
||
|
||
Synthetic, and shaped to catch real problems rather than flatter the pipeline:
|
||
|
||
- **Two rows per patient.** `patient_id` is the `group_column` and the config sets
|
||
`split_identity_level: 1`, so folds split by patient. If grouping ever breaks,
|
||
the two eyes of one patient land on both sides of a split and the score jumps
|
||
suspiciously — that is the bug this shape is here to expose.
|
||
- **Signal is modest and noisy.** The label depends on `age`, `iop`, `cdr` and
|
||
`rnfl` through a logistic model with added noise, so a healthy run lands around
|
||
0.85–0.95 AUC. A perfectly separable toy would score 1.0 even with a broken
|
||
model and tell you nothing.
|
||
- **`noise_feat` is pure noise** and `notes` is free text listed in `exclude_cols`.
|
||
Both are here so that column handling is actually exercised.
|
||
- **`site` is categorical**, covering the `cat_cols` path.
|
||
|
||
Regenerate with:
|
||
|
||
```bash
|
||
python scripts/make_smoke_data.py
|
||
```
|
||
|
||
## Submitting it
|
||
|
||
```bash
|
||
curl -X POST https://hypertower.example.com/jobs \
|
||
-H "x-token: $HT_ADMIN_TOKEN" -H 'content-type: application/json' \
|
||
-d '{
|
||
"run_name": "tabular_smoke",
|
||
"args": ["--config", "configs/tabular_smoke.json"],
|
||
"repo": "https://git.sticknife.com/ryan/hypertower-sample.git",
|
||
"ref": "main",
|
||
"constraints": {"device": "cpu", "min_memory_mb": 2000}
|
||
}'
|
||
```
|
||
|
||
Submission needs the **admin** token; a provider token cannot queue jobs.
|
||
|
||
`device: cpu` is deliberate — this model is a small MLP on ten-ish features, so a
|
||
GPU buys nothing, and constraining it to CPU means the job can be picked up by any
|
||
provider rather than waiting on a GPU box.
|
||
|
||
Jobs dedup on `(args, repo, ref)`. Re-submitting the same config at the same ref
|
||
returns the original `job_id` instead of queueing a duplicate; commit a change and
|
||
submit against the new ref to get a genuinely new run.
|
||
|
||
## Adding a real dataset
|
||
|
||
Keep large data **out** of git. The provider runs `git clean -fd` without `-x`
|
||
before every job, so gitignored paths survive between runs on the same machine —
|
||
put real data under `datasets/` (already ignored) and stage it onto the provider
|
||
once, rather than cloning it every job.
|