# hypertower-sample

A minimal **job repo** for HyperTower — the smallest thing that exercises the
repo-backed job flow end to end.

A job repo carries *configs and data*, not the framework. When a job names this
repo and a ref, the provider clones it, checks out that exact commit, and runs
with the checkout as the working directory. HyperTower itself comes from the
provider's own install via `PYTHONPATH`. That is why `configs/tabular_smoke.json`
can reference `data/smoke/labels.csv` as a plain relative path.

```
job: { repo: <this repo>, ref: main,
       args: ["--config", "configs/tabular_smoke.json"] }

provider:  git clone/fetch → checkout <ref> --detach → reset --hard → clean -fd
           cd <checkout> && python -m hypertower_provider.orchestrator --config …
           tar results/ → POST /results/<job_id>
```

## Contents

| Path | What |
|---|---|
| `configs/tabular_smoke.json` | Tabular binary classification, 3 folds, small epoch counts |
| `data/smoke/labels.csv` | 400 rows, 200 patients, 24% positive |
| `scripts/make_smoke_data.py` | Regenerates the CSV deterministically |

## The dataset

Synthetic, and shaped to catch real problems rather than flatter the pipeline:

- **Two rows per patient.** `patient_id` is the `group_column` and the config sets
  `split_identity_level: 1`, so folds split by patient. If grouping ever breaks,
  the two eyes of one patient land on both sides of a split and the score jumps
  suspiciously — that is the bug this shape is here to expose.
- **Signal is modest and noisy.** The label depends on `age`, `iop`, `cdr` and
  `rnfl` through a logistic model with added noise, so a healthy run lands around
  0.85–0.95 AUC. A perfectly separable toy would score 1.0 even with a broken
  model and tell you nothing.
- **`noise_feat` is pure noise** and `notes` is free text listed in `exclude_cols`.
  Both are here so that column handling is actually exercised.
- **`site` is categorical**, covering the `cat_cols` path.

Regenerate with:

```bash
python scripts/make_smoke_data.py
```

## Submitting it

```bash
curl -X POST https://hypertower.example.com/jobs \
  -H "x-token: $HT_ADMIN_TOKEN" -H 'content-type: application/json' \
  -d '{
        "run_name": "tabular_smoke",
        "args": ["--config", "configs/tabular_smoke.json"],
        "repo": "https://git.sticknife.com/ryan/hypertower-sample.git",
        "ref":  "main",
        "constraints": {"device": "cpu", "min_memory_mb": 2000}
      }'
```

Submission needs the **admin** token; a provider token cannot queue jobs.

`device: cpu` is deliberate — this model is a small MLP on ten-ish features, so a
GPU buys nothing, and constraining it to CPU means the job can be picked up by any
provider rather than waiting on a GPU box.

Jobs dedup on `(args, repo, ref)`. Re-submitting the same config at the same ref
returns the original `job_id` instead of queueing a duplicate; commit a change and
submit against the new ref to get a genuinely new run.

## Adding a real dataset

Keep large data **out** of git. The provider runs `git clean -fd` without `-x`
before every job, so gitignored paths survive between runs on the same machine —
put real data under `datasets/` (already ignored) and stage it onto the provider
once, rather than cloning it every job.