Infrastructure that cannot rewrite the science

Every research run starts from clean commit-addressed source, pinned model and instrument artifacts, a private internet-off kernel, explicit hardware verification, and pulled provenance. The remote job may execute a reviewed plan; it may not edit metrics, select thresholds, inspect meta-test, or change the measuring instrument.

The result bundle must prove the source commit, archive hash, model or organ hash, runtime manifest, hardware contract, and output identity. A provider status of “complete” is not enough.

Kaggle CPU smoke, generated bootstrap, Qwen CPU, and the parallel scheduler have passed end to end. Colab CLI CPU sessions have also been verified. The standing agreement records CPU and a historical 2×T4 path as verified, but the current Research Guide archives every GPU profile and prohibits new GPU submissions. This distinction matters: “verified once” is not the same as “active executor.”

The R20 campaign uses CPU because its exact TorchAO INT8 recipe, frozen organ hash, and gradient contract were verified there. Faster hardware is not automatically a valid substitution.

CPU

ACTIVE + VERIFIEDKaggle and Colab executor paths.

2×T4

HISTORICALLY VERIFIED, ARCHIVEDStanding evidence retained; current guide says do not submit GPU work.

P100 / L4

ARCHIVED / INACTIVENot admitted for current research submissions.

TPU

NOT WIREDNo approved executor contract.

The runner pool separates reviewed plans from execution. Account leases and owner locks prevent two schedulers from claiming the same capacity. Submission pacing, per-provider limits, durable state, output collection, and manifest validation make interrupted campaigns resumable without inventing new experimental decisions.

The R20 campaign exposed three production-grade failure modes: an argument routed to the wrong CLI layer, an orchestrator crash-loop after terminal scheduler failure, and stale owner locks left by dead processes. The fixes added explicit bootstrap routing, a terminal-job guard, and stale-lock cleanup.

Three of five first-wave R20 jobs exceeded Kaggle’s CPU window. The correct response was not to classify the science as null or tune the metric. The campaign preserved the same seeds, outer steps, optimizer, instrument, and runtime, then halved the developmental task count in replacement jobs to fit the admitted execution envelope.

One checkpoint completed after the local scheduler timeout and was recovered manually only after its runtime manifest and output identity passed validation. The incident also calibrated the scheduler timeout independently from Kaggle’s hard limit.

Acceleration in Oczy means more than parallel jobs. It means reducing avoidable reruns while preserving the firewall: versioned source datasets, pinned offline wheels, runtime manifests, quantization recipes, model hashes, deterministic task shards, and systemd-backed recovery.

That infrastructure can run 90 calibration shards later. It cannot decide whether the resulting distribution is good enough, unseal meta-test, or approve its own candidate. Throughput scales; authority does not.