Skip to content

Dedicated remote builders

Dedicated hosts have builder- names and register no GitHub runners. The reviewed shape is n4-highcpu-16: 16 vCPU, 32 GiB RAM, one unprivileged Docker builder limited to an aggregate 16 CPUs and 28 GiB, with no additional swap. Two OxiDex worktrees can compile concurrently inside that container, sharing its limits and Cargo downloads while using separate target directories.

The image installs Rust 1.99.0 (full compiler commit b940084d7eb6a299eb4bfeb8e34901bc051e7ac4), rustfmt, clippy, Zig 0.15.2, cargo-zigbuild 0.23.4, cbindgen 0.29.2, Just, nextest, Python, uv, Perl, rsync, LLVM, and native build dependencies. App builds run as UID/GID 1001 and receive no Docker socket. SSH lands on the host; the uploader can sudo only the fixed /usr/local/bin/oxidex-remote-build launcher.

The website tool additions pin Node 24.21.0 and Playwright 1.63.0, including Chromium revision 1243. The package lock verifies npm archives; the official Node archive has a fixed SHA-256. Browser downloads and Linux libraries are installed during image construction. Browser files live at the root-owned /opt/playwright-browsers path, which UID 1001 can read but not write,, selected by PLAYWRIGHT_BROWSERS_PATH. The tool probe checks the browser executable and shared libraries as UID 1001. The website audit must separately prove an actual browser launch and rendering. These source additions need a reviewed image build and actual tool proof before an existing host is upgraded; they do not establish that website tools are installed on previously launched builders.

Render once, launch with one command

Prepare the private JSON configuration below and a separate explicitly approved public uploader mapping file. Do not use a private SSH key. The mapping must contain exactly schema (root-approved-ssh-mapping-v1), principal, username, numeric uid (at least 1000 and not 1001), identity_receipt_sha256, the complete single-line Ed25519 public_key, and public_key_sha256 (SHA-256 of that complete key string). The renderer validates the selected mapping and computes its canonical checksum. Selecting the file does not approve a new account or key; use the mapping you already approved. Identity receipts and key mappings are deployment inputs, not repository defaults. The renderer supplies the fixed host, container, and baseline disk settings. Explicit settings still pass through the same strict policy checks.

{
  "instance_name": "builder-your-unique-name",
  "project": "your-project",
  "zone": "asia-east2-a"
}

Render locally with the approved mapping file:

python3 src/render_builder_startup.py --config /private/builder-config.json \
  --uploader-mapping /private/approved-uploader.json \
  --output-directory /private/builder-payload
bash /private/builder-payload/launch-command.sh

The deployment identity defaults to instance_name in the configuration. Use --deployment-id only when you need a distinct identity in the startup receipt. Existing configurations can continue embedding authorized_uploader and its canonical authorized_uploader_sha256 instead. The renderer refuses a mapping file combined with either embedded field, so it cannot silently replace an identity.

The second command executes one gcloud compute instances create invocation. It passes the complete embedded payload as startup-script metadata. The Debian 13 GCE image runs this startup mechanism; a separate cloud-init agent is not required. cloud-config.yaml contains the equivalent payload for images with a cloud-init agent, but that alternative execution path has not been live-tested.

Startup verifies fresh-host identity and the absence of existing runners/work, mounts the cache disk, installs Docker, stops Docker and containerd, points both at the cache disk and restarts them, builds, and pins the immutable image, starts aggregate admission, checks actual tool/cgroup/cache observations, enrolls the approved restricted uploader, and writes a completion marker. No manual SSH installation or repository checkout is needed on the host. Rendering refuses an existing output directory; fresh-host startup must not be replayed to upgrade an occupied VM. Replacements need a new name and matching rendered payload.

Verify startup and readiness

Startup records two files on the builder:

/var/lib/oxidex-builder-bootstrap.complete
/var/lib/oxidex-builder-readiness.json

The readiness JSON joins the deployment, boot, source hashes, image and container, actual cgroup limits, tool results, and the mounted cache identity. It sets workload_qualified to false: tool installation isn't compilation, cache-reuse, concurrency, or full qualification proof. Run the intended workload and keep its result before you declare the host ready for that workload.

The standalone image includes:

  • Rust 1.99.0, rustfmt, clippy, and Linux musl and Darwin cross targets.
  • Build tools, Clang, CMake, pkg-config, OpenSSL, and archive development libraries.
  • Git, SSH, GnuPG, Perl, Python 3.11, rsync, GNU time, and compression tools.
  • just, cargo-nextest, cbindgen 0.29.2, Zig 0.15.2, cargo-zigbuild 0.23.4, uv, and LLVM tools.

Some auxiliary tool installs resolve versions during the image build. Use the actual tool receipt and the immutable image ID rather than assuming that every dependency has a fixed version.

The older shared-host image in builder/Dockerfile still pins Rust 1.97.1. Ordinary runner images don't establish the standalone builder's toolchain, because workflows select their Rust version separately. Provisioning a new standalone builder doesn't upgrade existing runner containers.

Configure the build client

The renderer also writes cargo-remote-env.toml with the project, instance name, zone, and uploader username. Merge those entries into the existing [env] table in OxiDex's .cargo/config.toml; do not replace unrelated settings.

After startup, add the actual numeric GCE instance ID and the absolute paths to your private SSH key and trusted known-hosts file using the commented entries. These values cannot be inferred from a launch plan. The snippet contains no key material and does not claim the new VM is ready. OxiDex verifies the identity, SSH trust, installed launcher, and resource admission before executing work. Then use just build; use just build-local for an explicit local build.

Storage, access, and draining

The launch command uses a 40 GiB Hyperdisk Balanced boot disk and a 100 GiB Hyperdisk Balanced cache disk through NVMe, with 3,000 IOPS and 140 MiB/s baseline provisioning. For the cache disk only, the renderer also accepts the explicitly reviewed 4,000 IOPS / 250 MiB/s pair; price that pair before launching. The boot disk is always 3,000 IOPS / 140 MiB/s. Compute and both disks were quoted at $2.573746849/day in asia-east2-a on 2026-10-05; this is a historical quote, not a promise of future pricing or Spot availability.

Cargo downloads and per-project targets persist on /mnt/runner-data. The cache disk has auto-delete=no, so it survives instance deletion and continues to incur storage charges. Recovery must verify the disk and ownership; startup refuses to format an existing unidentified filesystem. Use direct authenticated SSH/SCP with the established host key for oxidex-uploader. gcloud compute ssh/scp can enroll the account into broader guest permissions and is not the restricted uploader transport.

The launcher prepares a project and starts the allowed build command in an admitted container; the Docker admission wrapper takes the aggregate lease and cache lock. The launcher's cleanup empties the project's source directory after it checks the locks and container mounts. Artifact export isn't a launcher step. Draining refuses new builds and lets actual active builds, leases and cache/upload locks finish before replacement. Persistent Cargo downloads and targets are not deleted after each build. GitHub runner drains are separate from this lifecycle.

Observed validation

On 2026-10-05, the dedicated host completed concurrent cold Linux native command-line builds on OxiDex source 429d6f26ec08e703cba2e5a3712b098998d6778d: debug 45.15 seconds and release 189.80 seconds, total 190.16 seconds, aggregate charged memory peak 7.13 GiB, zero OOM kills, and zero throttled CPU periods. Both artifacts were independently hashed. The ordinary just build-debug-remote recipe also passed source synchronization, compilation, verified download, and source cleanup with targets retained on client 01c5b1fcbb30e2938de244addb979e127288a7c8.

A later monolithic expression-catalog compiler exceeded the 28 GiB cap; its chunked-function repair passed the same 608 expressions and 16,821-probe population at 6.21 GiB with zero OOM events. These are exact workload proofs, not evidence that all workloads fit. The initial launch required Debian service/sudo parser and target-parent fixes included in the current payload. A second untouched launch of that final payload was not performed.

The canceled 24-hour runner monitoring study is not part of this provisioning.

Runtime updates

Routine remote builds do not require a runtime update. To upgrade an existing builder while preserving its jobs and caches, use the separate builder runtime update guide. It covers draining, backup, paired-client finish, and recovery. Do not replay fresh-host startup on an occupied builder.