An 8-subcommand harness — plan / verify / preflight / freeze-runtime / capture / encode / analyse / report — where each stage refuses to run until its predecessor verifies.
Research Engineering
I build the harness, the pipeline and the verifier.
I build the systems that make a research result trustworthy rather than merely plausible: staged experiment pipelines, deterministic execution, automated testing, and reproducibility and artifact-validation tooling that fails closed rather than open. The same programme appears on the main portfolio; here it is ordered around the engineering first.
Experiment orchestrationDeterministic pipelinesReproducibility & provenanceAutomated testing
What I'd bring
Infrastructure an employer can inspect.
Each of these is a concrete piece of shipped engineering, not a slogan, and each one shows up in one of the two public repositories.
Seeds derived via HMAC-SHA256 rather than ambient randomness, so a resampling run reproduces exactly; environment-lock verification exits non-zero on drift.
Artifacts are published with O_EXCL + O_NOFOLLOW against a directory file descriptor, then fsync'd, with openat-style descriptor-relative path walking to close the TOCTOU window.
Manifests verify 12 root files, 120 nested supplement files and 20 shipped Python files; checksum and environment-lock scripts both exit non-zero on any mismatch.
The canonical reproducibility package carries thousands of test functions, run under pytest and GitHub Actions.
Evidence
The evaluation harness and staged execution.
- Problem
- How do you stop a large evaluation run from silently producing invalid data when any one of many stages could fail quietly?
- Built
- A srop_mech_eval Python package exposing plan / verify / preflight / freeze-runtime / capture / encode / analyse / report as ordered subcommands, where a stage refuses to run until its predecessor verifies.
- Scale
- 20,096 model jobs executed against a frozen schedule with zero failures; 65,536 layer-feature coordinates captured and encoded in a separate mechanistic run.
- Methods
- Ordered subcommands with hard verification gates between stages, pinned environments, and deterministic HMAC-SHA256-derived seeding so any stage can be re-run exactly.
- Result
- The staged design caught what a single-script run would have hidden: the harness completed almost every case (98.4% procedural closure) while its own reliability gate correctly identified that the underlying measurement was not yet trustworthy. What this establishes: the orchestration worked as designed. What it does not establish: anything by itself about the model behaviour being measured — that is a separate, already-reported result.
Evidence
The reproducibility and release-verification system.
- Problem
- Once a research artifact is published, how does anyone — including me, a year later — know the numbers in the paper are the numbers the code actually produces?
- Built
- A verification system: SHA-256 manifests, write-once publication via O_EXCL/O_NOFOLLOW against a directory file descriptor, checksum scripts, and environment-lock verification, all exiting non-zero on any mismatch.
- Scale
- Manifests verify 12 root files, 120 nested supplement files and 20 shipped Python files; two predecessor packages of 86 and 201 files carried forward byte-preserved, zero modified.
- Methods
- Descriptor-relative (openat-style) path walking to close the TOCTOU window between check and use; statistics reimplemented directly on NumPy so verification does not depend on scipy/statsmodels/sklearn version drift.
- Result
- An independent numerical pass reproduced the published figures directly from the shipped code and manifests. What this establishes: byte- and number-level reproducibility of the release. What it does not establish: the validity of the underlying research claims, which is a separate, already-reported result.
Capability stack
What the infrastructure is built from.
Grouped by what each tool was used for, not as a logo wall — every entry appears in code that ships in a public repository.
Experiment orchestration
- Staged subcommand pipelines
- Ordered stage verification
- Pinned model & environment revisions
- Python 3.11
Reproducibility & provenance
- SHA-256 manifests
- O_EXCL / O_NOFOLLOW write-once publication
- openat-style descriptor-relative paths
- Environment lockfiles
- HMAC-SHA256 derived seeding
Tooling & testing
- pytest
- GitHub Actions
- Thousands of test functions
- Checksum & drift verification scripts
Contact
Open to research-engineering work.
If the work is building the thing that makes a result trustworthy rather than just plausible, I would like to hear about it.