← Back to blog

Building RunnerLens

A build can pass without the repository containing everything it needs. On a hosted CI runner, compilers, runtimes, SDKs, and package managers are already there. A workflow can call one directly and keep moving. Nothing in the repository says that tool was part of the build.

I kept thinking about that gap. When a runner image changes, a build might break because a tool disappeared, moved, or changed version. The workflow file tells me what commands ran, but it does not always tell me which parts of the machine those commands quietly relied on.

RunnerLens is my attempt to make that dependency visible. It watches the tools a build actually invokes, records what it can learn about them, and writes the observations to a Runner Dependency Receipt. The idea is simple: leave behind enough evidence to inspect what the job used, instead of treating the runner as an opaque box.

What counts as a dependency?

There is a tempting version of this problem where a tool scans a runner, lists everything installed, and calls that a dependency report. But an installed tool is not necessarily part of a build. The more useful question is narrower: what did this job actually execute?

That distinction shapes the project. RunnerLens focuses on observed executable use and the available evidence around each tool, such as its name, version, path, and likely origin. Some of that evidence will be incomplete. I would rather make that uncertainty visible than guess at provenance and make the receipt look more certain than it is.

runnerlens run -- make
runnerlens show runnerlens-receipt.json
runnerlens compare baseline-receipt.json runnerlens-receipt.json
runnerlens impact runnerlens-receipt.json

The useful test

A receipt is only useful if it helps explain a real change. If a runner image update removes or changes a tool, the record should help answer whether a workflow used it, and where that dependency came from. If it cannot help with that, it is just another report to ignore.

The first target is GitHub-hosted Ubuntu runners. Process observation, dependency resolution, origin classification, receipt comparison, and runner-image impact analysis are the pieces I am working through. The project is early, and its coverage will have limits. I want those limits to be legible too.

That is the problem I want RunnerLens to make a little less invisible: not every package on a machine, but the tools a build reached for without declaring them.

RunnerLens on GitHub →

— Jad