Probe the machine, don't guess it

2026-07-21 · Caldrix Engineering

Most inference stacks identify hardware the way a driver does: read the device ID, look it up in a table, load the preset someone tuned months ago on a machine that was not yours. That assumes a device name determines performance — which stopped being true once the same GPU shipped with three memory configurations and a host whose PCIe topology decides whether offload is a good idea.

Inferno probes instead. Before it plans anything it asks the machine in front of it: which CPU instruction set, which cache-line size, how many cores, which GPU and which runtime. The battery will grow to measure achievable bandwidth rather than the number on the spec sheet, hierarchy depth and the cost of moving a tensor across every link between host and device.

That is what lets a compiler stop guessing. KV-cache page size is a tradeoff against the cache line it will be walked through. Whether a megakernel pays will depend on occupancy on this device at this batch size. Whether to offload is a question about a specific interconnect, and the honest answer is often no on hardware whose marketing page says yes.

There is a second reason, and it is not performance. If the compiler only knows what it probed, it can only claim what it probed. caldrix doctor reports what it found and nothing else — no inferred capabilities, no feature flags derived from a device name. When a target is unsupported you get told which probe failed. We would rather ship a tool that says "I could not measure this" than one that guesses confidently and leaves you to find out in production.

The cost is a short probe on first run, cached afterwards. The return is a compiler arguing from evidence.