interp-engine, Neuronpedia's New Interpretability Engine, Tested on a Mac

On August 31 the team behind Neuronpedia, the open-source interpretability platform, announced the engine it had already moved into production after serving the wrong tensor from a Gemma model for a while with no alarm. interp-engine (Apache-2.0) gives each measurement point inside a model one name across model families and records cross-engine checks for 27 of its 34 point types; each model and engine covers a subset. I audited the code and ran it on a laptop, the first outside run I could find: where the names match, it agreed with TransformerLens to 5.3e-4; the advertised speed was measured on a datacenter GPU. Read this if you have ever trusted a hook name. You will learn how a wrong tensor passes for a right one, what "checked" tolerates, which hardware gets the speed, and the one rule that prevents the mistake today.

• The mistake is easy to make, and I measured it. On gemma-2-2b, the raw MLP output and the tensor TransformerLens calls hook_mlp_out agree at cosine 0.87: wrong, and plausible. On gpt2 they are identical, which is how code tested on one model is wrong on the other.
• Where the names match, the engines agree. On my Mac, interp-engine and TransformerLens differ by at most 5.3e-4 across twenty fp32 comparisons, a steering vector pushed through either produces the same twenty tokens, and at fp16 interp-engine matched plain transformers hooks bit for bit across 1,608 comparisons and 24 generations.
• "Over 40x" is eight concurrent requests on a datacenter GPU with fixed taps. One stream is 6.9×. My gemma-2-2b laptop run got no speedup, and left to its defaults the engine would have run it on my CPU.
• A green cell has a tolerance. Against TransformerLens it means cosine 0.99 and relative error 0.5. The committed table has 35 models, not the "50+" the README says, and DeepSeek-V4-Flash fails.
• The validator will outlast the engine. Neuronpedia moved three services onto interp-engine in one commit; TransformerLens now wraps native Hugging Face models and nnsight now serves traces through vLLM; the project's distinctive piece is the cross-engine comparison.
• The rule: on any model with a norm after the sublayer, ask for the contribution point, never the raw output, unless you mean it.

The reason a wrong number survives is that nothing crashes. A model is a stack of layers with places between them where you can tap the numbers passing through. One common workflow reads a tap and trains a small model on it, a sparse autoencoder, that turns those numbers into features a person can name. The autoencoder learns one tap. Feed it a different tap from the same block and it returns something with the right shape and the wrong meaning, and the charts downstream still look valid. Which tap a name points to depends on the model family, so code that was correct on Llama reads the wrong tap on Gemma without a warning.

That is what happened at Neuronpedia. Its old server translated a TransformerLens hook name into the raw output for gemma-2-2b and fed it to a Gemma Scope autoencoder trained on the normed one. The autoencoder's own reconstruction error said so, 9.8 where 0.26 was expected, with 8 active features instead of 85. The endpoint returned zeros and the autoencoder produced no features on the very text its dashboards were built from. The engine that replaced that server documents the failure in its hook-mapping guide.

A wrong tensor with the right name

The mistake is possible because Gemma's blocks are built differently from Llama's, and one tool's name hides the difference. A Gemma-2 block normalizes both before and after each sublayer, what the code calls a sandwich norm, and only the normed output is added to the residual stream. TransformerLens applies that second norm before both of its block-level hooks; its attention comment reads: We do it before the hook so hook_attn_out captures "that which is added to the residual stream", and the MLP path follows the same order. On a Llama-shaped block there is no second norm, so the raw MLP output and the residual contribution are the same tensor and the name is safe. On Gemma they are two tensors, and the name picks one of them without telling you.

interp-engine's answer is to name both. mlp_out is the raw module output. mlp_out_post is what gets added. On families without a post-sublayer norm the second aliases the first, so asking for the contribution point is safe everywhere. The architecture facts detect the norm structurally, on a real block. Checking whether the model is called Gemma would not do: Gemma-1 has none of these norms and VaultGemma removes them.

The two highlighted rows are where the mistake happens. TransformerLens and nnterp default to different sides of the norm, both call their choice the MLP output, and the mapping between them is model-dependent. interp-engine ships a mapper that translates hook names in both directions and refuses names that have no faithful equivalent, such as a norm's hook_normalized, which TransformerLens fires between the scale and the gain and which no Hugging Face module ever outputs. A name is not a tensor. Use the model-aware mapper when you can, and when you cannot, ask for the contribution point.

One address, three engines

The next two sections are for people who will run the thing; the rest of the article does not depend on them. The same request can run three ways, and only the static path delivers the advertised single-stream gain.

The eager backend is plain Hugging Face. load_model loads the checkpoint, the point registry turns Address("mlp_out_post", 4) into a module, a tensor side and an optional transform, and run_with_cache installs ordinary PyTorch hooks around one forward pass. Nothing about this path needs a GPU. It is a conventional hook-based capture path, with a fixed vocabulary on top and tests that reconstruct the sandwich equations to make sure raw and post outputs stay distinct.

vLLM is where the engineering is. A hook fires while a CUDA graph is being recorded and never again during replay, so the hooked backend runs vLLM with enforce_eager=True. Prefix caching stays on; capture and steering requests get a unique cache salt so a steered prefix is never reused, while plain generation shares prefixes as usual. That keeps 28 of the 34 points reachable and gives up the single-stream gain: gemma-2-2b decodes at 31.5 tokens per second hooked against 30.9 eager, though batching still lifts eight requests to 226 aggregate against 30.1. The static backend makes the opposite trade. It wraps the modules before graph capture, so graph capture records a copy_ into a capture buffer and an add_ from a steering buffer and replays them with the graph. A self-test writes a sentinel and checks it survives replay. The price is that you declare the taps at load time, hold extra VRAM for the buffers, and lose most of the batch window, from 16,384 tokens to 1,024.

The launch post says "Over 40x the throughput vs HF transformers." The 41× is the eight-request aggregate on the static backend, where eager serializes requests that vLLM batches. One stream is 6.9×. Static capture still costs vLLM 40% of its own decode speed, and a single capture or a logit lens is slower through vLLM than through eager, because the tensor has to cross a worker boundary. The benchmark is one B200, bf16, interp-engine 1.2.0 on vLLM 0.26.0, run on August 19. The audited release is 1.5.1 and now requires vLLM 0.28.0. The eager path runs on CPU, MPS or CUDA; the published speed evidence applies to datacenter CUDA with taps you declare up front.

What a green cell means

The part of the repository I would keep if the engine vanished is the validator. It is also where "checked against the other tools" turns from a slogan into a number with a tolerance. It runs each model through six engine paths, eager, hooked vLLM, static vLLM, TransformerLens 2 and 3, and nnsight, records the point-engine pairs each path supports, and leaves unsupported and unasked cells explicit. The thresholds decide what a green cell means. A raw Hugging Face pair passes at a maximum absolute error of 0.002 and cosine similarity of 0.9999. A pair involving TransformerLens or a fused kernel normally passes at cosine 0.99 and relative error 0.5, and named checkpoint waivers can lower the cosine gate. A tolerance miss becomes a warning. A missing signal, a shape mismatch, or cosine below 0.5 is a failure. The validator samples the first, middle and last layers, adds three-quarter depth for trunks with at least 16 layers, and may add an early attending layer for hybrid trunks. It compares 27 of the 34 points; seven, including resid_pre and attn_probs, are excluded.

Every stored cell for gemma-2-2b is green, and the detail file shows what green tolerates. The vLLM column is bf16 against an fp32 reference. Whole-tensor cosines sit above 0.997; the worst single token on final_norm sits at 0.961. gpt2 is fp32 everywhere and differs nowhere. DeepSeek-V4-Flash fails on both vLLM backends at its last layer, with mlp_out near cosine 0.235, and because no other engine runs that model the table cannot say which side is wrong. Gemma-4 12B and 26B carry warnings on vLLM that the bug registry does not explain, while 31B passes.

Read green as "within tolerance, on that engine and that vLLM, on that day." It is useful evidence, and it is still not a certificate. For the model you care about, rerun the comparison before you trust a point.

Whose layer is this

Back to the plain-language question: does this change who owns the way we look inside models? On August 21, Neuronpedia's repository took a 1,119-file commit whose message reads "interp-engine: Migrated inference, autointerp and graph services to new engine." The inference README now says the engine "replaced the previous TransformerLens + nnsight stack," and the graph service depends on interp-engine with no TransformerLens in its manifest. In January the same site said nnsight powered several of its inference backends. That is the maintainer thesis in one move: interpretability at production scale runs inside a serving engine, and the semantic layer that names the points belongs to whoever runs it.

The skeptic's case is that this is Neuronpedia's hosting convenience and researchers will keep their tools. It has evidence. Nothing differentiates through vLLM; the worker runs under inference mode and its kernels have no backward. The vLLM worker exposes a closed list of remote calls, so an arbitrary patching function cannot cross into it, though the eager backend does accept callables and dotted module paths. TransformerLens is not retreating: version 3.8.1 shipped the day after the launch post, and its TransformerBridge now wraps native Hugging Face models where the legacy HookedTransformer path used its own model implementations. nnsight 0.7.0 ships its own vLLM server. Both are moving toward the design interp-engine started from.

I read the field as segmenting rather than switching: interp-engine is built for repeatable, concurrent, product-facing inference, while TransformerLens keeps the research surface and nnsight keeps programmable traces and remote execution. The contest that is left is over who names the points, and that is why the validator is the durable piece. It already records a real production failure, six execution paths, and version-pinned diffs. Whether it becomes a standard other engines score against is speculative; no outside project uses its vocabulary yet, and when I checked on September 1 the repository had no issues and no pull requests from anyone but the maintainer and Dependabot.

The maintenance picture is two weeks old. Thirty-one commits since August 20, one human author under two git identities, thirteen commits with Cursor co-author trailers, seven release tags in twelve days. The vLLM integration monkeypatches the worker's load_model, reaches into private model-runner attributes, and pins nothing above its floor. The README says Gemma 4 needs transformers 5.14.1; the Gemma-4 validator cell ran on 5.16.1. Pin the engine, vLLM, transformers and torch together, and replay your own parity before you move any of them. The layer is contested and young; the comparison table is the part worth keeping whatever happens to the engine. It is also the third issue in a row where the important change happens inside a runtime, after Herdr and Bun 1.4.

What runs on my Mac

I care about this engine because I run the workload it serves. My research harness extracts contrastive activation vectors from gemma-2-2b through TransformerLens 3.5.1 on Apple Silicon and injects them at resid_post to measure how many directions a frozen model can hold at once. Every number I have depends on reading the tensor I think I am reading. So the question I could answer on a laptop was the one the validator answers on a B200: does the eager backend agree with the tool I already trust, and does the mistake show up when I pair the names naively?

Automatic device selection would have put gemma-2-2b on the CPU. The checkpoint is bf16-native and the engine treats MPS as unsafe for bf16 weights, so unless you ask for the device and a dtype explicitly you get a correct, slow run and no warning about speed. I asked for MPS and fp32. TransformerLens prints a warning for that same device: "MPS backend may produce silently incorrect results" on the PyTorch I have. The two tools disagree about my machine, and only the numbers can settle it.

The numbers broke the tie. I captured four points at five layers on both models with one 29-token prompt, both engines in fp32 on MPS, TransformerLens loaded without weight folding so the tensors are comparable. Where the names mean the same tensor, the two engines agree to floating-point noise: the largest absolute difference across twenty gemma-2-2b comparisons is 5.3e-4, at the last layer's resid_post. Then I paired the names the way Neuronpedia's old server did, raw mlp_out against blocks.N.hook_mlp_out, and the same run reproduced the mistake.

A cosine of 0.87 is the kind of wrong that goes unnoticed: far from random, with the right shape, and nothing downstream refuses it. On gpt2 the same naive pairing is exact, which is why code that was tested on gpt2 is wrong on Gemma without noticing. The maximum absolute difference on the naive gemma rows runs from 15 to 272; on the matched rows it never passes 0.001.

Steering held too. I built a contrastive vector the way my harness does, six sentiment pairs, mean difference at the final token of resid_post, layer 13 on gemma-2-2b and layer 6 on gpt2, and injected it at four times the vector from the last prompt token onward. interp-engine's steer takes that mask as a list of excluded prompt positions and applies the delta to every generated token; a hand-written TransformerLens hook did the same arithmetic. Next-token logits differed by at most 7.7e-5 with the same argmax, and twenty greedy tokens came out identical on both models. At that strength the text is already degenerate, which is the point: the two engines agree even where the model has stopped making sense.

Speed on a laptop is the part the benchmark cannot tell you, so I measured it: a 128-token prompt, 64 greedy tokens, fp32 on MPS, one warm-up and three timed runs, capturing resid_post at one layer throughout. The two engines capture differently. interp-engine generates and then runs one extra forward over the finished sequence to collect the point; TransformerLens hooks each position during cached decoding. The numbers are medians in tokens per second, with the sampled peak of Metal driver memory beside them.

Nobody gets the B200 story here, and nobody should expect to. On gemma-2-2b, interp-engine's generate-then-recapture path is 19% slower than bare transformers and 3.3× faster than this TransformerLens per-position caching loop, while staying 4.5 gigabytes under it on a machine with 24 GiB. Those are two capture strategies doing different work, not a ranking of engines. On gpt2 interp-engine's own decode loop beats generate outright.

The earlier draft of this article named one test that would move Apple Silicon from "unverified" to "verified for that configuration," so I ran it. At fp16 on this M5, against plain transformers hooks as the independent comparator, with eight prompts at each of 64, 512 and 2,048 tokens, every applicable point at layers 0, 13, 19 and 25, and 32 greedy tokens per prompt: 19 points on gemma-2-2b, 1,608 comparisons, maximum absolute difference 0, every position cosine 1.0, 24 of 24 generations identical. The gpt2 control, 18 points and 1,008 comparisons, was identical too. The only values that were not finite were the masked positions in attn_scores, which both sides set to negative infinity in the same 8,290 places. Runtime was 40 minutes; peak Metal memory 13.3 GB on both sides.

The test found one defect on the way. Asking interp-engine for fp16 on MPS hands transformers a device_map of mps, and on this machine that streamed load of a bf16 checkpoint into fp16 crashed with a segmentation fault inside transformers' threaded weight materialization, or hung. The fp32 load through the same path had worked all evening. Loading on the CPU and moving the model afterwards works, and because the engine reads its device from the parameters, capture and generation then run on MPS. That is the route the sweep took, it is recorded beside the results, and it is the first thing I would report upstream.

What changed in my harness is small and specific. My vectors come from resid_post, which is the one name every engine agrees on, so the mistake was not in my extraction. It is waiting in the next phase, where I move from raw directions to a Gemma Scope SAE basis and the SAE's declared hook is exactly the kind of name that means two tensors. I now translate every hook string through the mapper before I trust it, and I keep the naive-pairing check in the test suite as a tripwire.

Who should use it

Try it now if you serve capture or residual steering to many concurrent callers on Linux with a CUDA card, you can fix the taps you need, and you can pin the whole stack and replay parity before upgrades. The verified configurations are Qwen3-4B on an A40 and a B200. Try the eager backend now if you consume block-level hook names from someone else's SAE, transcoder or lens and want a mapper that refuses to guess. Try it on Apple Silicon for correctness: at fp16 it matched plain transformers bit for bit on gemma-2-2b and gpt2 on my machine, provided you load on the CPU and move the model.

Wait if you expected a speedup on a Mac, a free Colab, or a consumer GPU; no vLLM or static result is verified there, and on my Mac the fp16 eager path decoded at half the speed of plain generate. Wait if your research needs the fast path to do what eager does: gradients through the model, arbitrary patching, head-level tensors across GPUs, or neuron-basis points on a sparse MoE. Wait if you need exact parity on the architectures the table flags, DeepSeek-V4-Flash and Gemma-4 12B and 26B.

One result would still change this verdict. A bf16 Qwen3-4B comparison on one RTX 4090 with the same pinned stack, 8,192-token context, prompts, generation length and acceptance check in eager and static modes, holding at least 5× one-stream and 20× eight-request aggregate throughput with no cross-request contamination, would make the headline relevant to hardware that researchers own. The Apple Silicon test I would have asked for is the one above, and it passed.

A hook name is a claim about which tensor you get. This engine is the first one I have used that writes the claim down and checks it.

interp-engine belongs in a CUDA serving stack today and in any harness that consumes hook names, and it does not yet justify the speed claim on the hardware most researchers have.