Same Hook Name,
Different Tensor

interp-engine is the inference engine behind Neuronpedia, released under Apache-2.0 and announced on August 31. It gives 34 hook points one name each across model families, checks them against TransformerLens and nnsight on 35 committed models, and runs them inside vLLM for throughput. Its correctness claim is stronger than its speed claim. The speed needs a CUDA machine and a fixed set of taps. The parity table can be checked on a laptop, and I checked it on mine.

The best argument for the engine is a number that looked fine. Neuronpedia's old inference server translated TransformerLens's blocks.4.hook_mlp_out into the raw MLP output for gemma-2-2b and fed it to a Gemma Scope sparse autoencoder that had been trained on a different tensor. Nothing raised. The SAE's reconstruction error came back at 9.8 instead of 0.26, worse than predicting the mean, with 8 active features instead of the declared 85. The endpoint returned zeros, and a whole SAE source stopped firing on the very text its dashboards were built from. The maintainers tell this story themselves in the hook-mapping guide.

A wrong tensor with the right name

A Gemma-2 block is a sandwich. Each sublayer has a norm before it and a second norm after it, and only the normed output is added to the residual stream. TransformerLens's block-level hook_mlp_out fires after that second norm; the source comment says it does so "so hook_attn_out captures that which is added." On a Llama-shaped block there is no second norm, so the raw MLP output and the residual contribution are the same tensor and the name is safe. On Gemma they are two tensors, and the name picks one of them without telling you.

interp-engine's answer is to spell both. mlp_out is the raw module output. mlp_out_post is what gets added. On families without a post-sublayer norm the second aliases the first, so asking for the contribution point is safe everywhere. The architecture facts detect the norm structurally, on a real block. Checking whether the model is called Gemma would not do: Gemma-1 has none of these norms and VaultGemma removes them.

The two highlighted rows are the trap. TransformerLens and nnterp default to different sides of the norm, both call their choice the MLP output, and the mapping between them is model-dependent. interp-engine ships a mapper that translates hook names in both directions and refuses names that have no faithful equivalent, such as a norm's hook_normalized, which TransformerLens fires between the scale and the gain and which no Hugging Face module ever outputs.

One address, three engines

The eager backend is plain Hugging Face. load_model loads the checkpoint, the point registry turns Address("mlp_out_post", 4) into a module, a tensor side and an optional transform, and run_with_cache installs ordinary PyTorch hooks around one forward pass. Nothing about this path needs a GPU. It is the same trick every interpretability library uses, with a fixed vocabulary on top and tests that reconstruct the sandwich equations to make sure raw and post outputs stay distinct.

vLLM is where the engineering is. A hook fires while a CUDA graph is being recorded and never again during replay, so the hooked backend runs vLLM with enforce_eager=True and prefix caching off, or salted per request when a steer would poison a cached prefix. That keeps every point reachable and gives up most of what vLLM is for: gemma-2-2b decodes at 31.5 tokens per second hooked against 30.9 eager. The static backend goes the other way. It wraps the modules before graph capture so that a copy_ into a capture buffer and an add_ from a steering buffer are recorded into the graph and replayed with it. A self-test writes a sentinel and checks it survives replay. The price is that you declare the taps at load time, hold extra VRAM for the buffers, and lose most of the batch window, from 16,384 tokens to 1,024.

Read the headline against that table. The launch post says "over 40x the throughput vs HF transformers." The 41× is the eight-request aggregate on the static backend, where eager serializes requests that vLLM batches. One stream is 6.9×. Static capture still costs vLLM 40% of its own decode speed, and a single capture or a logit lens is slower through vLLM than through eager, because the tensor has to cross a worker boundary. The benchmark is one B200, bf16, interp-engine 1.2.0 on vLLM 0.26, run on August 19. The audited release is 1.5.1 and now requires vLLM 0.28.

What a green cell means

The part of the repository I would keep if the engine vanished is the validator. It runs each model through eager, hooked vLLM, static vLLM, TransformerLens 2 and 3, and nnsight, and commits a per-point comparison. The thresholds are the fine print. A raw Hugging Face pair passes at a maximum absolute error of 0.002 and cosine similarity of 0.9999. A pair involving TransformerLens or a fused kernel passes at cosine 0.99 and relative error 0.5, falls to a warning below that, and fails only when shapes differ or cosine drops under 0.5. Layers are sampled at the first, middle and last block, plus three-quarter depth. Twenty-seven of the 34 points are compared; resid_pre, mlp_in, attn_probs and four others are not.

gemma-2-2b is green across the board, and the detail file shows what green tolerates. The vLLM column is bf16 against an fp32 reference. Whole-tensor cosines sit above 0.997; the worst single token on final_norm sits at 0.961. gpt2 is fp32 everywhere and differs nowhere. DeepSeek-V4-Flash fails on both vLLM backends at its last layer, with mlp_out at cosine 0.235, and because no other engine runs that model the table cannot say which side is wrong. Gemma-4 12B and 26B carry warnings on vLLM that the bug registry does not explain, while 31B passes.

Whose layer is this

On August 21, Neuronpedia's repository took a 1,119-file commit whose message reads "Migrated inference, autointerp and graph services to new engine." The inference README now says the engine "replaced the previous TransformerLens + nnsight stack," and the graph service depends on interp-engine with no TransformerLens in its manifest. In January the same site said nnsight powered its backends. That is the maintainer thesis in one move: interpretability at production scale runs inside a serving engine, and the semantic layer that names the points belongs to whoever runs it.

The skeptic's case is that this is Neuronpedia's hosting convenience and researchers will keep their tools. It has evidence. Nothing differentiates through vLLM; the worker runs under inference mode and its kernels have no backward. The vLLM worker exposes a closed list of remote calls, so an arbitrary patching function cannot cross into it, though the eager backend does accept callables and dotted module paths. TransformerLens is not retreating: version 3.8.1 shipped the day after the launch post, and its TransformerBridge now wraps native Hugging Face models where version 2 reimplemented them. nnsight 0.7.0 ships its own vLLM server. The incumbents are converging on the same design from the other side.

I read the field as segmenting rather than switching. interp-engine wants repeatable, concurrent, product-facing inference. TransformerLens wants the research surface. nnsight wants programmable traces and remote execution. The contest that is left is over who names the points, and that is why the validator is the durable piece. It already records a real production failure, six execution paths, and version-pinned diffs. Whether it becomes a standard other engines score against is speculative; no outside project uses its vocabulary yet, and the repository had zero issues and zero pull requests from anyone else when I checked.

The maintenance picture is thin in the way a two-week-old project is thin. Thirty-one commits since August 20, one human author, thirteen commits co-signed by the Cursor agent, seven releases in twelve days. The vLLM integration monkeypatches the worker's load_model, reaches into private model-runner attributes, and pins nothing above its floor. The README says Gemma 4 needs transformers 5.14.1; the Gemma-4 validator cell ran on 5.16.1. Pin the engine, vLLM, transformers and torch together, and replay your own parity before you move any of them.

What runs on my Mac

I care about this engine because I run the workload it serves. My research harness extracts contrastive activation vectors from gemma-2-2b through TransformerLens 3.5.1 on Apple Silicon and injects them at resid_post to measure how many directions a frozen model can hold at once. Every number I have depends on reading the tensor I think I am reading. So the question I could answer on a laptop was the one the validator answers on a B200: does the eager backend agree with the tool I already trust, and does the trap show up when I pair the names naively?

Two things surprised me before a single forward pass. First, automatic device selection would have put gemma-2-2b on the CPU. The checkpoint is bf16-native and the engine treats MPS as unsafe for bf16 weights, so unless you ask for the device and a dtype explicitly you get a correct, slow run and no warning about speed. I asked for MPS and fp32. Second, TransformerLens greets that same device with a warning that "MPS backend may produce silently incorrect results" on the PyTorch I have. Two tools, two different opinions about my machine, and no third party to break the tie except the numbers.

The numbers broke the tie. I captured four points at five layers on both models with one 29-token prompt, both engines in fp32 on MPS, TransformerLens loaded without weight folding so the tensors are comparable. Where the names mean the same tensor, the two engines agree to floating-point noise: the largest absolute difference across twenty gemma-2-2b comparisons is 5.3e-4, at the last layer's resid_post. Then I paired the names the way Neuronpedia's old server did, raw mlp_out against blocks.N.hook_mlp_out, and the same run reproduced the trap.

A cosine of 0.87 is the dangerous kind of wrong. It is far from random, it has the right shape, and an SAE trained on the other tensor will encode it into something that looks like sparse features. On gpt2 the same naive pairing is exact, which is why code that was tested on gpt2 carries the bug to Gemma without noticing. The maximum absolute difference on the naive gemma rows runs from 15 to 272; on the matched rows it never passes 0.001.

Steering held too. I built a contrastive vector the way my harness does, six sentiment pairs, mean difference at the final token of resid_post, layer 13 on gemma-2-2b and layer 6 on gpt2, and injected it at four times its own norm from the last prompt token onward. interp-engine's steer takes that mask as a list of excluded prompt positions and applies the delta to every generated token; a hand-written TransformerLens hook did the same arithmetic. Next-token logits differed by at most 7.7e-5 with the same argmax, and twenty greedy tokens came out identical on both models. At that strength the text is already degenerate, which is the point: the two engines agree even where the model has stopped making sense.

Speed on a laptop is the part the benchmark cannot tell you, so I measured it: a 128-token prompt, 64 greedy tokens, fp32 on MPS, one warm-up and three timed runs, capturing resid_post at one layer throughout. The two engines capture differently. interp-engine generates and then runs one extra forward over the finished sequence to collect the point; TransformerLens hooks each position during cached decoding. The numbers are medians in tokens per second, with the sampled peak of Metal driver memory beside them.

Nobody gets the B200 story here, and nobody should expect to. What the table does say is that the eager backend costs almost nothing over bare transformers on the model I use, and that my current harness is the slow one. On gpt2 interp-engine's own decode loop beats generate outright. On gemma-2-2b it gives back a fifth of the speed for the recapture pass and stays 4.6 gigabytes under TransformerLens on a machine with 24.

What changed in my harness is small and specific. My vectors come from resid_post, which is the one name every engine agrees on, so the trap was not in my extraction. It is waiting in the next phase, where I move from raw directions to a Gemma Scope SAE basis and the SAE's declared hook is exactly the kind of name that means two tensors. I now translate every hook string through the mapper before I trust it, and I keep the naive-pairing check in the test suite as a tripwire.

Who should use it

Try it now if you serve capture or residual steering to many concurrent callers on Linux with a CUDA card, you can fix the taps you need, and you can pin the whole stack and replay parity before upgrades. The verified configurations are Qwen3-4B on an A40 and a B200. Try the eager backend now if you consume block-level hook names from someone else's SAE, transcoder or lens and want a mapper that refuses to guess.

Wait if you expected the speedup on a Mac, a free Colab, or a consumer GPU; nothing is verified there. Wait if your work needs gradients through the model, arbitrary patching inside the fast path, head-level tensors on more than one GPU, neuron-basis points on a sparse MoE, or exact parity on DeepSeek-V4-Flash or Gemma-4 12B and 26B.

Two results would change this verdict. A fp16 eager run on Apple Silicon across every applicable point at four layers and three prompt lengths, passing at mean cosine 0.9999 and max absolute error 0.002 with identical greedy tokens, would move Mac users from "unverified" to "supported." A Qwen3-4B static run on one RTX 4090 at 8,192 context, holding at least 5× one-stream and 20× aggregate over eager with no cross-request contamination, would make the headline relevant to hardware researchers own.

A hook name is a promise about a tensor. This engine is the first one I have used that writes the promise down and checks it.

interp-engine earns a place in a CUDA serving stack today and a place in any harness that consumes hook names, and it does not yet earn the speed story on the hardware most researchers have. This is the third issue in a row where the interesting move happens inside a runtime: Herdr turned a terminal multiplexer into one, Bun 1.4 pulled package jobs into one, and here a serving engine becomes the bench interpretability has to run on. The next issue stays there: DFlash 2, a block-diffusion drafter for speculative decoding that also only started to matter once it landed inside vLLM and llama.cpp, and that I can run on this Mac through MLX.