Training a discovery SAE on Qwen3-4B

Decompose one layer of a 4B language model into 40,960 browsable, human-labeled concepts with a single sparse-autoencoder run

Building	A general-purpose sparse autoencoder that turns Qwen3-4B into a browsable atlas of 40,960 labeled concepts
Models	Qwen3-4B (subject) · Claude Sonnet (autointerp labeler + intruder-test judge)
Topics	Language Models · Sparse autoencoders · Autointerpretability

Try it yourself. Open the finished run in your own workspace and pick up where it leaves off.

A language model does not store its concepts one-per-neuron. Individual neurons fire for many unrelated things at once, so reading a model's internal state directly is hard. This recipe takes the standard remedy and runs it end to end: train a sparse autoencoder (SAE) on one layer of Qwen3-4B, which re-expresses the layer's activity as a wide dictionary of features that each fire sparsely, then have an LLM label every feature and check that the labels hold up. The output is a discovery atlas, a general-purpose dictionary meant for browsing what the model represents, rather than a dictionary tuned for one behavior.

A labeled dictionary is what makes a model's internals usable rather than merely visible. Once each concept has a name, the features become building blocks for concrete work: locating where the model represents a topic, monitoring whether a behavior is active as it generates, or steering its output by turning a concept up or down. That is far more tractable than the raw alternative, where a single activation vector blends thousands of concepts into numbers no person can read. Silico trains SAEs with autointerp natively: a single scoped experiment harvests the activations, trains the dictionary, and labels every feature, so producing an atlas is a matter of pointing the run at a model rather than stitching a toolchain together.

This recipe produces that dictionary for Qwen3-4B and answers the practical question underneath it: how much compute a usable one actually takes. The atlas below is a diverse slice of the finished dictionary. Search the concept labels, then click any feature to read the real passages it fires on, with the triggering token highlighted.

40,960 features, 0.76 of held-out variance, 0% dead at a modest 200-million-token budget: at expansion factor 16, where dead features are the usual failure, this run left none, and every feature carries a label that passes a blind check.

This recipe builds a healthy, fully-labeled feature dictionary for Qwen3-4B at layer 18, then publishes it as the browsable atlas above.

Key takeaways

An SAE turns entangled activations into a readable interface. A layer's activation is a dense vector that mixes many concepts at once; a sparse autoencoder re-expresses it as a short list of named features, which is what makes a model's internals something a person can inspect, monitor, or steer.
A dictionary's health is measurable, so check it before trusting it. Three numbers say whether an SAE is usable: how much variance it reconstructs, how sparse it is (features active per token), and how many features are dead. A dictionary that reconstructs poorly or leaves many features dead is not yet a reliable lens.
A label is a hypothesis until it is verified. Autointerp names every feature cheaply, but the names are only trustworthy once an independent check, such as a blind intruder test, confirms they predict where the feature actually fires.

Sparse autoencoders in a nutshell

As a language model reads text, every token turns into a long list of numbers, an activation vector, that holds what the model currently has in mind. For Qwen3-4B that list is 2,560 numbers wide at each layer. The trouble is that these numbers do not line up neatly with concepts. The model tracks far more distinct ideas than it has numbers to hold them, so it packs several unrelated concepts into overlapping combinations of the same numbers. This packing is called superposition, and it is the reason staring at a raw activation vector tells a person almost nothing.

A sparse autoencoder (SAE) is a small network trained to undo that packing. It helps to picture it as a translator with a large menu of concepts, its dictionary. For each activation vector that comes in, the SAE picks out the handful of menu items that best explain it and reports how strongly each one is present. Those menu items are the features, and because only a few are ever active at once, the description is sparse: a short, readable list of named ingredients in place of 2,560 entangled numbers.

The SAE learns this menu on its own, with no labels. It is an autoencoder, which means it is trained to compress each activation into that short sparse list and then rebuild the original vector from it. When the rebuilt vector lands close to the real one, the menu items it leaned on must have captured what mattered. Two knobs set the shape of the menu. The expansion factor says how many more dictionary entries there are than input numbers (16 times as many here, giving 40,960 features), on the bet that a bigger menu splits concepts more finely. The sparsity budget, k, fixes how many features may be active on a single token (64 here). This recipe uses a BatchTopK SAE, a variant that simply keeps the k largest feature activations and zeroes the rest.

How a sparse autoencoder works. The encoder rewrites a dense layer-18 activation (2,560 numbers, each mixing many concepts) as a short list of active entries drawn from a 40,960-feature dictionary, and the decoder rebuilds the original vector from that sparse list. Only about 64 features fire on any given token (k=64), and each active feature stands for a single human-readable concept.

A menu of 40,960 entries is only useful once the entries are named, so the final step labels them automatically. For each feature, an LLM reads the passages where that feature fires most strongly and writes a short description of what they share, for example "buttermilk as a recipe ingredient" or "aircraft carrier references". This autointerpretation pass is what turns the atlas from anonymous vectors into the searchable concepts the browser above is built on. The main thing to watch for is a dead feature, a menu entry that never fires on any text and so carries no meaning, and a healthy dictionary keeps that count low.

Inputs & dependencies. Qwen/Qwen3-4B (36 layers, residual stream 2,560 wide), a general text mix of 80% FineWeb and 20% UltraChat at sequence length 4,096, and an LLM (Claude Sonnet) for labeling. Qwen3-4B fits on a single GPU, which routes the run to the fast single-GPU harvest-and-train path.
Method. Harvest residual-stream activations at layer 18, train one BatchTopK SAE at expansion factor 16 (40,960 features) with k=64, then autointerp-label every feature and gate on reconstruction, sparsity, and dead-feature rate.
Results & applications. A labeled, browsable dictionary of the model's concepts, usable for inspecting what a layer represents and as the foundation for later probing or steering work.

How Silico verifies the dictionary

Training an SAE is the easy part; knowing whether the result is trustworthy is the point. A dictionary can reconstruct the layer poorly, waste capacity on features that never fire, or carry labels that read well but do not match where the feature actually activates. Silico checks both the dictionary's health and its labels in the same experiment that trains it, on held-out activations the SAE never saw, and gates the run on those checks before reporting a result.

Health, reconstruction and sparsity. The first check asks whether the sparse code still captures the layer. On held-out activations the dictionary reconstructs 0.76 of the variance while keeping only about 64 features active per token (L0), and the run is gated to pass both a reconstruction floor and its sparsity target.
Health, no dead features. The second check counts features that never fire, which are wasted dictionary capacity. Here 0% of the 40,960 features are dead, so every entry carries some meaning, and a high dead rate is exactly the failure the gate is there to catch.
Label quality, a blind intruder test. Autointerp labels every feature, but a label is only trustworthy if it predicts where the feature fires. Silico verifies this with a blind intruder test: shown the label and four passages, a judge that has not seen the training must spot the odd one out. It succeeded 95% of the time (93 of 98 trials, against 25% chance), and the labels carry 29,687 distinct strings, so concepts are named specifically rather than collapsing onto a few generic tags.
What the checks do not cover. The gates confirm the dictionary is healthy and its labels hold up, not that it is complete. This is one layer (18, roughly mid-depth) at one expansion factor, sized for exploration, and 0.76 variance explained leaves a real reconstruction gap. The atlas is a discovery tool, not a validated basis for any specific downstream claim.

Every feature earns its place. The firing rate of all 40,960 features on held-out text, on a log scale. The distribution is a single hump peaking near 0.08% of tokens, so most features are highly selective, firing on a small, specific slice of text. None sit at zero: 0% of the dictionary is dead.

Follow-up ideas

Does a wider dictionary find sharper concepts? Fork the run and sweep expansion factor across 8, 16, and 32 at a matched token budget, watching the dead-feature rate and intruder score to see where wider stops paying off.

Do different layers carry different kinds of concepts? Point the same recipe at a second layer, one earlier and one later than 18, and compare the atlases to see which concepts appear where in the network.

What can the atlas do beyond browsing? Build a probe or a steering vector on top of a labeled feature and test whether the concept the label names actually drives the model's behavior.