Verbalizing the weekday manifold

Reading Llama-3.1-8B's weekday geometry with a SelfIE adapter: a ring the model can name day by day but cannot interpolate, with a small causal subspace that shows the geometry is real.

Building	A SelfIE read-out of a model's weekday geometry — a nameable ring plus a causal-subspace test
Models	Llama-3.1-8B (layer 19) · SelfIE scalar-affine adapter · Goodfire SAE · weekday-mean control
Topics	Language models · Neural geometry · Circuits & causal tracing

A language model stores a concept like the day of the week as a direction in activation space, not as words. A small SelfIE adapter lets the model read one of its own activations back in plain text, so its internal geometry can be questioned directly: name this point, and the one next to it.

Pointed at the seven weekdays, the read-out reveals a model that both reveals and resists. It holds the weekdays as a continuous calendar-ordered ring and names every real day on it, yet it cannot voice a point between two days. It snaps to the nearest weekday instead.

The ring is not a projection artifact: the first two principal components capture 62% of the residual-stream variance. An interchange test confirms the geometry does causal work, moving the model's output day 85.2% of the time, but only once the day has been committed to the sentence.

This recipe builds a faithful self-interpretation read-out and turns it on a concept with known structure: a replicated SelfIE adapter, a weekday manifold the model can name day by day but not interpolate, and a causal subspace test that shows the geometry is load-bearing.

Key takeaways

• A model can name the landmarks of its own geometry without describing the space between them: continuous structure, discrete verbalization.
• Self-interpretation is most trustworthy when it rests on a faithful, replicated read-out rather than a bespoke one.
• A correlate becomes a mechanism only under a causal test. An interchange swap of the subspace shows where, and when, the geometry actually drives the output.

SelfIE in a nutshell

A trained model carries information in its activations, but those activations are vectors, not words. SelfIE (Self-Interpretation of Embeddings, arXiv 2602.10352) closes that gap by training a small scalar-affine adapter that injects an activation back into the model's own prompt, so the model describes, in plain text, what the activation encodes. The thread builds directly on that adapter, applied to Llama-3.1-8B at layer 19 over the Goodfire sparse-autoencoder feature set.

The first step was to reproduce the adapter, so every later claim rests on a faithful read-out rather than a custom one. The replication matches the paper on its own metric. Coverage lands at 88.1 against the paper's 87.7, and Hitrate reproduces the regime at 62.7, a few points above the reported value.

These numbers measure latent controllability, not autointerpretation accuracy: they ask whether steering a latent changes the model's self-description, not whether a separate judge agrees with it. Of 4,541 latents, 1,783 (39%) reach a perfect hit rate at their best scale, and 11.9% never fire, which accounts for the entire gap below full coverage.

In short:

• Inputs and dependencies. Llama-3.1-8B, layer-19 residual activations, the Goodfire SAE feature set, and the SelfIE scalar-affine adapter.
• Method. Inject a target activation into the model's prompt through the adapter and read back the model's own description.
• Result. A faithful read-out that recovers the paper's controllability numbers, ready to point at a new concept.

Recipe goal: can the model verbalize its own weekday geometry, and is that geometry causal?

The research objective of this thread was to turn the SelfIE read-out on a concept with known structure, the seven weekdays, and ask two questions: can the model put its own internal geometry into words, and is that geometry doing causal work rather than sitting as a passive correlate.

The evidence builds in steps:

• The activations form a ring. Weekday activations at layer 19 lie on a calendar-ordered circle. The first two principal components capture 62% of the variance in the full residual stream, so the ring is most of the structure, not a projection artifact.
• The model names each day. At the seven weekday centroids the adapter reads back the correct day. Read "Saturday" off the Saturday centroid, "Sunday" off the Sunday centroid, and so on.
• It cannot name the points in between. Move to an intermediate point on the ring, halfway between two days, and the model snaps to the nearest real weekday. A point a fifth of the way from Saturday to Sunday reads back as "the seventh day of the week, which is Sunday," never as a blend. The geometry is continuous, but the verbalization is discrete. This is the result the explorer above is built to show: switch the prompt frame and injection scale, then click around the ring and read the continuations.
• The geometry is causal, but only downstream. A DAS (Distributed Alignment Search) interchange test swaps the weekday subspace between examples and checks whether the model's output day follows. At the weekday token itself the swap does essentially nothing, with interchange accuracy of 0.000 and 0.008 against a chance level of 0.143. One position downstream, at the punctuation after the day-bearing sentence and at the final read-out position, the same swap moves the output cleanly: 0.852 at the punctuation and 0.732 at the read-out. The weekday is causally encoded, but it becomes swappable only after the sentence that states it.
• A small subspace carries it. Locking the causal direction to a 16-dimensional subspace keeps a per-day signal worth about 24.5% of the activation norm, against roughly 8.5% for the earlier diluted version. Patching with that subspace reads back the correct day and tracks the ring, while a weekday-mean control reads a single fixed default day, which confirms the naming is genuine subspace content rather than an artifact of the patch.

The result the model both reveals and resists: it holds the weekday as a continuous ring, names every real day on it, and yet cannot voice a point between two days. The causal test shows the ring is load-bearing, but only once the day has been committed to the sentence.

Related recipes

Getting Started with Neural GeometryA worked tutorial: find a structured concept in a model, prove it is real, and steer along it. Discovering the geometry of emotionsA map of emotion inside a language model — a polar readout plus a steering dial. Replicating ROME's Causal TracingRecovering ROME's causal-tracing sites from Meng et al. (2022) with a native restoration primitive.