Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#3850
Fires on “ boy”story 0.38% of tokensto add?<|assistant_end|><|bos|>One day, a strong boy named Tom
-
#3851
Fires on “!”agent 0.84% of tokensa short story about a boat.<|user_end|><|assistant_start|>Sure! One day
-
#3852
Fires on “s”story 0.29% of tokensshell
-
#3853
Fires on “em”agent 0.18% of tokenswent down, they all went home with happy memories of
-
#3854
Fires on “p”agent 2.99% of tokensthe tree and see the world from a different perspective
-
#3855
Fires on “ a”story 0.58% of tokens<|bos|>One day, a little ball
-
#3856
Fires on “test”agent 0.67% of tokens`, `test_mathlib.py` and `test_stats
-
#3857
Fires on “,”worksheet 1.60% of tokens):⏎ acc = 0⏎ for i in range(n
-
#3858
Fires on “ g”story 1.08% of tokenslittle girl called Daisy. Daisy really wanted to try riding
-
#3859
Fires on “ trees”story 1.52% of tokensday she would go for a walk and see the trees in the
-
#3860
Fires on “ be”story 0.47% of tokenslet you play with the boat. But be careful near
-
#3861
Fires on “ see”story 0.84% of tokensbox with a bow on it. They wanted to see what was
-
#3862
Fires on “ me”agent 0.43% of tokensthe other boats.<|assistant_end|><|user_start|>Can you tell me a story
-
#3863
Fires on “ call”agent 0.50% of tokens<|tool_start|>Traceback (most recent call last):
-
#3864
Fires on “2”worksheet 0.69% of tokens61178+43771:
-
#3865
Fires on “,”story 0.55% of tokens495<|assistant_end|><|bos|>Once upon a time, in a
-
#3866
Fires on “ of”story 0.16% of tokenshappy family. Mia was very proud of her picture
-
#3867
Fires on “ not”story 2.66% of tokens't want to
-
#3868
Fires on “.”story 1.01% of tokens, Tim! That is a very nice purple ball. Do you
-
#3869
Fires on “ everyone”story 0.70% of tokensSue and Tim learned that sharing with others can make everyone happy.
-
#3870
Fires on “⏎”python 1.39% of tokens`exclaim("python")` returns `"python!"`.⏎⏎```
-
#3871
Fires on “ who”story 7.73% of tokenslittle dog named Spot. Spot was a dependable dog who loved to
-
#3872
Fires on “ never”story 0.28% of tokensa little shark. He was nervous because he had never swum
-
#3873
Fires on “python”python 0.47% of tokensCat")⏎"cat"⏎⏎```python⏎def
-
#3874
Fires on “<|arg|>”agent 0.52% of tokenshell<|arg|>command=
-
#3875
Fires on “_”agent 0.51% of tokenstest_lists.py⏎from lists import keep_even,
-
#3876
Fires on “ you”story 1.11% of tokensparty outside?" Her mom said, "Yes, you can."
-
#3877
Fires on “ She”story 3.07% of tokensscared and hungry. Anna wanted to help it. She ran inside
-
#3878
Fires on “ll”agent 0.29% of tokensreplacement)<|tool_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command
-
#3879
Fires on “0”chat 0.89% of tokens56<|tool_end|><|assistant_start|>The answer is 756.<|assistant_end|>
-
#3880
Fires on “ was”story 0.62% of tokensplaces to explore.⏎One day, as he was walking through
-
#3881
Fires on “ down”story 0.84% of tokensmom said it was a good idea. They sat down and took
-
#3882
Fires on “ had”story 3.80% of tokensbut the acorn did not move. Then, Sam had an idea
-
#3883
Fires on “⏎”agent 1.29% of tokens: ⏎9: def factorial(n):⏎10
-
#3884
Fires on “ find”story 1.27% of tokensuby. Ruby loved to explore and find new exciting
-
#3885
Fires on “No”agent 0.58% of tokenswelcome("Noa"))
-
#3886
Fires on “)”python 1.04% of tokens2` instead of `return (a + b) /
-
#3887
Fires on “3”worksheet 0.64% of tokens8+0=8, 8336⏎
-
#3888
Fires on “Just”story 0.27% of tokensjust like the
-
#3889
Fires on “ climbed”story 0.29% of tokensdid not listen. She ran to the tree and climbed up.
-
#3890
Fires on “.”story 1.94% of tokensthey took the acorn to the river and washed it. The dirty
-
#3891
Fires on “,”story 0.86% of tokens. He lived in a small house with his mom, dad,
-
#3892
Fires on “8”worksheet 0.68% of tokens6 more. Now Tom has 43857
-
#3893
Fires on “ =”worksheet 0.44% of tokens0 to n, but it has `acc = acc +
-
#3894
Fires on “⏎”agent 0.54% of tokenspopular.<|assistant_end|><|bos|># notes.txt⏎Remember
-
#3895
Fires on “=”agent 1.30% of tokensreplacement)<|tool_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3
-
#3896
Fires on “⏎”python 0.80% of tokens(s):⏎ return s.lower()⏎⏎⏎def vowels
-
#3897
Fires on “ house”story 1.65% of tokensher elderly grandma. Her grandma lived in a small house with a
-
#3898
Fires on “ everything”story 0.85% of tokensEveryone was happy
-
#3899
Fires on “1”worksheet 0.26% of tokens6+77: 6+7=13,