Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#3750
Fires on “3”worksheet 1.67% of tokens33⏎So 529 + 4 =
-
#3751
Fires on “.”story 0.18% of tokens2 stars and found 176 more. Now Max
-
#3752
Fires on “⏎”worksheet 0.61% of tokens206 = 297.⏎Lily has
-
#3753
Fires on “.”chat 0.72% of tokenscalculator.<|assistant_end|><|bos|><|user_start|>Tell me a short story.<|user_end|><|assistant_start|>
-
#3754
Fires on “Where”story 0.62% of tokensThe bird said, "Hi, Will! Why are you
-
#3755
Fires on “0”agent 1.09% of tokens6, 0])` returns `2.666
-
#3756
Fires on “<|bos|>”chat 0.76% of tokensknow how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who
-
#3757
Fires on “ she”story 0.62% of tokens. Jane made the most beautiful paper airplane she had ever
-
#3758
Fires on “7”worksheet 0.93% of tokens1441.⏎24232 plus
-
#3759
Fires on “ tests”agent 0.44% of tokensa coding agent harness.<|system_end|><|user_start|>"Do the tests pass?"
-
#3760
Fires on “ ball”story 0.44% of tokens, "Let's play!"⏎Tim threw the ball to Sue
-
#3761
Fires on “ and”story 3.19% of tokensmouse were surprised. The nice lady took the diamond and made a
-
#3762
Fires on “ They”story 1.53% of tokenspass.<|assistant_end|><|bos|>Anna and Ben are twins. They like to
-
#3763
Fires on “⏎”worksheet 0.29% of tokens063 = 9101.⏎11
-
#3764
Fires on “8”worksheet 2.89% of tokens0: 9+0=9, 9⏎3
-
#3765
Fires on “4”worksheet 1.09% of tokens3+8=11, 155⏎4
-
#3766
Fires on “ a”story 1.40% of tokenssun. While he was hopping, he saw a strange o
-
#3767
Fires on “ and”story 1.97% of tokensOnce upon a time there were two children, Joe and Ella.
-
#3768
Fires on “0”worksheet 1.08% of tokens⏎50+00: 0+0+1
-
#3769
Fires on “5”agent 0.88% of tokens7: ⏎8:
-
#3770
Fires on “ "”story 0.48% of tokens⏎At the party, they played a game called "mat
-
#3771
Fires on “_”python 0.49% of tokensnumber_utils.py⏎def sum_up_to(
-
#3772
Fires on “What”chat 0.75% of tokensto tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>What is the
-
#3773
Fires on “ on”story 1.37% of tokensOne day, she saw a big, red envelope on the floor
-
#3774
Fires on “expression”chat 0.08% of tokensCompute 6+4<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=6
-
#3775
Fires on “ words”agent 0.86% of tokensreturn len(items)<|tool_call_end|><|assistant_end|><|tool_start|>Edited stats.py
-
#3776
Fires on “ the”story 2.04% of tokens<|bos|><|system_start|>Answer with the number only
-
#3777
Fires on “,”agent 0.39% of tokensit on her hair.⏎"Look, Tom, I found
-
#3778
Fires on “ b”python 0.47% of tokensif a < b:⏎
-
#3779
Fires on “ sat”story 4.54% of tokenswas happy to see the sunset too. They sat down on
-
#3780
Fires on “Find”agent 0.79% of tokensmini, a small and friendly assistant.<|system_end|><|user_start|>Find the sum
-
#3781
Fires on “ run”story 0.40% of tokenspy print "Hello, Eva!" and run run.py
-
#3782
Fires on “ named”agent 0.75% of tokensa big dog named Max.
-
#3783
Fires on “ ”python 1.50% of tokensa, b):⏎ return a % b == 0⏎
-
#3784
Fires on “`.”python 0.40% of tokensbut it has `return a + "" + b`.⏎⏎
-
#3785
Fires on “ to”chat 0.17% of tokenscan't help with that. I only know how to tell stories
-
#3786
Fires on “ =”agent 0.14% of tokens'double_al
-
#3787
Fires on “ a”story 3.12% of tokens.<|assistant_end|><|bos|>John was very excited because today was a special day
-
#3788
Fires on “ and”story 0.46% of tokenstruck nearby. The deaf man saw Tim and stopped.
-
#3789
Fires on “ ”worksheet 0.33% of tokens63778.⏎what is 45
-
#3790
Fires on “ we”story 0.26% of tokenswebf
-
#3791
Fires on “ Tim”story 1.79% of tokenswere many kids playing and laughing in the park. Spot was so
-
#3792
Fires on “ new”story 2.64% of tokenseveryone safe. Lily felt happy that she had helped her mommy.
-
#3793
Fires on “Good”worksheet 0.43% of tokens<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Bye<|user_end|><|assistant_start|>Goodbye!
-
#3794
Fires on “ project”agent 0.37% of tokenspy⏎test_greetings.py<|tool_end|><|assistant_start|>This project has
-
#3795
Fires on “:”agent 1.15% of tokens⏎3: ⏎4: ⏎5: def square
-
#3796
Fires on “.”story 7.03% of tokensgot on the stage, he saw many kids. The kids
-
#3797
Fires on “ careful”agent 0.75% of tokenstheir mom said. "But you have to be careful with other
-
#3798
Fires on “,”story 0.93% of tokensletter(s)`.<|assistant_end|><|bos|>Once upon a time, there was
-
#3799
Fires on “ only”chat 0.21% of tokensSorry, I'm a tiny model: I can only tell short