Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2946
Fires on “0”worksheet 1.40% of tokens=12, 29⏎000+8
-
#2947
Fires on “Once”story 0.24% of tokensTell me a story about a bird.<|user_end|><|assistant_start|>Once upon a
-
#2948
Fires on “ with”story 0.34% of tokensale universe. It was very big, filled with stars and
-
#2949
Fires on “ with”agent 0.45% of tokensb` with `return
-
#2950
Fires on “ language”chat 0.08% of tokens<|assistant_start|>My name is prelude. I'm a tiny language model:
-
#2951
Fires on “=”agent 2.24% of tokenshello.py⏎from strings import final_letter, count_
-
#2952
Fires on “_”agent 1.58% of tokensmax_item⏎⏎ Line 4: print(max_item
-
#2953
Fires on “ it”python 0.22% of tokensit was beautiful
-
#2954
Fires on “ said”story 1.19% of tokensSue went to her mom and said, "
-
#2955
Fires on “3”worksheet 1.11% of tokens⏎053+448: 3+8
-
#2956
Fires on “ of”story 0.17% of tokenswas happy he did not give up. The moral of the story
-
#2957
Fires on “ the”story 0.05% of tokensars of corn from the stalk and loved the way it
-
#2958
Fires on “ ”worksheet 0.92% of tokens03: 1+3=4, 4⏎
-
#2959
Fires on “ saw”story 1.86% of tokensand started to play with Buddy. Buddy's friends saw how kind
-
#2960
Fires on “ share”story 0.46% of tokensLuna that fighting was not good, and they should share the w
-
#2961
Fires on “ ”agent 1.12% of tokenspy, lines 1-2⏎1: def average_
-
#2962
Fires on “ question”agent 1.59% of tokensiet<|user_end|><|assistant_start|>
-
#2963
Fires on “ its”story 1.28% of tokensfriendly dog came up to him. The dog wagged its tail and
-
#2964
Fires on “ add”chat 0.59% of tokensis 7749.⏎If you add 6
-
#2965
Fires on “ wings”story 1.32% of tokensbe bigger!"⏎The silly bird stretched its wings and its
-
#2966
Fires on “⏎”agent 0.49% of tokensif n % 2 == 0]⏎⏎⏎def add
-
#2967
Fires on “ can”chat 0.68% of tokensPython files in this project.<|assistant_end|><|bos|><|user_start|>What can you do
-
#2968
Fires on “⏎”worksheet 0.40% of tokens+ 8 = 7964.⏎If you
-
#2969
Fires on “ =”worksheet 0.82% of tokens771.⏎5 + 78 = 8
-
#2970
Fires on “ the”story 0.88% of tokensbig red ball on the clear street
-
#2971
Fires on “3”worksheet 1.15% of tokensa calendar, and they said thank you. They
-
#2972
Fires on “ edit”agent 0.54% of tokensis 97.<|assistant_end|><|bos|><|system_start|>tools: edit, gl
-
#2973
Fires on “ `”worksheet 0.62% of tokenses and saving the day.<|bos|>Bug: `increment`
-
#2974
Fires on “=”agent 0.35% of tokens<|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>pattern=def has
-
#2975
Fires on “:”worksheet 0.93% of tokens⏎7068+1608: 8
-
#2976
Fires on “ in”python 0.24% of tokensin nums]
-
#2977
Fires on “last”python 0.41% of tokens(nums)⏎```⏎⏎Bug: `first_letter
-
#2978
Fires on “What”worksheet 0.45% of tokens= 115585.⏎If you add
-
#2979
Fires on “Yes”story 0.18% of tokenswalk with you?" The seal said, "Yes, let
-
#2980
Fires on “r”agent 0.21% of tokensabsolute<|arg|>newString=magnitude<|arg|>replaceAll=true<|tool_call_end|>
-
#2981
Fires on “=”python 1.06% of tokens+1265: 6+5=11
-
#2982
Fires on “ they”agent 0.69% of tokensanymore. He shared his sugar with his friends and they all had
-
#2983
Fires on “'m”agent 1.17% of tokensize this article for me<|user_end|><|assistant_start|>I'm sorry,
-
#2984
Fires on “ an”agent 0.94% of tokenswrite, execute<|system_end|><|system_start|>You are an AI agent
-
#2985
Fires on “py”agent 0.23% of tokensmost recent call last):⏎ File "run.py", line
-
#2986
Fires on “ way”story 0.48% of tokens! Tom was sad because he could not find a way to get
-
#2987
Fires on “ =”chat 0.81% of tokens49.⏎6 + 810 = 8
-
#2988
Fires on “ are”chat 0.35% of tokensa laptop by mini-lab.<|assistant_end|><|user_start|>Who are you?
-
#2989
Fires on “ friends”story 1.09% of tokenswere happy and clapped their paws. They loved being friends.<|bos|>
-
#2990
Fires on “⏎”python 0.41% of tokenss):⏎ return s[-1]⏎```⏎⏎Fix
-
#2991
Fires on “ fell”story 1.02% of tokenslisten and she fell asle
-
#2992
Fires on “ a”story 0.79% of tokensa happy little girl named Emma went to play in a big ch
-
#2993
Fires on “ print”agent 8.14% of tokensmodule>⏎ print(der
-
#2994
Fires on “ them”story 0.47% of tokensThey let the bug go in the garden and watched it fly away
-
#2995
Fires on “ ”agent 0.19% of tokens⏎ File "test_textlib.py", line 6,