Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2146
Fires on “:”worksheet 0.68% of tokens<|assistant_start|>`stats.py` defines 2 functions: `first
-
#2147
Fires on “ me”chat 0.11% of tokensstep or with a calculator.<|assistant_end|><|bos|><|user_start|>Tell me a story
-
#2148
Fires on “,”story 0.33% of tokensin it.⏎When Spot jumped in the mud, he heard
-
#2149
Fires on “ of”agent 0.71% of tokensheld out his hand which had a melted piece of candy in
-
#2150
Fires on “ his”agent 0.64% of tokensdecided to share the fruit with all their jungle friends
-
#2151
Fires on “ ”worksheet 0.59% of tokens7536 and 8, you get 47
-
#2152
Fires on “ nums”python 0.36% of tokens]` instead of `return [n for n in nums if n
-
#2153
Fires on “7”worksheet 0.43% of tokens7+8
-
#2154
Fires on “ >”python 2.24% of tokens⏎⏎⏎def positive(x):⏎ return x > 0
-
#2155
Fires on “ trained”agent 0.25% of tokens<|assistant_start|>I'm prelude, a very small language model trained from scratch
-
#2156
Fires on “ m”story 0.66% of tokens, why are you worried?"⏎"I just mention
-
#2157
Fires on “ would”story 1.04% of tokensin her soup. She smelled it. It did not smell bad
-
#2158
Fires on “,”agent 0.22% of tokens.py<|tool_call_end|><|assistant_end|><|tool_start|>Read file greetings.py, lines
-
#2159
Fires on “ a”story 0.37% of tokensmake a big pile. Max and Lily were having a lot of
-
#2160
Fires on “ of”story 3.78% of tokensbe your friend!" They played together and had lots of fun.
-
#2161
Fires on “_”agent 0.63% of tokens<|assistant_start|>Renamed `final_letter` to `new_final_
-
#2162
Fires on “i”agent 0.76% of tokensexplore the outdoors, especially the
-
#2163
Fires on “assert”story 0.28% of tokenswanted to go to a fun party. He put on his best
-
#2164
Fires on “What”chat 0.40% of tokensequals 74.⏎What's
-
#2165
Fires on “ have”chat 0.28% of tokens4 more cookies. How many cookies does Sue have now?
-
#2166
Fires on “ ”agent 1.56% of tokens3 main.py⏎9⏎[2, 0,
-
#2167
Fires on “ Tim”story 0.15% of tokensOnce upon a time, there was a boy named Tim. He
-
#2168
Fires on “1”agent 0.50% of tokens5+08: 5+8=13,
-
#2169
Fires on “ "”story 0.45% of tokens.⏎"Hi, Sam!" said Tim. "Do you
-
#2170
Fires on “.”story 0.22% of tokens.<|assistant_end|><|bos|>
-
#2171
Fires on “ ”worksheet 3.98% of tokens970 and 9812 is 13
-
#2172
Fires on “ with”agent 0.21% of tokenswas red and very warm. She wore it all the time.
-
#2173
Fires on “ me”story 0.74% of tokensand help mom. And
-
#2174
Fires on “ only”agent 0.41% of tokens, I don't know about that. I can only tell short
-
#2175
Fires on “ return”python 0.28% of tokens⏎```⏎⏎Bug: `cube` should return x cubed
-
#2176
Fires on “⏎ ”agent 0.21% of tokens8, in <module>⏎ print(
-
#2177
Fires on “ it”agent 0.56% of tokens08.<|assistant_end|><|user_start|>Add 61 to it.<|user_end|>
-
#2178
Fires on “ es”agent 3.90% of tokensteacher discussed new words. He was espec
-
#2179
Fires on “ very”story 0.90% of tokensjoined them and they all danced together. They were all happy and
-
#2180
Fires on “ She”story 0.66% of tokens, she found a mysterious box under her bed. She was very
-
#2181
Fires on “#”story 0.80% of tokensin nums]⏎⏎# README.md⏎# Sc
-
#2182
Fires on “ ch”agent 0.60% of tokensBeing selfish and mean makes you and others unhappy.
-
#2183
Fires on “⏎”python 2.10% of tokenstests⏎⏎$ python
-
#2184
Fires on “ ancient”agent 0.55% of tokens<|user_end|><|assistant_start|>Once upon a time, there was an ancient tree in
-
#2185
Fires on “ moment”agent 0.21% of tokensstrong."⏎The wise old tree thought for a moment and said
-
#2186
Fires on “ favorite”story 0.21% of tokensto make animals with the clay. It was her favorite thing to
-
#2187
Fires on “1”worksheet 0.84% of tokens⏎5+7: 5+7=12,
-
#2188
Fires on “'t”story 0.22% of tokenssaid, "Don't worry,
-
#2189
Fires on “ fun”story 0.60% of tokensand put it on her head. It looked so pretty! Lily
-
#2190
Fires on “ so”story 0.24% of tokenshouse. They wanted to climb it and see what was on the
-
#2191
Fires on “ how”story 0.38% of tokensI'm prelude, and I know how to tell
-
#2192
Fires on “1”worksheet 0.36% of tokens3886: 6+6=12,
-
#2193
Fires on “3”worksheet 1.51% of tokens5269⏎3+0: 3+0
-
#2194
Fires on “x”agent 1.00% of tokenspy:⏎ Line 9: def count(items):⏎
-
#2195
Fires on “ we”story 1.68% of tokensto find the weight!