Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#191
Fires on “ felt”story 0.47% of tokenspretty view from the sky. Tim felt ashamed because
-
#192
Fires on “ day”story 0.84% of tokensin nums]⏎⏎⏎def has_item(items, x
-
#193
Fires on “<|assistant_start|>”agent 0.97% of tokensyou plan properly, you can achieve anything
-
#194
Fires on “,”agent 0.53% of tokensutils.py⏎from number_utils import twice, smaller,
-
#195
Fires on “n”agent 0.36% of tokens<|bos|><|user_start|>Can you book a table for two tonight?
-
#196
Fires on “ be”story 0.40% of tokensthe mugs to the living room. But be careful,
-
#197
Fires on “ a”story 0.32% of tokensplaying together.<|bos|>Once upon a time, in a big forest
-
#198
Fires on “ had”story 2.02% of tokensgoing up and down the slide. They laughed and had a good
-
#199
Fires on “,”python 0.67% of tokenstail⏎⏎assert tail([8, 2
-
#200
Fires on “Edited”agent 0.34% of tokensnewString= return a ** b<|tool_call_end|><|assistant_end|><|tool_start|>Edited helpers.
-
#201
Fires on “```”python 0.56% of tokensreturn "Hello, " + name + "!"⏎```⏎⏎
-
#202
Fires on “ ex”agent 0.67% of tokensch, websearch, write, exec
-
#203
Fires on “ passed”agent 0.76% of tokenspython3 test_items.py⏎all tests passed⏎<|bos|>
-
#204
Fires on “!”story 0.50% of tokens. She laughed and said, "Silly me! This is
-
#205
Fires on “ so”story 1.11% of tokensMax. He wanted to have an amazing adventure, so he decided
-
#206
Fires on “ upon”story 0.50% of tokensell me a story about a boat<|user_end|><|assistant_start|>Once upon a time
-
#208
Fires on “The”chat 0.29% of tokens0<|tool_call_end|><|assistant_end|><|tool_start|>1018<|tool_end|><|assistant_start|>The answer is
-
#209
Fires on “ to”story 0.29% of tokensnamed Max. Max was a hero dog who loved to help people
-
#210
Fires on “print”story 0.36% of tokensdifference⏎from helpers import min_item⏎⏎print(min
-
#211
Fires on “0”story 0.68% of tokens([2, 4, 1, 0,
-
#212
Fires on “+”worksheet 1.98% of tokens143?⏎04048+43
-
#213
Fires on “ always”story 0.36% of tokens. He was jolly, loved to play and was always having fun
-
#214
Fires on “path”agent 0.46% of tokenswhat does negate return?<|user_end|><|assistant_start|><|tool_call_start|>read<|arg|>path=ops
-
#215
Fires on “",”story 1.21% of tokenscan list, read
-
#216
Fires on “<|arg|>”agent 0.41% of tokens, fix the bug<|user_end|><|assistant_start|><|tool_call_start|>glob<|arg|>pattern=**/
-
#217
Fires on “Bug”agent 0.57% of tokens<|bos|>Bug: `
-
#218
Fires on “ help”chat 0.15% of tokens?<|user_end|><|assistant_start|>I'm sorry, I can't help with that
-
#219
Fires on “1”worksheet 0.25% of tokens7+72: 7+2+1=1
-
#220
Fires on “ you”story 0.31% of tokensI live in the rich part of town. Do you want to
-
#221
Fires on “.”agent 0.49% of tokensBut then, something unexpected happened. The fire
-
#222
Fires on “,”python 1.25% of tokensPython files: `calc
-
#223
Fires on “ ran”story 0.69% of tokensthem. They went to the
-
#224
Fires on “ and”agent 0.27% of tokens's leaves.⏎Lily took some paper and put it
-
#225
Fires on “s”agent 0.72% of tokens.py fail? Fix it."<|user_end|><|assistant_start|><|tool_call_start|>shell
-
#226
Fires on “ dog”story 1.65% of tokensbuzzing around the meadow. His bee friends were
-
#227
Fires on “ a”story 0.38% of tokenswhen she went exploring.<|bos|>One day, a little boy
-
#228
Fires on “ and”story 0.95% of tokensfence! Tom was sad because he could
-
#229
Fires on “+”worksheet 0.57% of tokens: 4+2+1=
-
#230
Fires on “ is”worksheet 0.24% of tokens63 = 917.⏎What is 7
-
#231
Fires on “ag”agent 0.37% of tokens, subagent,
-
#232
Fires on “ soon”agent 0.59% of tokens!<|user_end|><|assistant_start|>Bye! Come back for another story soon.<|assistant_end|>
-
#233
Fires on “-”chat 0.24% of tokens<|user_end|><|assistant_start|>Max was a nosy three-year-old.
-
#234
Fires on “py”agent 0.44% of tokenspy:⏎
-
#235
Fires on “ s”agent 0.72% of tokensin `strings.py`: it used `return s[-1
-
#236
Fires on “ not”story 0.81% of tokensCome back!" Ben shouted.⏎But Lily did not hear him
-
#237
Fires on “ you”worksheet 0.23% of tokens<|assistant_end|><|user_start|>Hello<|user_end|><|assistant_start|>Hi there! Would you like a
-
#238
Fires on “ climbed”story 1.61% of tokensidea. She pushed a chair to the window and climbed on it
-
#239
Fires on “ story”story 0.46% of tokens't change his mind.⏎The moral of the story is if
-
#240
Fires on “ friend”story 0.66% of tokensient to start the game.⏎Tim called his friend, Sam
-
#241
Fires on “8”worksheet 1.61% of tokens10808⏎So 40489