Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#342
Fires on “`”python 0.45% of tokens` should return
-
#343
Fires on “ and”chat 0.27% of tokens, and a funny tail. Lily was not scared. She
-
#344
Fires on “ another”story 0.49% of tokens<|user_end|><|assistant_start|>Sure! Bye! Come back for another story soon
-
#345
Fires on “ didn”story 1.50% of tokensasked Tim for the coin. Tim was shy and didn't want
-
#346
Fires on “ it”agent 0.37% of tokenslist` should return the average of nums, but it has `
-
#347
Fires on “ man”story 0.92% of tokensspeeding away. Joe chased after him and he managed
-
#348
Fires on “7”worksheet 1.34% of tokens0022: 3+2=5,
-
#349
Fires on “(”agent 0.41% of tokens⏎ result = 1⏎ for i in range(1,
-
#350
Fires on “ on”story 1.14% of tokensput the cushion on her bed
-
#351
Fires on “⏎”worksheet 2.09% of tokens: 8+1=9, 9⏎61
-
#352
Fires on “'re”worksheet 0.72% of tokens.<|assistant_end|><|bos|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
-
#353
Fires on “ c”story 1.33% of tokensfull. Tim was proud of the meal he cooked
-
#354
Fires on “Thank”agent 0.55% of tokensto the thief. She said, "Thanks for
-
#355
Fires on “=”worksheet 0.35% of tokens0+100: 0+0=0,
-
#356
Fires on “?”worksheet 0.65% of tokens's 4 plus 41402?⏎0
-
#357
Fires on “<|think_end|>”chat 0.64% of tokens1=10, 1006<|think_end|>The answer
-
#358
Fires on “,”agent 0.33% of tokensSpanish: good morning.<|user_end|><|assistant_start|>Sorry, I'm
-
#359
Fires on “ by”agent 0.43% of tokensby step or
-
#360
Fires on “3”worksheet 1.33% of tokens: 6+8=14, 4⏎0
-
#361
Fires on “ done”story 1.51% of tokensWhen they were
-
#362
Fires on “ text”python 0.35% of tokensfrom text import count
-
#363
Fires on “1”worksheet 0.64% of tokens<|user_end|><|assistant_start|><|think_start|>9+1: 9+1=1
-
#364
Fires on “ we”agent 1.27% of tokensmiddle of the room. His mum said, "We can un
-
#365
Fires on “ a”story 0.59% of tokensabout a dragon.<|user_end|><|assistant_start|>Once upon a time,
-
#366
Fires on “ number”python 1.46% of tokens⏎ for i in range
-
#367
Fires on “banana”python 0.89% of tokensFor example, `exclaim("python")` returns `"python!"`.
-
#368
Fires on “ for”chat 0.63% of tokensyou add 38 and 327 for me?
-
#369
Fires on “ collar”story 1.08% of tokenswas blue and had a thick shirt. He
-
#370
Fires on “1”worksheet 0.33% of tokens0+97: 0+7+1=8
-
#371
Fires on “most”agent 0.34% of tokenstest_calc.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call
-
#372
Fires on “=”worksheet 0.40% of tokens3+422: 3+2=5,
-
#373
Fires on “,”story 0.65% of tokens, but he wanted to get his ball. So, he thought
-
#374
Fires on “TEXT”agent 0.59% of tokens"cloud"))⏎print(last_letter(TEXT))⏎
-
#375
Fires on “ in”agent 0.38% of tokensOnce upon a time, in a big
-
#376
Fires on “ with”story 0.32% of tokensood morning<|user_end|><|assistant_start|>Hi! How can I help with your code
-
#377
Fires on “As”story 0.30% of tokensuel in the car and they drove off. As they raced
-
#378
Fires on “y”chat 0.27% of tokens<|bos|>Once upon a time, there was a loyal dog
-
#379
Fires on “,”agent 0.81% of tokensetch, websearch, write, exe
-
#380
Fires on “.”python 0.39% of tokensn in nums]⏎⏎$ python3 app.py⏎
-
#381
Fires on “ of”agent 0.30% of tokenstoo small for the car. So, he thought of a plan
-
#382
Fires on “ friend”story 0.84% of tokens. He was very sad. Then, a kind cat named Luna
-
#383
Fires on “ '”agent 1.11% of tokensains' is not defined. Did you mean: 'contains'?
-
#384
Fires on “Once”story 0.20% of tokens<|bos|><|user_start|>Tell me a short story.<|user_end|><|assistant_start|>Once upon a
-
#385
Fires on “,”agent 0.53% of tokens: edit, glob, grep, question, read,
-
#386
Fires on “`”python 0.23% of tokens()⏎```⏎⏎Bug: `times` should return
-
#387
Fires on “2”worksheet 2.05% of tokens+1=16, 1691<|think_end|>The
-
#388
Fires on “to”agent 1.33% of tokenscount_vowels(s
-
#389
Fires on “)`.”python 1.32% of tokens1]))⏎print(next_number(3))⏎⏎
-
#390
Fires on “ you”story 0.46% of tokens"Hi, Bunny!" said Spot. "Do you want to
-
#391
Fires on “command”agent 0.32% of tokens<|tool_call_start|>shell<|arg|>command=