Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#3550
Fires on “ was”story 0.39% of tokenspuppy and said, "Wow, your puppy is so cute!
-
#3551
Fires on “test”agent 0.48% of tokens<|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>`test_tools
-
#3552
Fires on “ no”story 0.05% of tokensthe white ball in its beak.⏎"Oh no!" said
-
#3553
Fires on “ and”story 0.10% of tokensated their houses. They wondered what was on the other side.
-
#3554
Fires on “ as”story 0.42% of tokensnear the alligator, so he ran away as fast as
-
#3555
Fires on “ when”story 0.43% of tokensturn. They learned that sometimes, good things happen when you try
-
#3556
Fires on “4”worksheet 0.14% of tokens, 402
-
#3557
Fires on “ named”story 0.67% of tokenssquared.<|assistant_end|><|bos|>One day, a grumpy cat named Tom went
-
#3558
Fires on “ "”agent 0.33% of tokensive. What do you do?" Tiny replied, "Hi B
-
#3559
Fires on “`”python 0.27% of tokens):⏎ return s + "!"⏎```⏎⏎`exclaim`
-
#3560
Fires on “ It”story 0.25% of tokensfrom a fierce
-
#3561
Fires on “4”worksheet 2.93% of tokens8493⏎0457+849
-
#3562
Fires on “<|tool_end|>”agent 1.18% of tokens'triangle'?⏎⏎Exited with code 1<|tool_end|><|assistant_start|><|tool_call_start|>
-
#3563
Fires on “ returns”python 0.62% of tokens]`.⏎⏎`divides(a, b)` returns True if
-
#3564
Fires on “```”python 0.77% of tokens(items):⏎ return items[0]⏎```⏎⏎
-
#3565
Fires on “,”worksheet 1.44% of tokens: 7+9+1=17, 7
-
#3566
Fires on “ not”story 0.38% of tokens"Mom said we should not take things that are not ours
-
#3567
Fires on “ talk”story 0.77% of tokensand the big blue bird sat in their nest and talked about their
-
#3568
Fires on “ for”story 0.21% of tokensto the house.⏎Bill thanked his neighbor for the inv
-
#3569
Fires on “ as”story 0.38% of tokensnear the alligator, so he ran away as fast as
-
#3570
Fires on “("”python 0.35% of tokenssize("hello") == 5⏎assert initial("banana")
-
#3571
Fires on “ so”story 1.30% of tokensbig, thick bird named Bobo. Bobo loved to soar high
-
#3572
Fires on “ belonged”agent 0.44% of tokenspassport. We need to find who it belongs to
-
#3573
Fires on “'s”story 0.28% of tokenssandwich under the tree. Tim's mom said
-
#3574
Fires on “'s”agent 0.22% of tokens<|think_end|>114<|assistant_end|><|bos|><|user_start|>Hey, what's 4
-
#3575
Fires on “,”story 0.36% of tokensin their hive.<|bos|>Once upon a time, there was
-
#3576
Fires on “<|bos|>”chat 0.66% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#3577
Fires on “Tom”story 0.61% of tokensSally went to Tom's house and said, "Tom, I
-
#3578
Fires on “'m”agent 0.41% of tokens<|bos|><|user_start|>What can you do?<|user_end|><|assistant_start|>I'm prelude:
-
#3579
Fires on “ look”story 0.93% of tokensto know what it was, so he went to look.⏎
-
#3580
Fires on “):”agent 0.71% of tokenspy⏎def whisper(s):⏎ return
-
#3581
Fires on “ upon”agent 0.82% of tokenslike warm, soft wool.<|bos|><|user_start|>Once upon a time
-
#3582
Fires on “⏎”agent 1.27% of tokensanswer is 7.<|assistant_end|><|user_start|>What if you add
-
#3583
Fires on “ ”python 0.59% of tokens8, 1, 3]) == 8⏎
-
#3584
Fires on “ They”agent 0.34% of tokensrake away and started to play. They both had
-
#3585
Fires on “ to”story 0.23% of tokenss," their mom said. "But you have to be careful
-
#3586
Fires on “6”worksheet 0.79% of tokens+59: 7+9=16,
-
#3587
Fires on “<|bos|>”chat 1.12% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#3588
Fires on “#”python 0.21% of tokensin nums]⏎⏎# README.md⏎# Sc
-
#3589
Fires on “5”story 0.38% of tokensthanked the old lady and promised to come
-
#3590
Fires on “ the”story 0.78% of tokensto school. She saw a big blackboard in her class.
-
#3591
Fires on “,”story 0.78% of tokens, "Can I be in the parade, please?"
-
#3592
Fires on “I”story 0.47% of tokensinto a beautiful bird! The bird said, "I am a
-
#3593
Fires on “0”worksheet 1.48% of tokens84⏎5+0: 5+0=5
-
#3594
Fires on “<|assistant_end|>”agent 1.74% of tokensThank you<|user_end|><|assistant_start|>Sure! You're welcome!<|assistant_end|><|bos|><|system_start|>
-
#3595
Fires on “print”python 0.35% of tokensprint(larger(9, 7))⏎print(sub
-
#3596
Fires on “5”worksheet 0.78% of tokens<|assistant_end|><|bos|>Tom has 87274 apples.
-
#3597
Fires on “ a”story 0.54% of tokensanswer is 8.<|assistant_end|><|bos|>One day, a little girl
-
#3598
Fires on “ there”story 0.27% of tokens, pour, and taste the food. But there was one
-
#3599
Fires on “7”story 0.38% of tokens7770