Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#3800
Fires on “ night”story 0.49% of tokensanswer is 87.<|assistant_end|><|bos|><|user_start|>Good night!<|user_end|>
-
#3801
Fires on “ lived”agent 1.23% of tokensand never let anything get lost again. And they lived happily ever
-
#3802
Fires on “ time”story 0.43% of tokens61799.<|bos|>Once upon a time, there
-
#3803
Fires on “1”story 0.26% of tokens7: 3+7=10,
-
#3804
Fires on “ fur”story 1.19% of tokenslived a boy named Tim. Tim had an orange flute that
-
#3805
Fires on “[”agent 0.37% of tokensevery number in nums, but it has `return [n *
-
#3806
Fires on “ you”chat 0.40% of tokenswith big smiles on their faces.<|bos|><|user_start|>Could you tell me
-
#3807
Fires on “loud”python 0.88% of tokens("moon")` returns `4`.⏎⏎`loud(s
-
#3808
Fires on “ to”chat 0.30% of tokensSue. Sue loved to act. She would pretend to be a
-
#3809
Fires on “0”worksheet 1.24% of tokens11, 1⏎14+00:
-
#3810
Fires on “ help”story 0.47% of tokensGood morning!<|user_end|><|assistant_start|>Hello! How can I help you today
-
#3811
Fires on “3”python 0.32% of tokens[8]⏎4.3333333
-
#3812
Fires on “`”agent 0.67% of tokens8, 7])⏎3⏎<|bos|>`min_
-
#3813
Fires on “,”story 0.18% of tokens<|bos|>Once upon a time, in a small town, there lived
-
#3814
Fires on “ careful”story 1.69% of tokensto be careful when sailing and to not go out in bad
-
#3815
Fires on “("”worksheet 0.28% of tokenssize("hello") == 5⏎assert initial("banana")
-
#3816
Fires on “ wanted”story 0.36% of tokensrace. They wanted to see who was the fastest.⏎
-
#3817
Fires on “a”python 0.62% of tokens⏎ return acc⏎⏎⏎def is_divisible(a, b
-
#3818
Fires on “<|arg|>”agent 1.44% of tokens.py<|arg|>oldString=head<|arg|>newString=first<|arg|>replaceAll=
-
#3819
Fires on “ for”story 0.18% of tokenson her shoulder. One day, Lily went for a walk
-
#3820
Fires on “ hello”agent 0.69% of tokens* x⏎<|tool_call_end|><|assistant_end|><|tool_start|>Created file successfully: lib.py
-
#3821
Fires on “lists”agent 1.16% of tokensis 88.<|assistant_end|><|bos|># test_lists.py
-
#3822
Fires on “5”worksheet 1.49% of tokens635.⏎I need to know 541
-
#3823
Fires on “ has”worksheet 0.27% of tokensget 21940.⏎Mia has 3
-
#3824
Fires on “s”python 1.11% of tokens1⏎ return k⏎⏎⏎def initial(s):⏎
-
#3825
Fires on “ how”chat 0.26% of tokensI can't help with that. I only know how to tell
-
#3826
Fires on “ park”story 0.49% of tokens`.<|assistant_end|><|bos|>Lily and Ben are playing in the park with their
-
#3827
Fires on “ result”agent 0.93% of tokens⏎⏎⏎def add_all(nums):⏎ result =
-
#3828
Fires on “ but”python 0.05% of tokensitem` should return the biggest item of nums, but it has
-
#3829
Fires on “ mess”story 0.60% of tokenshis feet, crossed his arms and said firmly
-
#3830
Fires on “ about”agent 0.52% of tokenss.<|user_end|><|assistant_start|>Sorry, I don't know about that.
-
#3831
Fires on “Once”story 0.38% of tokenswith `final_letter(s)`.<|assistant_end|><|bos|>Once upon a
-
#3832
Fires on “<|bos|>”python 0.49% of tokens3 app.py⏎81⏎t⏎<|bos|>Once upon
-
#3833
Fires on “ lots”story 0.19% of tokensseal and the crab joined the party and had lots of fun
-
#3834
Fires on “ So”story 0.33% of tokenslooks like a fun place to play!" So, he
-
#3835
Fires on “ that”story 5.43% of tokenswas a big hot sun. It was so hot that everyone looked
-
#3836
Fires on “s”python 0.75% of tokens<|assistant_end|><|bos|>`keep_even(nums)` keeps only the
-
#3837
Fires on “ has”worksheet 0.44% of tokens08 more. Now Anna has 2
-
#3838
Fires on “<|bos|>”chat 0.91% of tokensa story, or some numbers to add?<|assistant_end|><|bos|>If you
-
#3839
Fires on “⏎”python 0.74% of tokenshead(items)`.<|assistant_end|><|bos|># mathlib.py⏎def fact
-
#3840
Fires on “⏎”worksheet 0.45% of tokens. "I can grant you one wish," said the fairy
-
#3841
Fires on “ he”story 0.50% of tokensstone and started to polish it. As he polished
-
#3842
Fires on “:”story 0.45% of tokens." Lily closed her eyes and counted. "One, two,
-
#3843
Fires on “sum”agent 1.02% of tokens' is not defined. Did you mean: 'squared'?⏎
-
#3844
Fires on “ `”agent 0.77% of tokens<|bos|>Bug: `max_
-
#3845
Fires on “6”agent 0.29% of tokens66.⏎
-
#3846
Fires on “ag”agent 0.43% of tokens3, 4, 3]) == [0,
-
#3847
Fires on “7”worksheet 1.39% of tokens7, 76⏎00+86:
-
#3848
Fires on “ Lucy”worksheet 1.91% of tokens87 stickers and found 0 more. Now Lucy has
-
#3849
Fires on “ into”story 0.73% of tokensbetter.⏎Lily's mom heard them and came into the room