Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2696
Fires on “_”agent 0.35% of tokensabs_value(3))⏎⏎⏎def min_of(
-
#2697
Fires on “ now”agent 0.23% of tokensa % b == 0`. The tests pass now.<|assistant_end|>
-
#2698
Fires on “ He”story 0.62% of tokensHe liked to
-
#2699
Fires on “.”story 0.78% of tokenssmall town, there lived a cute dog named Max. Max was
-
#2700
Fires on “1”worksheet 0.40% of tokens⏎5+5: 5+5=10,
-
#2701
Fires on “c”agent 1.05% of tokenscute<|system_end|>
-
#2702
Fires on “1”worksheet 0.74% of tokens: 8+2+1=11,
-
#2703
Fires on “⏎”python 1.34% of tokensif n % 2 == 0]⏎⏎# test
-
#2704
Fires on “7”worksheet 0.94% of tokens+9+1=14, 145<|think_end|>
-
#2705
Fires on “ you”agent 0.80% of tokensHi!<|user_end|><|assistant_start|>Hi there! Would you like a
-
#2706
Fires on “ with”story 1.03% of tokensby step or with a calculator
-
#2707
Fires on “ first”python 0.78% of tokensitems[0]`.⏎⏎```python⏎def first(items
-
#2708
Fires on “ was”story 0.58% of tokensa story.<|user_end|><|assistant_start|>Once upon a time there was a little
-
#2709
Fires on “2”worksheet 2.16% of tokens24424 = 82567
-
#2710
Fires on “ help”story 0.39% of tokenswas a young boy named Tim. Tim loved to help his mom
-
#2711
Fires on “g”agent 0.70% of tokenseverything around them.
-
#2712
Fires on “ but”story 0.40% of tokenscloset, too! Lucy was scared at first, but the mouse
-
#2713
Fires on “ are”story 0.57% of tokens`main.py`.<|assistant_end|><|bos|>Lily and Ben are playing in
-
#2714
Fires on “ of”story 0.14% of tokensball made him so happy.⏎At the end of the day
-
#2715
Fires on “ some”agent 0.29% of tokensthere! Would you like a story, or some numbers to
-
#2716
Fires on “ filled”story 0.35% of tokensparty.⏎Mia was amazed! The room was filled with balloons
-
#2717
Fires on “<|tool_call_start|>”agent 1.15% of tokens7, 6, 0]))<|tool_end|><|assistant_start|><|tool_call_start|>edit
-
#2718
Fires on “:”chat 0.24% of tokens?<|user_end|><|assistant_start|>Sorry, I'm a tiny model: I can
-
#2719
Fires on “ it”story 0.90% of tokensTim watched carefully and tried to do it too.
-
#2720
Fires on “ to”story 1.55% of tokenssunny day, a little girl named Mia was eager to go to
-
#2721
Fires on “0”worksheet 1.81% of tokens50?⏎146+050:
-
#2722
Fires on “7”worksheet 2.94% of tokens, 70071⏎So 656
-
#2723
Fires on “ said”story 0.69% of tokenshis mother told him that he
-
#2724
Fires on “ happy”story 0.72% of tokenshave something to do. And the rich man was happy that he
-
#2725
Fires on “print”chat 0.16% of tokensprint(sub_one(12))⏎print(contains
-
#2726
Fires on “core”agent 1.02% of tokenstest_text.py
-
#2727
Fires on “ d”story 0.37% of tokensa big red button. He said, "I dare to
-
#2728
Fires on “1”worksheet 0.30% of tokensHi!<|user_end|><|assistant_start|>Hi there! Would you like a
-
#2729
Fires on “ soon”agent 0.40% of tokens!<|user_end|><|assistant_start|>Bye! Come back for another story soon.<|assistant_end|>
-
#2730
Fires on “ Lucy”worksheet 3.74% of tokens4 shells and found 42 more. Now Lucy has
-
#2731
Fires on “:”agent 1.16% of tokensc in s:⏎ if
-
#2732
Fires on “ step”agent 0.71% of tokensshort stories for children and add numbers, step by step or with
-
#2733
Fires on “ came”story 0.97% of tokensmom, and she came to see
-
#2734
Fires on “ b”python 0.36% of tokens: `difference` should subtract b from a
-
#2735
Fires on “ ”worksheet 0.11% of tokens7: 8+7=15, 15
-
#2736
Fires on “ question”agent 0.40% of tokenstools: edit, glob, grep, question, read
-
#2737
Fires on “ a”story 0.22% of tokenstell me a story?<|user_end|><|assistant_start|>Once upon a time,
-
#2738
Fires on “ for”story 0.71% of tokens<|bos|>One day, a cat named Jazz went for a walk
-
#2739
Fires on “ see”story 3.21% of tokensa gift from Grandma!" She opened the box and inside were yummy
-
#2740
Fires on “ wanted”story 0.98% of tokenscar was missing a tire.⏎Tim wanted to help
-
#2741
Fires on “un”story 0.34% of tokenssay sorry. They find another car in the trunk.
-
#2742
Fires on “.”agent 1.50% of tokenspython3 hello.py<|tool_call_end|><|assistant_end|><|tool_start|>2⏎False⏎<|tool_end|>
-
#2743
Fires on “[”agent 0.88% of tokens`: it used `return s[1:
-
#2744
Fires on “ smiled”story 0.79% of tokenssmiled and said
-
#2745
Fires on “ This”story 0.66% of tokenswent to the park again with the big ball. This time,