Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#893
Fires on “ he”story 2.02% of tokensbe clean, so he decided to
-
#894
Fires on “(”agent 0.37% of tokens0, 2]⏎print(head(SCORES))
-
#895
Fires on “ qu”agent 0.40% of tokens.<|bos|>One day, Fred the cat was feeling quite nosy
-
#896
Fires on “ called”story 0.23% of tokensshe walked through the trees, she heard a voice calling to
-
#897
Fires on “ decrement”agent 0.95% of tokensfrom text import count_vowels
-
#898
Fires on “⏎”agent 0.68% of tokenscan have my delicious mouse!"<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>
-
#899
Fires on “ hide”worksheet 0.41% of tokenstheir gray dog, Max. They like to play hide and seek
-
#900
Fires on “ with”agent 1.08% of tokens00⏎ ^^^^^^^^^^^^^^^^^⏎AssertionError⏎⏎Exited with code
-
#901
Fires on “ was”story 0.42% of tokenswas big and
-
#902
Fires on “ ready”story 2.32% of tokensand he is angry. He tells them to get down and go
-
#903
Fires on “ to”story 5.48% of tokensHe found a big stick and thought, "I can make a
-
#904
Fires on “ arith”agent 1.03% of tokenscoding agent harness.<|system_end|><|user_start|>"What functions does text.py
-
#905
Fires on “se”agent 1.27% of tokenswebfetch
-
#906
Fires on “1”worksheet 0.35% of tokens3886: 6+6=12,
-
#907
Fires on “g”agent 0.84% of tokensthe tests and fix any bug."<|user_end|><|assistant_start|><|tool_call_start|>glob
-
#908
Fires on “ near”story 0.23% of tokenslittle girl named Lucy found a thick cardboard box in her room
-
#909
Fires on “⏎”agent 0.29% of tokensprint(subtract(6, 8))⏎⏎# test
-
#910
Fires on “ important”worksheet 0.23% of tokensof the chicken scratch. But he did learn an important lesson in
-
#911
Fires on “,”story 0.45% of tokens<|assistant_end|><|bos|>Once upon a time, there was
-
#912
Fires on “```”python 0.76% of tokens4, 2])⏎4⏎<|bos|>```python⏎
-
#913
Fires on “c”agent 0.72% of tokenswent to the park to do some exercise.
-
#914
Fires on “3”agent 0.40% of tokens1382
-
#915
Fires on “ and”story 0.40% of tokenswill die and not come back."⏎Lily and Ben felt
-
#916
Fires on “0”worksheet 1.69% of tokens93?⏎09+
-
#917
Fires on “1”worksheet 0.20% of tokens0039: 4+9=13,
-
#918
Fires on “ stuck”story 0.56% of tokenswith her friends. They saw a big red ball stuck high up
-
#919
Fires on “ "”story 1.15% of tokenswas friends with a fast jet that could fly high in the sky
-
#920
Fires on “ She”story 0.88% of tokensmy eggs." Lily and Tom were very surprised. They did not
-
#921
Fires on “ return”agent 0.70% of tokenss):⏎2: return s.
-
#922
Fires on “,”worksheet 1.87% of tokens+86: 0+6=6, 6
-
#923
Fires on “:”story 0.56% of tokenshappier.⏎The moral of the story is: List
-
#924
Fires on “ sad”story 0.33% of tokensthere was an ugly tree. The ugly tree was sad because all
-
#925
Fires on “```”python 0.27% of tokensROBOT⏎78⏎<|bos|>```python⏎
-
#926
Fires on “ home”story 0.43% of tokensk in her room. She wanted to make her room clean,
-
#927
Fires on “ and”story 0.53% of tokensthe others if they wanted to move to a bigger and better place
-
#928
Fires on “p”agent 0.32% of tokens<|bos|><|system_start|>tools: edit, glob, grep, question
-
#929
Fires on “ He”story 5.56% of tokensTim saw a hard twig on the ground. He picked it
-
#930
Fires on “ met”story 0.25% of tokensthe wind.⏎While sailing, the boat met a little
-
#931
Fires on “ need”story 0.21% of tokenssays he is still a good boy, but he needs to
-
#932
Fires on “"”story 1.32% of tokensvoices.⏎John called out, "Hello,
-
#933
Fires on “ too”story 0.13% of tokensful dancing made everyone in the house want to dance too. So
-
#934
Fires on “⏎ ”agent 0.92% of tokensup the hill!" Tom says. "It will be fun!"
-
#935
Fires on “ re”story 0.44% of tokensmore fun and adventurous days together, and they always remained
-
#936
Fires on “ something”story 0.22% of tokenswandered around his house and outside, looking for something to eat
-
#937
Fires on “it”agent 0.93% of tokensreturn s.lower()<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path
-
#938
Fires on “ you”story 2.03% of tokens"I want to have a marriage when I grow up
-
#939
Fires on “ the”story 1.10% of tokensThey are strong and brave. They are almost at the top.
-
#940
Fires on “0”worksheet 0.51% of tokens6 and 41148.⏎000
-
#941
Fires on “ each”story 0.25% of tokenswere the best of friends, and they always helped each other.
-
#942
Fires on “ k”agent 0.68% of tokens":⏎5: k = k +