Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1795
Fires on “ len”python 0.32% of tokens⏎def count_items(items):⏎ return len(items
-
#1796
Fires on “ little”story 0.53% of tokensnot want to move.⏎One day, a little bird flew
-
#1797
Fires on “ helping”story 0.94% of tokensThey all hugged and said thank you to Kitty for helping.⏎
-
#1798
Fires on “ an”story 1.63% of tokensthe kids were sad. But then, Anna had an idea.
-
#1799
Fires on “One”story 0.55% of tokens<|bos|>One day,
-
#1800
Fires on “ated”story 0.21% of tokensfor Pinky. Soon, Pinky could jump just as high
-
#1801
Fires on “ to”story 2.90% of tokensand waved goodbye to the weird
-
#1802
Fires on “ over”story 0.62% of tokenstaking part in the game.⏎The girl was overwhe
-
#1803
Fires on “ and”story 0.70% of tokens. They found many old things there, like toys and books and
-
#1804
Fires on “ you”story 0.53% of tokenswas a joke. We are sorry. Are you mad at
-
#1805
Fires on “ moral”agent 0.44% of tokensone and continued to study together.⏎The moral of this
-
#1806
Fires on “ day”worksheet 0.76% of tokenseach other can solve problems. And from that day on,
-
#1807
Fires on “ walk”agent 0.48% of tokensday, a little boy named Tim went for a walk in the
-
#1808
Fires on “ little”story 0.57% of tokens<|bos|>One day, a little ball named
-
#1809
Fires on “<|bos|>”story 1.03% of tokens<|bos|>Once upon
-
#1810
Fires on “ and”story 2.04% of tokensHe walked up to the big cat and blin
-
#1811
Fires on “appy”agent 0.18% of tokens<|user_start|>Great, thanks!<|user_end|><|assistant_start|>Happy to help
-
#1812
Fires on “ what”chat 0.50% of tokens2 = 261.⏎Hey, what's
-
#1813
Fires on “ a”story 0.94% of tokensit!" The ant cried.⏎Just then, a big bird
-
#1814
Fires on “ saw”story 1.62% of tokensinto the living room. She heard the noises and saw Lily and
-
#1815
Fires on “,”python 0.18% of tokensreturns True if x is in items. For example, `has
-
#1816
Fires on “⏎”python 0.63% of tokens⏎ return "Hello, " + name + "?"⏎```⏎
-
#1817
Fires on “0”worksheet 0.73% of tokens8⏎00+63: 0+3
-
#1818
Fires on “)`”python 1.68% of tokensexample, `mean(1, 12)` returns `
-
#1819
Fires on “No”agent 0.51% of tokensyour blocks, Lily?" Ben asks.⏎"No, you
-
#1820
Fires on “<|tool_end|>”agent 0.84% of tokens: print(backwards(TEXT))<|tool_end|><|assistant_start|><|tool_call_start|>
-
#1821
Fires on “'s”agent 0.37% of tokens7 = 4933.<|bos|>What's 3
-
#1822
Fires on “=”worksheet 0.33% of tokens026: 1+6+1=8,
-
#1823
Fires on “ =”agent 0.40% of tokens49.⏎6 + 810 = 8
-
#1824
Fires on “0”worksheet 0.82% of tokens962⏎8026 + 0 =
-
#1825
Fires on “ all”story 1.56% of tokensin surprise as he flew through the air! They all clapped and
-
#1826
Fires on “[”python 1.48% of tokens[1]
-
#1827
Fires on “<|tool_call_start|>”agent 0.71% of tokens'?⏎⏎Exited with code 1<|tool_end|><|assistant_start|><|tool_call_start|>read<|arg|>
-
#1828
Fires on “ had”story 0.82% of tokenswant my hands to hurt." Mom smiled and gave Lily warm gl
-
#1829
Fires on “<|bos|>”chat 0.65% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#1830
Fires on “ the”story 0.18% of tokens, yummy food, and happy music. Everyone in the town loved
-
#1831
Fires on “ i”python 0.60% of tokensn + 1):⏎ acc = acc + i⏎ return
-
#1832
Fires on “,”worksheet 1.90% of tokensn in nums]⏎⏎⏎def contains(items, x):
-
#1833
Fires on “ mean”story 0.17% of tokensupper_ase' is not defined. Did you mean: '
-
#1834
Fires on “ wait”story 0.19% of tokensfor you," Sam replied. "You have to wait until you
-
#1835
Fires on “<|bos|>”chat 0.64% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#1836
Fires on “ know”story 0.92% of tokensHe was a very curious boy and always wanted to know everything.
-
#1837
Fires on “ example”python 0.36% of tokens`subtract` subtracts b from a. For example, `
-
#1838
Fires on “ They”story 1.66% of tokensThe colors were friends and liked to play together. They made the
-
#1839
Fires on “One”story 0.95% of tokensseen before.⏎Tom played with the new ball all day.
-
#1840
Fires on “ a”agent 0.83% of tokensif c in "
-
#1841
Fires on “pattern”agent 0.63% of tokensare here?<|user_end|><|assistant_start|><|tool_call_start|>glob<|arg|>pattern=**/*.py
-
#1842
Fires on “ ”agent 0.30% of tokens(a, b)` returns a and b separated by a
-
#1843
Fires on “0”worksheet 0.79% of tokens8 = ?⏎79447+000
-
#1844
Fires on “ `”python 0.58% of tokensa + " " + b⏎```⏎⏎`combine`