Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1745
Fires on “1”worksheet 0.42% of tokens⏎1+0: 1+0=1,
-
#1746
Fires on “ig”agent 0.94% of tokensher. "My name is Sally," the figure said
-
#1747
Fires on “ will”story 0.73% of tokensyourself. The moral of the story is that sharing will bring more
-
#1748
Fires on “"”agent 0.41% of tokensa script say.py that prints "Good morning" and run
-
#1749
Fires on “ said”story 0.26% of tokenssaid they needed
-
#1750
Fires on “Once”story 0.24% of tokenstext.py`, line 1.<|assistant_end|><|bos|>Once upon a
-
#1751
Fires on “1”python 0.26% of tokens([4, 5, 5, 4],
-
#1752
Fires on “8”chat 0.85% of tokens<|system_end|><|user_start|>Hey, what's 8 plus
-
#1753
Fires on “8”chat 0.78% of tokens.⏎I need to know 61349 +
-
#1754
Fires on “ent”agent 1.64% of tokensve Lily a yummy meal. She made a menu with
-
#1755
Fires on “2”agent 0.76% of tokens=print(max_item([5, 2,
-
#1756
Fires on “ more”python 2.50% of tokenstests passed")⏎⏎$ python3 test_utils.py
-
#1757
Fires on “t”agent 0.54% of tokensreplaceAll=true
-
#1758
Fires on “⏎”worksheet 0.67% of tokens0+1=8, 801⏎16
-
#1759
Fires on “1”agent 0.88% of tokens, but it has `result = result + 1`.⏎
-
#1760
Fires on “ ”worksheet 0.82% of tokens+0: 9+0=9, 96
-
#1761
Fires on “,”agent 0.56% of tokenstests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit, glob
-
#1762
Fires on “ ”worksheet 0.63% of tokens= 7874.⏎8 + 5 =
-
#1763
Fires on “ little”story 0.62% of tokensSam was a little boy.
-
#1764
Fires on “ can”story 0.34% of tokens.<|assistant_end|><|user_start|>Hello!<|user_end|><|assistant_start|>Hello! How can I help
-
#1765
Fires on “ they”story 1.65% of tokensvery much.⏎One day, her mom says they have to
-
#1766
Fires on “3”agent 0.30% of tokens<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3 hello.
-
#1767
Fires on “ like”chat 0.13% of tokens<|user_start|>Hello<|user_end|><|assistant_start|>Hi there! Would you like a story
-
#1768
Fires on “ b”python 0.25% of tokens))⏎⏎⏎def min_of(a, b):⏎
-
#1769
Fires on “'t”story 1.07% of tokensfelt sad and didn't want to
-
#1770
Fires on “ came”story 0.34% of tokensThen, something strange happened. A big wind came and blew
-
#1771
Fires on “,”story 2.04% of tokenspark with her mom. She played on the swings, the slide
-
#1772
Fires on “ jumped”story 0.51% of tokenswas purple! Lily was very surprised.⏎Lily jumped in the
-
#1773
Fires on “0”python 1.06% of tokens⏎ROBOT⏎```<|assistant_end|><|bos|>Bug: `
-
#1774
Fires on “<|bos|>”chat 0.83% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#1775
Fires on “ we”python 0.48% of tokenswebf
-
#1776
Fires on “:”agent 0.49% of tokens, mean monster had won.<|bos|><|user_start|>Could you tell me what
-
#1777
Fires on “ on”story 1.17% of tokensride up and down the street, feeling the wind in his face
-
#1778
Fires on “ the”story 0.81% of tokensbig red ball on the clear street
-
#1779
Fires on “ it”story 1.65% of tokenssaw a shiny nail on the ground. He picked it up and
-
#1780
Fires on “<|bos|>”chat 0.87% of tokensknow how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools
-
#1781
Fires on “5”worksheet 0.38% of tokens4⏎7+5: 7+5+1
-
#1782
Fires on “.”agent 0.80% of tokens<|assistant_start|><|tool_call_start|>read<|arg|>path=lib.py<|tool_call_end|>
-
#1783
Fires on “ with”story 2.16% of tokensTim liked to watch the fish
-
#1784
Fires on “.”agent 0.31% of tokensmusic. She saw the people in the seats. She felt
-
#1785
Fires on “ and”story 0.16% of tokens9.⏎If you add 635 and 9
-
#1786
Fires on “Why”story 0.44% of tokenssmiled and said, "That looks sweet Tom. Why don't
-
#1787
Fires on “ with”agent 1.42% of tokens` with `max
-
#1788
Fires on “2”worksheet 0.36% of tokens.⏎what is 91283+88
-
#1789
Fires on “ to”story 0.31% of tokens. They were so happy to have found something new to play with
-
#1790
Fires on “ her”story 0.44% of tokensand hugged Lily. They
-
#1791
Fires on “ how”agent 0.25% of tokens, and I know how to tell little stories and how to add
-
#1792
Fires on “ town”story 0.63% of tokensthe family. Tim felt sad for the ra
-
#1793
Fires on “("”python 0.10% of tokensUSIC") == "music"⏎print("all tests
-
#1794
Fires on “ be”agent 0.40% of tokens! If you win, you will be the best swimmer."