Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1895
Fires on “+”worksheet 0.83% of tokens00 + 2.⏎200+00
-
#1896
Fires on “s”agent 0.16% of tokenspy fails, please fix it<|user_end|><|assistant_start|><|tool_call_start|>shell
-
#1897
Fires on “e”agent 0.64% of tokens, websearch, write, execute
-
#1898
Fires on “ ”agent 0.30% of tokensk<|tool_end|><|assistant_start|><|tool_call_start|>edit
-
#1899
Fires on “ Every”story 0.13% of tokens.<|bos|>Pete was a happy rat. Every morning,
-
#1900
Fires on “ you”agent 0.59% of tokensyou add 1 and 341, you get
-
#1901
Fires on “=”agent 0.65% of tokens<|arg|>oldString=welcome<|arg|>newString=greet<|arg|>replaceAll=tr
-
#1902
Fires on “ answer”chat 0.48% of tokens<|tool_call_end|><|assistant_end|><|tool_start|>50<|tool_end|><|assistant_start|>The answer is
-
#1903
Fires on “5”worksheet 0.87% of tokens8.⏎What is 10857 plus
-
#1904
Fires on “ good”story 0.46% of tokensthe wall. They were both very happy and became good friends.
-
#1905
Fires on “ How”chat 0.12% of tokens. Anna gets 872 more stars. How many stars
-
#1906
Fires on “1”agent 0.23% of tokens^^^^^^⏎AssertionError⏎⏎Exited with code 1<|tool_end|><|assistant_start|>
-
#1907
Fires on “_”agent 1.16% of tokens$ python3 test_lib.
-
#1908
Fires on “ loud”story 0.68% of tokensballs make a loud noise
-
#1909
Fires on “ divides”python 0.52% of tokens⏎<|bos|># demo.py⏎from calc import divides, decrement
-
#1910
Fires on “ +”worksheet 0.75% of tokens33.<|bos|>What's 370 + 1
-
#1911
Fires on “Hi”chat 0.24% of tokensin this project.<|assistant_end|><|bos|><|user_start|>Hey<|user_end|><|assistant_start|>Hi there!
-
#1912
Fires on “0”worksheet 0.37% of tokens4 + 103?<|user_end|><|assistant_start|><|think_start|>00
-
#1913
Fires on “1”worksheet 0.29% of tokens<|assistant_end|><|bos|>Tom has 87274 apples
-
#1914
Fires on “ated”agent 0.22% of tokensa and b separated by a
-
#1915
Fires on “ How”agent 0.22% of tokenshelpful assistant.<|system_end|><|user_start|>Hello<|user_end|><|assistant_start|>Hello! How can I
-
#1916
Fires on “ home”story 0.88% of tokensleton away and said goodbye. Tom went to bed and had
-
#1917
Fires on “ toy”story 1.55% of tokenswas a little girl named Sue. Sue had a pet cat named
-
#1918
Fires on “1”worksheet 0.59% of tokens701+0006: 1+6
-
#1919
Fires on “,”story 1.04% of tokens?" Sam replied, "My desk is broken, and I
-
#1920
Fires on “def”agent 1.03% of tokens("lemon")⏎2⏎⏎```python⏎def divides(
-
#1921
Fires on “ there”story 0.51% of tokensa story?<|user_end|><|assistant_start|>Once upon a time, there was a
-
#1922
Fires on “ t”story 0.32% of tokenswarmer. From then on, the stars would travel
-
#1923
Fires on “ ”worksheet 0.62% of tokens5, 56⏎7+8: 7+
-
#1924
Fires on “ <”agent 0.37% of tokens⏎ if x < 0
-
#1925
Fires on “ could”story 0.65% of tokensgo!" Tim began to run as fast as he could. He
-
#1926
Fires on “0”worksheet 0.60% of tokens94 = 850.⏎0 plus
-
#1927
Fires on “Let”story 0.63% of tokensthought it was so funny that it started
-
#1928
Fires on “ bird”story 1.92% of tokensit warm and fed it. Soon, the bird grew big
-
#1929
Fires on “=”worksheet 0.47% of tokens: 7+9+1=17
-
#1930
Fires on “ to”story 1.98% of tokensrolling towards them. Tom was scared and did not know what
-
#1931
Fires on “-”agent 1.98% of tokensmax(nums
-
#1932
Fires on “"”agent 0.16% of tokensin OpenCode, a coding agent harness.<|system_end|><|user_start|>"There is
-
#1933
Fires on “ l”story 0.58% of tokensdone swinging, she saw a dull hose lying on
-
#1934
Fires on “ used”agent 0.43% of tokens_many` in `items.py`: it used `return
-
#1935
Fires on “list”agent 1.47% of tokensagent harness.<|system_end|><|user_start|>Add a function average_list(nums
-
#1936
Fires on “ ”worksheet 0.64% of tokenswheat.<|bos|><|user_start|>What is 4 plus 7?
-
#1937
Fires on “he”agent 0.36% of tokens)<|tool_end|><|assistant_start|><|tool_call_start|>shell<|arg|>
-
#1938
Fires on “_”agent 0.42% of tokensdouble_al' is not defined. Did you mean: '
-
#1939
Fires on “([”python 0.42% of tokenslargest number in nums. For example, `biggest([4,
-
#1940
Fires on “ each”agent 0.38% of tokenswere the best of friends, and they always helped each other.
-
#1941
Fires on “ But”story 0.34% of tokens"The apples belong to someone else."⏎"But they look
-
#1942
Fires on “ stories”story 0.83% of tokens'm a tiny model: I can only tell short stories and add
-
#1943
Fires on “ a”story 0.73% of tokens.<|assistant_end|><|bos|>Once upon a time, there was a nosy bee
-
#1944
Fires on “ new”story 0.26% of tokensnew file tools