prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#1845
Fires on “ sk”
agent 0.92% of tokens

water was swirling around in the puddle, twirling
#1846
Fires on “ stopped”
story 0.54% of tokens

round thing started to crack. The bunnies stopped and looked
#1847
Fires on “ friends”
story 0.28% of tokens

The bird, Tim, and the car all became friends and played
#1848
Fires on “,”
story 0.31% of tokens

and shapes of blocks.⏎"Look, Lily, I made
#1849
Fires on “ surprised”
story 0.33% of tokens

They are my eggs." Lily and Tom were very surprised. They
#1850
Fires on “ short”
chat 0.21% of tokens

don't know about that. I can only tell short stories and
#1851
Fires on “9”
worksheet 4.38% of tokens

: 3+0=3, 36<|think_end|>The
#1852
Fires on “e”
agent 0.56% of tokens

girl who lived on a gloomy day. She yearned
#1853
Fires on “ from”
agent 0.29% of tokens

asked him to forgive him. He said he wanted to
#1854
Fires on “+”
worksheet 3.56% of tokens

457.⏎00019+34
#1855
Fires on “One”
story 0.30% of tokens

was always ready to lend a paw.⏎One day,
#1856
Fires on “ of”
story 0.77% of tokens

.⏎At the party, they played a game called "m
#1857
Fires on “ prints”
agent 0.31% of tokens

ldes` should be `includes`. It now prints:⏎
#1858
Fires on “ is”
story 0.70% of tokens

the park to play again.<|bos|>What is 4
#1859
Fires on “from”
python 0.16% of tokens

))⏎⏎# test_ops.py⏎from ops import
#1860
Fires on “Read”
agent 0.39% of tokens

read<|arg|>path=utils.py<|tool_call_end|><|assistant_end|><|tool_start|>Read file utils
#1861
Fires on “ you”
chat 0.64% of tokens

What a lovely bunny, Anna. Where did you find him
#1862
Fires on “ tell”
story 1.39% of tokens

who are hurt. Tim felt regret for not listening to
#1863
Fires on “oldString”
agent 0.42% of tokens

<|tool_call_start|>edit<|arg|>path=demo.py<|arg|>oldString=plus
#1864
Fires on “=”
agent 0.48% of tokens

replacement)<|tool_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3
#1865
Fires on “ each”
story 0.72% of tokens

that he could help his mom and dad talk to each other.
#1866
Fires on “ on”
story 1.64% of tokens

axi was very adventurous. It liked to go on long r
#1867
Fires on “1”
worksheet 0.46% of tokens

49: 3+9+1=13,
#1868
Fires on “.”
agent 0.84% of tokens

9804.⏎9
#1869
Fires on “⏎⏎”
agent 0.23% of tokens

(items, x):⏎ return x in items⏎⏎⏎def
#1870
Fires on “assert”
agent 0.95% of tokens

== "bananabananabanana"⏎assert last_
#1871
Fires on “ were”
story 0.96% of tokens

Max played together with the ball, and they were very happy
#1872
Fires on “ small”
python 0.21% of tokens

5<|assistant_end|><|bos|>Once upon a time, in a small house near
#1873
Fires on “ ”
worksheet 1.29% of tokens

<|assistant_start|><|think_start|>549+000: 9+
#1874
Fires on “ at”
story 0.45% of tokens

loved to play at the park. Her favorite thing at the park
#1875
Fires on “ have”
agent 1.14% of tokens

we can share the shell, Ben. But you have to help
#1876
Fires on “A”
worksheet 0.50% of tokens

.md⏎# Playground⏎⏎A small Python
#1877
Fires on “The”
agent 0.35% of tokens

85 = 584.⏎The sum of
#1878
Fires on “path”
agent 0.69% of tokens

and she liked playing with her doll. She worked very hard and
#1879
Fires on “ loved”
story 0.48% of tokens

, there was a little girl named Mia. Mia loved her toy
#1880
Fires on “ is”
chat 1.38% of tokens

<|user_start|>What is 6
#1881
Fires on “ you”
story 0.76% of tokens

port was big and noisy. There were many people and plan
#1882
Fires on “,”
story 0.73% of tokens

't know where to look. She swam and swam, looking under
#1883
Fires on “ like”
story 0.24% of tokens

tree. Its branches were empty and it looked like it had
#1884
Fires on “utils”
agent 0.58% of tokens

and b.<|user_end|><|assistant_start|><|tool_call_start|>read<|arg|>path=utils.py
#1885
Fires on “ little”
story 0.61% of tokens

Let's go in the kayak!" The little boy was
#1886
Fires on “NUMBERS”
agent 0.59% of tokens

_utils import larger, sub_one⏎⏎COUNT =
#1887
Fires on “!”
agent 0.32% of tokens

<|user_start|>Once upon a time...<|user_end|><|assistant_start|>Sure! Once
#1888
Fires on “ trees”
story 0.95% of tokens

home. He liked to look at the flowers, trees, and
#1889
Fires on “ +”
python 0.55% of tokens

+ b)
#1890
Fires on “ it”
story 0.78% of tokens

that she loved very much. She took the teddy bear with her
#1891
Fires on “Once”
story 0.40% of tokens

I want a story about a dragon!<|user_end|><|assistant_start|>Once upon a
#1892
Fires on “<|system_start|>”
agent 0.28% of tokens

you add 9 and 840, you get
#1893
Fires on “+”
worksheet 1.23% of tokens

his fixed toy with the wealthy mouse. They became best friends and
#1894
Fires on “ call”
story 0.13% of tokens

call last):