prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#1695
Fires on “ of”
story 2.33% of tokens

while his neighbor opened the case and showed him all the amazing things
#1696
Fires on “ on”
story 0.19% of tokens

was a little boy named Tim. Tim was going on a mar
#1697
Fires on “ made”
story 0.73% of tokens

smelly sock belonged to Sam. They washed it and made it clean
#1698
Fires on “ n”
python 0.31% of tokens

if c in "
#1699
Fires on “1”
chat 0.77% of tokens

<|think_start|>101+010: 1+0
#1700
Fires on “expression”
agent 0.53% of tokens

s up into a tall tree. She was scared and cried for
#1701
Fires on “<|user_start|>”
agent 2.47% of tokens

for a story!<|assistant_end|><|user_start|>What's
#1702
Fires on “#”
chat 0.41% of tokens

48.<|assistant_end|><|bos|># README.md⏎# Tool
#1703
Fires on “ never”
story 0.24% of tokens

One day, Tim found a big fruit he had never seen before
#1704
Fires on “ happy”
story 0.37% of tokens

ring sound was Daisy's bell. Lily was so happy to find
#1705
Fires on “ came”
story 0.23% of tokens

something strange happened.⏎A big, friendly dog came running up
#1706
Fires on “ high”
story 0.44% of tokens

get the cone, but it was too high for him
#1707
Fires on “f”
python 0.75% of tokens

arith import half, negate⏎⏎print(half(5
#1708
Fires on “9”
agent 0.92% of tokens

999⏎
#1709
Fires on “ her”
story 1.17% of tokens

much nicer home, and he asked the others if they
#1710
Fires on “,”
worksheet 0.73% of tokens

I'm a tiny language model
#1711
Fires on “,”
story 1.00% of tokens

Lily, I made a big tower!" Ben says, showing
#1712
Fires on “ of”
worksheet 0.78% of tokens

and started to count right away. He counted one, two,
#1713
Fires on “ are”
agent 0.53% of tokens

, write, execute<|system_end|><|system_start|>You are an AI
#1714
Fires on “ how”
agent 0.36% of tokens

pick it up.⏎But then the wind blew even harder
#1715
Fires on “+”
worksheet 0.74% of tokens

many flowers does Ben have now?⏎23+01
#1716
Fires on “ is”
chat 0.70% of tokens

6<|tool_call_end|><|assistant_end|><|tool_start|>18<|tool_end|><|assistant_start|>The answer is 1
#1717
Fires on “ c”
python 0.67% of tokens

0⏎ for c in s:⏎ if c in "
#1718
Fires on “command”
agent 0.20% of tokens

fix it.<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python
#1719
Fires on “ill”
agent 0.51% of tokens

grep, question, read, shell, skill, sub
#1720
Fires on “```”
python 0.76% of tokens

])` returns `4`.⏎⏎```python⏎
#1721
Fires on “print”
agent 1.47% of tokens

rite<|arg|>path=demo.py<|arg|>content=print("W
#1722
Fires on “ and”
agent 0.04% of tokens

You are mini, a small and friendly assistant
#1723
Fires on “ the”
story 0.14% of tokens

his friend, Birdy the bird,
#1724
Fires on “2”
python 0.67% of tokens

_odd(x):⏎ return x % 2 ==
#1725
Fires on “ and”
story 0.27% of tokens

park every day.<|bos|>One day, a cat and a dog
#1726
Fires on “ named”
story 0.89% of tokens

a little girl named Lily heard
#1727
Fires on “`”
python 0.37% of tokens

Bug: `mean_list` should return
#1728
Fires on “ the”
story 0.09% of tokens

went for a walk. The sun was shining and the weather
#1729
Fires on “2”
agent 1.33% of tokens

min(nums)⏎3:
#1730
Fires on “1”
worksheet 1.21% of tokens

053: 8+3=11,
#1731
Fires on “ and”
story 1.22% of tokens

friends, to the trees, to the sun, and to anyone
#1732
Fires on “<|bos|>”
chat 1.11% of tokens

day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
#1733
Fires on “ look”
story 0.62% of tokens

, Mia spotted something purple.⏎"Mommy, look! It
#1734
Fires on “test”
agent 1.01% of tokens

910<|think_end|>910<|assistant_end|><|bos|># test_textlib
#1735
Fires on “ with”
agent 0.42% of tokens

b` with `return
#1736
Fires on “1”
agent 0.13% of tokens

rue<|tool_call_end|><|assistant_end|><|tool_start|>Edited run.py (2 replacements)
#1737
Fires on “<|bos|>”
chat 0.66% of tokens

a > b:`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Start
#1738
Fires on “ory”
agent 0.25% of tokens

ery, memory<|system_end|><|system_start|>
#1739
Fires on “ with”
story 0.16% of tokens

park with his mom. They saw a big tree with lots of
#1740
Fires on “ tell”
story 0.88% of tokens

The river is fast and deep. Mom and Dad told us not
#1741
Fires on “ [”
agent 7.77% of tokens

⏎8: ⏎9: NUMBERS = [1,
#1742
Fires on “ count”
python 1.24% of tokens

== "HELLO"⏎assert first_letter
#1743
Fires on “2”
chat 0.89% of tokens

have now?<|user_end|><|assistant_start|>The answer is 121.
#1744
Fires on “ him”
story 0.63% of tokens

and gave it a hug