prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#40
Fires on “ it”
story 0.59% of tokens

decided to hide the music from Max. He thought this would make
#41
Fires on “ nice”
chat 0.52% of tokens

bye!<|user_end|><|assistant_start|>Goodbye! Have a nice day.
#42
Fires on “ the”
story 0.33% of tokens

thought of a plan. He asked all his friends to
#43
Fires on “ coding”
agent 0.89% of tokens

a coding agent harness
#44
Fires on “ her”
story 1.72% of tokens

, a little girl named Lily went to play in her yard.
#45
Fires on “ in”
story 0.25% of tokens

on it.
#46
Fires on “ ”
worksheet 1.06% of tokens

<|assistant_end|><|bos|>What's 7819 + 85
#47
Fires on “ They”
story 3.72% of tokens

the hill. Tim was proud of Max, and they played in
#48
Fires on “ project”
agent 5.98% of tokens

calc.py⏎calc.py<|tool_end|><|assistant_start|>This project has
#49
Fires on “ of”
agent 0.72% of tokens

sum of the
#50
Fires on “ liked”
story 0.66% of tokens

girl called Amy. She was a happy girl who liked to play
#51
Fires on “.”
agent 1.26% of tokens

small Python project.⏎⏎# test_listlib.py⏎
#52
Fires on “ went”
story 2.05% of tokens

hungry. He wanted to get some food. He went to the
#53
Fires on “ welcome”
agent 0.27% of tokens

<|system_end|><|user_start|>Thank you<|user_end|><|assistant_start|>Sure! You're welcome!<|assistant_end|>
#54
Fires on “ ”
agent 0.54% of tokens

: 5+
#55
Fires on “0”
worksheet 0.97% of tokens

263 flowers and found 1465 more
#56
Fires on “ help”
agent 0.73% of tokens

"Don't be sad, Sue. I will help you find
#57
Fires on “ sum”
chat 0.22% of tokens

playing. The big cat said, "I am tired." He
#58
Fires on “ and”
story 8.69% of tokens

me replace the hole in the boat?" Sam nodded and said,
#59
Fires on “9”
worksheet 1.65% of tokens

⏎So 573 + 1897 =
#60
Fires on “,”
story 1.83% of tokens

ice to meet you, Tim."
#61
Fires on “ main”
python 0.53% of tokens

the Python files in this project.<|assistant_end|><|bos|># app.py
#62
Fires on “ to”
agent 0.22% of tokens

Would you like a story, or some numbers to add?
#63
Fires on “ numbers”
chat 0.15% of tokens

<|assistant_start|>Hi there! Would you like a story, or some
#64
Fires on “<|tool_call_start|>”
agent 1.08% of tokens

test_number_utils.py"<|user_end|><|assistant_start|><|tool_call_start|>she
#65
Fires on “ outside”
agent 0.17% of tokens

. Spot was a dependable dog who loved to play outside. One
#66
Fires on “ you”
story 0.51% of tokens

ran away. I just got scared. Will you forgive me
#67
Fires on “1”
agent 0.78% of tokens

⏎Fix: replace `result = result + 1` with
#68
Fires on “Hi”
chat 1.24% of tokens

in this project.<|assistant_end|><|bos|><|user_start|>Hey<|user_end|><|assistant_start|>Hi there!
#69
Fires on “ dad”
story 0.11% of tokens

to her mom and dad. They
#70
Fires on “ x”
python 0.99% of tokens

⏎⏎`is_positive` returns True if x is positive
#71
Fires on “ sometimes”
story 0.50% of tokens

and Tom were happy and safe. They learned that sometimes, something
#72
Fires on “ in”
agent 0.47% of tokens

run, create, edit and fix the Python files in this project
#73
Fires on “ friend”
python 0.65% of tokens

Tim was happy to have a new friend.⏎
#74
Fires on “ mom”
agent 0.20% of tokens

mom. Mia
#75
Fires on “ the”
story 1.74% of tokens

thought of a plan. He asked all his friends to
#76
Fires on “)”
agent 0.44% of tokens

/ 2` with `return sum(nums) / len
#77
Fires on “6”
worksheet 0.33% of tokens

8299 toys and found 51615
#78
Fires on “ brother”
story 2.39% of tokens

fun!" she exclaimed.⏎Her older brother, Sam
#79
Fires on “ files”
agent 0.07% of tokens

, run, create, edit and fix the Python files in this
#80
Fires on “n”
agent 1.27% of tokens

surprised because it hadn't t
#81
Fires on “ to”
story 0.35% of tokens

Max came to the house
#82
Fires on “ only”
agent 0.43% of tokens

sorry, I can't help with that. I only know how
#83
Fires on “ he”
agent 0.68% of tokens

on a folder. "Ouch!" she said.
#84
Fires on “ we”
story 0.57% of tokens

webf
#85
Fires on “(”
python 1.04% of tokens

and add numbers.<|assistant_end|><|bos|>`abs_value(x)`
#86
Fires on “ you”
worksheet 0.59% of tokens

9 = 654.⏎If you add
#87
Fires on “ ”
story 0.77% of tokens

9: 8+9=17, 17
#88
Fires on “ lived”
story 0.42% of tokens

upon a time, in a small town, there lived a little
#89
Fires on “,”
agent 1.44% of tokens

⏎Mom hugged Sara and said, "I