prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#3299
Fires on “.”
story 0.62% of tokens

Once upon a time, there was a little seed. The seed
#3300
Fires on “ a”
agent 0.57% of tokens

children and add numbers, step by step or with a calculator.
#3301
Fires on “ and”
story 2.14% of tokens

⏎Poppy thought for a moment and had an
#3302
Fires on “!”
chat 1.09% of tokens

project.<|assistant_end|><|bos|><|user_start|>Hey<|user_end|><|assistant_start|>Hi there! Would
#3303
Fires on “,”
agent 4.03% of tokens

etch, websearch, write, exe
#3304
Fires on “ Mia”
story 0.69% of tokens

listen to each other.⏎"Stop it, girls!"
#3305
Fires on “ last”
agent 0.51% of tokens

.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call last):⏎
#3306
Fires on “ with”
story 0.57% of tokens

went to the park. She saw a big tree with lots of
#3307
Fires on “,”
agent 2.26% of tokens

very sad. The unique box of colors was gone, and Lily
#3308
Fires on “ main”
story 0.59% of tokens

):⏎ return s.lower()⏎⏎# hello.py
#3309
Fires on “ was”
story 1.12% of tokens

the park. He was very glad. The sun was warm and
#3310
Fires on “ and”
story 0.95% of tokens

. Mia looked everywhere for it, but she could not find it
#3311
Fires on “ numbers”
chat 0.60% of tokens

language model: I can tell short stories and add numbers.<|assistant_end|>
#3312
Fires on “⏎”
story 0.38% of tokens

. He could balance many toys in his hands.⏎One day
#3313
Fires on “ idea”
story 1.11% of tokens

kids were sad. But then, Anna had an idea. "
#3314
Fires on “ `”
agent 0.60% of tokens

knife and put it in her bag to keep everyone safe.
#3315
Fires on “ this”
story 0.32% of tokens

take a bite.⏎"Mmm, this is yummy
#3316
Fires on “ small”
python 0.13% of tokens

final_letter("sun") == "n"⏎assert exclaim
#3317
Fires on “ big”
story 0.16% of tokens

⏎ return "Hello, " + name + "!"⏎⏎⏎U
#3318
Fires on “ touch”
story 1.25% of tokens

take your balls and go away. And don't touch my machine
#3319
Fires on “:”
agent 0.27% of tokens

% b == 0⏎```⏎⏎Bug: `sum
#3320
Fires on “ ”
worksheet 0.66% of tokens

Tim had 3946 flowers and found 31
#3321
Fires on “ very”
story 0.12% of tokens

named Tim was learning to whistle. He tried very hard,
#3322
Fires on “p”
agent 0.15% of tokens

find is_divisible?"<|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>pattern
#3323
Fires on “ if”
worksheet 0.70% of tokens

<|bos|>```python⏎def magnitude(x):⏎ if x <
#3324
Fires on “1”
worksheet 0.45% of tokens

+053: 7+3=10,
#3325
Fires on “<|system_start|>”
agent 0.75% of tokens

<|bos|><|system_start|>Answer in
#3326
Fires on “ warm”
story 0.91% of tokens

outside to enjoy the nice weather
#3327
Fires on “ fun”
story 0.51% of tokens

oth of them were happy and laughing. What a great day!
#3328
Fires on “1”
worksheet 1.26% of tokens

⏎00+63: 0+3+1
#3329
Fires on “ to”
agent 0.20% of tokens

(1 replacement)<|tool_end|><|assistant_start|>Set `NUMBER` to `8
#3330
Fires on “ you”
story 0.07% of tokens

Hi!<|user_end|><|assistant_start|>Hi there! Would you like a
#3331
Fires on “ must”
agent 1.05% of tokens

not find oldString in text.py. It must match exactly
#3332
Fires on “,”
agent 1.28% of tokens

ob, grep, question, read, shell, skill
#3333
Fires on “:”
worksheet 1.30% of tokens

65+00: 5
#3334
Fires on “,”
story 0.33% of tokens

He said, "Stop
#3335
Fires on “<|assistant_start|>”
agent 0.42% of tokens

utils.py⏎test_greetings.py<|tool_end|><|assistant_start|><|tool_call_start|>s
#3336
Fires on “ model”
story 0.23% of tokens

morning.<|user_end|><|assistant_start|>Sorry, I'm a tiny model: I
#3337
Fires on “!”
chat 0.16% of tokens

<|user_start|>thanks<|user_end|><|assistant_start|>Happy to help!<|assistant_end|><|user_start|>
#3338
Fires on “ upon”
story 0.53% of tokens

together.<|bos|>Once upon a time
#3339
Fires on “ and”
story 0.64% of tokens

performance. She danced her way home, happy and content
#3340
Fires on “ have”
story 0.02% of tokens

't want to do what you want them to
#3341
Fires on “ They”
story 0.70% of tokens

8.<|assistant_end|><|bos|>Tom and Sam are friends. They like to
#3342
Fires on “py”
agent 0.88% of tokens

rue<|tool_call_end|><|assistant_end|><|tool_start|>Edited test_mathlib.py (2
#3343
Fires on “ there”
story 0.59% of tokens

this project.<|assistant_end|><|bos|><|user_start|>Hey<|user_end|><|assistant_start|>Hi there! W
#3344
Fires on “ops”
python 0.45% of tokens

⏎# test_number_utils
#3345
Fires on “ named”
agent 0.46% of tokens

, Kitty heard a sound. It was a dog named Doggy.
#3346
Fires on “<|tool_call_end|>”
agent 0.61% of tokens

<|user_end|><|assistant_start|><|tool_call_start|>read<|arg|>path=hello.py<|tool_call_end|><|assistant_end|><|tool_start|>
#3347
Fires on “ friend”
story 1.11% of tokens

just a toy I use
#3348
Fires on “ ”
agent 0.22% of tokens

For example, `raise_to(3, 1)`