prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#2346
Fires on “+”
worksheet 0.60% of tokens

3+1818: 3+8+
#2347
Fires on “ in”
agent 0.48% of tokens

"aeio":` instead of `if c in "aeiou
#2348
Fires on “ add”
chat 0.18% of tokens

that. I only know how to tell stories and add numbers.
#2349
Fires on “ the”
story 0.33% of tokens

ving pieces of the fruit to everyone.⏎At the end of
#2350
Fires on “ dragon”
story 0.28% of tokens

```⏎<|bos|><|user_start|>Tell me a story about a dragon.<|user_end|>
#2351
Fires on “<|assistant_start|>”
agent 2.69% of tokens

n * 2 for n in nums]<|tool_end|><|assistant_start|>`smallest
#2352
Fires on “5”
worksheet 0.46% of tokens

83466 plus 4537 is
#2353
Fires on “ is”
agent 1.16% of tokens

lines 1-10⏎1: def is_odd
#2354
Fires on “,”
agent 1.01% of tokens

, question, read,
#2355
Fires on “One”
story 0.70% of tokens

had lots of fun and were very happy.⏎One day,
#2356
Fires on “`”
agent 0.72% of tokens

⏎ if x < 0:⏎
#2357
Fires on “After”
story 0.73% of tokens

the little boy didn't have an answer.⏎After trying for
#2358
Fires on “ add”
story 0.39% of tokens

was so peaceful. Sally found a rock to sit on and she
#2359
Fires on “ friend”
story 0.55% of tokens

. Kitty lived in a small house with her best friend, a
#2360
Fires on “ a”
story 0.88% of tokens

me a story about a dragon<|user_end|><|assistant_start|>Once upon a time,
#2361
Fires on “,”
chat 0.39% of tokens

the Eiffel Tower?<|user_end|><|assistant_start|>Sorry, I'm
#2362
Fires on “ name”
story 0.14% of tokens

!<|assistant_end|><|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude
#2363
Fires on “ tiny”
chat 0.25% of tokens

stand for?<|user_end|><|assistant_start|>Sorry, I'm a tiny model:
#2364
Fires on “ ”
agent 1.39% of tokens

1: def size(s):⏎2: return len
#2365
Fires on “Could”
chat 0.79% of tokens

is 64510.<|bos|><|user_start|>Could you tell
#2366
Fires on “ worry”
agent 0.51% of tokens

⏎After a while, the monkey began to feel tired. He
#2367
Fires on “ a”
story 0.28% of tokens

Once upon a time, there was a large cat
#2368
Fires on “ happy”
story 0.84% of tokens

aby brother went back to sleep, and everyone was happy.<|bos|>
#2369
Fires on “ a”
chat 0.46% of tokens

<|assistant_end|><|user_start|>Bye<|user_end|><|assistant_start|>Goodbye! Have a nice day
#2370
Fires on “ the”
story 0.37% of tokens

w of the fields, the trees, and the sky.
#2371
Fires on “ "”
story 0.06% of tokens

uacky asked his friend, Froggy, "Can we
#2372
Fires on “!”
story 0.32% of tokens

came over and said, "Wow, Tim! That is a
#2373
Fires on “print”
agent 0.42% of tokens

print(gret("Max"))<|arg|>newString=print(greet
#2374
Fires on “7”
chat 2.56% of tokens

567+249<|user_end|><|assistant_start|><|think_start|>567
#2375
Fires on “<|bos|>”
chat 0.74% of tokens

day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
#2376
Fires on “1”
agent 0.66% of tokens

return b⏎⏎⏎print(min_of(10,
#2377
Fires on “.”
story 1.14% of tokens

, the sky got dark and it started to rain. Sue began
#2378
Fires on “)`”
agent 0.97% of tokens

`-3`.⏎⏎`doubled(nums)` returns every
#2379
Fires on “⏎ ”
python 0.40% of tokens

.py<|arg|>content=def head(items):⏎ return items
#2380
Fires on “=”
agent 0.45% of tokens

<|assistant_start|><|think_start|>7+1: 7+1=8,
#2381
Fires on “ his”
python 0.29% of tokens

. He had a big, heavy toy box in his room.
#2382
Fires on “ little”
story 1.54% of tokens

kind. He wanted to help the bird. So
#2383
Fires on “ helpers”
story 0.23% of tokens

<|assistant_end|><|bos|># test_helpers.py⏎from helpers import first
#2384
Fires on “ each”
worksheet 0.73% of tokens

go back to the big space.⏎They saw each other and
#2385
Fires on “Hello”
agent 0.19% of tokens

python⏎def hello(name):⏎ return "Hello, "
#2386
Fires on “,”
worksheet 0.17% of tokens

52 = 261.⏎Hey, what's
#2387
Fires on “`”
python 0.57% of tokens

ent, we
#2388
Fires on “ careful”
story 0.27% of tokens

let you play with the boat. But be careful near the
#2389
Fires on “Once”
story 0.37% of tokens

and `test_stats.py`.<|assistant_end|><|bos|>Once upon a
#2390
Fires on “utils”
agent 0.54% of tokens

add numbers.<|assistant_end|><|bos|># test_number_utils.py
#2391
Fires on “=”
chat 0.91% of tokens

0+600: 0+0=0,
#2392
Fires on “ ”
story 0.48% of tokens

3]))⏎⏎$ python3 main.py⏎1⏎
#2393
Fires on “,”
story 1.82% of tokens

⏎Max's friend, a little girl named Lily, saw him
#2394
Fires on “=”
worksheet 0.70% of tokens

0+600: 0+0=0,
#2395
Fires on “⏎”
story 1.01% of tokens

think and help his friends when they were lost.⏎One day