prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#1245
Fires on “ a”
story 0.43% of tokens

<|user_start|>Can you tell me a story about a car?
#1246
Fires on “ tests”
story 0.99% of tokens

's mom came and saw the fight. She told Max
#1247
Fires on “ stickers”
story 0.35% of tokens

has 4 stickers. Ben gets 5 more stickers. How
#1248
Fires on “main”
agent 2.72% of tokens

2]))<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path=script.py
#1249
Fires on “ v”
agent 0.56% of tokens

, laughing and having fun, knowing they learned a valu
#1250
Fires on “ gl”
agent 2.14% of tokens

very happy.<|assistant_end|><|bos|><|system_start|>tools: edit, glob,
#1251
Fires on “)`”
worksheet 0.78% of tokens

x. For example, `opposite(11)` returns `
#1252
Fires on “ good”
story 1.12% of tokens

fun. The pink bunny and the little girl became good friends.
#1253
Fires on “⏎”
story 0.45% of tokens

.py⏎from arith import half, negate⏎⏎print(
#1254
Fires on “ Lily”
story 0.42% of tokens

Max ran to his friend, a little girl named Lily, to
#1255
Fires on “ on”
story 0.13% of tokens

they had another great day together. From that day on, Tom
#1256
Fires on “ eat”
story 1.12% of tokens

why don't you come inside my house and have some hot chocolate
#1257
Fires on “ to”
worksheet 0.10% of tokens

68074.⏎I need to know
#1258
Fires on “ in”
agent 0.56% of tokens

return [n * 2 for n in nums]
#1259
Fires on “ in”
story 0.23% of tokens

At the park
#1260
Fires on “ put”
story 0.50% of tokens

and put it back
#1261
Fires on “ you”
agent 0.51% of tokens

a calendar, and they said thank you. They
#1262
Fires on “ give”
story 0.50% of tokens

but he fell once more. He did not give up,
#1263
Fires on “,”
story 0.36% of tokens

<|assistant_end|><|bos|>Once upon a time, there was
#1264
Fires on “ on”
story 0.47% of tokens

together with the yarn. The moral of the story is that
#1265
Fires on “'s”
story 0.41% of tokens

⏎In the end, Amy learned that sometimes it's important to
#1266
Fires on “ very”
story 0.35% of tokens

Lily loved her pink bed very much.
#1267
Fires on “ replace”
python 0.16% of tokens

" " + a⏎```⏎⏎Fix: replace `return
#1268
Fires on “0”
worksheet 0.52% of tokens

, 9⏎0082
#1269
Fires on “ from”
worksheet 0.77% of tokens

⏎As they searched, a little squirrel came out from a tree
#1270
Fires on “ doll”
agent 0.54% of tokens

her baby doll with her. She put her baby doll on a
#1271
Fires on “`”
python 0.70% of tokens

<|bos|>Bug: `max_
#1272
Fires on “ would”
story 0.33% of tokens

and wanted to slide down it. Tim thought it would be fun
#1273
Fires on “ Mia”
chat 0.31% of tokens

.<|assistant_end|><|bos|>Tim has 86 shells. Tim gets
#1274
Fires on “ years”
story 0.40% of tokens

boy called David. He was only three years old,
#1275
Fires on “Jack”
story 4.80% of tokens

thanked Jack's parents for bringing him.⏎Jack felt proud
#1276
Fires on “ girl”
story 1.36% of tokens

<|bos|>Once upon a time, there was a little girl who lived
#1277
Fires on “:”
python 0.12% of tokens

<|bos|>Bug: `max
#1278
Fires on “ time”
agent 0.89% of tokens

lots of fun. After a while, it was time to go
#1279
Fires on “⏎”
agent 0.87% of tokens

⏎>>> sum_up_to(6)⏎21
#1280
Fires on “ her”
story 0.69% of tokens

They have some bread in their bag.
#1281
Fires on “ ”
agent 3.37% of tokens

all day. The other dog saw this and felt envious
#1282
Fires on “("”
story 0.52% of tokens

tiny language model trained from scratch on a
#1283
Fires on “ (”
agent 0.39% of tokens

true<|tool_call_end|><|assistant_end|><|tool_start|>Edited items.py (1 replacement
#1284
Fires on “ +”
chat 1.46% of tokens

= 295.⏎Calculate 6 + 8
#1285
Fires on “ that”
agent 0.28% of tokens

, and the squirrel felt better. They both learned that when you
#1286
Fires on “0”
agent 0.49% of tokens

the trees. He was jumping and swinging from branch to branch
#1287
Fires on “ of”
story 0.44% of tokens

, Max. Maybe it will come back someday."
#1288
Fires on “1”
worksheet 1.39% of tokens

0: 2+0=2, 2⏎0
#1289
Fires on “=”
agent 0.41% of tokens

<|assistant_start|><|think_start|>7+2: 7+2=9,
#1290
Fires on “ He”
story 1.10% of tokens

He knew he had been mean to his sister. He tried to
#1291
Fires on “1”
worksheet 2.18% of tokens

What is 1037 plus 280?
#1292
Fires on “<|bos|>”
chat 0.69% of tokens

played more games at the camp.<|assistant_end|><|bos|><|user_start|>Sue
#1293
Fires on “ to”
python 0.21% of tokens

played until the sun went down and it was time to go home
#1294
Fires on “ me”
chat 0.42% of tokens

she set about searching for the person she was meant to