Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1395
Fires on “<|tool_call_end|>”agent 0.35% of tokens<|tool_call_end|><|assistant_end|><|tool_start|>
-
#1396
Fires on “ line”agent 0.28% of tokenscall last):⏎ File "app.py", line 4
-
#1397
Fires on “ help”story 0.31% of tokensHe was a compassionate helper. He liked to help people.
-
#1398
Fires on “]))”agent 1.34% of tokens0, 0]))⏎ ^^^^^^^^
-
#1399
Fires on “`”story 0.19% of tokens⏎⏎Fix: replace `return x * x` with `
-
#1400
Fires on “ about”story 0.62% of tokensbox.⏎Luna went to the box and tried to open
-
#1401
Fires on “ surprised”story 1.99% of tokensnew skin!" Lily said. Sue was very surprised but happy
-
#1402
Fires on “ ”python 0.37% of tokenslist([5, 7, 4, 3])
-
#1403
Fires on “3”worksheet 0.39% of tokens9: 8+9
-
#1404
Fires on “⏎”python 0.35% of tokensonly.<|system_end|>
-
#1405
Fires on “g”agent 0.48% of tokenslike to build towers and houses and bridges with
-
#1406
Fires on “ a”story 0.13% of tokens!<|user_end|><|assistant_start|>One day, a big cat and a little dog
-
#1407
Fires on “ have”chat 0.11% of tokens6 more stickers. How many stickers does Tim have now?
-
#1408
Fires on “ ”worksheet 2.26% of tokens+ 62486 = 14
-
#1409
Fires on “<|assistant_start|>”agent 0.74% of tokens<|system_end|><|user_start|>What files are in this project?<|user_end|><|assistant_start|><|tool_call_start|>g
-
#1410
Fires on “ try”story 0.04% of tokenstry to find
-
#1411
Fires on “ to”story 0.31% of tokensThen they take the blocks and go back to the living
-
#1412
Fires on “.”story 2.38% of tokens⏎In the town, there was a big race. All the
-
#1413
Fires on “=”worksheet 0.70% of tokenstreasure!"⏎They put on their pirate costumes
-
#1414
Fires on “ a”story 0.76% of tokensOne day, a cat named Jazz went for a walk.
-
#1415
Fires on “ only”chat 0.68% of tokensSorry, I'm a tiny model: I can only tell short
-
#1416
Fires on “ we”story 0.66% of tokens. I was mean. Can we be friends
-
#1417
Fires on “ n”agent 0.74% of tokens9: return [n + 2 for n in nums
-
#1418
Fires on “from”agent 0.28% of tokens_utils.py (1 replacement)<|tool_end|><|assistant_start|>Added `absolute
-
#1419
Fires on “ short”agent 0.30% of tokens<|user_end|><|assistant_start|>I'm prelude: I can tell you short stories for
-
#1420
Fires on “ up”story 0.28% of tokenscoat on.⏎The next morning, Mia woke up and her
-
#1421
Fires on “est”agent 0.06% of tokens.py`.<|assistant_end|><|bos|><|user_start|>What does DNA stand for?
-
#1422
Fires on “+”worksheet 0.28% of tokens+0006: 4+6+1=
-
#1423
Fires on “ sitting”story 0.73% of tokenswent to the pond. She saw a big frog sitting on a
-
#1424
Fires on “!”chat 0.27% of tokensreat, thanks!<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|bos|>
-
#1425
Fires on “ with”story 1.61% of tokenswhen the sun went down, they all went home with happy m
-
#1426
Fires on “ happened”story 0.13% of tokenswanted to be rich.⏎Then, something unexpected happened. The
-
#1427
Fires on “=”worksheet 0.61% of tokens799: 3+9+1=13
-
#1428
Fires on “ I”chat 0.51% of tokensWorld Cup?<|user_end|><|assistant_start|>Sorry, I can't
-
#1429
Fires on “ help”story 1.11% of tokens, "Don't worry, Mia. I will help you fix
-
#1430
Fires on “ and”chat 0.08% of tokens5.<|bos|><|system_start|>You are mini, a small and friendly assistant
-
#1431
Fires on “ mom”story 0.21% of tokens. Bop hopped away, excited to tell his family about his
-
#1432
Fires on “(”agent 0.52% of tokens)<|tool_end|><|assistant_start|>Added `total(nums)`
-
#1433
Fires on “ they”story 2.01% of tokensyour crown."⏎They walked and walked until they found a
-
#1434
Fires on “Answer”chat 0.21% of tokens⏎<|bos|><|system_start|>Answer with the
-
#1435
Fires on “ Tom”story 0.11% of tokenswe can find something fun."⏎"OK, Tom!" Lily
-
#1436
Fires on “ find”story 1.46% of tokensfind out.
-
#1437
Fires on “ ”worksheet 0.44% of tokens=6, 6193⏎So 57
-
#1438
Fires on “ to”chat 0.28% of tokens<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>I need to know
-
#1439
Fires on “ upon”story 0.53% of tokens.<|assistant_end|><|bos|>Once upon a time
-
#1440
Fires on “ ”agent 0.95% of tokenscore import minus⏎⏎assert minus(8, 12
-
#1441
Fires on “.”story 1.59% of tokenslight, wet and dry, and loud and soft. They also
-
#1442
Fires on “2”agent 4.69% of tokensscript.py<|tool_call_end|><|assistant_end|><|tool_start|>True⏎9⏎28⏎
-
#1443
Fires on “ for”agent 0.69% of tokensthat they were kind and gentle. He felt sorry for Anna and
-
#1444
Fires on “+”worksheet 0.51% of tokensit.⏎One day, the bird met a kind friend.