Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#442
Fires on “ said”story 4.15% of tokensbathe in the sun too.⏎Red said, "
-
#443
Fires on “ couldn”story 1.24% of tokensand his mom were done, he couldn't believe
-
#444
Fires on “ didn”story 0.60% of tokenstree and got hurt. They were very sad and didn't play
-
#445
Fires on “ test”agent 0.49% of tokens910<|think_end|>910<|assistant_end|><|bos|># test_textlib
-
#446
Fires on “ at”story 0.59% of tokenstable. They looked at the pl
-
#447
Fires on “print”agent 0.41% of tokenspy⏎from math_utils import product⏎⏎print(product
-
#448
Fires on “ watched”story 0.43% of tokens, he sat on a bench and watched the cake
-
#449
Fires on “6”python 1.36% of tokens666666666665⏎
-
#450
Fires on “What”story 0.38% of tokensbroken colors. She said, "Oh no, what happened?"
-
#451
Fires on “,”agent 0.39% of tokens, web
-
#452
Fires on “ tree”story 0.54% of tokensunder a big tree and shared
-
#453
Fires on “ play”story 0.75% of tokensMia liked to play with tr
-
#454
Fires on “ play”story 0.51% of tokensIt was sad because it had no friends to play with.
-
#455
Fires on “<|assistant_end|>”chat 0.55% of tokensonly know how to tell stories and add numbers.<|assistant_end|><|user_start|>S
-
#456
Fires on “6”python 0.71% of tokens⏎assert sum_to(2) == 3⏎assert
-
#457
Fires on “8”worksheet 2.65% of tokenshad 5 toys and found 41886 more
-
#458
Fires on “ of”story 1.32% of tokensended up in a forest. She got out of the car
-
#459
Fires on “,"”story 0.43% of tokenssaw a tiny bug on a leaf. "Hi," said Spot
-
#460
Fires on “ said”story 1.12% of tokenssaid they needed
-
#461
Fires on “ the”story 0.58% of tokensSTAR⏎banana⏎<|tool_end|><|assistant_start|>Fixed the typo in
-
#462
Fires on “Bug”python 0.45% of tokens⏎ROBOT⏎```<|assistant_end|><|bos|>Bug: `
-
#463
Fires on “ asked”story 0.39% of tokensthought of a plan. He asked all his
-
#464
Fires on “'re”chat 1.48% of tokens`.⏎<|bos|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
-
#465
Fires on “ dream”agent 1.24% of tokensand Tim hugged each other and went to sleep, dreaming of
-
#466
Fires on “ big”story 0.45% of tokensliked to fish with their dad. They had a big rod and
-
#467
Fires on “ made”story 1.29% of tokensnot going to let you show mummy what you made".⏎
-
#468
Fires on “<|tool_call_start|>”agent 0.44% of tokens<|tool_call_start|>read<|arg|>
-
#469
Fires on “ ”worksheet 0.51% of tokenswent away.<|bos|>Calculate 200 + 2.
-
#470
Fires on “One”story 0.25% of tokensLucy for making it look so pretty.<|bos|>One day,
-
#471
Fires on “.”chat 1.27% of tokens10.⏎Max has 6 toys. Max gets
-
#472
Fires on “ he”story 1.10% of tokenscame to the wise owl. He was sad because he could not
-
#473
Fires on “ AI”agent 0.59% of tokensto sleep. She took her baby doll everywhere she went.
-
#474
Fires on “ and”chat 0.53% of tokensyou help me?<|user_end|><|assistant_start|>I'm prelude, and I know
-
#475
Fires on “ if”python 1.66% of tokens⏎ if x <
-
#476
Fires on “ She”story 3.39% of tokensthe tape to stick colorful papers together. She made a
-
#477
Fires on “ double”python 0.99% of tokens):⏎ return items[-1]⏎⏎⏎def double_all
-
#478
Fires on “=”worksheet 0.54% of tokens704: 7+4+1=12
-
#479
Fires on “ each”agent 0.40% of tokensthey hid their toys and snacks. They trusted each other and
-
#480
Fires on “ if”story 0.79% of tokenseven more fun
-
#481
Fires on “ unexpected”story 0.70% of tokensthe hole with sand.⏎But then, something unexpected happened.
-
#482
Fires on “6”worksheet 0.89% of tokens6.⏎Can you add 72366 and
-
#483
Fires on “ tail”agent 1.23% of tokensand buy a new red ball!" Max wagged his tail and barked
-
#484
Fires on “,”story 0.55% of tokenswith the fixed toy all day.⏎But then, something unexpected
-
#485
Fires on “ are”agent 0.34% of tokens73865.<|bos|><|system_start|>You are mini,
-
#486
Fires on “<|tool_end|>”agent 0.35% of tokens<|assistant_end|><|tool_start|>Edited main.py (2 replacements)<|tool_end|><|assistant_start|><|tool_call_start|>
-
#487
Fires on “ number”agent 0.57% of tokenstail(items)`.<|assistant_end|><|bos|><|system_start|>Answer with the number only.
-
#488
Fires on “ ”worksheet 0.55% of tokens: 1+2+1=4, 44
-
#489
Fires on “'s”chat 0.69% of tokens<|system_end|><|user_start|>Hey, what's 8
-
#490
Fires on “ can”agent 1.38% of tokenscan you help me
-
#491
Fires on “⏎”worksheet 3.63% of tokens+8=14, 4928⏎5