Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#3500
Fires on “Fix”agent 0.40% of tokens⏎ return s.lower()⏎```⏎⏎Fix: replace
-
#3501
Fires on “b”agent 1.23% of tokensshell, skill, subagent, webfet
-
#3502
Fires on “,”agent 1.66% of tokens. I can list, read, run, create, edit and
-
#3503
Fires on “ him”story 1.93% of tokensThey walked to Mr. Brown's house and gave him the letter
-
#3504
Fires on “ call”agent 0.22% of tokenscalc.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call last):
-
#3505
Fires on “NUMBERS”agent 0.48% of tokensreturn x % 2 == 1⏎⏎⏎NUMBER =
-
#3506
Fires on “.”python 0.11% of tokens_vowels` returns the number of vowels in s. For example
-
#3507
Fires on “ and”story 1.39% of tokensto dance and sing!
-
#3508
Fires on “ worked”story 0.28% of tokens. From then on, he understood that if everyone worked together and
-
#3509
Fires on “NameError”agent 0.39% of tokens_veen(NUMBERS))⏎ ^^^^^^^^^⏎NameError: name
-
#3510
Fires on “1”worksheet 0.16% of tokens+6: 5+6+1=12,
-
#3511
Fires on “y”agent 0.66% of tokens<|bos|>Once upon a time, there was a loyal dog
-
#3512
Fires on “ gets”agent 0.66% of tokens⏎<|bos|><|user_start|>Sue has 3 cookies. Sue gets 4
-
#3513
Fires on “ a”chat 0.74% of tokens<|assistant_end|><|bos|><|user_start|>Can you tell me a story about a dragon?
-
#3514
Fires on “ lots”agent 0.75% of tokenswent outside. The park was very big and had lots of fun
-
#3515
Fires on “h”agent 0.98% of tokens's play a new game. It is called 'bathe
-
#3516
Fires on “nums”agent 0.60% of tokenspass now.<|assistant_end|><|bos|>`double_all(nums)` double
-
#3517
Fires on “1”agent 0.67% of tokens5, 8, 8])⏎[14,
-
#3518
Fires on “ time”story 0.42% of tokensold lady was happy.<|bos|>Once upon a time, there
-
#3519
Fires on “ was”story 0.65% of tokensbox, and now we can play together!" Tim was very surprised
-
#3520
Fires on “)`”python 0.63% of tokens.<|assistant_end|><|bos|>`keep_even(nums)` keeps only
-
#3521
Fires on “ next”agent 0.44% of tokensblocks too, but Ben had taken them all. He
-
#3522
Fires on “cat”story 1.80% of tokensfrom words import loud⏎⏎TEXT = "robot"⏎
-
#3523
Fires on “5”chat 1.22% of tokens6.<|bos|><|user_start|>What's 5 + 500
-
#3524
Fires on “ beach”story 1.21% of tokensdoubled.<|assistant_end|><|bos|>Tom and Lily went to the beach with their
-
#3525
Fires on “ named”story 0.15% of tokens<|assistant_start|>Once upon a time, there was a girl named Lily.
-
#3526
Fires on “ function”agent 0.82% of tokens) == "bananabanana"⏎⏎⏎words.py
-
#3527
Fires on “ "”story 1.11% of tokensWhy are you sad, Sue?" Sue said, "I lost
-
#3528
Fires on “ she”story 0.19% of tokens!"⏎Mommy looked up. "Oh my," she said.
-
#3529
Fires on “it”agent 0.73% of tokens: return a * b<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path
-
#3530
Fires on “:”agent 0.93% of tokenspy, lines 1-10⏎1: def combine
-
#3531
Fires on “'t”worksheet 0.59% of tokensTim. His mom smiled and said, "Don't worry,
-
#3532
Fires on “ best”story 0.23% of tokenssurprised. He played with the dog and they became best friends.
-
#3533
Fires on “ get”story 3.07% of tokensnot know what was inside. They all wanted to open it.
-
#3534
Fires on “ lived”story 0.18% of tokenswas a big farm. On this farm, there lived a graceful
-
#3535
Fires on “One”story 0.08% of tokens82347 apples.<|bos|>One day,
-
#3536
Fires on “ to”story 0.33% of tokenswill be mean to you!" Tim did not want to surre
-
#3537
Fires on “ wanted”story 1.04% of tokensa vine hanging from a tree. She wanted to reach
-
#3538
Fires on “ about”agent 0.40% of tokensme.<|user_end|><|assistant_start|>Sorry, I don't know about that.
-
#3539
Fires on “ away”story 0.42% of tokensunexpected. A strong wind blew Tim's cape away, and
-
#3540
Fires on “ed”agent 0.91% of tokens: return x > 0<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>
-
#3541
Fires on “,”story 0.90% of tokensdad talk to each other. From that day on, Tim loved
-
#3542
Fires on “.”agent 1.07% of tokens<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3 hello.
-
#3543
Fires on “ with”agent 0.47% of tokensis 1534.<|bos|><|system_start|>Answer with the number
-
#3544
Fires on “ waited”agent 0.83% of tokensBut Ben is curious and sneaky. He waits until
-
#3545
Fires on “,”agent 0.41% of tokensched!<|bos|><|system_start|>tools: edit, glob, grep
-
#3546
Fires on “ You”agent 0.47% of tokenswas amazed and said, "Wow, bunny! You really did
-
#3547
Fires on “'s”story 0.26% of tokensdoor and he was in! Max couldn't believe it! He
-
#3548
Fires on “def”python 0.54% of tokens(6)⏎True⏎⏎```python⏎def factorial(
-
#3549
Fires on “ her”agent 0.87% of tokensit with dirt. He thinks the mole will be his friend.