Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#4050
Fires on “4”worksheet 0.64% of tokens8641 + 35620 =
-
#4051
Fires on “ decrement”agent 1.92% of tokensagent harness.<|system_end|><|user_start|>Rename the function decrement to decrement_v
-
#4052
Fires on “ an”story 0.18% of tokensHe felt sad.⏎Then, Pandy had an idea.
-
#4053
Fires on “ +”worksheet 3.51% of tokens= 0⏎ for i in range(n + 1
-
#4054
Fires on “ looking”story 1.34% of tokensalways remembered his
-
#4055
Fires on “ assistant”agent 0.56% of tokens<|system_start|>tools: calculator<|system_end|><|system_start|>You are a helpful assistant.<|system_end|>
-
#4056
Fires on “0”worksheet 0.63% of tokens1=8, 8031⏎0+7
-
#4057
Fires on “While”story 0.83% of tokensto the park with her mom to play.⏎While playing,
-
#4058
Fires on “ It”story 0.87% of tokensup, she saw something scary near the tree. It was a
-
#4059
Fires on “ min”agent 0.44% of tokensnums):⏎ return min(nums
-
#4060
Fires on “ called”story 0.66% of tokens.⏎All of sudden, her mom called her back inside
-
#4061
Fires on “1”story 0.87% of tokens1608: 8+8=16,
-
#4062
Fires on “ of”story 0.05% of tokensfriends in the yard. They were having a lot of fun.
-
#4063
Fires on “1”worksheet 0.46% of tokens0+97: 0+7+1=8
-
#4064
Fires on “ that”story 0.32% of tokensThe small leaf was sad. The small leaf learned that sometimes you
-
#4065
Fires on “,”story 0.74% of tokens<|bos|>One day, a little
-
#4066
Fires on “ are”chat 0.28% of tokensTry asking me for a story!<|assistant_end|><|user_start|>What are you good
-
#4067
Fires on “ are”story 0.62% of tokensand to anyone who would listen. She thought she was very smart
-
#4068
Fires on “What”chat 0.24% of tokensthe Python files in this project.<|assistant_end|><|bos|><|user_start|>What can you
-
#4069
Fires on “from”python 0.20% of tokens3⏎<|bos|># test_calc.py⏎from calc import
-
#4070
Fires on “ add”agent 0.80% of tokenskeyhole and twisted it. And, sure enough,
-
#4071
Fires on “,”agent 0.91% of tokens<|bos|><|system_start|>tools: edit, glob, grep
-
#4072
Fires on “ he”story 0.75% of tokenstired. Tom wanted to help the frog, so he took it
-
#4073
Fires on “ a”story 0.72% of tokensme a story about a dragon<|user_end|><|assistant_start|>Once upon a time,
-
#4074
Fires on “⏎”agent 0.21% of tokens⏎⏎$ python3 app.py⏎music⏎e⏎
-
#4075
Fires on “=”worksheet 0.37% of tokens2+1: 2+1+1=4,
-
#4076
Fires on “ being”story 0.60% of tokens⏎Lily and Ben learned that it's important to be careful and
-
#4077
Fires on “ about”python 0.20% of tokenss.<|user_end|><|assistant_start|>Sorry, I don't know about that.
-
#4078
Fires on “l”agent 0.10% of tokensFix the failing test.<|user_end|><|assistant_start|><|tool_call_start|>glob<|arg|>
-
#4079
Fires on “ defined”agent 1.16% of tokensdefined in `
-
#4080
Fires on “ Come”python 0.42% of tokensSure!" Max said. "Come on!"
-
#4081
Fires on “ She”story 2.40% of tokensa little girl named Sue went to the park. She saw a
-
#4082
Fires on “ They”story 1.44% of tokenssky and white clouds. They smiled and waved. They thought it
-
#4083
Fires on “⏎”python 0.37% of tokens(nums):⏎ return min(nums)⏎⏎$ python
-
#4084
Fires on “("”python 0.64% of tokenssize("hello") == 5⏎assert initial("banana")
-
#4085
Fires on “ loud”story 1.36% of tokensfast and makes a loud sound.
-
#4086
Fires on “ ”worksheet 4.66% of tokens01 more. Now Sue has 14113
-
#4087
Fires on “ But”story 0.32% of tokens. "And we can play more games together. But only if
-
#4088
Fires on “ closed”story 0.61% of tokensa rest. He lay down in the grass and closed his eyes
-
#4089
Fires on “f”agent 1.23% of tokensdon't want to, it's for your safety.
-
#4090
Fires on “ Mia”agent 0.71% of tokens.<|assistant_end|><|bos|>Tim has 86 shells. Tim gets
-
#4091
Fires on “ar”agent 0.98% of tokens, webfetch, websearch,
-
#4092
Fires on “Sam”story 0.79% of tokens59 = 18591⏎Tom has
-
#4093
Fires on “<|user_end|>”chat 0.85% of tokens<|user_end|><|assistant_start|><|think_start|>
-
#4094
Fires on “Created”agent 0.42% of tokens<|tool_start|>Created file successfully: lib.py<|tool_end|><|assistant_start|>Created `lib
-
#4095
Fires on “'s”worksheet 0.76% of tokens2 is 885.⏎What's 4