Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2996
Fires on “_”agent 0.67% of tokensis_positive(
-
#2997
Fires on “ce”agent 0.11% of tokens. She said, "Puppy, accept this
-
#2998
Fires on “ of”worksheet 0.58% of tokenstoo small for the car. So, he thought of a plan
-
#2999
Fires on “ box”story 0.54% of tokensthis is a gift from Grandma!" She opened the box and inside
-
#3000
Fires on “ belong”story 1.11% of tokensnot ours," Ben said. "The apples belong to someone
-
#3001
Fires on “'t”story 0.88% of tokensto eat. Sam was very sad. He didn't know what
-
#3002
Fires on “ Lily”story 0.28% of tokensupon a time, there was a little girl named Lily. She
-
#3003
Fires on “ very”chat 0.10% of tokensto the garden.⏎The moral of this story is that when
-
#3004
Fires on “ a”story 0.22% of tokensgo there and
-
#3005
Fires on “ returns”python 1.18% of tokens⏎⏎Bug: `sum_list` should return the sum
-
#3007
Fires on “ day”story 0.53% of tokensmade everyone around her feel good.⏎One sunny day, Lily
-
#3008
Fires on “ a”story 2.04% of tokenswoods. He saw a big tree with a pattern
-
#3009
Fires on “ sub”agent 0.38% of tokens, question, read, shell, skill, subagent
-
#3010
Fires on “ for”story 1.11% of tokensto the house.⏎Bill thanked his neighbor for the inv
-
#3011
Fires on “ started”story 0.45% of tokensstarted to race
-
#3012
Fires on “ to”story 0.36% of tokenshe could not find his lost treasure. He started to feel very
-
#3013
Fires on “py”story 0.86% of tokenspy:⏎
-
#3014
Fires on “=”agent 0.45% of tokens1<|tool_end|><|assistant_start|><|tool_call_start|>read<|arg|>path=run.
-
#3015
Fires on “ big”story 0.80% of tokensbig smile.
-
#3016
Fires on “ did”story 0.41% of tokensfa.⏎But Anna is careless. She does not look
-
#3017
Fires on “l”agent 0.27% of tokensCan you run the tests?"<|user_end|><|assistant_start|><|tool_call_start|>glob<|arg|>
-
#3018
Fires on “ cubed”python 1.24% of tokens, b):⏎ return a * b⏎⏎⏎def sum_
-
#3019
Fires on “ closer”story 0.79% of tokensom when he heard a loud noise. He walked closer to the
-
#3020
Fires on “ friendly”story 0.58% of tokensup when something strange happened.⏎A big, friendly dog came
-
#3021
Fires on “ute”agent 0.81% of tokens, write, execute<|system_end|><|system_start|>
-
#3022
Fires on “⏎”python 0.11% of tokens, but it has `return s + s`.⏎⏎```
-
#3023
Fires on “1”agent 0.21% of tokens6+38: 6+8=14,
-
#3024
Fires on “ started”agent 0.36% of tokensthe toy anywhere. The little bird started to cry
-
#3025
Fires on “,”agent 0.20% of tokensanswer is 479.<|assistant_end|><|user_start|>Hey, what's
-
#3026
Fires on “ could”story 1.02% of tokens. Ollie and Timmy fell into the water. They could not swim
-
#3027
Fires on “ Then”story 0.40% of tokensat the sun, it's bright and fun. Then you put
-
#3028
Fires on “ all”story 0.49% of tokensand bigger. It was not a tiny fish after all. It
-
#3029
Fires on “,”story 1.24% of tokensplayed with her friends. They ran, jumped,
-
#3031
Fires on “ you”python 0.21% of tokens`strings.py`.<|assistant_end|><|bos|><|user_start|>See you later!
-
#3032
Fires on “<|user_end|>”agent 1.73% of tokensharness.<|system_end|><|user_start|>Where can I find odd?<|user_end|><|assistant_start|><|tool_call_start|>
-
#3033
Fires on “How”chat 0.28% of tokensstep by step or with a calculator.<|assistant_end|><|user_start|>How can you
-
#3034
Fires on “([”python 0.53% of tokenslargest number in nums. For example, `biggest([4,
-
#3035
Fires on “ +”python 0.57% of tokensoldString= return "Hello, " + name + "!"<|arg|>
-
#3036
Fires on “.”story 3.72% of tokensand didn't know what to do.⏎Do
-
#3037
Fires on “<|bos|>”chat 0.83% of tokenseven when things don't go as planned.<|bos|>Once upon
-
#3038
Fires on “ Tim”story 0.67% of tokenstail very fast. The big, scary bird saw Max's tail
-
#3039
Fires on “3”agent 0.51% of tokensa % b == 0⏎⏎$ python3 script.
-
#3040
Fires on “mean”agent 0.53% of tokensis 14.<|assistant_end|><|bos|>Bug: `mean` should
-
#3041
Fires on “ he”story 0.44% of tokenshe couldn't
-
#3042
Fires on “ it”python 0.16% of tokenscubed` should return the cube of x, but it has `
-
#3043
Fires on “("”python 0.66% of tokens"HELLO"⏎assert initial("python")
-
#3044
Fires on “)`”python 0.43% of tokensx. For example, `magnitude(-2)` returns `
-
#3045
Fires on “ again”story 0.74% of tokensto find me, my green hat was too clear against the
-
#3046
Fires on “ her”story 1.40% of tokens. He was going to have his favorite food for dinner. His
-
#3047
Fires on “ `”agent 0.66% of tokensit has `return len