Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1093
Fires on “`:”agent 0.44% of tokens<|assistant_start|>Fixed `shout` in `utils.py`: it used
-
#1094
Fires on “ ”worksheet 1.22% of tokens2⏎assert sum_to(6) == 21
-
#1095
Fires on “ has”chat 0.54% of tokens<|assistant_end|><|bos|>Tom has 8
-
#1096
Fires on “newString”agent 0.48% of tokenstest_arith.py<|arg|>oldString=positive<|arg|>newString=is
-
#1097
Fires on “ behind”story 1.47% of tokens, but the dot followed it.
-
#1098
Fires on “e”agent 0.86% of tokens, write, execute
-
#1099
Fires on “ tests”python 3.84% of tokensPython files in
-
#1100
Fires on “ ”worksheet 1.29% of tokens109878.⏎Calculate 57
-
#1101
Fires on “ knew”story 0.31% of tokenssaw the beautiful universe together. They knew they would
-
#1102
Fires on “ at”story 0.24% of tokensit's better," he says.⏎Lily looks at the bowl
-
#1103
Fires on “ he”story 1.48% of tokensto talk to you anymore. I will find someone who likes to
-
#1104
Fires on “ toys”worksheet 0.11% of tokensmore. Now Anna has 1062 toys.⏎
-
#1105
Fires on “ o”story 0.79% of tokenswhich made his dad chuckle. He happily obl
-
#1106
Fires on “,”agent 0.50% of tokensLine 3: assert add_all([9, 6
-
#1107
Fires on “ and”story 3.04% of tokensBen reply.⏎They eat the cookies. They are tasty
-
#1108
Fires on “("”worksheet 0.55% of tokensjoin_words, welcome⏎⏎print(welcome("Sam"))
-
#1109
Fires on “ mean”agent 2.57% of tokensutils.py:⏎ Line 7: def raise_to
-
#1110
Fires on “ are”story 0.37% of tokenswhy are you
-
#1111
Fires on “J”story 1.25% of tokenscave were happy and thanked Joy.⏎Joy
-
#1113
Fires on “ was”story 0.38% of tokens, the surprise was not broken.<|bos|>Once there was a man
-
#1114
Fires on “`.”python 0.44% of tokens, but it has `return x not in items`.⏎⏎
-
#1115
Fires on “ night”story 0.41% of tokensthe person she was meant to help. That night she had
-
#1116
Fires on “ "”story 0.42% of tokensand took the white ball in its beak.⏎"Oh no
-
#1117
Fires on “8”worksheet 0.70% of tokens66534.⏎Anna has 815
-
#1118
Fires on “ it”story 0.41% of tokensase. It will be fun."⏎Lily stops crying.
-
#1119
Fires on “0”worksheet 0.94% of tokens06+160: 6+0+1
-
#1120
Fires on “<|tool_call_end|>”agent 0.97% of tokens_sum_list<|arg|>replaceAll=true<|tool_call_end|><|assistant_end|><|tool_start|>
-
#1121
Fires on “.”agent 0.56% of tokens": false}⏎⏎# test_textlib.py⏎
-
#1122
Fires on “ with”story 0.60% of tokensa smart idea.⏎The squirrel told Bob to use the shiny
-
#1123
Fires on “ you”chat 0.32% of tokens<|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Can you tell me
-
#1124
Fires on “ day”story 0.60% of tokensThey would run, jump, and laugh together all day long.
-
#1125
Fires on “#”chat 0.57% of tokenspy`.<|assistant_end|><|bos|># tools.
-
#1126
Fires on “7”worksheet 0.72% of tokens547=⏎
-
#1127
Fires on “⏎”python 0.53% of tokenspy⏎from tools import max_item, combine⏎⏎print
-
#1128
Fires on “ has”python 0.33% of tokensreturn the average of a and b, but it has `return
-
#1129
Fires on “<|system_end|>”chat 1.68% of tokens`.<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I
-
#1130
Fires on “ every”agent 1.24% of tokens⏎<|bos|><|system_start|>Start every answer with
-
#1131
Fires on “,”story 0.43% of tokens<|bos|>Once upon a time, in a small town, there was
-
#1132
Fires on “ with”agent 0.30% of tokens⏎Fix: replace `return min(nums)` with `return
-
#1133
Fires on “ to”agent 0.54% of tokensHe felt a little sorry. He did not mean to upset Lily
-
#1134
Fires on “ a”story 0.20% of tokensing very low and out of sight. After a few min
-
#1135
Fires on “ that”story 0.25% of tokenslittle girl named Amy. Amy had a toy rod that she loved
-
#1136
Fires on “⏎⏎”python 0.75% of tokensLily nodded. She was sleepy. She held Mom's hand.
-
#1137
Fires on “ ”worksheet 1.19% of tokensof 43 and 530 is 57
-
#1138
Fires on “ walked”story 0.80% of tokensto bake her treats.⏎She walked and walked until she
-
#1139
Fires on “ run”agent 0.26% of tokensoy.⏎Joy learned a valuable
-
#1140
Fires on “ is”chat 0.31% of tokensflowers does Tim have now?<|user_end|><|assistant_start|>The answer is 2
-
#1141
Fires on “.”agent 1.19% of tokens.py. It adds
-
#1142
Fires on “ to”story 0.19% of tokenscake on the kitchen
-
#1143
Fires on “Once”story 0.62% of tokens<|bos|>Once upon a