Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2296
Fires on “'s”story 0.54% of tokens<|system_end|><|user_start|>Hey, what's 8
-
#2297
Fires on “)”agent 0.53% of tokenss, but it has `return len(s) +
-
#2298
Fires on “1”agent 0.25% of tokens5+08: 5+8=13,
-
#2299
Fires on “ to”agent 0.19% of tokens, a coding agent harness.<|system_end|><|user_start|>Rename head to first everywhere
-
#2300
Fires on “=”agent 0.51% of tokenspy please<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3
-
#2301
Fires on “ you”chat 0.38% of tokens<|user_start|>What are you good at
-
#2302
Fires on “ (”agent 0.71% of tokens"apple"<|tool_call_end|><|assistant_end|><|tool_start|>Edited main.py (1 replacement
-
#2303
Fires on “1”worksheet 0.43% of tokens5+5: 5+5=10,
-
#2304
Fires on “ very”story 0.57% of tokensplay hide and seek with Max, because he is very good at
-
#2305
Fires on “What”chat 2.00% of tokens= 39589.<|bos|><|user_start|>What is the
-
#2306
Fires on “ defines”agent 0.97% of tokens" + b<|tool_end|><|assistant_start|>`greetings.py` defines 2
-
#2307
Fires on “ lived”agent 0.16% of tokens, there was a little green bug. The bug lived in a
-
#2308
Fires on “ it”story 1.26% of tokensball dropped to the ground, and the dog picked it up with
-
#2309
Fires on “The”chat 0.30% of tokens1=10, 103<|think_end|>The answer is
-
#2310
Fires on “A”python 0.74% of tokens# README.md⏎# Tiny utils⏎⏎A small Python
-
#2311
Fires on “8”worksheet 0.56% of tokensWhat is 9852 + 3806
-
#2312
Fires on “+”worksheet 0.27% of tokens04+62: 4+2+1=
-
#2313
Fires on “ I”chat 0.31% of tokenssaid, "We are sorry, big bird. We will leave
-
#2314
Fires on “Once”story 0.95% of tokens<|bos|>Once upon a
-
#2315
Fires on “⏎”python 0.83% of tokens, 4, 3, 3])⏎3⏎
-
#2316
Fires on “ squared”agent 0.83% of tokens<|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>pattern=def doubled<|tool_call_end|><|assistant_end|>
-
#2317
Fires on “ tests”agent 0.28% of tokens$ python3 test_tools.py⏎all tests passed⏎
-
#2318
Fires on “ sh”agent 0.15% of tokenstled back to the ground.⏎The sun shone bright
-
#2319
Fires on “assert”python 0.26% of tokensimport smaller, sum_two, halve⏎⏎assert smaller(
-
#2320
Fires on “")”python 0.45% of tokensLO"⏎assert first_letter("robot") == "
-
#2321
Fires on “ fly”story 1.06% of tokenswing got better. One day, the bird could fly again.
-
#2322
Fires on “ `”agent 0.55% of tokenshello` in `greetings.
-
#2323
Fires on “:”agent 0.14% of tokenspy<|tool_end|><|assistant_start|>This project has 3 Python files: `main
-
#2324
Fires on “ and”story 0.24% of tokensflower on a tall stem. It was red and yellow and
-
#2325
Fires on “ was”story 2.08% of tokenstree was toppl
-
#2326
Fires on “ behind”story 0.94% of tokenshe saw his friend, Alex, hiding behind the couch
-
#2327
Fires on “<|assistant_start|>”agent 1.27% of tokens<|assistant_start|>The answer
-
#2328
Fires on “ day”story 0.20% of tokensplayed together, and from that day on,
-
#2329
Fires on “s”agent 0.78% of tokensdef larger(a, b):⏎⏎
-
#2330
Fires on “ Now”worksheet 0.09% of tokens8 stars and found 664 more. Now Tim has
-
#2331
Fires on “ c”python 0.16% of tokensc in s:⏎ if c in "
-
#2332
Fires on “nums”agent 1.19% of tokens(x)` and `max_of(a, b)`.
-
#2333
Fires on “ and”agent 1.02% of tokensworked hard and fast.⏎The rain stopped, and the water
-
#2334
Fires on “ill”agent 0.43% of tokensread, shell, skill, sub
-
#2335
Fires on “ and”story 0.80% of tokens⏎Poppy thought for a moment and had an
-
#2336
Fires on “,”worksheet 0.85% of tokens: 4+6+1=11, 1
-
#2337
Fires on “⏎ ”python 0.68% of tokens<|arg|>content=def evens(nums):⏎ return [
-
#2338
Fires on “:”worksheet 2.67% of tokens0055+69667: 5
-
#2339
Fires on “ there”story 0.22% of tokensa big lake. Near the lake, there lived a happy
-
#2340
Fires on “ got”story 0.28% of tokenslaughed and screamed as they got wet.
-
#2341
Fires on “ home”story 4.45% of tokens, you are my new friend. Let's go home together!"
-
#2342
Fires on “ def”agent 0.50% of tokens3: ⏎4: ⏎5: def initial(
-
#2343
Fires on “("”python 0.78% of tokenss.lower()⏎```⏎⏎>>> whisper("PY
-
#2344
Fires on “ the”worksheet 0.69% of tokensby step or with a calculator.<|assistant_end|><|bos|>Find the sum of
-
#2345
Fires on “ me”worksheet 0.23% of tokenshome to get warm.<|bos|><|user_start|>Could you tell me what