Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#3449
Fires on “ next”agent 0.78% of tokens, lines 1-2⏎1: def next_number
-
#3450
Fires on “ apples”worksheet 1.02% of tokens561.⏎Tim had 94 stars and found
-
#3451
Fires on “ ”agent 1.49% of tokens8: 8+8=16, 16
-
#3452
Fires on “4”agent 0.36% of tokensfile hello.py, lines 1-14⏎1
-
#3453
Fires on “ in”story 0.74% of tokenswas a fun race every year. All the people in the village
-
#3454
Fires on “ a”agent 0.11% of tokensoutside. One day, she decided to go for a walk in
-
#3455
Fires on “<|tool_call_start|>”agent 1.56% of tokensIt works! and run it.<|user_end|><|assistant_start|><|tool_call_start|>write
-
#3457
Fires on “ all”story 0.48% of tokensadventures she could have with her toys. And they all lived happily
-
#3458
Fires on “ soon”story 0.54% of tokensfirst because the refrigerator seemed so big. But as soon as she
-
#3459
Fires on “ rain”story 1.15% of tokensbirds were singing.⏎Suddenly, the sky turned dark and the
-
#3460
Fires on “ in”story 0.46% of tokensbeans. Suddenly, an angry man walked in and yelled
-
#3461
Fires on “ day”story 1.12% of tokensTim ate his ice cream. It was a fun day.<|bos|>
-
#3462
Fires on “ +”agent 0.66% of tokenswords(a, b):⏎ return b + " "
-
#3463
Fires on “1”worksheet 0.56% of tokens8082⏎So 6245 +
-
#3464
Fires on “8”python 0.85% of tokens11⏎assert absolute(8) == 8⏎assert
-
#3465
Fires on “3”worksheet 2.25% of tokens70, you get 15423.⏎
-
#3466
Fires on “What”agent 0.26% of tokens40 = 8747.⏎What's
-
#3467
Fires on “ch”agent 0.33% of tokensill, subagent, webfetch, we
-
#3468
Fires on “ ”worksheet 0.74% of tokens7: 8+7=15, 15
-
#3469
Fires on “ eat”story 0.88% of tokenswent outside. Tim was happy to eat outside and watch the jet
-
#3470
Fires on “ I”story 0.95% of tokens"That's very kind of you, Anna. I'm sure
-
#3471
Fires on “,”worksheet 1.57% of tokensnums)⏎```⏎⏎>>> biggest([7, 9
-
#3472
Fires on “ want”story 0.32% of tokensdidn't want her daddy
-
#3473
Fires on “ it”python 0.59% of tokens, we can share the shell, Ben. But you have to
-
#3474
Fires on “he”worksheet 0.50% of tokensOpenCode, a coding agent harness.<|system_end|><|user_start|>Check the
-
#3475
Fires on “ then”story 0.38% of tokens. They were having a lot of fun.⏎Suddenly, a
-
#3476
Fires on “ ”worksheet 1.72% of tokens: 0+0+1=1, 10
-
#3477
Fires on “9”chat 0.59% of tokens99 + 4?<|user_end|><|assistant_start|><|think_start|>99+
-
#3478
Fires on “ because”story 0.20% of tokensexcite?" Will said, "I am excite because I like
-
#3479
Fires on “.”agent 0.73% of tokenscoding agent harness.<|system_end|><|user_start|>"test_words.py is
-
#3480
Fires on “+”worksheet 1.27% of tokens09 plus 5?⏎409+00
-
#3481
Fires on “ there”story 0.25% of tokensbe fun too.<|bos|>Once upon a time, there was a
-
#3482
Fires on “ many”chat 0.10% of tokenscoins. Lily gets 8 more coins. How many coins does
-
#3483
Fires on “ she”story 0.17% of tokensshe held her
-
#3484
Fires on “Could”chat 1.33% of tokenstell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Could you tell
-
#3485
Fires on “,”python 0.23% of tokenscombine` should join a and b with a space, but it
-
#3486
Fires on “ have”story 0.08% of tokensthere lived a boy named Tom, who wanted to have nice toys
-
#3487
Fires on “re”agent 0.18% of tokensmean to my_mean.<|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>
-
#3488
Fires on “ when”agent 0.67% of tokens. When she was
-
#3489
Fires on “(”python 0.60% of tokens're welcome!<|assistant_end|><|bos|>```python⏎def tail(items):
-
#3490
Fires on “3”worksheet 0.60% of tokens3+0: 3+0=3,
-
#3491
Fires on “ good”agent 0.18% of tokensthe wall. They were both very happy and became good friends.
-
#3492
Fires on “print”agent 0.14% of tokens<|arg|>path=greeting.py<|arg|>content=print("Hello
-
#3493
Fires on “ their”story 0.45% of tokensnew friends played games, sang songs, and clapped their hands.
-
#3494
Fires on “0”worksheet 1.57% of tokens7⏎000+600:
-
#3495
Fires on “0”python 0.30% of tokens_value(x):⏎ if x < 0:⏎
-
#3496
Fires on “ =”agent 0.77% of tokens= 1
-
#3497
Fires on “Look”story 0.12% of tokensaround in the air.⏎"Look, Amy
-
#3498
Fires on “ mom”story 0.39% of tokensnamed Lily went to the airport with her mom and dad
-
#3499
Fires on “.”agent 0.72% of tokens4 balls and found 7145 more. Now Tom