Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#392
Fires on “ test”story 0.45% of tokens6, 14]⏎$ python3 test_helpers
-
#393
Fires on “=”worksheet 0.89% of tokens+76: 7+6+1=14
-
#394
Fires on “ the”story 2.17% of tokensnod at the car.
-
#395
Fires on “.”story 0.75% of tokens66.⏎If
-
#396
Fires on “(”python 1.29% of tokenscalc import positive, product⏎⏎print(product(7,
-
#397
Fires on “it”agent 0.21% of tokensthe flowers, but the toy was not there either.
-
#398
Fires on “:”agent 1.33% of tokens2: return s[::-1]⏎3: ⏎
-
#399
Fires on “ is”chat 0.43% of tokenscookies does Sue have now?<|user_end|><|assistant_start|>The answer is 7
-
#400
Fires on “ to”chat 0.31% of tokensa small and friendly assistant.<|system_end|><|user_start|>I need to know
-
#401
Fires on “ new”story 0.48% of tokensfor a walk. Kitty loved to explore and see new things.
-
#402
Fires on “ files”agent 0.31% of tokens, run, create, edit and fix the Python files in this
-
#403
Fires on “:”agent 0.29% of tokensGoodbye!<|user_end|><|assistant_start|>Sure! B
-
#404
Fires on “ar”agent 0.72% of tokensbsearch,
-
#405
Fires on “,”python 1.18% of tokens<|bos|>One day, a little
-
#406
Fires on “,”agent 1.20% of tokens, subagent, webfetch, web
-
#407
Fires on “=”story 0.40% of tokens2+1: 2+1+1=4,
-
#408
Fires on “If”story 0.37% of tokens910<|think_end|>910<|assistant_end|><|bos|># test_textlib
-
#409
Fires on “ day”story 0.48% of tokenstime to go
-
#410
Fires on “ k”agent 0.54% of tokens: acc = acc
-
#411
Fires on “ at”agent 0.16% of tokens15.<|assistant_end|><|user_start|>what are you good at<|user_end|><|assistant_start|>
-
#412
Fires on “("”python 0.90% of tokenss.lower()⏎```⏎⏎>>> whisper("PY
-
#413
Fires on “ gr”story 0.41% of tokensBut her mom said that June had to earn the
-
#414
Fires on “,”agent 0.40% of tokensleft. The
-
#415
Fires on “ to”story 0.42% of tokens! She was so scared and didn't know what to do.
-
#416
Fires on “.”story 1.06% of tokensThey learned that working together can make hard things easy.<|bos|>Once
-
#417
Fires on “ s”agent 1.42% of tokensreturn items[0]<|arg|>newString= return items[-1
-
#418
Fires on “ a”story 0.36% of tokensmaking it look so pretty.<|bos|>One day, a little cat
-
#419
Fires on “<|bos|>”chat 0.70% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#420
Fires on “ get”story 1.10% of tokensumber and he is angry. He tells them to get down and
-
#421
Fires on “1”worksheet 0.61% of tokens7902 plus 71 equals 779
-
#422
Fires on “ and”python 0.13% of tokensand hugged Lily
-
#423
Fires on “ x”python 0.66% of tokens⏎⏎`halve(x)` returns half of x, round
-
#424
Fires on “ no”story 0.51% of tokensand wags his tail.⏎"Oh no, Max
-
#425
Fires on “ pond”story 3.49% of tokensthey needed to go to the grocery store first.
-
#426
Fires on “1”worksheet 1.10% of tokens4+6+1=11, 13⏎
-
#427
Fires on “<|system_end|>”chat 0.13% of tokensabilities<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Find
-
#428
Fires on “ upper”python 0.87% of tokens⏎⏎`upper_case` returns s in upper case.
-
#429
Fires on “ b”agent 0.48% of tokensb:⏎
-
#430
Fires on “+”worksheet 0.56% of tokens⏎4+0: 4+0=
-
#431
Fires on “`”agent 0.44% of tokensFix: replace `return items[0]` with `
-
#432
Fires on “AssertionError”agent 1.39% of tokens⏎ ^^^^^^^^^^^^^^^^^^^^^^⏎AssertionError⏎⏎
-
#433
Fires on “,”agent 2.27% of tokens<module>⏎ assert min_item([2, 6
-
#434
Fires on “ll”agent 0.87% of tokensapp.py print?<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command
-
#435
Fires on “ and”worksheet 0.76% of tokens<|system_end|><|user_start|>Find the sum of 759 and 8
-
#436
Fires on “A”python 0.47% of tokensOUD"))⏎print(quiet("STAR"))
-
#437
Fires on “ more”agent 0.71% of tokens75 balls and found 6333 more. Now
-
#438
Fires on “_”python 0.31% of tokensabs_value(3))⏎⏎⏎def min_of(
-
#439
Fires on “_”agent 0.41% of tokens⏎78⏎<|bos|>```python⏎def mean_list(
-
#440
Fires on “<|arg|>”agent 0.95% of tokensdoes quiet do?"<|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>pattern=
-
#441
Fires on “=”agent 1.17% of tokenspath=lists.py<|arg|>oldString=head<|arg|>