Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#742
Fires on “⏎ ”agent 0.39% of tokens<|tool_start|>Found 1 matches⏎utils.py:⏎ Line
-
#743
Fires on “ the”agent 0.52% of tokens<|system_start|>You are a title generator.<|system_end|><|user_start|>Run the test su
-
#744
Fires on “ day”story 0.67% of tokensthe bottle all day. He
-
#745
Fires on “ k”python 0.51% of tokenss:⏎ if c in "aeiou":⏎ k = k
-
#746
Fires on “(”agent 0.16% of tokens3, 9]))<|arg|>newString=print(head([
-
#747
Fires on “ `”python 0.57% of tokens<|bos|>Bug: `max_
-
#748
Fires on “ any”story 0.72% of tokens, and when she was bored. She danced all the time.
-
#749
Fires on “<|tool_start|>”agent 0.69% of tokens<|tool_call_start|>glob<|arg|>pattern=**/*<|tool_call_end|><|assistant_end|><|tool_start|>README
-
#750
Fires on “<|user_end|>”chat 1.43% of tokensharness.<|system_end|><|user_start|>Where can I find odd?<|user_end|><|assistant_start|><|tool_call_start|>
-
#751
Fires on “ should”python 0.22% of tokens"!"⏎```⏎⏎Bug: `triangle` should add up
-
#752
Fires on “ opened”agent 0.59% of tokens's house and knocked on the door. When Tim opened the door
-
#753
Fires on “product”python 0.70% of tokens<|bos|>Bug: `add` should
-
#754
Fires on “ be”agent 0.07% of tokenslwoer_case` should be `lower
-
#755
Fires on “ n”worksheet 0.48% of tokens: return [n for n in nums if n %
-
#756
Fires on “ story”chat 0.29% of tokensin one short sentence.<|system_end|><|user_start|>I want a story about a
-
#757
Fires on “ but”agent 0.49% of tokenshe saw that it was not just one ball, but two!
-
#758
Fires on “,”story 0.88% of tokensSue went to her mom and said, "Mom
-
#759
Fires on “Could”chat 0.61% of tokens. I can change into any animal I want. Thank you for
-
#760
Fires on “2”worksheet 0.36% of tokens3 stars. Mia gets 62214 more stars
-
#761
Fires on “ that”story 0.24% of tokens?<|user_end|><|assistant_start|>Sorry, I don't know about that. I
-
#762
Fires on “ pretty”story 0.49% of tokensSpot loved to play outside. They put on their warm coats
-
#763
Fires on “ there”story 0.30% of tokensOnce upon a time, in a small house, there was
-
#764
Fires on “ and”chat 0.11% of tokensyou help me?<|user_end|><|assistant_start|>I'm prelude, and I know
-
#765
Fires on “ toys”worksheet 0.71% of tokens. Sue gets 5 more toys. How many toys does Sue
-
#766
Fires on “ leaves”story 4.08% of tokenswith her. Lily gave him food and water and toys and hugs
-
#767
Fires on “ very”story 2.72% of tokenstall, and Mia was very small.
-
#768
Fires on “(”python 0.79% of tokensin nums]⏎⏎⏎def add_all(nums):
-
#769
Fires on “ not”story 0.19% of tokensdon't steal that nut!" But the squirrel did not listen.
-
#770
Fires on “+”worksheet 1.78% of tokens987?⏎56350+00
-
#771
Fires on “ his”story 1.34% of tokenspinched the balloon string with its beak.
-
#772
Fires on “ ”worksheet 0.27% of tokens7: 8+7=15, 15
-
#773
Fires on “Once”story 0.29% of tokens<|tool_end|><|assistant_start|>Created `hello.py`.<|assistant_end|><|bos|>Once upon a
-
#774
Fires on “2”agent 1.05% of tokens8, 7, 2, 2]⏎
-
#775
Fires on “1”worksheet 1.15% of tokens+556: 5+6=11,
-
#776
Fires on “3”python 0.17% of tokens6666666666666
-
#777
Fires on “ `"”python 0.58% of tokens. For example, `flip("python")` returns `"no
-
#778
Fires on “ ”worksheet 0.52% of tokens94109 more. Now Tom has 94
-
#779
Fires on “ with”story 1.61% of tokenswanted to be friends with the unusual
-
#780
Fires on “tiger”agent 1.04% of tokensOT"⏎assert combine("tiger", "tiger") ==
-
#781
Fires on “,”python 0.26% of tokensarith.py⏎from arith import positive, smaller, product⏎
-
#782
Fires on “ of”story 0.22% of tokenswanted to reach the vine so she asked one of the forest
-
#783
Fires on “ was”story 0.95% of tokenswas a messy room. In the room, there was a boy
-
#784
Fires on “The”agent 1.13% of tokens+ 4<|tool_call_end|><|assistant_end|><|tool_start|>10<|tool_end|><|assistant_start|>The answer is
-
#785
Fires on “<|assistant_end|>”agent 2.83% of tokens⏎lib.py⏎test_lib.py<|tool_end|><|assistant_start|>This
-
#786
Fires on “=”worksheet 0.38% of tokens1+0: 1+0+1=2,
-
#787
Fires on “2”agent 0.51% of tokensx):⏎9: return x % 2 ==
-
#788
Fires on “ "”agent 0.25% of tokens, they played a game. They called the game "Find the
-
#789
Fires on “ Tom”story 2.87% of tokensand showed it to his mom. "Look, Mom," he
-
#790
Fires on “ How”worksheet 0.42% of tokens. Anna gets 872 more stars. How many stars
-
#791
Fires on “ end”story 0.44% of tokenswatch the truck load the rocks!"⏎The man in the truck