Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1595
Fires on “ were”agent 0.26% of tokensupon a time, in a beautiful land, there were many colors
-
#1596
Fires on “```”python 0.78% of tokens])` returns `4`.⏎⏎```python⏎
-
#1597
Fires on “ and”agent 0.31% of tokensprelude: I can tell you short stories for children and add numbers
-
#1598
Fires on “0”worksheet 0.84% of tokens548: 8+8=16,
-
#1599
Fires on “(”python 1.54% of tokens`.<|assistant_end|><|bos|># tools.py⏎def repeat(s,
-
#1600
Fires on “#”worksheet 0.43% of tokensin nums]⏎⏎# README.md⏎# Sc
-
#1601
Fires on “-”agent 0.35% of tokens!⏎Mother and Father took their three-year
-
#1602
Fires on “from”python 0.21% of tokens]⏎```⏎<|bos|># demo.py⏎from calc import
-
#1603
Fires on “2”worksheet 0.99% of tokens⏎1+2: 1+2+1
-
#1604
Fires on “ "”story 0.63% of tokensCan I climb the tree?" His mom said, "Yes,
-
#1605
Fires on “ time”story 0.23% of tokens, and they were happy.<|bos|>Once upon a time, there
-
#1606
Fires on “⏎”worksheet 0.77% of tokens6+4=10, 0⏎57
-
#1607
Fires on “ showed”story 1.54% of tokensa sound like little rocks moving.⏎Lily showed the or
-
#1608
Fires on “ Amy”story 0.46% of tokensmom smiled and said, "It's okay, Amy. I
-
#1609
Fires on “(”worksheet 1.15% of tokens6) == 216⏎assert cubed(4)
-
#1610
Fires on “1”agent 0.17% of tokens6+77: 6+7=13,
-
#1611
Fires on “def”python 0.28% of tokensall tests passed⏎<|bos|># mathlib.py⏎def product(
-
#1612
Fires on “ return”story 0.35% of tokensreturn -x
-
#1613
Fires on “<|bos|>”chat 0.64% of tokensbe fewer disagreements.<|bos|><|system_start|>tools
-
#1614
Fires on “<|user_end|>”chat 1.64% of tokensWhen did World War II end?<|user_end|><|assistant_start|>
-
#1615
Fires on “Don”python 0.27% of tokensstarted to cry. Mr. Bear said, "Don't worry
-
#1616
Fires on “⏎”worksheet 0.80% of tokenspopular.<|assistant_end|><|bos|># notes.txt⏎Remember
-
#1617
Fires on “ H”agent 0.45% of tokensLily smiled and picked up her doll.⏎"Ha!"
-
#1618
Fires on “ or”story 0.61% of tokensHi there! Would you like a story, or some numbers
-
#1619
Fires on “1”agent 1.56% of tokensRead file words.py, lines 1-10⏎
-
#1620
Fires on “ =”agent 0.45% of tokens92 shells.⏎6 + 26 = 3
-
#1621
Fires on “<|tool_start|>”agent 2.25% of tokens<|assistant_end|><|tool_start|>Edited number
-
#1622
Fires on “ p”story 0.39% of tokensluffy was so happy and it piped
-
#1623
Fires on “ ever”story 0.66% of tokenslaughed. And they lived happily ever after.
-
#1624
Fires on “ had”story 0.50% of tokensMia." They played with the toy car together and had lots of
-
#1625
Fires on “ b”python 0.55% of tokensdef biggest(nums):⏎ return max(nums)⏎⏎
-
#1626
Fires on “1”worksheet 0.36% of tokens⏎5+5: 5+5=10,
-
#1627
Fires on “9”worksheet 1.12% of tokens548: 8+8=16,
-
#1628
Fires on “=”worksheet 0.39% of tokens25+81: 5+1=6,
-
#1629
Fires on “ short”chat 0.26% of tokensdon't know about that. I can only tell short stories and
-
#1630
Fires on “ Line”agent 0.69% of tokensFound 6 matches⏎listlib.py:⏎ Line 1
-
#1631
Fires on “ go”story 1.07% of tokens. They see a big hill. They want to go up the
-
#1632
Fires on “⏎”story 0.88% of tokensold and he was always getting into silly trouble.⏎Today
-
#1633
Fires on “it”agent 0.49% of tokens.py (2 replacements)<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path
-
#1634
Fires on “ fix”agent 0.06% of tokensharness.<|system_end|><|user_start|>the tests are failing, fix the bug
-
#1635
Fires on “Once”python 0.92% of tokens<|bos|>Once upon a
-
#1636
Fires on “ by”agent 0.45% of tokensMia was an independent girl who liked to do things by herself.
-
#1637
Fires on “ other”agent 0.71% of tokensThey were happy to be friends and to help each other.<|bos|>
-
#1638
Fires on “.”story 1.03% of tokenswith her best friend, a little girl named Lily. They loved
-
#1639
Fires on “ Can”story 0.46% of tokens, "Wow, that's a cool toy! Can I play
-
#1640
Fires on “ max”agent 0.62% of tokensa coding agent harness.<|system_end|><|user_start|>Rename cubed to cubed_v
-
#1641
Fires on “ sad”agent 0.79% of tokensa bird with an empty nest. The bird looked sad. Jane
-
#1642
Fires on “4”chat 0.71% of tokens8 coins.⏎7 plus 3964 is
-
#1643
Fires on “ of”agent 1.13% of tokens2910.⏎Find the sum of 4
-
#1644
Fires on “ the”agent 0.22% of tokens# notes.txt⏎Remember to run the tests.