Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1545
Fires on “my”agent 0.47% of tokensarith.py<|arg|>oldString=opposite<|arg|>newString=my_opposite
-
#1546
Fires on “ "”agent 0.76% of tokens<|tool_start|>Traceback (most recent call last):⏎ File "test_
-
#1547
Fires on “ was”story 0.40% of tokenssaw a big jug on the table. It was full of
-
#1548
Fires on “ do”chat 1.34% of tokensgarden. He said "Patch! What do you think
-
#1549
Fires on “ did”story 0.52% of tokensand did not get
-
#1550
Fires on “7”worksheet 1.18% of tokens07⏎0+1: 0+1=1
-
#1551
Fires on “ head”python 1.25% of tokensdef head(items
-
#1552
Fires on “ tell”chat 0.98% of tokens. I'm a tiny language model: I can tell short stories
-
#1553
Fires on “ was”story 1.47% of tokens.<|bos|>One day, a little boy named Timmy was curious about
-
#1554
Fires on “ tests”agent 0.87% of tokens3 test_lists.py<|tool_call_end|><|assistant_end|><|tool_start|>all tests passed⏎
-
#1555
Fires on “ to”story 0.61% of tokensto play anymore
-
#1556
Fires on “ file”agent 0.35% of tokens<|arg|>path=utils.py<|tool_call_end|><|assistant_end|><|tool_start|>Read file utils.
-
#1557
Fires on “ s”python 0.99% of tokensk⏎⏎⏎def initial(s):⏎ return s[0
-
#1558
Fires on “ =”python 0.63% of tokenssum_up_to(n):⏎ acc = 0
-
#1559
Fires on “ and”story 1.34% of tokensspital with her mom. She was feeling sick and wanted to
-
#1560
Fires on “ became”story 0.55% of tokens. Max and the bird became good friends
-
#1561
Fires on “'s”story 0.65% of tokensit. She hugs him and says, "It's okay,
-
#1562
Fires on “,”agent 0.62% of tokensob, grep, question, read, shell, skill
-
#1563
Fires on “ you”story 0.22% of tokensHi!<|user_end|><|assistant_start|>Hi there! Would you like a
-
#1564
Fires on “ sum”python 0.37% of tokens= return sum(nums
-
#1565
Fires on “assert”story 1.32% of tokensfound the big red tree. It was his home! He was
-
#1566
Fires on “ He”story 0.91% of tokensup late. The bird was sad. It could not
-
#1567
Fires on “larger”agent 0.49% of tokensone⏎⏎COUNT = 6⏎print(larger(9
-
#1568
Fires on “ his”story 0.98% of tokensto play with his friends,
-
#1569
Fires on “ill”agent 0.80% of tokensums, but he didn't know the rhythm
-
#1570
Fires on “ from”story 0.31% of tokensaround and sees Ben and Max peeking out from behind the
-
#1571
Fires on “ s”agent 0.46% of tokens` should reverse s, but it has `return s[1
-
#1572
Fires on “1”worksheet 0.64% of tokensyou add 66972 and 19,
-
#1573
Fires on “ of”story 0.25% of tokensTim, you can play with the hook. But be careful not
-
#1574
Fires on “return”python 0.39% of tokensaverage of a and b, but it has `return (a
-
#1575
Fires on “ many”worksheet 0.63% of tokensballs. Sam gets 2 more balls. How many balls does
-
#1576
Fires on “ soon”story 0.47% of tokensall the numbers from 1 to n. For example, `
-
#1577
Fires on “ close”story 0.55% of tokensake wanted to see it. He wanted to get close.⏎
-
#1578
Fires on “0”worksheet 0.65% of tokens3+3180: 3+0=3
-
#1579
Fires on “ now”chat 0.32% of tokens2 more books. How many books does Lucy have now?<|user_end|>
-
#1580
Fires on “ a”story 1.51% of tokensTuna had a special gift
-
#1581
Fires on “ up”story 0.36% of tokenssaw stairs. They went up and up and up. She
-
#1582
Fires on “ how”story 0.76% of tokens't know how to eat
-
#1583
Fires on “ he”story 0.57% of tokenswith a long string. Bob was surprised, but he was happy
-
#1584
Fires on “,”story 0.43% of tokens: `half` should return half of x, rounded
-
#1585
Fires on “ picked”story 0.56% of tokens, Lily, a key!" Ben said. He picked it up
-
#1586
Fires on “.”story 0.43% of tokensdried in the sun. They played some more and had a
-
#1587
Fires on “ about”chat 0.19% of tokens<|user_start|>Can you tell me a story about a car
-
#1588
Fires on “ ”worksheet 0.42% of tokens+ 62486 = 14
-
#1589
Fires on “ called”story 0.77% of tokenshugged them. He was sorry and worried. He called for help
-
#1590
Fires on “ want”story 0.66% of tokensI will be mean to you!" Tim did not want to sur
-
#1591
Fires on “⏎”story 0.58% of tokensto fly. The bird wanted to relax.⏎The bird
-
#1592
Fires on “print”agent 0.10% of tokensprint(gret("Max"))<|arg|>newString=print(greet
-
#1593
Fires on “ure”agent 1.16% of tokenson the counting board. She soon figured out
-
#1594
Fires on “ %”python 0.51% of tokens⏎ return [n for n in nums if n % 2