Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2596
Fires on “_”python 0.18% of tokens3)⏎-3⏎<|bos|># test_calc.
-
#2597
Fires on “ only”chat 0.41% of tokensbecame good friends.<|bos|><|system_start|>Answer with the number only.<|system_end|>
-
#2598
Fires on “ with”story 0.27% of tokensshe loved very much. She took the teddy bear with her everywhere
-
#2599
Fires on “ who”story 0.45% of tokensOnce upon a time, there was a little girl who lived on
-
#2600
Fires on “⏎”worksheet 0.39% of tokens67⏎52
-
#2601
Fires on “<|user_start|>”chat 0.84% of tokenslittle ant.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Hi<|user_end|>
-
#2602
Fires on “6”worksheet 0.70% of tokens1=13, 132509⏎
-
#2603
Fires on “ everything”story 0.36% of tokenspersistent. She loved to explore and touch everything! One
-
#2604
Fires on “ When”worksheet 0.72% of tokenstrees and waited for the lion to come out. When the lion
-
#2605
Fires on “("”python 0.98% of tokens3) == "appleappleapple"⏎print("all tests
-
#2606
Fires on “ plus”worksheet 0.65% of tokens9880.⏎877 plus 1
-
#2607
Fires on “3”chat 0.70% of tokens3+1=4, 48<|think_end|>48<|assistant_end|>
-
#2608
Fires on “ed”agent 0.67% of tokensk<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>
-
#2609
Fires on “ the”agent 1.98% of tokens.py with a function upper_case that returns s in upper
-
#2610
Fires on “ one”agent 0.72% of tokensters in the world.<|bos|><|system_start|>Answer in one short sentence
-
#2611
Fires on “.”worksheet 0.70% of tokens1248.<|assistant_end|><|bos|># README.md⏎
-
#2612
Fires on “ matches”agent 0.79% of tokensquiet<|tool_call_end|><|assistant_end|><|tool_start|>Found 1 matches⏎utils
-
#2613
Fires on “ soon”agent 0.38% of tokensBye<|user_end|><|assistant_start|>Bye! Come back for another story soon.<|assistant_end|>
-
#2614
Fires on “ went”worksheet 0.45% of tokensThey both played together in the park until the sun went down.
-
#2615
Fires on “ she”story 1.61% of tokensher blouse very much.⏎One day, Lily went to
-
#2616
Fires on “items”worksheet 0.23% of tokens⏎False⏎⏎```python⏎def contains(items, x
-
#2617
Fires on “ broke”story 1.57% of tokensMax!" Ben says, frowning. "You broke our tower
-
#2618
Fires on “ wagged”story 1.07% of tokens, friendly dog came up to him. The dog wagged its tail
-
#2619
Fires on “1”worksheet 1.00% of tokenstools: calculator<|system_end|><|user_start|>Can you add 51 and
-
#2620
Fires on “ only”agent 0.83% of tokens>⏎ assert lower_case("PYTHON")
-
#2621
Fires on “=”agent 0.84% of tokenseven numbers of nums<|user_end|><|assistant_start|><|tool_call_start|>read<|arg|>path=hello.
-
#2622
Fires on “ a”chat 0.56% of tokensOne day, a cat named Jazz went for a walk.
-
#2623
Fires on “ didn”story 1.04% of tokensthat Mr Chirp was very small and he didn't want
-
#2624
Fires on “ the”chat 0.67% of tokenscareful and not be foolish. They played together for the rest of
-
#2625
Fires on “ friends”story 0.99% of tokens1540<|bos|>Tom and Mia were best friends. They
-
#2626
Fires on “ add”agent 0.53% of tokenscalculator<|system_end|><|user_start|>Tell me the latest sports sc
-
#2627
Fires on “ sunny”story 0.70% of tokensto be naughty and they became friends.<|bos|>One sunny day,
-
#2628
Fires on “ to”story 0.52% of tokensher very own dinosaur!<|bos|>Lily and Mom went to the m
-
#2629
Fires on “ scratch”chat 0.25% of tokens'm prelude, a very small language model trained from scratch on a
-
#2630
Fires on “path”agent 2.34% of tokens15⏎<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path=lists
-
#2631
Fires on “ about”story 0.58% of tokenswanted to get his ball. So, he thought of a plan
-
#2632
Fires on “ would”agent 0.35% of tokenswould fly around
-
#2633
Fires on “=”worksheet 0.29% of tokens+94: 7+4+1=12
-
#2634
Fires on “`.”agent 0.59% of tokensof `return s * n`. The tests
-
#2635
Fires on “ says”story 0.38% of tokensgo closer and look at the nests," Tom said.⏎
-
#2636
Fires on “EXT”agent 0.50% of tokensoy.⏎Joy learned a valuable
-
#2637
Fires on “ dog”story 0.42% of tokens<|assistant_start|>Once upon a time, there was a little dog named Spot
-
#2638
Fires on “ and”story 0.25% of tokenshelp us make yummy food!" The oven felt happy and important.
-
#2639
Fires on “print”python 0.28% of tokens⏎print(final_letter("apple"))⏎print(echo
-
#2640
Fires on “ instead”agent 0.25% of tokenspy`: it used `if a < b:` instead of `
-
#2641
Fires on “4”worksheet 0.16% of tokens6+0014: 6+4+1
-
#2642
Fires on “ c”python 1.14% of tokens(s):⏎ k = 0⏎ for c in s
-
#2643
Fires on “ n”agent 0.43% of tokens= return [n * 2 for n in nums
-
#2644
Fires on “ time”story 0.22% of tokensher yummy muffin.<|bos|>Once upon a time, there
-
#2645
Fires on “ answer”agent 0.62% of tokens87.<|user_end|><|assistant_start|>The answer is