Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2496
Fires on “Can”story 0.72% of tokensa yellow ball on the floor.⏎"Can we eat
-
#2497
Fires on “ Tim”story 0.81% of tokensLily has
-
#2498
Fires on “ where”story 0.70% of tokens.⏎At the party, there was a game where the kids
-
#2499
Fires on “ about”agent 0.30% of tokensand add numbers.<|assistant_end|><|user_start|>Tell me a story about a cat
-
#2500
Fires on “ In”story 0.83% of tokenstree. The bird was not mad at Tim. In fact,
-
#2501
Fires on “a”python 0.24% of tokens⏎assert first_letter("apple") == "a"⏎
-
#2502
Fires on “+”worksheet 1.34% of tokens+1=6, 6⏎90+03
-
#2503
Fires on “$”agent 0.23% of tokenstest_greetings.py⏎all tests passed⏎$ python3
-
#2504
Fires on “ played”python 0.39% of tokensplayed together every
-
#2505
Fires on “2”agent 1.48% of tokensin <module>⏎ assert max_item([0,
-
#2506
Fires on “<|user_start|>”chat 0.49% of tokens<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Please add
-
#2507
Fires on “+”worksheet 0.72% of tokens, 9⏎3+3: 3+3=
-
#2508
Fires on “-”chat 0.25% of tokenslanguage model trained from scratch on a laptop by mini-lab.
-
#2509
Fires on “ have”agent 0.70% of tokens7 more books. How many books does Sue have now?
-
#2510
Fires on “That”story 1.02% of tokenslaughed and laughed.⏎"That was fun
-
#2511
Fires on “Good”chat 0.38% of tokensimport echo, bigger⏎⏎print(echo("tiger",
-
#2512
Fires on “Do”chat 0.29% of tokensstory!<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|system_start|>Do not use
-
#2513
Fires on “⏎”agent 0.16% of tokens<|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>
-
#2514
Fires on “ ”worksheet 0.75% of tokens1: 9+1=10, 0⏎
-
#2515
Fires on “0”worksheet 0.47% of tokens8+7=15, 5⏎000
-
#2516
Fires on “ liked”story 0.45% of tokensevery day.<|bos|>Ben is a big boy who likes to help
-
#2517
Fires on “ saw”story 0.46% of tokenshis sack with him. In the park, Tim saw a little
-
#2518
Fires on “ as”agent 0.44% of tokensas fast as she had
-
#2519
Fires on “:”worksheet 0.90% of tokens13⏎359+000: 9
-
#2520
Fires on “3”worksheet 0.07% of tokens6 + 58681 = 586
-
#2521
Fires on “ Mia”story 1.57% of tokensand play together, you can have more fun." Tom and Tim
-
#2522
Fires on “=”agent 0.37% of tokenspy please<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3
-
#2523
Fires on “0”agent 0.57% of tokens0031
-
#2524
Fires on “ ex”agent 0.52% of tokenshis lesson and now always makes sure he gives himself extra time
-
#2525
Fires on “<|tool_call_end|>”agent 1.43% of tokens<|tool_call_start|>read<|arg|>path=words.py<|tool_call_end|><|assistant_end|><|tool_start|>
-
#2526
Fires on “,”agent 2.05% of tokensHe would pick up pretty rocks, shiny buttons, and soft
-
#2527
Fires on “ two”story 0.32% of tokensday.<|bos|>Once upon a time, there were two friends called
-
#2528
Fires on “ saw”agent 0.82% of tokensown game.⏎Then, a wise old man saw them qu
-
#2529
Fires on “ inside”story 0.20% of tokensloved very much. She took the teddy bear with her everywhere she
-
#2530
Fires on “ sees”agent 0.60% of tokenscrying and comes to see what is wrong. She sees the be
-
#2531
Fires on “python”python 0.39% of tokens+ b) / 3`.⏎⏎```python⏎def
-
#2532
Fires on “ project”agent 0.80% of tokens<|user_end|><|assistant_start|>Hi! What should we do in this project?<|assistant_end|>
-
#2533
Fires on “3”python 0.35% of tokens.py⏎noom⏎6⏎$ python3 test_
-
#2534
Fires on “ very”story 0.59% of tokensBilly gave a big smile and nodded, feeling very excited.
-
#2535
Fires on “<|arg|>”chat 0.26% of tokens<|assistant_end|><|user_start|>Plus 9?<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=
-
#2536
Fires on “ of”story 0.25% of tokensclassroom became friends again. The moral of the story
-
#2537
Fires on “ tell”story 0.58% of tokensAnswer with the number only.<|system_end|><|user_start|>Could you tell me what
-
#2538
Fires on “ example”python 0.64% of tokensall the numbers from 1 to n. For example, `
-
#2539
Fires on “8”agent 0.71% of tokensthe calculator.<|system_end|><|user_start|>Lily has 636 shells.
-
#2540
Fires on “ another”worksheet 0.53% of tokensyou later!<|user_end|><|assistant_start|>Bye! Come back for another story soon
-
#2541
Fires on “!”story 3.62% of tokenspicked up Tim's vest instead of the ball!⏎The
-
#2542
Fires on “+”worksheet 0.32% of tokens08+27: 8+7+1=
-
#2543
Fires on “ ==”agent 1.53% of tokens, 7, 5], 0) == False⏎
-
#2544
Fires on “")”python 0.80% of tokens]) == 9⏎print("all tests passed")⏎⏎
-
#2545
Fires on “,”python 0.27% of tokens: `fact` should return the factorial of n, but it