Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2746
Fires on “ s”story 1.43% of tokenswere busy with Lily, and then he snea
-
#2747
Fires on “ to”story 1.96% of tokensplayed until the sun went down and it was time to go home
-
#2748
Fires on “ =”agent 0.42% of tokensin range(n + 1):⏎ acc = acc +
-
#2749
Fires on “ ”worksheet 0.31% of tokens+ 62486 = 14
-
#2750
Fires on “⏎ ”python 1.10% of tokens// 2⏎⏎⏎def odd(x):⏎ return x
-
#2751
Fires on “("”python 0.46% of tokenssize("hello") == 5⏎assert initial("banana")
-
#2752
Fires on “,”story 0.10% of tokens<|assistant_start|>Hi there! Would you like a story, or some
-
#2753
Fires on “ but”python 2.01% of tokensall` should return every number of nums doubled, but it has
-
#2754
Fires on “ learned”story 0.48% of tokens!" They both ate the toast and felt happy. After their meal
-
#2755
Fires on “⏎”python 0.99% of tokens]))⏎print(quiet("Tree"))⏎⏎$ python
-
#2756
Fires on “ob”agent 1.40% of tokensfiles<|user_end|><|assistant_start|><|tool_call_start|>glob<|arg|>
-
#2757
Fires on “ scared”story 1.37% of tokenssaw a big, terrible bug. Lily was scared and ran
-
#2758
Fires on “=”worksheet 0.50% of tokens496: 5+6+1=12
-
#2759
Fires on “ and”story 1.43% of tokensto shine. The charming festival was filled with joy and laughter
-
#2760
Fires on “("”agent 0.66% of tokens"HELLO"⏎assert initial("python")
-
#2761
Fires on “ around”story 0.81% of tokens. We will find them."⏎They all looked around the park
-
#2762
Fires on “ ”worksheet 0.48% of tokens12.⏎7311 + 92
-
#2763
Fires on “9”worksheet 0.31% of tokens493, you get 7499.⏎
-
#2764
Fires on “Sorry”chat 0.15% of tokenshelp me with my taxes?<|user_end|><|assistant_start|>Sorry, I
-
#2765
Fires on “ little”story 0.34% of tokens<|bos|>One day, a little ball named
-
#2766
Fires on “ to”story 0.72% of tokensa plan. He told Sam to find a
-
#2767
Fires on “ is”story 0.72% of tokensfor helping them. Fin learned that being different was not bad.
-
#2768
Fires on “ need”story 0.35% of tokensis 189.<|assistant_end|><|bos|><|user_start|>I need to know
-
#2769
Fires on “#”agent 0.30% of tokensin nums]⏎⏎# README.md⏎# Sc
-
#2770
Fires on “ue”agent 0.75% of tokens_halve<|arg|>replaceAll=true<|tool_call_end|><|assistant_end|>
-
#2771
Fires on “ outside”story 0.25% of tokensto play.⏎One day Millie went outside to play
-
#2772
Fires on “ old”story 0.27% of tokensnot able to pick it up.⏎A wise old man who
-
#2773
Fires on “ his”story 0.54% of tokensto go to a fun party. He put on his best hat
-
#2774
Fires on “ result”agent 0.47% of tokensfor n in nums:⏎10: result = result
-
#2775
Fires on “ that”story 0.30% of tokensOne day, she found a weird, round toy that she had
-
#2776
Fires on “⏎”python 0.80% of tokens([8, 8, 5])⏎[8,
-
#2777
Fires on “ got”story 0.55% of tokens.<|bos|>Ben has 23 flowers. Ben gets 1
-
#2778
Fires on “ are”agent 0.75% of tokenson the ocean, so they decided to build a boat
-
#2779
Fires on “<|user_start|>”chat 0.58% of tokensbest of friends forever.<|bos|><|user_start|>Write a
-
#2780
Fires on “#”python 0.08% of tokens(SCORES))⏎⏎# README.md⏎# Word
-
#2781
Fires on “c”agent 0.49% of tokenswent to the park to do some exercise.
-
#2782
Fires on “ and”story 0.50% of tokensit will die and not come back."⏎Lily and Ben
-
#2783
Fires on “ and”story 0.42% of tokensa girl called Sam. She was three years old and her parents
-
#2784
Fires on “ make”story 0.65% of tokenssafe. And from that day on, Lily always made sure her
-
#2785
Fires on “What”chat 1.57% of tokens8 = 2353.<|bos|><|user_start|>What is
-
#2786
Fires on “:”python 0.29% of tokens⏎ return x⏎<|tool_call_end|><|assistant_end|><|tool_start|>Created file successfully: math_
-
#2787
Fires on “.”agent 0.73% of tokensand started to count right away. He counted one, two,
-
#2788
Fires on “ not”story 2.70% of tokenssad. Bouncy was poor and did not have any
-
#2789
Fires on “ you”chat 0.53% of tokensand went home to get warm.<|bos|><|user_start|>Could you tell me
-
#2790
Fires on “ s”story 0.37% of tokensBen says, "OK. One, two, three..."
-
#2791
Fires on “'t”story 0.53% of tokensI don't know about
-
#2792
Fires on “1”worksheet 2.71% of tokens$ python3 main.py⏎tigertiger⏎11⏎
-
#2793
Fires on “0”worksheet 0.59% of tokens=15, 5⏎9221+0
-
#2794
Fires on “,”agent 0.50% of tokens8⏎<|bos|><|system_start|>tools: edit, glob, grep
-
#2795
Fires on “ said”story 0.74% of tokenson a folder. "Ouch!" she said. Amy