Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#592
Fires on “ after”agent 0.42% of tokensown little castle. And they all lived happily ever after.<|bos|>
-
#593
Fires on “)”python 0.16% of tokens, smaller, product⏎⏎assert positive(-4) == False
-
#594
Fires on “ loud”story 0.36% of tokensstarted to climb the wall, but he heard a loud bark.
-
#595
Fires on “("”agent 0.23% of tokenspath=say.py<|arg|>content=print("Hi there
-
#596
Fires on “⏎”python 0.22% of tokens5, 7]) == [8]⏎assert sum
-
#597
Fires on “ mom”story 1.43% of tokensher Mommy for help. However, her Mommy was always
-
#598
Fires on “⏎”worksheet 1.28% of tokens9+1=10, 0⏎35
-
#599
Fires on “⏎”agent 1.48% of tokens3 main.py⏎40⏎12⏎<|bos|><|system_start|>
-
#600
Fires on “positive”python 1.09% of tokensabs_value, divides⏎⏎assert abs_value(-4
-
#601
Fires on “ a”story 0.55% of tokensIt was a calm day
-
#602
Fires on “ each”story 0.02% of tokensWe have to remain in a line and follow each other.
-
#603
Fires on “ ”worksheet 0.22% of tokens84674 more. Now Max has 84
-
#604
Fires on “ They”story 1.77% of tokens<|assistant_end|><|bos|>Lily and Ben are playing with blocks. They like to
-
#605
Fires on “ Every”agent 0.50% of tokens.<|bos|>Pete was a happy rat. Every morning,
-
#606
Fires on “ many”python 0.47% of tokens03 more stickers. How many stickers does
-
#607
Fires on “ big”story 2.36% of tokenshe got an idea. He found a big, dry branch and
-
#608
Fires on “ said”story 0.36% of tokenshere! Look at this vine!" Tim shouted.⏎
-
#609
Fires on “<|bos|>”story 0.98% of tokens<|bos|>Once upon
-
#610
Fires on “ is”agent 0.22% of tokensb):⏎<|tool_end|><|assistant_start|>`lower_of` is defined in
-
#611
Fires on “ "”story 0.46% of tokensHey, cat, stop that!" Ben shouted. "Those
-
#612
Fires on “ something”worksheet 0.86% of tokensbut he wanted to be rich.⏎Then, something unexpected happened
-
#613
Fires on “ found”worksheet 0.19% of tokens06.⏎Anna had 7 stickers and found 6
-
#614
Fires on “ found”story 0.79% of tokens. One day, while they were playing, Spot found some d
-
#615
Fires on “ent”agent 0.18% of tokens, read, shell, skill, subagent, we
-
#616
Fires on “2”worksheet 0.31% of tokens8 stickers. Mia gets 97692 more stickers
-
#617
Fires on “ forgot”story 0.98% of tokensmouse kept insisting
-
#618
Fires on “-”agent 0.45% of tokensa story!<|user_end|><|assistant_start|>Max was a nosy three-year
-
#619
Fires on “ and”story 7.96% of tokenshurt her head.⏎Her mom ran to her and called for
-
#620
Fires on “ of”story 0.60% of tokenshad a lot of fun with
-
#621
Fires on “ for”story 0.28% of tokens<|bos|>One day, a cat named Jazz went for a walk
-
#622
Fires on “ day”story 0.35% of tokensand find something new and exciting to explore. One day the rabbit
-
#623
Fires on “S”python 0.21% of tokens<|user_start|>Deploy this to AWS<|user_end|><|assistant_start|>
-
#624
Fires on “ ”python 0.85% of tokensharness.<|system_end|><|user_start|>Set SCORES to [2, 3,
-
#625
Fires on “=”python 0.26% of tokens+3806: 2+6=8,
-
#626
Fires on “ waited”story 0.80% of tokensbig, blue chair in the living room. He waited and waited
-
#627
Fires on “.”agent 1.07% of tokens(1, 12)` returns `6.5`.
-
#628
Fires on “ This”story 0.38% of tokensboxes. In each box, they drew something that happened in
-
#629
Fires on “Once”story 0.57% of tokenspy`.<|assistant_end|><|bos|>Once upon a
-
#630
Fires on “'m”agent 0.59% of tokensThat's very kind of you, Anna. I'm sure the
-
#631
Fires on “ took”story 0.30% of tokensplay with the ball and the net. They both took turns throw
-
#632
Fires on “.”story 0.85% of tokensso it flew fast and saved the bug. The big
-
#633
Fires on “ to”story 0.56% of tokenshe couldn't find it
-
#634
Fires on “most”agent 0.29% of tokens.<|bos|>Pete was a happy rat. Every morning,
-
#635
Fires on “ g”agent 0.87% of tokensgrater is sharp." Tom nods and grates
-
#636
Fires on “ things”story 0.26% of tokensYou have to be careful with things like this
-
#637
Fires on “ unexpected”agent 0.36% of tokensthe hole with sand.⏎But then, something unexpected happened.
-
#638
Fires on “ day”story 0.12% of tokensfor their kind words. From that day on the
-
#639
Fires on “```”python 0.46% of tokens])` returns `4`.⏎⏎```python⏎
-
#640
Fires on “#”story 0.20% of tokensin nums]⏎⏎# README.md⏎# Sc
-
#641
Fires on “ like”story 0.72% of tokens"Hi, Spot!" said Lucy. "I like your wa