prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#2896
Fires on “0”
worksheet 1.17% of tokens

0003: 0+3
#2897
Fires on “⏎”
python 0.24% of tokens

from words import exclaim, echo, last_letter⏎⏎assert
#2898
Fires on “The”
chat 1.90% of tokens

0<|tool_call_end|><|assistant_end|><|tool_start|>1018<|tool_end|><|assistant_start|>The answer is
#2899
Fires on “4”
agent 0.33% of tokens

43.⏎
#2900
Fires on “:”
python 0.62% of tokens

% b == 1⏎```⏎⏎Fix: replace `
#2901
Fires on “("”
python 0.54% of tokens

("star") == "star!"⏎assert echo("banana",
#2902
Fires on “8”
worksheet 1.20% of tokens

29 = 84.⏎432
#2903
Fires on “ saw”
story 2.67% of tokens

saw the ball getting bigger
#2904
Fires on “("”
agent 0.39% of tokens

size("hello") == 5⏎assert initial("banana")
#2905
Fires on “ c”
chat 0.24% of tokens

then, the ball hit Tom's paw and it caused
#2906
Fires on “ [”
python 0.58% of tokens

def doubled(nums):⏎ return [n *
#2907
Fires on “ a”
python 0.40% of tokens

w York<|user_end|><|assistant_start|>Sorry, I'm a tiny model
#2908
Fires on “ answer”
chat 0.28% of tokens

The train was happy to help.⏎On the way, the
#2909
Fires on “⏎”
python 0.69% of tokens

return a % b == 1⏎```⏎⏎Fix:
#2910
Fires on “<|user_end|>”
agent 0.84% of tokens

<|system_end|><|user_start|>"Can you run demo.py?"<|user_end|><|assistant_start|><|tool_call_start|>
#2911
Fires on “ that”
story 0.49% of tokens

.⏎"Mom said we should not take things that are not
#2912
Fires on “<|system_start|>”
agent 1.22% of tokens

story, or some numbers to add?<|assistant_end|><|bos|><|system_start|>tools:
#2913
Fires on “ Tom”
story 0.44% of tokens

great day.<|assistant_end|><|bos|>Lily loved her dog, Spot. Spot
#2914
Fires on “ but”
story 0.32% of tokens

the ant.⏎The ant tried and tried, but the leaf
#2915
Fires on “ver”
agent 1.92% of tokens

main.py<|arg|>oldString=print(decrement(5
#2916
Fires on “ play”
story 0.67% of tokens

little cat named Quick went out to play. He
#2917
Fires on “ unexpected”
story 0.25% of tokens

ari was winning. But then, something unexpected happened.
#2918
Fires on “ unexpected”
story 0.13% of tokens

the hole with sand.⏎But then, something unexpected happened.
#2919
Fires on “⏎”
python 1.02% of tokens

:⏎ result = result + n⏎ return result⏎```⏎
#2920
Fires on “ returns”
python 0.69% of tokens

<|bos|>Bug: `max_of` should return the bigger
#2921
Fires on “Once”
story 0.43% of tokens

with `final_letter(s)`.<|assistant_end|><|bos|>Once upon a
#2922
Fires on “ forgot”
story 0.35% of tokens

up and followed his mom to the kitchen. He forgot about the
#2923
Fires on “A”
story 0.03% of tokens

for helping them. Fin learned that being different was not bad.
#2924
Fires on “3”
worksheet 2.77% of tokens

?⏎8167+0004:
#2925
Fires on “ saw”
story 0.63% of tokens

. As she looked out of her window, she noticed a jar
#2926
Fires on “,”
agent 0.30% of tokens

small Python projects. I can list, read, run,
#2927
Fires on “0”
worksheet 1.22% of tokens

0+2=2, 2973⏎0
#2928
Fires on “<|assistant_start|>”
chat 0.23% of tokens

another story soon.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>Happy
#2929
Fires on “0”
worksheet 0.46% of tokens

1.⏎40124+000
#2930
Fires on “ sub”
agent 2.32% of tokens

subagent
#2931
Fires on “ ex”
agent 0.55% of tokens

exec
#2932
Fires on “ mom”
story 0.44% of tokens

Will I get my trumpet back, mom?" Anna
#2933
Fires on “1”
worksheet 1.63% of tokens

0+97: 0+7+1=8
#2934
Fires on “ ”
worksheet 0.47% of tokens

you tell me what 6 +
#2935
Fires on “ should”
python 0.22% of tokens

```⏎⏎Bug: `only_even` should keep only
#2936
Fires on “A”
agent 0.51% of tokens

README.md⏎# Toolbox⏎⏎A small Python
#2937
Fires on “c”
agent 0.49% of tokens

answer is 7.<|assistant_end|><|user_start|>What if you add
#2938
Fires on “ being”
agent 0.52% of tokens

stay in his tree. The bird thanked her for being his friend
#2939
Fires on “ walked”
agent 0.30% of tokens

to collect food for his family. He hopped and jumped, looking
#2940
Fires on “.”
worksheet 0.64% of tokens

girl named Lily. She had a toy named Mr. Bear.
#2941
Fires on “oldString”
agent 0.48% of tokens

<|tool_call_start|>edit<|arg|>path=items.py<|arg|>oldString=
#2942
Fires on “ Every”
chat 1.90% of tokens

there a little boy, who was very happy. Every morning he
#2943
Fires on “ never”
story 0.84% of tokens

she found a weird, round toy that she had never seen before
#2944
Fires on “ loved”
story 0.31% of tokens

Tim. He had a black toy car that he loved very much
#2945
Fires on “⏎”
worksheet 0.22% of tokens

67 = 54704.⏎0 +