prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#3048
Fires on “py”
agent 1.08% of tokens

<|tool_call_start|>read<|arg|>path=words.py<|tool_call_end|><|assistant_end|>
#3049
Fires on “ to”
agent 0.51% of tokens

toy. Amy was sad and didn't know what to do.
#3050
Fires on “ now”
agent 0.36% of tokens

leopard, "I have to go home now." The
#3051
Fires on “ came”
agent 0.37% of tokens

Create mathlib.py with a function triangle that adds up the numbers
#3052
Fires on “ and”
story 2.25% of tokens

was very surprised. She waved the magic stick, and all the
#3053
Fires on “ not”
story 1.35% of tokens

little bird. The bird was helpless and could not fly.
#3054
Fires on “ on”
story 0.30% of tokens

, Zoom saw a big, foolish green truck on the mountain
#3055
Fires on “ore”
agent 0.41% of tokens

Tell me the latest sports scores.
#3056
Fires on “1”
agent 0.73% of tokens

- b<|tool_call_end|><|assistant_end|><|tool_start|>Edited calc.py (1 replacement)
#3057
Fires on “a”
python 0.75% of tokens

welcome("Noa"))⏎
#3058
Fires on “ utils”
agent 1.71% of tokens

home sad.<|bos|># utils.py
#3059
Fires on “])”
python 0.33% of tokens

5, 3, 4, 3]) == [
#3060
Fires on “⏎”
python 0.27% of tokens

tiger", "robot")` returns `"tiger robot"`.⏎⏎`
#3061
Fires on “ an”
story 0.42% of tokens

He felt sad.⏎Then, Pandy had an idea.
#3062
Fires on “⏎”
python 0.07% of tokens

kind to everyone. All the kids at school liked
#3063
Fires on “1”
python 0.20% of tokens

([8, 2, 2, 4,
#3064
Fires on “(”
python 0.62% of tokens

chars(s):⏎ return len(s)
#3065
Fires on “⏎”
story 0.77% of tokens

ine and bandages and make them feel better.⏎One day
#3066
Fires on “Yes”
worksheet 0.32% of tokens

I climb the tree?" His mom said, "Yes, let
#3067
Fires on “ stick”
story 0.45% of tokens

had an idea. He ran and found a long stick. P
#3068
Fires on “ tired”
story 0.51% of tokens

, Max and Tim went back home. They were tired from their
#3069
Fires on “ there”
chat 0.65% of tokens

<|user_start|>"Why does
#3070
Fires on “ until”
story 0.06% of tokens

having lots of fun. They played all evening until the sun
#3071
Fires on “ AI”
agent 0.18% of tokens

, execute<|system_end|><|system_start|>You are an AI agent running
#3072
Fires on “ help”
story 0.40% of tokens

.⏎Tim asked his mom, "Can you help me get
#3073
Fires on “nums”
agent 0.48% of tokens

<|assistant_end|><|bos|>```python⏎def double_all(nums):⏎
#3074
Fires on “ tell”
chat 0.06% of tokens

sure to only
#3075
Fires on “ time”
story 0.31% of tokens

old lady was happy.<|bos|>Once upon a time, there
#3076
Fires on “'re”
agent 0.46% of tokens

tools: calculator<|system_end|><|user_start|>Thank you<|user_end|><|assistant_start|>You're welcome!
#3077
Fires on “ on”
story 1.26% of tokens

wanted to go to a fun party. He put on his best
#3078
Fires on “ other”
story 0.69% of tokens

tree. The ugly tree was sad because all the other trees were
#3079
Fires on “ feel”
story 0.37% of tokens

is a spray for my face. It makes me feel nice."
#3080
Fires on “assert”
agent 0.77% of tokens

, 11) == 19⏎assert is_
#3081
Fires on “<|bos|>”
chat 0.39% of tokens

nums) / len(nums)⏎```⏎<|bos|><|user_start|>What
#3082
Fires on “ `”
agent 0.29% of tokens

1 replacement)<|tool_end|><|assistant_start|>Renamed `initial` to `first_
#3083
Fires on “ great”
agent 0.53% of tokens

of the chicken scratch. But he did learn an important lesson in
#3084
Fires on “4”
worksheet 2.26% of tokens

6 cookies.⏎1114 plus 49 equals
#3085
Fires on “0”
chat 0.46% of tokens

is 7.<|assistant_end|><|bos|>The sum of 277
#3086
Fires on “1”
agent 0.49% of tokens

8+12: 8+2=10,
#3087
Fires on “ successfully”
agent 0.62% of tokens

+ name + "!"⏎<|tool_call_end|><|assistant_end|><|tool_start|>Created file successfully: core
#3088
Fires on “ name”
chat 0.88% of tokens

says, "I am a swan. My name is Lily
#3089
Fires on “,”
story 0.43% of tokens

Sue went to her mom and said, "Mom
#3090
Fires on “ tests”
python 0.27% of tokens

"Hello, Noa!"⏎print("all tests passed")
#3091
Fires on “b”
agent 1.03% of tokens

bug jumped high into the air! Tim did not expect
#3092
Fires on “ ”
worksheet 0.70% of tokens

80: 0+0=0, 00
#3093
Fires on “⏎”
python 0.58% of tokens

⏎assert is_odd(3) == True⏎print("
#3094
Fires on “ you”
chat 0.33% of tokens

?<|user_end|><|assistant_start|>I'm prelude: I can tell you short stories
#3095
Fires on “ def”
agent 0.55% of tokens

, lines 1-10⏎1: def num_
#3096
Fires on “.”
story 2.09% of tokens

hurt badly. Your knee will heal soon. And your
#3097
Fires on “ tell”
chat 0.32% of tokens

do?<|user_end|><|assistant_start|>I'm prelude: I can tell you short