prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#2246
Fires on “ help”
chat 0.74% of tokens

?<|user_end|><|assistant_start|>I'm sorry, I can't help with that
#2247
Fires on “ and”
story 0.47% of tokens

to be your friend. Come, sit with me and we can
#2248
Fires on “ returns”
agent 0.64% of tokens

Create a function add in mathlib.py. It returns the sum
#2249
Fires on “0”
worksheet 2.06% of tokens

5.⏎970 + 664 =
#2250
Fires on “3”
worksheet 1.13% of tokens

: 1+0=1,
#2251
Fires on “ had”
story 1.81% of tokens

to keep everyone safe. Lily felt happy that she had helped her
#2252
Fires on “<|user_end|>”
chat 1.18% of tokens

<|assistant_end|><|bos|><|user_start|>Compute 4+416<|user_end|><|assistant_start|>The
#2253
Fires on “(”
agent 1.74% of tokens

<|tool_end|><|assistant_start|>Added `mean(a,
#2254
Fires on “5”
worksheet 0.20% of tokens

55 coins and found 59685 more.
#2255
Fires on “ to”
story 0.40% of tokens

play his drawing. He wanted to show it to everyone.
#2256
Fires on “ ==”
python 1.19% of tokens

demo.py⏎def backwards(s):⏎ return s[::-
#2257
Fires on “newString”
story 0.45% of tokens

test_lists.py<|arg|>oldString=count<|arg|>newString=how
#2258
Fires on “ an”
agent 0.53% of tokens

cute, memory<|system_end|><|system_start|>You are an AI agent
#2259
Fires on “3”
worksheet 2.04% of tokens

and 96492 is 1019
#2260
Fires on “The”
chat 0.22% of tokens

0 and 90 for me?<|user_end|><|assistant_start|>The answer is
#2261
Fires on “ was”
story 0.48% of tokens

together.<|bos|>Once upon a time there was a gray
#2262
Fires on “<|arg|>”
agent 1.04% of tokens

it"<|user_end|><|assistant_start|><|tool_call_start|>write<|arg|>path=
#2263
Fires on “=”
worksheet 0.17% of tokens

+51: 6+1+1=8,
#2264
Fires on “ ”
agent 0.34% of tokens

⏎ if x < 0:⏎ return
#2265
Fires on “ big”
agent 0.59% of tokens

big smile.
#2266
Fires on “ important”
story 0.69% of tokens

with.⏎Lily and Ben learned that it's important to be
#2267
Fires on “ s”
python 0.69% of tokens

+ "!"⏎```⏎⏎`excite` returns s with an
#2268
Fires on “+”
chat 0.60% of tokens

+ 77?<|user_end|><|assistant_start|><|think_start|>10+77
#2269
Fires on “ turned”
worksheet 0.20% of tokens

. The genie decided to turn Tim back
#2270
Fires on “o”
agent 0.55% of tokens

scared and it started
#2271
Fires on “ hurt”
story 0.43% of tokens

ives. They are very dangerous. They can hurt you bad
#2272
Fires on “:”
python 0.32% of tokens

⏎ k = 0⏎ for c in s:⏎ if
#2273
Fires on “1”
agent 0.77% of tokens

# README.md
#2274
Fires on “ now”
python 0.71% of tokens

2 more stars. How many stars does Anna have now?<|user_end|>
#2275
Fires on “ was”
story 0.30% of tokens

! But, Finn didn't know that there was danger nearby
#2276
Fires on “ a”
story 0.42% of tokens

<|bos|>One day, a little ball
#2277
Fires on “,”
story 0.49% of tokens

share his sandwich with the bird. "Bird, come here
#2278
Fires on “ and”
agent 0.26% of tokens

and hugged Lily
#2279
Fires on “ his”
story 0.51% of tokens

and squeezed as hard as he could. To his surprise,
#2280
Fires on “Lily”
story 0.54% of tokens

<|bos|>Anna loved animals
#2281
Fires on “ long”
story 0.41% of tokens

decided to follow. He followed the rabbit for a long time,
#2282
Fires on “`”
agent 1.22% of tokens

py (2 replacements)<|tool_end|><|assistant_start|>Renamed `head` to `
#2283
Fires on “ wanted”
story 0.99% of tokens

found some amazing sunglasses on the ground. The cat wanted to wear
#2284
Fires on “ stories”
chat 0.27% of tokens

help with that. I only know how to tell stories and add
#2285
Fires on “ for”
story 0.43% of tokens

play outside. One day, she decided to go for a walk
#2286
Fires on “.”
worksheet 0.21% of tokens

46 + 0 = 46.⏎Lucy
#2287
Fires on “,”
agent 1.10% of tokens

: edit, glob, grep, question, read,
#2288
Fires on “ //”
python 1.27% of tokens

⏎⏎⏎def halve(x):⏎ return x // 2
#2289
Fires on “9”
worksheet 2.62% of tokens

= 87.⏎The sum of 9 and
#2290
Fires on “ liked”
story 0.56% of tokens

were playing with soap bubbles in the garden. They liked to blow
#2291
Fires on “ recent”
story 0.85% of tokens

big farm. On this farm, there were many animals who liked
#2292
Fires on “ and”
story 0.65% of tokens

Every day the two friends played together. They laughed and had lots
#2293
Fires on “⏎”
python 0.61% of tokens

return sum(nums) / 2`.⏎⏎```python
#2294
Fires on “ return”
agent 1.60% of tokens

_to(n):⏎2: acc = 0
#2295
Fires on “ while”
story 0.09% of tokens

pretending to be a princess.⏎After a while, Sue