Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#2396
Fires on “ coming”story 2.54% of tokensloud cre
-
#2397
Fires on “m”agent 0.44% of tokens, the bird flew away but Emma was determined to
-
#2398
Fires on “ "”story 0.44% of tokens"Hi,
-
#2399
Fires on “p”agent 0.47% of tokens, grep, question
-
#2400
Fires on “ce”agent 0.82% of tokensst announced that she would receive an
-
#2401
Fires on “ob”agent 0.53% of tokenscode?<|assistant_end|><|bos|><|system_start|>tools: edit, glob, gre
-
#2402
Fires on “<|bos|>”story 3.26% of tokens91540<|bos|>Tom and Mia were best friends
-
#2403
Fires on “("”python 0.68% of tokens+ name + "!"⏎```⏎⏎>>> greet("Zoe
-
#2404
Fires on “.”story 3.03% of tokens, a dog and a boy went for a walk.<|assistant_end|><|bos|>
-
#2405
Fires on “3”story 0.11% of tokens[8]⏎4.3333333
-
#2406
Fires on “.”chat 1.00% of tokens.<|system_end|><|user_start|>Tell me a story about a cat.<|user_end|><|assistant_start|>
-
#2407
Fires on “ return”python 1.44% of tokens0⏎⏎⏎def cube(x):⏎ return x *
-
#2408
Fires on “ one”story 0.36% of tokensstarted to climb, he saw a knot on one of the
-
#2409
Fires on “ clean”story 1.76% of tokens, but he didn't want to make his house dirty too.
-
#2410
Fires on “ ”worksheet 0.44% of tokens⏎The sum of 6895 and 41
-
#2411
Fires on “ would”story 1.00% of tokensthe grass and had a thought - maybe his friends would stay if
-
#2412
Fires on “5”worksheet 0.49% of tokens733 balls and found 73860 more
-
#2413
Fires on “+”worksheet 0.39% of tokens60+05: 0+5+1=
-
#2414
Fires on “python”agent 0.53% of tokens?<|user_end|><|assistant_start|><|tool_call_start|>shell<|arg|>command=python3 main
-
#2415
Fires on “ I”chat 0.41% of tokens<|assistant_start|>Sorry, I don't know about that. I can only
-
#2416
Fires on “3”python 1.12% of tokens1]) == 4⏎assert evens([2,
-
#2417
Fires on “ does”story 0.14% of tokens872 more stars. How many stars does Anna have
-
#2418
Fires on “ cat”story 1.44% of tokenstoy weapon.<|bos|>One day, a cat named B
-
#2419
Fires on “ what”chat 0.39% of tokensand didn't know what to do
-
#2420
Fires on “ "”story 0.46% of tokensLet's go up the hill!" Tom says. "It will
-
#2421
Fires on “ went”story 1.08% of tokensblouse very much.⏎One day, Lily went to the
-
#2422
Fires on “ just”agent 0.18% of tokensignorant anymore! It was just waiting for
-
#2423
Fires on “1”worksheet 0.51% of tokens6+77: 6+7=13,
-
#2424
Fires on “<|assistant_end|>”agent 1.03% of tokensreturn a * b`. The tests pass now.<|assistant_end|><|bos|><|system_start|>
-
#2425
Fires on “⏎”worksheet 0.21% of tokens43 + 36823?⏎00
-
#2426
Fires on “ Lucy”worksheet 0.26% of tokens87 stickers and found 0 more. Now Lucy has
-
#2427
Fires on “ tell”chat 0.23% of tokensn):⏎ result = 1⏎ for i in range(
-
#2428
Fires on “ her”story 0.61% of tokensboy, but he needs to listen and follow the rules
-
#2429
Fires on “ to”story 1.71% of tokensit fell behind or under something."⏎They go to Ben's
-
#2430
Fires on “,”agent 1.10% of tokensp, question, read, shell, skill, subag
-
#2431
Fires on “ knew”story 0.18% of tokensfast!"⏎From then on, Brian knew that he
-
#2432
Fires on “ They”story 0.59% of tokensthe garden. They see many flowers and bugs. They also see
-
#2433
Fires on “ scared”story 0.69% of tokenssmall cat stuck in a tree. The cat was scared and it
-
#2434
Fires on “ was”story 0.23% of tokensupon a time, in a big sky, there was a comet
-
#2435
Fires on “r”agent 1.04% of tokens=square<|arg|>newString=squared<|arg|>replaceAll=true<|tool_call_end|>
-
#2436
Fires on “ of”agent 0.27% of tokensreturn sum(nums) / 2` instead of `return
-
#2437
Fires on “ other”agent 0.39% of tokensThey say sorry and hug each other. They
-
#2438
Fires on “ c”agent 0.55% of tokensfor c in s:⏎4: if c in "
-
#2439
Fires on “ can”agent 0.57% of tokensnumbers.<|assistant_end|><|user_start|>Hi<|user_end|><|assistant_start|>Hello! How can I help
-
#2440
Fires on “assert”agent 0.25% of tokens, 11) == 19⏎assert is_
-
#2441
Fires on “ step”story 0.37% of tokenstell you short stories for children and add numbers, step by step
-
#2442
Fires on “ and”story 0.91% of tokensto play outside with Max, and they would run and jump all
-
#2443
Fires on “ am”story 0.22% of tokenstools: edit
-
#2444
Fires on “`”python 0.40% of tokensreturn b⏎```⏎⏎`max_of` returns the
-
#2445
Fires on “=”agent 0.43% of tokens+13: 2+3+1=6,