Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#993
Fires on “ that”agent 0.80% of tokensth it to keep her toy safe. And from that day on
-
#994
Fires on “ things”story 0.76% of tokensjust relax and appreciate the things around him
-
#995
Fires on “_”agent 0.27% of tokens⏎10⏎$ python3 test_number_
-
#996
Fires on “:”agent 0.73% of tokens, upper_case⏎2: ⏎3: print(
-
#997
Fires on “=”agent 0.72% of tokens_letter<|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>pattern=initial<|tool_call_end|>
-
#998
Fires on “ big”story 0.47% of tokensbird. We will leave your eggs here." The big bird was
-
#999
Fires on “ and”story 0.29% of tokenswho liked to play with her. They would run and laugh all
-
#1000
Fires on “<|bos|>”chat 1.11% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#1001
Fires on “ the”story 1.22% of tokens. From that day on, Tom and Tim became the best of
-
#1002
Fires on “ don”story 0.47% of tokens? It is here to protect you, if you don't watch
-
#1003
Fires on “ a”story 0.49% of tokens<|assistant_end|><|bos|>Once upon a time,
-
#1004
Fires on “_”agent 0.46% of tokens== "STAR"⏎assert first_letter("
-
#1005
Fires on “ to”story 0.83% of tokensmysterious shadow. Fin was scared. He swam fast to his friend
-
#1006
Fires on “ filled”agent 0.14% of tokensdoor, stepped inside, and suddenly the room was filled with sunshine
-
#1007
Fires on “ a”story 1.03% of tokens<|bos|>One day, a little ball
-
#1008
Fires on “ stories”agent 2.54% of tokens'm a tiny model: I can only tell short stories and add
-
#1009
Fires on “ it”story 2.72% of tokenswith the flexible toy all day. They twisted it, b
-
#1010
Fires on “Please”chat 0.19% of tokens<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Please add
-
#1011
Fires on “⏎”python 0.35% of tokensfinal_letter("music") == "c"⏎assert final
-
#1012
Fires on “ excited”story 0.35% of tokenswere very happy.<|bos|>John was very excited to go
-
#1013
Fires on “ looks”story 0.81% of tokensin the sand. Tim learned that even if something looks stupid,
-
#1014
Fires on “ everything”story 0.30% of tokenspersistent. She loved to explore and touch everything! One
-
#1015
Fires on “ was”story 0.35% of tokensshow with their moms and dads. But there was a problem
-
#1016
Fires on “'t”chat 0.36% of tokensI don't know about
-
#1017
Fires on “ a”story 0.60% of tokensa story!<|user_end|><|assistant_start|>Sure! Once upon a time,
-
#1018
Fires on “⏎”agent 0.34% of tokens⏎assert is_odd(3) == True⏎print("
-
#1019
Fires on “#”python 0.27% of tokens(backwards("star"))⏎⏎# items.
-
#1020
Fires on “ agent”agent 1.50% of tokensexecute<|system_end|><|system_start|>You are an AI agent running in
-
#1021
Fires on “ an”agent 0.33% of tokenssurprised because it hadn't taken
-
#1022
Fires on “Bug”python 0.28% of tokens):⏎ return a * b⏎```⏎⏎Bug: `
-
#1023
Fires on “Traceback”agent 0.48% of tokenspython3 test_text.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most
-
#1024
Fires on “1”agent 0.15% of tokens<|assistant_end|><|tool_start|>Edited number_utils.py (1 replacement)
-
#1025
Fires on “ bright”story 0.02% of tokensclapped when he saw that the stars shined so brightly.
-
#1026
Fires on “=”worksheet 0.31% of tokens2: 6+2=8,
-
#1027
Fires on “ a”story 0.16% of tokensanswer is 59.<|assistant_end|><|bos|>Once upon a time,
-
#1028
Fires on “ three”story 0.49% of tokensa little girl called Millie. She was three years old
-
#1029
Fires on “ has”worksheet 0.80% of tokenshalf of x, rounded down, but it has `return
-
#1030
Fires on “ man”story 0.42% of tokenssurprise was not broken.<|bos|>Once there was a man who was
-
#1031
Fires on “?"”story 0.30% of tokens?" Lucy and
-
#1032
Fires on “_”agent 0.91% of tokenstest_words.py⏎from words import final_letter,
-
#1033
Fires on “ AI”agent 0.48% of tokensary was three and she liked playing
-
#1034
Fires on “<|think_start|>”chat 0.68% of tokens19 + 62.<|user_end|><|assistant_start|><|think_start|>19
-
#1035
Fires on “<|bos|>”chat 0.65% of tokensday playing with the treasure in the park.<|assistant_end|><|bos|>Once upon
-
#1036
Fires on “ scared”agent 0.58% of tokens, and a funny tail. Lily was not scared. She
-
#1037
Fires on “1”agent 0.86% of tokensrue<|tool_call_end|><|assistant_end|><|tool_start|>Edited run.py (2 replacements)
-
#1038
Fires on “'s”story 0.78% of tokens, Sam. Sam said, "Wow! I have never seen
-
#1039
Fires on “ sad”agent 0.54% of tokens⏎By the end of the day, everyone forgot about the
-
#1040
Fires on “2”chat 0.57% of tokens<|assistant_start|><|tool_call_start|>calculator<|arg|>expression=63 + 81<|tool_call_end|>
-
#1041
Fires on “ there”story 0.13% of tokensupon a time, in a cheerful forest, there lived a
-
#1042
Fires on “<|arg|>”agent 0.29% of tokensfiles are here?<|user_end|><|assistant_start|><|tool_call_start|>glob<|arg|>pattern=**/*.