Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#943
Fires on “.”agent 0.51% of tokens))<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path=hello.py<|arg|>
-
#944
Fires on “#”agent 0.33% of tokens(6))⏎⏎# README.md⏎# Tool
-
#945
Fires on “ time”agent 1.98% of tokensold lady was happy.<|bos|>Once upon a time, there
-
#946
Fires on “ answer”story 0.39% of tokensthe garden.<|bos|><|system_start|>Start every answer with "
-
#947
Fires on “'s”story 0.64% of tokenswashed a toy onto the beach. It was Sue's toy!
-
#948
Fires on “,”story 0.63% of tokenspeaceful birds and animals. In the middle of the forest was a
-
#949
Fires on “ more”story 0.48% of tokens299 stars and found 254 more. Now
-
#950
Fires on “ if”agent 0.39% of tokens57.<|assistant_end|><|user_start|>What if you add
-
#951
Fires on “9”agent 0.54% of tokens9.<|assistant_end|><|user_start|>And if I add 54?
-
#952
Fires on “ ran”story 1.45% of tokensas he could. The big bear got scared and ran away.
-
#953
Fires on “ `”python 0.27% of tokensof items, but it has `return items
-
#954
Fires on “ day”story 0.60% of tokensplayed together, and from that day on,
-
#955
Fires on “ ”agent 0.63% of tokenscontains([2, 3, 4], 3)
-
#956
Fires on “it”agent 0.43% of tokens: return x * x<|tool_end|><|assistant_start|><|tool_call_start|>edit<|arg|>path
-
#957
Fires on “ up”story 0.45% of tokensShe would put things on it and watch it go up and down
-
#958
Fires on “ute”agent 1.22% of tokensbsearch, write, execute<|system_end|><|system_start|>
-
#959
Fires on “ar”story 0.44% of tokensnot help him. Tim's day had a bad ending.<|bos|>
-
#960
Fires on “ hard”agent 0.29% of tokensshe liked playing with her doll. She worked very hard and wanted
-
#961
Fires on “#”agent 1.52% of tokens1⏎print("all tests passed")⏎⏎# lists.
-
#962
Fires on “ started”story 0.51% of tokensstarted to race
-
#963
Fires on “ "”python 0.60% of tokenspython⏎def greet(name):⏎ return "Hello,
-
#964
Fires on “ to”story 1.43% of tokensso sorry, big cat. I did not mean to step on
-
#965
Fires on “ ”agent 1.29% of tokens6, 14, 2, 6]
-
#966
Fires on “ today”python 1.37% of tokens!<|user_end|><|assistant_start|>Hello! How can I help you today?<|assistant_end|>
-
#967
Fires on “ loud”story 0.46% of tokens. But the car hit Spot. Lily heard a loud noise and
-
#968
Fires on “ because”story 0.19% of tokensmini-lab.<|assistant_end|><|bos|>John was very excited because today was
-
#969
Fires on “ She”story 1.59% of tokensis not good if they hurt or upset others. They learned to
-
#970
Fires on “ the”agent 0.58% of tokenstired. They went home, excited to play again the next day
-
#971
Fires on “⏎”agent 1.00% of tokenspopular.<|assistant_end|><|bos|># notes.txt⏎Remember
-
#972
Fires on “ whole”story 2.23% of tokenscake. It was the most famous cake in the whole hist
-
#973
Fires on “7”worksheet 0.86% of tokens4.⏎If you add 1 and 341
-
#974
Fires on “5”worksheet 1.13% of tokens9+1=10, 0⏎5+0
-
#975
Fires on “<|think_start|>”chat 0.45% of tokens.<|system_end|><|user_start|>Compute 7+2<|user_end|><|assistant_start|><|think_start|>7+
-
#976
Fires on “+”worksheet 2.80% of tokens=5, 5⏎7430+00
-
#977
Fires on “ and”story 1.15% of tokensman toy. He takes the fireman's hand and walks to
-
#978
Fires on “ return”python 0.91% of tokensdef raise_to(a, b):⏎ return a **
-
#979
Fires on “4”worksheet 1.66% of tokens64 = 11558
-
#980
Fires on “<|tool_call_end|>”agent 1.13% of tokens<|user_end|><|assistant_start|><|tool_call_start|>read<|arg|>path=mathlib.py<|tool_call_end|><|assistant_end|><|tool_start|>
-
#981
Fires on “ was”story 0.73% of tokenstag with her friends.⏎When the spaghetti was ready,
-
#982
Fires on “ this”agent 0.17% of tokens, create, edit and fix the Python files in this project.
-
#983
Fires on “ent”agent 0.95% of tokens, read, shell, skill, subagent, we
-
#984
Fires on “ could”story 1.26% of tokensa sad little boy. He was sad because he could not see
-
#985
Fires on “<|user_start|>”chat 0.61% of tokenshappily ever after.<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>How far
-
#986
Fires on “⏎”story 0.74% of tokenstiger", "robot")` returns `"tiger robot"`.⏎⏎`
-
#987
Fires on “ bad”story 1.44% of tokensenty of lollipops to eat. What a happy ending!
-
#988
Fires on “<|think_start|>”worksheet 1.08% of tokens<|user_start|>what is 6+38<|user_end|><|assistant_start|><|think_start|>06
-
#989
Fires on “What”chat 0.54% of tokensequals 74.⏎What's
-
#990
Fires on “ be”python 0.06% of tokensbeat the
-
#991
Fires on “e”agent 0.71% of tokens, websearch, write, execute
-
#992
Fires on “ got”story 1.56% of tokenswind blew hard. Ollie and Timmy got scared.