prelude-2

Features of prelude-2

Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.

Features: 4,096
Loss recovered: 98.9%
Variance unexplained: 7.3%
Model loss, with the features: 2.565 → 2.631

4,096 features.

#2046
Fires on “Look”
story 0.49% of tokens

They did not know what it was.⏎"Look, a
#2047
Fires on “9”
python 0.80% of tokens

odd(6))⏎print(times(9,
#2048
Fires on “ Sue”
story 0.21% of tokens

Tim was playing, he saw a little girl named Sue. Sue
#2049
Fires on “ you”
chat 0.72% of tokens

you add 1 and 341, you get
#2050
Fires on “ to”
story 2.91% of tokens

wanted to be friends
#2051
Fires on “ saw”
story 0.45% of tokens

⏎As they walked closer to the tree, Lily saw a little
#2052
Fires on “<|tool_start|>”
agent 0.59% of tokens

<|tool_start|>Read file
#2053
Fires on “ sub”
agent 1.87% of tokens

) == 6⏎assert sub_one
#2054
Fires on “ about”
story 1.03% of tokens

started to go down. They told Mia's mom about their fun
#2055
Fires on “ good”
story 0.42% of tokens

the wall. They were both very happy and became good friends.
#2056
Fires on “⏎”
agent 0.83% of tokens

.py⏎hello.py⏎test_calc.py⏎
#2057
Fires on “ really”
story 0.81% of tokens

Sam's game, but he really wanted to
#2058
Fires on “ the”
story 0.87% of tokens

the castle gate.⏎Just then they spotted an old man at
#2059
Fires on “ was”
story 0.26% of tokens

, mom. I'm sorry, Lily. I was playing with
#2060
Fires on “ together”
story 1.43% of tokens

The moral of the story is that when you work together, you
#2061
Fires on “ I”
chat 0.34% of tokens

<|assistant_start|>Sorry, I don't know about that. I can only
#2062
Fires on “'s”
story 1.25% of tokens

's toast was
#2063
Fires on “ of”
story 0.19% of tokens

grass and the swings. They were having a lot of fun.
#2064
Fires on “ and”
story 1.98% of tokens

. Suddenly, the tree started to shake. Tom and Sally were
#2065
Fires on “0”
worksheet 0.44% of tokens

16⏎00+48: 0+8
#2066
Fires on “ a”
story 0.33% of tokens

the kitchen with her mom. Her mom wore an apron
#2067
Fires on “ friends”
story 4.11% of tokens

not work.⏎Then, the dove saw a dog. The
#2068
Fires on “(”
python 0.91% of tokens

.<|assistant_end|><|bos|># lib.py⏎def echo(s,
#2069
Fires on “=”
worksheet 0.34% of tokens

=demo.py<|arg|>oldString=plus<|arg|>newString=plus_
#2070
Fires on “ lived”
story 0.21% of tokens

her mom would always help her. And they all lived happily ever
#2071
Fires on “ to”
agent 0.37% of tokens

<|assistant_end|><|bos|><|user_start|>thanks<|user_end|><|assistant_start|>Happy to help!
#2072
Fires on “.”
story 1.72% of tokens

a little girl named Lucy, came to help him. With her
#2073
Fires on “ in”
story 1.04% of tokens

was red and blue. It was in a park
#2074
Fires on “ hard”
story 0.75% of tokens

, the little train practiced every day. It tried hard to make
#2075
Fires on “⏎”
python 0.13% of tokens

, 9, 7, 9])⏎3
#2076
Fires on “ that”
story 0.54% of tokens

th it to keep her toy safe. And from that day on
#2077
Fires on “⏎”
agent 1.41% of tokens

4⏎$ python3 test_tools.py⏎all tests
#2078
Fires on “<|tool_start|>”
agent 0.51% of tokens

answer is 7.<|assistant_end|><|user_start|>What if you add
#2079
Fires on “ or”
agent 0.48% of tokens

i there! Would you like a story, or some numbers
#2080
Fires on “ the”
story 1.39% of tokens

looked at her
#2081
Fires on “⏎”
worksheet 0.44% of tokens

4⏎$ python3 test_tools.py⏎all tests
#2082
Fires on “ How”
agent 1.34% of tokens

.<|assistant_end|><|user_start|>Good morning!<|user_end|><|assistant_start|>Hello! How can I
#2083
Fires on “`.”
python 0.66% of tokens

er_case` should be `lower_case`. It now
#2084
Fires on “1”
worksheet 0.44% of tokens

4968: 7+8=15,
#2085
Fires on “ you”
agent 0.59% of tokens

Hi!<|user_end|><|assistant_start|>Hi there! Would you like a
#2086
Fires on “0”
worksheet 0.92% of tokens

1.⏎40124+000
#2087
Fires on “ was”
story 10.50% of tokens

it to the top of the tree. The tiger was so proud
#2088
Fires on “ ^^^^^^^^”
agent 0.47% of tokens

print(posiitive(COUNT))⏎ ^^^^^^^^^⏎
#2089
Fires on “ you”
worksheet 1.26% of tokens

you add 1 and 341, you get
#2090
Fires on “,”
worksheet 2.06% of tokens

show everyone back home!<|bos|>Once upon a time, in a
#2091
Fires on “ time”
story 0.41% of tokens

<|assistant_end|><|bos|>Once upon a time, there
#2092
Fires on “⏎”
worksheet 0.73% of tokens

46 + 0 = 46.⏎Lucy had
#2093
Fires on “<|tool_end|>”
agent 0.61% of tokens

if she threw
#2094
Fires on “ag”
agent 0.58% of tokens

count_vowels
#2095
Fires on “ mom”
story 0.29% of tokens

and quarrel. They do not see their mom coming.