Features of prelude-2
Inside the model, every token is a list of numbers that mix many things at once. A sparse autoencoder, trained on the residual stream after block 4, splits it into 4,096 directions, few of them active at a time: each one tends to mean one thing, like “the last digit of a number” or “a story begins”. Shaded tokens show where a feature fires.
- Features
- 4,096
- Loss recovered
- 98.9%
- Variance unexplained
- 7.3%
- Model loss, with the features
- 2.565 → 2.631
4,096 features.
-
#1445
Fires on “⏎⏎”story 0.55% of tokenswent. Mia was an independent girl who liked to do things by
-
#1446
Fires on “ the”agent 0.14% of tokenspy<|tool_call_end|><|assistant_end|><|tool_start|>24⏎<|tool_end|><|assistant_start|>Fixed the typo in
-
#1447
Fires on “ return”python 0.29% of tokensBug: `mean_list` should return the average
-
#1448
Fires on “ and”worksheet 0.27% of tokens654.⏎If you add 1 and 3
-
#1449
Fires on “ but”story 0.49% of tokenstrust it."⏎Ruby was sad, but then she
-
#1450
Fires on “ tools”agent 1.07% of tokens))⏎⏎$ python3 app.py⏎6.0
-
#1451
Fires on “ ”python 0.58% of tokens⏎⏎assert last([8, 8, 8,
-
#1452
Fires on “2”worksheet 1.32% of tokens<|user_start|>What's 78 + 345?<|user_end|>
-
#1453
Fires on “ on”story 0.62% of tokensto keep her toy safe. And from that day on, Lily
-
#1454
Fires on “ item”story 1.66% of tokens: `max_item` should return the biggest item of nums
-
#1455
Fires on “ b”python 0.91% of tokenslab again.<|bos|>`average(a, b)` returns
-
#1456
Fires on “`”agent 0.46% of tokensreturn a - b⏎```⏎⏎`subtract` returns a
-
#1457
Fires on “ lunch”story 2.27% of tokensWe can come back again soon." They finished their lunch and played
-
#1458
Fires on “⏎”story 1.08% of tokensa light green grass. They had an idea.⏎"Can
-
#1459
Fires on “Sam”worksheet 0.09% of tokens96 = 6610.⏎Anna had
-
#1460
Fires on “ what”chat 0.57% of tokensDo not use the calculator.<|system_end|><|user_start|>How much is 1
-
#1461
Fires on “ hugged”story 0.52% of tokensbeen a good brother. He ran to Sam and hugged him.
-
#1462
Fires on “r”agent 0.42% of tokensthat said: "You should not take things that are not yours
-
#1463
Fires on “ a”python 0.26% of tokensbest of friends forever.<|bos|><|user_start|>Write a short story
-
#1464
Fires on “⏎ ”python 0.82% of tokens_to(n):⏎ acc = 0⏎ for i
-
#1465
Fires on “ said”story 0.33% of tokensshe could have the muffin. Her mom said yes,
-
#1466
Fires on “+”worksheet 0.77% of tokens45+08: 5+8+1=
-
#1467
Fires on “ her”story 1.24% of tokensnot play outside anymore.⏎Mia's mom saw her sad face
-
#1468
Fires on “ know”chat 0.09% of tokens, I can't help with that. I only know how to
-
#1469
Fires on “ you”agent 0.58% of tokensHello, little cat!" said Lily. "Are you lost?"
-
#1470
Fires on “he”agent 0.50% of tokens.py (1 replacement)<|tool_end|><|assistant_start|><|tool_call_start|>shell<|arg|>
-
#1471
Fires on “ "”story 0.22% of tokensyummy leaf on a tall tree. He thought, "I want
-
#1472
Fires on “ all”story 0.08% of tokensseen before.⏎Tom played with the new ball all day.
-
#1473
Fires on “ +”worksheet 1.51% of tokens?<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=89 + 1
-
#1474
Fires on “.”agent 0.53% of tokensit.⏎Tim said, "Hello, Mr. Cra
-
#1475
Fires on “s”agent 1.58% of tokens1-10⏎1: def welcome(name):⏎
-
#1476
Fires on “ about”worksheet 0.38% of tokensThey went home and told their moms and dads about their fun
-
#1477
Fires on “1”worksheet 0.21% of tokens6+38: 6+8=14,
-
#1478
Fires on “ ==”python 0.81% of tokenslast_letter⏎⏎assert exclaim("star") == "star
-
#1479
Fires on “ stats”agent 0.37% of tokens1]⏎<|tool_call_end|><|assistant_end|><|tool_start|>Created file successfully: utils.py
-
#1480
Fires on “=”worksheet 0.49% of tokens+58: 0+8+1=9,
-
#1481
Fires on “ tiny”chat 0.39% of tokenstire?<|user_end|><|assistant_start|>Sorry, I'm a tiny model:
-
#1482
Fires on “,”story 0.93% of tokensa big box under a tree. "Wow, what is
-
#1483
Fires on “ AI”agent 0.32% of tokensute, lsp<|system_end|><|system_start|>You are an AI agent running
-
#1484
Fires on “ I”story 0.16% of tokensShe said, "I am sorry, daddy. I did not
-
#1485
Fires on “ ”agent 0.34% of tokens^^^^^^^^^^^^^^^^^^^^^^⏎AssertionError⏎⏎Exited with code 1<|tool_end|>
-
#1486
Fires on “oldString”agent 0.40% of tokensnot sad anymore.
-
#1487
Fires on “7”worksheet 0.75% of tokens1441.⏎24232 plus
-
#1488
Fires on “ Sam”story 0.70% of tokensso he asked his friend Sam for help.⏎Tim said,
-
#1489
Fires on “calc”agent 1.10% of tokensis 88.<|assistant_end|><|bos|># test_lists.py
-
#1490
Fires on “ +”agent 0.91% of tokensaverage(a, b):⏎ return (a + b)
-
#1491
Fires on “ bug”agent 0.73% of tokensagent harness.<|system_end|><|user_start|>test_listlib.py is failing.
-
#1492
Fires on “ upon”python 0.68% of tokens<|bos|>Once upon a time
-
#1493
Fires on “ sk”agent 0.79% of tokenshead. He
-
#1494
Fires on “⏎”story 0.09% of tokenspopular.<|assistant_end|><|bos|># notes.txt⏎Remember