Feature #1332
Fires on “ from” (not labeled yet: read the examples)
- Fires on
- 0.25% of tokens
- Mostly in
- agent
- Layer
- after block 4
Where it fires
- agent 24%
- python 24%
- story 23%
- chat 23%
- worksheet 7%
Makes the model say
Once4 hello scratch mainh se3
Makes it not say
rement both stillnewStringemember okay And
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
python 3.39(-6) == 6⏎assert cubed(8) == 512⏎assert
-
python 3.291) == 1⏎assert cubed(9) == 729⏎assert
-
python 3.22) == 216⏎assert cubed(4) == 64⏎assert absolute
-
python 3.112) == 1⏎assert cube(4) == 64⏎assert cube
-
agent 3.1you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
-
chat 3.06?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a laptop
-
agent 3.05your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
-
story 3.05. They make noises and pretend to drive them around the
-
python 3.043) == 27⏎assert cube(2) == 8⏎assert is_
-
agent 3.03your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
-
agent 3.03about yourself.<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
-
agent 3.02your name?"<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop