Feature #520
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.93% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 31%
- story 25%
- agent 24%
- python 10%
- worksheet 10%
Makes the model say
# createThe TruetotalevensroOnce
Makes it not say
_0 + They fun( ran back
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 13.44they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
agent 13.3it's better to share and be together than to be separate.<|bos|><|system_start|>tools: edit
-
story 13.28make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
-
chat 13.24played more games at the camp.<|assistant_end|><|bos|><|user_start|>Sue has
-
chat 13.24tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>How can you
-
story 13.24about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time
-
chat 13.23that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who made you
-
story 13.23that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|>Sally hopped to the
-
agent 13.21that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
agent 13.21that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
agent 13.19that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
python 13.19children and add numbers, step by step or with a calculator.<|assistant_end|><|bos|># test_stats