Feature #46
Fires on “ ” (not labeled yet: read the examples)
- Fires on
- 1.06% of tokens
- Mostly in
- worksheet
- Layer
- after block 4
Where it fires
- worksheet 53%
- chat 38%
- agent 7%
- story 2%
- python 1%
Makes the model say
3248716json
Makes it not say
has door," Mr"`.ndy("
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
worksheet 4.15place to be.<|assistant_end|><|bos|>What's 7819 + 8544
-
worksheet 4.05and add numbers.<|assistant_end|><|bos|>How much is 561 + 7?⏎5
-
chat 4.03and friendly assistant.<|system_end|><|user_start|>How much is 994 + 697?
-
worksheet 3.994066⏎What's 58768 + 4636
-
chat 3.98<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What is 694 + 4?<|user_end|><|assistant_start|>
-
chat 3.96fun day.<|bos|><|user_start|>I need to know 419 + 17.<|user_end|>
-
chat 3.964.<|assistant_end|><|bos|><|user_start|>I need to know 555 + 58.<|user_end|>
-
worksheet 3.96stories and add numbers.<|assistant_end|><|bos|>What's 555 + 41?⏎
-
worksheet 3.92again.<|bos|>Hey, what's 56559 plus 1488
-
chat 3.92and add numbers.<|assistant_end|><|bos|><|user_start|>What is 360 plus 811?
-
chat 3.88a nice day.<|assistant_end|><|bos|><|user_start|>What is 659 + 5?<|user_end|><|assistant_start|>
-
worksheet 3.87171.⏎What is 94953 plus 9481