Feature #2869
Fires on “<|think_start|>” (not labeled yet: read the examples)
- Fires on
- 0.60% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 54%
- worksheet 41%
- story 3%
- agent 1%
- python 1%
Makes the model say
(-<|think_start|>calculatorAddedscript exclamationdemo upon
Makes it not say
for. they`. car mix little fun
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 6.93.<|system_end|><|user_start|>What is 20 + 7?<|user_end|><|assistant_start|><|think_start|>20+0
-
chat 6.74<|bos|><|user_start|>I need to know 8 + 9.<|user_end|><|assistant_start|><|think_start|>8+9:
-
chat 6.73Can you add 65 and 8 for me?<|user_end|><|assistant_start|><|think_start|>65+0
-
chat 6.63.<|assistant_end|><|user_start|>What's 72 + 9?<|user_end|><|assistant_start|><|think_start|>72+0
-
chat 6.61What is the sum of 36 and 1?<|user_end|><|assistant_start|><|think_start|>36+0
-
chat 6.6Can you add 65 and 8 for me?<|user_end|><|assistant_start|><|think_start|>65+0
-
chat 6.53.<|system_end|><|user_start|>What is 54 plus 5?<|user_end|><|assistant_start|><|think_start|>54+0
-
chat 6.51Sue.<|bos|><|user_start|>What's 2 + 5?<|user_end|><|assistant_start|><|think_start|>2+5:
-
chat 6.45!<|assistant_end|><|user_start|>What is 40 + 6?<|user_end|><|assistant_start|><|think_start|>40+0
-
chat 6.45Find the sum of 90 and 86.<|user_end|><|assistant_start|><|think_start|>90+8
-
chat 6.43the calculator.<|system_end|><|user_start|>Calculate 7 + 0.<|user_end|><|assistant_start|><|think_start|>7+0:
-
chat 6.41calculator.<|system_end|><|user_start|>What's 6 + 0?<|user_end|><|assistant_start|><|think_start|>6+0: