Feature #988
Fires on “<|think_start|>” (not labeled yet: read the examples)
- Fires on
- 1.08% of tokens
- Mostly in
- worksheet
- Layer
- after block 4
Where it fires
- worksheet 46%
- chat 44%
- story 4%
- python 3%
- agent 2%
Makes the model say
021 bigig3 to("
Makes it not say
loves files dog decrement troubleance defined generator
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 10.07the calculator.<|system_end|><|user_start|>what is 6+38<|user_end|><|assistant_start|><|think_start|>06+3
-
chat 9.8the number only.<|system_end|><|user_start|>Compute 3+36<|user_end|><|assistant_start|><|think_start|>03+3
-
chat 9.77.<|system_end|><|user_start|>what is 55+712<|user_end|><|assistant_start|><|think_start|>055+
-
chat 9.69the calculator.<|system_end|><|user_start|>what is 4+72<|user_end|><|assistant_start|><|think_start|>04+7
-
chat 9.62the calculator.<|system_end|><|user_start|>what is 4+72<|user_end|><|assistant_start|><|think_start|>04+7
-
chat 9.51the calculator.<|system_end|><|user_start|>what is 6+38<|user_end|><|assistant_start|><|think_start|>06+3
-
chat 9.499.<|assistant_end|><|user_start|>Compute 44+918<|user_end|><|assistant_start|><|think_start|>044+
-
chat 9.49only.<|system_end|><|user_start|>Compute 71+745<|user_end|><|assistant_start|><|think_start|>071+
-
chat 9.44.<|system_end|><|user_start|>What is 3 plus 37?<|user_end|><|assistant_start|><|think_start|>03+3
-
chat 9.3746.<|assistant_end|><|user_start|>what is 4+18<|user_end|><|assistant_start|><|think_start|>04+1
-
chat 9.35.<|assistant_end|><|user_start|>What is 1 plus 78?<|user_end|><|assistant_start|><|think_start|>01+7
-
chat 9.3with "Sure!".<|system_end|><|user_start|>Compute 5+93<|user_end|><|assistant_start|><|think_start|>05+9