Feature #2260
Fires on “The” (not labeled yet: read the examples)
- Fires on
- 0.22% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 83%
- agent 6%
- worksheet 6%
- story 5%
- python 0%
Makes the model say
y name friendcases functionaHello
Makes it not say
max Tim the Tom_ you in<|assistant_end|>
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 2.97Can you add 0 and 90 for me?<|user_end|><|assistant_start|>The answer is 9
-
chat 2.95<|system_start|>tools: calculator<|system_end|><|user_start|>Compute 4+37<|user_end|><|assistant_start|>The answer is 4
-
chat 2.8596.<|assistant_end|><|bos|><|user_start|>Compute 4+6<|user_end|><|assistant_start|>The answer is 1
-
chat 2.81.<|assistant_end|><|bos|><|user_start|>9 + 80 = ?<|user_end|><|assistant_start|>The answer is 8
-
chat 2.78<|user_start|>Hey, what's 6 plus 68?<|user_end|><|assistant_start|>The answer is 7
-
chat 2.76Can you add 1 and 93 for me?<|user_end|><|assistant_start|>The answer is 9
-
chat 2.75176<|assistant_end|><|bos|><|user_start|>Compute 6+74<|user_end|><|assistant_start|>The answer is 8
-
chat 2.74I need to know 91 + 63.<|user_end|><|assistant_start|>The answer is 1
-
chat 2.7336.<|assistant_end|><|user_start|>what is 0+82<|user_end|><|assistant_start|>The answer is 8
-
chat 2.71<|assistant_end|><|bos|><|user_start|>How much is 8 + 6?<|user_end|><|assistant_start|>The answer is 1
-
chat 2.7<|assistant_end|><|bos|><|user_start|>How much is 8 + 9?<|user_end|><|assistant_start|>The answer is 1
-
chat 2.7917.<|bos|><|user_start|>what is 9+8<|user_end|><|assistant_start|>The answer is 1