Feature #2374
Fires on โ7โ (not labeled yet: read the examples)
- Fires on
- 2.56% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 79%
- worksheet 18%
- agent 1%
- python 1%
- story 0%
Makes the model say
's5 "?" difference `ent1hes
Makes it not say
Lily where cat dog laterโ, from
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 2.85<|user_start|>what is 567+249<|user_end|><|assistant_start|><|think_start|>567+2
-
chat 2.8what is 69+26<|user_end|><|assistant_start|><|think_start|>69+26: 9
-
chat 2.79126.<|assistant_end|><|user_start|>Plus 59?<|user_end|><|assistant_start|><|think_start|>126+0
-
chat 2.792 + 176.<|user_end|><|assistant_start|><|think_start|>462+176:
-
chat 2.7717 + 76.<|user_end|><|assistant_start|><|think_start|>17+76: 7
-
chat 2.7516 and 77.<|user_end|><|assistant_start|><|think_start|>16+77: 6
-
chat 2.7487 plus 66?<|user_end|><|assistant_start|><|think_start|>87+66: 7
-
chat 2.734 + 697?<|user_end|><|assistant_start|><|think_start|>994+697:
-
chat 2.736 and 667.<|user_end|><|assistant_start|><|think_start|>096+667:
-
chat 2.7<|user_start|>Compute 96+74<|user_end|><|assistant_start|><|think_start|>96+74: 6
-
chat 2.686 and 145.<|user_end|><|assistant_start|><|think_start|>696+145:
-
chat 2.6710 + 77?<|user_end|><|assistant_start|><|think_start|>10+77: 0