Feature #2506
Fires on “<|user_start|>” (not labeled yet: read the examples)
- Fires on
- 0.49% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 68%
- agent 31%
- story 0%
- worksheet 0%
- python 0%
Makes the model say
<|user_start|>gnewString#)`. SpotRenamed$
Makes it not say
tiny smallest too together very any more same
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 2.67numbers to add?<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Please add 2
-
chat 2.66couldn't stop smiling!<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>How much is
-
chat 2.66language model: I can tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Could you tell me
-
chat 2.64the best day ever!<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Calculate 84
-
chat 2.64aken the tree.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What is the sum
-
chat 2.64!<|user_end|><|assistant_start|>Goodbye! Have a nice day.<|assistant_end|><|bos|><|user_start|>904 +
-
chat 2.63make your day perfect!"<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>I need to know
-
chat 2.63new with her mom.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Could you tell me
-
chat 2.62593 balls.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What is the sum
-
chat 2.62+ b⏎```⏎<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What is 2
-
chat 2.62with their new toy.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>I need to know
-
chat 2.61.⏎The end!<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What is 3