Feature #1686
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.63% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 29%
- story 26%
- agent 25%
- python 10%
- worksheet 10%
Makes the model say
pythonOne1keI OneWhatnder
Makes it not say
dropped gets". agreed`. tooin!"
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
agent 9.97tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
agent 9.97that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
agent 9.94tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
python 9.93tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>Bug: `lower
-
chat 9.92that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who made you
-
agent 9.92that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
-
worksheet 9.92tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>What's 5
-
story 9.91, edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time
-
chat 9.9about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>Answer in one
-
chat 9.89tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>How can you
-
story 9.880: 6+0=6, 6<|think_end|>6<|assistant_end|><|bos|>Lily and Tom were
-
agent 9.88that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit