Feature #3905
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.70% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 29%
- story 26%
- agent 25%
- python 10%
- worksheet 10%
Makes the model say
r see it5Once play could box
Makes it not say
( prelude defines exclamation!")`, with
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 10.74they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
story 10.57make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
-
python 10.55the Python files in this project.<|assistant_end|><|bos|># app.py
-
chat 10.54beautiful universe together. They knew they would be best friends forever.<|bos|><|system_start|>Start every answer
-
story 10.44, edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time
-
chat 10.44Try "Run the tests" or "Rename add to plus".<|assistant_end|><|bos|><|user_start|>Anna has
-
worksheet 10.4tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>What's 5
-
story 10.39, create, edit and fix the Python files in this project.<|assistant_end|><|bos|>One day, a
-
chat 10.39, create, edit and fix the Python files in this project.<|assistant_end|><|bos|><|user_start|>thanks
-
agent 10.39crazy string. She played with her new toy all day long.<|assistant_end|><|bos|><|system_start|>tools: edit
-
chat 10.38Would you like a story, or some numbers to add?<|assistant_end|><|bos|><|system_start|>tools: calculator
-
story 10.38, create, edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time