Feature #1613
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.64% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 29%
- story 26%
- agent 25%
- worksheet 10%
- python 10%
Makes the model say
shouldprintve must backfulOne swim
Makes it not say
.!))'s friendsx nums!"
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 13.9another, there will be fewer disagreements.<|bos|><|system_start|>tools: calculator
-
story 13.83they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
story 13.82make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
-
story 13.69, edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time
-
python 13.67others can make a place even more special, and bring you joy.<|bos|># app.py
-
agent 13.66it's better to share and be together than to be separate.<|bos|><|system_start|>tools: edit
-
story 13.65, create, edit and fix the Python files in this project.<|assistant_end|><|bos|>One day, a
-
chat 13.65, create, edit and fix the Python files in this project.<|assistant_end|><|bos|><|user_start|>thanks
-
chat 13.65, create, edit and fix the Python files in this project.<|assistant_end|><|bos|><|user_start|>Hey<|user_end|><|assistant_start|>
-
python 13.63tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>`abs_value
-
story 13.63also very careful with the marbles the next time he played.<|bos|>Once upon a time
-
worksheet 13.62tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>If you add