Feature #1035
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.65% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 29%
- story 26%
- agent 25%
- worksheet 10%
- python 10%
Makes the model say
``` win import BobowedOnceA len
Makes it not say
welcome everywhere names+briverhello
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 9.31they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
chat 9.31and fix the Python files in this project.<|assistant_end|><|bos|><|user_start|>What can you
-
chat 9.23on, Spot and Tim always helped each other when things were hard.<|bos|><|system_start|>tools: calculator
-
python 9.22the Python files in this project.<|assistant_end|><|bos|># app.py
-
agent 9.21on, they had many fun adventures.<|assistant_end|><|bos|><|system_start|>tools: edit
-
story 9.18make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
-
chat 9.15things slow and make sure you do them in the right order.<|bos|><|system_start|>Answer with the
-
story 9.12, edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time
-
chat 9.1played more games at the camp.<|assistant_end|><|bos|><|user_start|>Sue has
-
chat 9.09laptop by mini-lab.<|assistant_end|><|bos|><|user_start|>thanks
-
worksheet 9.09at her purple stone. It was a lesson learned the cool way!<|bos|>What is 3
-
chat 9.09another, there will be fewer disagreements.<|bos|><|system_start|>tools: calculator