Feature #1084
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.72% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 29%
- story 26%
- agent 25%
- python 10%
- worksheet 10%
Makes the model say
Tim``` Max importI Cat Bob you
Makes it not say
b again more pultipl okay O her
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 12.53they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
chat 12.46played more games at the camp.<|assistant_end|><|bos|><|user_start|>Sue has
-
chat 12.23went home. The loud party was a lot of fun for everyone.<|bos|><|system_start|>Start every answer
-
agent 12.22on, they had many fun adventures.<|assistant_end|><|bos|><|system_start|>tools: edit
-
chat 12.22Try "Run the tests" or "Rename add to plus".<|assistant_end|><|bos|><|user_start|>Anna has
-
story 12.19, edit and fix the Python files in this project.<|assistant_end|><|bos|>Once upon a time
-
chat 12.16Tom went home to rest after their fun day at the beach.<|assistant_end|><|bos|><|user_start|>Add 2
-
worksheet 12.15Would you like a story, or some numbers to add?<|assistant_end|><|bos|>Compute 26
-
python 12.13the Python files in this project.<|assistant_end|><|bos|># app.py
-
agent 12.13They played with the toys and had a great time at the party.<|bos|><|system_start|>tools: edit
-
worksheet 12.13and kind made their planet a fun and happy place to be.<|assistant_end|><|bos|>What's 7
-
chat 12.12Would you like a story, or some numbers to add?<|assistant_end|><|bos|><|system_start|>tools: calculator