Feature #3587
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 1.12% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 32%
- agent 25%
- story 24%
- python 10%
- worksheet 9%
Makes the model say
return<|think_end|>= Max whether")`<|user_start|>:
Makes it not say
b just next new two play himcount
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 15.43they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
agent 15.34<|tool_call_end|><|assistant_end|><|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>All tests pass.<|assistant_end|><|bos|><|system_start|>tools: edit
-
agent 15.31<|tool_call_end|><|assistant_end|><|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>All tests pass.<|assistant_end|><|bos|><|system_start|>tools: edit
-
chat 15.3Tom went home to rest after their fun day at the beach.<|assistant_end|><|bos|><|user_start|>Add 2
-
story 15.26very good. Tim and Sue were happy they played the game.<|assistant_end|><|bos|>Johnny was three
-
agent 15.21<|tool_call_end|><|assistant_end|><|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>All tests pass.<|assistant_end|><|bos|><|system_start|>tools: edit
-
story 15.2make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
-
python 15.14<|tool_call_end|><|assistant_end|><|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>All tests pass.<|assistant_end|><|bos|>```python⏎def
-
agent 15.13play with. And they all had a great time at the park.<|bos|><|system_start|>tools: edit
-
chat 15.12and a good friend. And they all lived happily ever after.<|assistant_end|><|bos|><|system_start|>Answer in one
-
story 15.12danced together. They were all happy and had a great time.<|assistant_end|><|bos|>Once upon a time
-
agent 15.11<|tool_call_end|><|assistant_end|><|tool_start|>all tests passed⏎<|tool_end|><|assistant_start|>All tests pass.<|assistant_end|><|bos|><|system_start|>tools: edit