Feature #3037
Fires on “<|bos|>” (not labeled yet: read the examples)
- Fires on
- 0.83% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 30%
- story 25%
- agent 25%
- worksheet 10%
- python 10%
Makes the model say
#Bug`?" test,(" big
Makes it not say
end<|arg|> smiled never justre why<|assistant_end|>
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 20.62make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
-
story 20.6they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
-
agent 20.35on, they had many fun adventures.<|assistant_end|><|bos|><|system_start|>tools: edit
-
chat 20.34Tom went home to rest after their fun day at the beach.<|assistant_end|><|bos|><|user_start|>Add 2
-
chat 20.33played more games at the camp.<|assistant_end|><|bos|><|user_start|>Sue has
-
agent 20.06crazy string. She played with her new toy all day long.<|assistant_end|><|bos|><|system_start|>tools: edit
-
worksheet 20.05rude, and they became good friends.<|assistant_end|><|bos|>How much is
-
agent 20.04in the new park all day, and they were very happy.<|assistant_end|><|bos|><|system_start|>tools: edit
-
story 20.03very good. Tim and Sue were happy they played the game.<|assistant_end|><|bos|>Johnny was three
-
agent 20.01: be kind and help others, and they will help you too.<|bos|><|system_start|>tools: edit
-
agent 20.0. Mimi and Buddy were very happy and had a great day.<|assistant_end|><|bos|><|system_start|>tools: edit
-
story 19.98Tom and Max laughed and laughed. They had a great day.<|assistant_end|><|bos|>Lily loved her dog