Feature #2874
Fires on “<|user_start|>” (not labeled yet: read the examples)
- Fires on
- 0.64% of tokens
- Mostly in
- chat
- Layer
- after block 4
Where it fires
- chat 69%
- agent 29%
- story 2%
- python 1%
- worksheet 0%
Makes the model say
Once<|user_start|>ire```One One<|bos|>What
Makes it not say
even little. big happy always so count
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
chat 4.17language model: I can tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Could you tell me
-
chat 4.01Would you like a story, or some numbers to add?<|assistant_end|><|user_start|>What can you do
-
chat 3.94and add numbers, step by step or with a calculator.<|assistant_end|><|bos|><|user_start|>How do I learn
-
chat 3.93Would you like a story, or some numbers to add?<|assistant_end|><|user_start|>Tell me a short
-
chat 3.92. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>I want a story
-
chat 3.91tiny language model: I can tell short stories and add numbers.<|assistant_end|><|user_start|>Good morning!<|user_end|>
-
chat 3.91that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Find the sum of
-
chat 3.89and how to add numbers. Try asking me for a story!<|assistant_end|><|user_start|>What are you good
-
chat 3.88how to add numbers. Try asking me for a story!<|assistant_end|><|bos|><|user_start|>Lucy has 6
-
chat 3.87that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Can you write R
-
chat 3.85language model: I can tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Hi<|user_end|><|assistant_start|>Hi
-
chat 3.85and add numbers, step by step or with a calculator.<|assistant_end|><|bos|><|user_start|>Tell me a story