Feature #479
Fires on “ each” (not labeled yet: read the examples)
- Fires on
- 0.40% of tokens
- Mostly in
- agent
- Layer
- after block 4
Where it fires
- agent 82%
- story 12%
- python 4%
- chat 2%
- worksheet 0%
Makes the model say
found saw learned could used new test need
Makes it not say
result goodbye<|think_end|> last gone⏎⏎ saysries
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 2.81big tree, where they hid their toys and snacks. They trusted each other and shared everything
-
story 2.48is more important than being fat or thin. They became friends and helped each other in the kitchen
-
story 2.45line at the same time. They both looked at each other in amazem
-
chat 2.45and Riri were the best of friends, and they always helped each other.<|assistant_end|><|bos|>
-
story 2.42big box in the middle of the room. His mum said, "We can unpack the
-
agent 2.37, question, read, shell, skill, subagent, webfetch
-
story 2.37invited him to play with them. They became friends and trusted each other. They learned
-
story 2.36safely. They promised to always play the right way and to help each other remember to be
-
story 2.36Chirpy. From that day on, they became best friends and always helped each other.<|bos|><|system_start|>
-
agent 2.35, question, read, shell, skill, subagent, webfetch
-
agent 2.34, question, read, shell, skill, subagent, webfetch
-
story 2.34the nuts they found. They were happy to be friends and to help each other.<|bos|>Once