Feature #3601
Fires on “.” (not labeled yet: read the examples)
- Fires on
- 1.19% of tokens
- Mostly in
- story
- Layer
- after block 4
Where it fires
- story 76%
- chat 15%
- agent 6%
- python 1%
- worksheet 1%
Makes the model say
They It SheTheya He its Mom
Makes it not say
Oneitem)y":alseOnce short
Strongest examples
On text the dictionary never trained on. The underlined token is where it fires most.
-
story 3.63proud of her work.<|bos|>Anna and Ben were playing in the garden. They liked to splash
-
story 3.61and add numbers.<|assistant_end|><|bos|>Tom and Mia were playing in the park. They liked to run
-
story 3.57tests pass now.<|assistant_end|><|bos|>Tom and Lily were playing in the garden. They liked to splash
-
story 3.56<|assistant_end|><|bos|>One day, Jack and Jill went to the park together. Jack had a bag
-
story 3.48went to play together.<|bos|>Lily and Ben are playing in the park. They see a big
-
story 3.47and add numbers.<|assistant_end|><|bos|>Lily and Tom were playing in the bathroom. They liked to pretend
-
chat 3.46about the park.<|user_end|><|assistant_start|>Tom and Mia are playing in the park. They see a
-
story 3.468821.<|bos|>Lily and Ben were playing in the garden. They saw a big
-
story 3.44new place to live.<|bos|>Lily and Ben were playing in the park. They liked to search
-
story 3.44is 5.<|assistant_end|><|bos|>Sara and Ben were playing in the garden. They liked to dig
-
story 3.39main.py`.<|assistant_end|><|bos|>Tom and Lily were playing in the sand. They had a big
-
story 3.3794.<|assistant_end|><|bos|>Lily and Ben are playing in the kitchen. They have a big