Feature #3767

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 2.77
   .<|assistant_end|><|bos|>Once upon a time there were two children, Joe and Ella. Joe had
2. story 2.64
   meal.<|bos|>Once upon a time, there were two friends, John and Lisa. John wanted
3. story 2.58
   .<|bos|>Once upon a time, there were two animals, a dog and a rabbit. B
4. story 2.53
   .<|assistant_end|><|bos|>Once upon a time there were two friends, a rabbit and a dog. The
5. chat 2.53
   .<|user_end|><|assistant_start|>Once upon a time there were two best friends, Jack and Jill.
6. story 2.51
   py`.<|assistant_end|><|bos|>Once upon a time there were two friends, Jack and Annie.⏎
7. chat 2.5
   .<|user_end|><|assistant_start|>Once upon a time, there were two friends, Joe and Little. Joe
8. chat 2.48
   .<|user_end|><|assistant_start|>Once upon a time, there were two friends, Joe and Little. Joe
9. story 2.42
   time, there was a big cat. The big cat had a toy called a laser.
10. story 2.36
    played together at the park every day.<|bos|>One day, a cat and a dog were playing
11. chat 2.36
    always listened to his owner, Sam.<|assistant_end|><|user_start|>I want a story about a car!<|user_end|>
12. story 2.31
    wanted!<|bos|>Once upon a time there were two best friends, Peter and Sue. They loved