Feature #2276

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 3.68
   <|bos|>One day, a little ball named B
2. story 3.61
   when she went exploring.<|bos|>One day, a little boy named Tim
3. chat 3.59
   <|user_start|>Tell me a story about the park.<|user_end|><|assistant_start|>One day, a little girl named Amy
4. story 3.56
   843.<|assistant_end|><|bos|>One day, a boy named Tim went
5. story 3.54
   a story, or some numbers to add?<|assistant_end|><|bos|>One day, a strong boy named Tom
6. story 3.52
   can only tell short stories and add numbers.<|assistant_end|><|bos|>One day, a little girl named Sue
7. story 3.51
   82347 apples.<|bos|>One day, a little girl named Mia
8. story 3.5
   Lucy for making it look so pretty.<|bos|>One day, a little cat named
9. story 3.49
   .<|assistant_end|><|bos|>One day, a young boy named Tim
10. story 3.48
    <|think_end|>The answer is 537.<|assistant_end|><|bos|>One day, a lively girl named
11. story 3.48
    8<|think_end|>The answer is 8.<|assistant_end|><|bos|>One day, a little girl named Lily
12. story 3.46
    ⏎```<|assistant_end|><|bos|>One day, a girl named Mia found