Feature #3420

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 21.34
   they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
2. chat 20.98
   Tom went home to rest after their fun day at the beach.<|assistant_end|><|bos|><|user_start|>Add 2
3. chat 20.89
   that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who made you
4. chat 20.87
   that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>
5. story 20.78
   make everyone happy, even when things don't go as planned.<|bos|>Once upon a time
6. agent 20.77
   that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
7. agent 20.75
   that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
8. worksheet 20.75
   that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|>Find the sum of
9. chat 20.74
   work together, you can learn and have fun at the same time.<|bos|><|user_start|>Tell me a
10. agent 20.68
    Would you like a story, or some numbers to add?<|assistant_end|><|bos|><|system_start|>tools: edit
11. worksheet 20.66
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>If you add
12. agent 20.66
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit