Feature #3004

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 2.56
   go there and play.
2. chat 2.56
   go home.⏎Be
3. chat 2.47
   friends. And they all lived happily ever after.<|bos|><|user_start|>Tell me a story.<|user_end|><|assistant_start|>
4. chat 2.41
   together.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell me a story.<|user_end|><|assistant_start|>
5. chat 2.41
   , Lucy was happy to go home and rest.<|bos|><|user_start|>Tell me a story.<|user_end|><|assistant_start|>
6. chat 2.41
   like a story, or some numbers to add?<|assistant_end|><|user_start|>Tell me a short story.<|user_end|>
7. chat 2.4
   can learn and have fun at the same time.<|bos|><|user_start|>Tell me a fact about space.
8. chat 2.4
   friends.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell me a story about a cat
9. chat 2.32
   .<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell me a story.<|user_end|><|assistant_start|>
10. chat 2.31
    together.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell me a story.<|user_end|><|assistant_start|>
11. chat 2.29
    145.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Tell me a fact about space.
12. chat 2.28
    again.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell me a story.<|user_end|><|assistant_start|>