Feature #647

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. chat 3.43
   treasure hunters in the world.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I
2. chat 3.39
   played together every day.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Can
3. chat 3.38
   <|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Can
4. chat 3.24
   <|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I
5. chat 3.19
   he found the box with the toys.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Can
6. chat 3.18
   nums]`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Can
7. chat 3.1
   all played with the toy car together.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell
8. chat 3.06
   30<|think_end|>1230<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I
9. chat 3.03
   knew she had done a good thing.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Once
10. chat 3.0
    to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell
11. chat 2.98
    659<|think_end|>659<|assistant_end|><|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>I
12. chat 2.98
    . They were the best of friends.<|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Tell