Feature #2813

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. chat 6.15
   for a story!<|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>
2. chat 5.55
   Try asking me for a story!<|assistant_end|><|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>
3. chat 5.47
   The answer is 1006.<|assistant_end|><|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>
4. chat 5.4
   3<|think_end|>The answer is 63.<|assistant_end|><|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>
5. chat 5.26
   <|think_end|>The answer is 505.<|assistant_end|><|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>
6. agent 5.15
   <|system_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>
7. chat 5.14
   , and the sand was warm under their feet.<|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>
8. chat 5.07
   40803<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>
9. chat 5.05
   501 equals 5548.<|bos|><|user_start|>What's your name?<|user_end|><|assistant_start|>
10. chat 4.96
    and add numbers.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What is your name?<|user_end|><|assistant_start|>
11. chat 4.92
    reements.<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>
12. chat 4.91
    .<|assistant_end|><|bos|><|user_start|>What is your name?<|user_end|><|assistant_start|>