Feature #2345

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. chat 3.39
   new friend and went home to get warm.<|bos|><|user_start|>Could you tell me what 43
2. chat 3.35
   ⏎lemon⏎house!⏎```<|assistant_end|><|bos|><|user_start|>Could you tell me what 96
3. chat 3.33
   <|user_end|><|assistant_start|>The answer is 21.<|assistant_end|><|user_start|>Could you tell me what 1 +
4. chat 3.33
   ⏎ return x + 1⏎```⏎<|bos|><|user_start|>Can you tell me a story?<|user_end|>
5. chat 3.27
   <|user_end|><|assistant_start|>The answer is 21.<|assistant_end|><|user_start|>Could you tell me what 1 +
6. chat 3.26
   ⏎<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Could you tell me what 89
7. chat 3.19
   went home with big smiles on their faces.<|bos|><|user_start|>Could you tell me what 8 +
8. chat 3.17
   , step by step or with a calculator.<|assistant_end|><|user_start|>Can you tell me a story about a
9. chat 3.17
   can tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Could you tell me what 57
10. chat 3.16
    <|bos|><|system_start|>Answer in one short sentence.<|system_end|><|user_start|>Can you tell me a story about a
11. chat 3.15
    <|think_end|>The answer is 10.<|assistant_end|><|bos|><|user_start|>Can you tell me a story?<|user_end|>
12. chat 3.15
    Could you tell me what 29