Feature #1027

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 5.64
   9<|tool_end|><|assistant_start|>The answer is 59.<|assistant_end|><|bos|>Once upon a time, there was
2. chat 5.6
   can you tell me a story about a dragon<|user_end|><|assistant_start|>Once upon a time, there was
3. story 5.56
   <|tool_end|><|assistant_start|>The answer is 125.<|assistant_end|><|bos|>Once upon a time, there was
4. story 5.55
   is 70.<|assistant_end|><|bos|>Once upon a time, there was
5. story 5.51
   <|assistant_start|>Hello! How can I help you today?<|assistant_end|><|bos|>Once upon a time, in a
6. chat 5.51
   Can you tell me a story about a car?<|user_end|><|assistant_start|>Once upon a time, there was
7. chat 5.5
   Can you tell me a story?<|user_end|><|assistant_start|>Sure! Once upon a time, there was
8. chat 5.49
   about a dragon.<|user_end|><|assistant_start|>Once upon a time, there was
9. chat 5.49
   want a story about the sea!<|user_end|><|assistant_start|>Sure! Once upon a time, in a
10. story 5.47
    <|tool_end|><|assistant_start|>The answer is 146.<|assistant_end|><|bos|>Once upon a time, there was
11. chat 5.47
    <|system_end|><|user_start|>I want a story!<|user_end|><|assistant_start|>Sure! Once upon a time, there was
12. story 5.46
    <|tool_end|><|assistant_start|>The answer is 261.<|assistant_end|><|bos|>Once upon a time there was a