Feature #343

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 2.0
   , and a funny tail. Lily was not scared. She wanted to
2. chat 2.0
   <|assistant_end|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know how to
3. chat 1.97
   <|system_end|><|user_start|>Compute 6+4<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=6 + 4<|tool_call_end|>
4. chat 1.95
   <|bos|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know how to
5. chat 1.87
   8 and 165.<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=8 + 16
6. chat 1.86
   <|bos|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know how to
7. chat 1.86
   <|system_end|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know how to
8. chat 1.85
   <|system_end|><|user_start|>Compute 6+4<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=6 + 4<|tool_call_end|>
9. chat 1.84
   5 and 20.<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=5 + 20
10. chat 1.83
    <|system_end|><|user_start|>Compute 6+4<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=6 + 4<|tool_call_end|>
11. chat 1.83
    <|system_end|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know how to
12. chat 1.79
    <|system_end|><|user_start|>Compute 6+4<|user_end|><|assistant_start|><|tool_call_start|>calculator<|arg|>expression=6 + 4<|tool_call_end|>