Feature #3962

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 2.82
   Add is_divisible to ops.py<|assistant_end|><|bos|>Once upon a time, in a small town
2. story 2.78
   only tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time, in a quiet little
3. story 2.78
   can tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time, there was a little
4. story 2.76
   how to tell stories and add numbers.<|assistant_end|><|bos|>Once upon a time, there was a little
5. story 2.74
   only tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time, there was a big
6. story 2.72
   can tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time, there was a naughty
7. story 2.71
   json` and `demo.py`.<|assistant_end|><|bos|>Once upon a time, there was a girl
8. story 2.7
   and `test_arith.py`.<|assistant_end|><|bos|>Once upon a time, there was a little
9. story 2.7
   s<|user_end|><|assistant_start|>Anytime!<|assistant_end|><|bos|>Once upon a time, there was a little
10. story 2.69
    how to tell stories and add numbers.<|assistant_end|><|bos|>Once upon a time, there was a monkey
11. story 2.68
    2=9, 9<|think_end|>9<|assistant_end|><|bos|>Once upon a time, in a small messy
12. story 2.68
    . Try asking me for a story!<|assistant_end|><|bos|>Once upon a time, there was a fat