Feature #1450

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. python 4.14
   _list(SCORES))⏎⏎$ python3 app.py⏎6.0⏎<|bos|>
2. python 3.74
   py`.<|assistant_end|><|bos|># tools.py⏎def
3. python 3.63
   <|tool_end|><|assistant_start|>The answer is 1656.<|assistant_end|><|bos|># listlib.py⏎def
4. python 3.61
   , 8, 3, 8, 2])⏎4.8⏎<|bos|>
5. python 3.57
   home sad.<|bos|># utils.py⏎def
6. python 3.56
   shadows on the wall. They were happy and safe.<|bos|># tools.py⏎def
7. python 3.53
   add numbers.<|assistant_end|><|bos|># listlib.py⏎def
8. python 3.52
   513<|think_end|>513<|assistant_end|><|bos|># tools.py⏎def
9. python 3.5
   8, 4, 6, 8])` returns `6.2`.⏎
10. python 3.49
    8, 6, 8]⏎22⏎<|bos|># utils.py⏎def
11. python 3.49
    2670 = 92679.<|bos|># lib.py⏎def
12. python 3.47
    list([0, 7, 5, 7])⏎4.75⏎