Feature #1370

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. story 6.35
   walked, a strong wind suddenly blew and made the trees bend and sway. It was
2. chat 6.05
   Sally," the figure said kindly. "Why did you slip?"⏎Lucy
3. story 6.03
   ses sight of Tom.
4. story 5.98
   the apple. She said, "Lily, be careful! You might slip and fall."
5. story 5.97
   his sack when his dad walked up to him.⏎"Hey there son. What are
6. story 5.94
   They were careful not to let the water in again. The ship stopped sinking. The
7. story 5.92
   if she promised to be a good girl. The girl promised and ever since then,
8. story 5.87
   the world, all thanks to the high mountain and the special symbol he
9. story 5.82
   't want to give up, so he decided to try and be very sneaky.⏎
10. story 5.76
    :⏎```⏎I love Python⏎```<|assistant_end|><|bos|>Two siblings,
11. agent 5.75
    s[-1]<|tool_end|>
12. python 5.75
    s`.⏎⏎```