Feature #2020

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. chat 7.96
   project.<|assistant_end|><|bos|><|user_start|>What can you do?<|user_end|><|assistant_start|>I'm prelude: I can tell
2. chat 7.57
   project.<|assistant_end|><|bos|><|user_start|>What can you do?<|user_end|><|assistant_start|>I'm prelude: I can tell
3. chat 7.4
   help!<|assistant_end|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know
4. chat 7.28
   85.<|assistant_end|><|user_start|>What can you do?<|user_end|><|assistant_start|>I'm prelude: I can tell
5. chat 7.14
   3.<|bos|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude, and I know
6. chat 7.08
   story soon.<|assistant_end|><|bos|><|user_start|>Who made you?<|user_end|><|assistant_start|>I'm prelude, a very small
7. chat 7.06
   <|bos|><|user_start|>What are you good at?<|user_end|><|assistant_start|>I'm prelude, and I know
8. chat 7.06
   <|bos|><|user_start|>What are you good at?<|user_end|><|assistant_start|>I'm prelude: I can tell
9. chat 7.02
   tools: calculator<|system_end|><|user_start|>What can you do?<|user_end|><|assistant_start|>I'm prelude, and I know
10. chat 7.01
    3<|assistant_end|><|bos|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude: I can tell
11. chat 6.99
    tools: calculator<|system_end|><|user_start|>What is your name?<|user_end|><|assistant_start|>I'm prelude, a very small
12. chat 6.99
    .<|assistant_end|><|bos|><|user_start|>How can you help me?<|user_end|><|assistant_start|>I'm prelude: I can tell