Feature #3747

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

1. chat 2.7
   <|bos|><|user_start|>Tell me about yourself.<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
2. chat 2.63
   !<|assistant_end|><|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
3. chat 2.62
   <|assistant_end|><|bos|><|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
4. chat 2.61
   !<|assistant_end|><|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
5. chat 2.57
   <|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
6. chat 2.56
   <|assistant_end|><|user_start|>tell me about yourself<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
7. chat 2.56
   <|bos|><|user_start|>What is your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
8. chat 2.53
   .<|assistant_end|><|user_start|>what's your name<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
9. chat 2.43
   .<|bos|><|user_start|>Who are you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
10. chat 2.4
    ?<|assistant_end|><|user_start|>who made you<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
11. chat 2.39
    <|system_end|><|user_start|>What is your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language
12. chat 2.37
    .<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language