Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #1371

Fires on “<|assistant_start|>” (not labeled yet: read the examples)

Fires on
0.84% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 60%
  • agent 36%
  • story 3%
  • worksheet 1%
  • python 0%

Makes the model say

<|tool_call_start|>Fixed<|think_start|><|system_end|>Create calculator mini files

Makes it not say

and. too to " with car that

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.1
    snack ever!"<|assistant_end|><|bos|><|user_start|>What should I invest in?<|user_end|><|assistant_start|>Sorry, I'm
  2. chat 4.01
    time at school.<|bos|><|user_start|>How far away is the moon?<|user_end|><|assistant_start|>Sorry, I don
  3. chat 4.0
    <|bos|><|user_start|>Can you book a table for two tonight?<|user_end|><|assistant_start|>Sorry, I'm
  4. chat 3.99
    <|assistant_end|><|bos|><|user_start|>Who wrote Romeo and Juliet?<|user_end|><|assistant_start|>Sorry, I don
  5. chat 3.98
    calculator<|system_end|><|user_start|>What is the tallest mountain in the world?<|user_end|><|assistant_start|>Sorry, I don
  6. chat 3.97
    ⏎<|bos|><|user_start|>Can you help me with my taxes?<|user_end|><|assistant_start|>Sorry, I'm
  7. chat 3.97
    .<|assistant_end|><|user_start|>summarize this article for me<|user_end|><|assistant_start|>I'm sorry,
  8. chat 3.96
    <|user_start|>Summarize this article for me.<|user_end|><|assistant_start|>Sorry, I'm
  9. chat 3.96
    Explain quantum physics to me.<|user_end|><|assistant_start|>Sorry, I'm
  10. chat 3.95
    <|system_start|>tools: calculator<|system_end|><|user_start|>How far away is the moon?<|user_end|><|assistant_start|>Sorry, I don
  11. chat 3.94
    <|user_start|>Who is the richest person in the world?<|user_end|><|assistant_start|>I'm sorry,
  12. chat 3.93
    <|user_start|>Who is the richest person in the world?<|user_end|><|assistant_start|>I'm sorry,