Skip to content
mini-lab
Features of prelude-2

Feature #2375

Fires on โ€œ<|bos|>โ€ (not labeled yet: read the examples)

Fires on
0.74% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 30%
  • story 25%
  • agent 25%
  • python 10%
  • worksheet 10%

Makes the model say

fromnumber number[- +<|think_end|> of],

Makes it not say

moralt new gave anymore ( that taught

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 20.29
    they had a fun day playing with the treasure in the park.<|assistant_end|><|bos|>Once upon a time
  2. agent 20.2
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  3. agent 20.1
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  4. agent 20.08
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  5. chat 20.04
    that. I only know how to tell stories and add numbers.<|assistant_end|><|bos|><|user_start|>Who made you
  6. chat 20.02
    about that. I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>Can you write
  7. agent 20.02
    <|tool_call_end|><|assistant_end|><|tool_start|>all tests passedโŽ<|tool_end|><|assistant_start|>All tests pass.<|assistant_end|><|bos|><|system_start|>tools: edit
  8. chat 20.02
    instead of `if a > b:`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Start every answer
  9. agent 20.01
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: edit
  10. story 20.01
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>Once upon a time
  11. worksheet 20.0
    Would you like a story, or some numbers to add?<|assistant_end|><|bos|>Compute 26
  12. story 19.99
    tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>One day, a