Skip to content
mini-lab
Features of prelude-2

Feature #2133

Fires on “<|user_start|>” (not labeled yet: read the examples)

Fires on
1.61% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 56%
  • agent 41%
  • python 2%
  • story 1%
  • worksheet 0%

Makes the model say

Oncegmd longer counts sun glue project

Makes it not say

."(" he little. princess bird

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.86
    to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What is your name
  2. chat 4.81
    , 48<|think_end|>48<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Tell me the l
  3. chat 4.8
    697<|think_end|>697<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Calculate 49
  4. chat 4.79
    sub_one(x)`.<|assistant_end|><|bos|><|user_start|>Who wrote Rome
  5. chat 4.75
    and fix the Python files in this project.<|assistant_end|><|bos|><|user_start|>What can you do
  6. chat 4.74
    now I can have my delicious mouse!"<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Add 63
  7. chat 4.73
    (nums)`.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Tell me a story
  8. chat 4.72
    648<|think_end|>648<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What is your name
  9. chat 4.71
    107<|think_end|>107<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Find the sum of
  10. chat 4.7
    use the calculator.<|system_end|><|user_start|>Calculate 2 +
  11. chat 4.7
    a small and friendly assistant.<|system_end|><|user_start|>How much is
  12. chat 4.7
    calc.py⏎all tests passed⏎<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Who is the pres