Skip to content
mini-lab
Features of prelude-2

Feature #2928

Fires on “<|assistant_start|>” (not labeled yet: read the examples)

Fires on
0.23% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 58%
  • agent 26%
  • story 13%
  • python 2%
  • worksheet 1%

Makes the model say

: little[ for forgot eachYouGood

Makes it not say

s defines was()` sentence SueoldString looking

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 1.83
    ! Come back for another story soon.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>Happy to help
  2. story 1.72
    good. But she thinks
  3. chat 1.71
    big forest.<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
  4. chat 1.71
    <|assistant_start|>The answer is 10.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>Happy to help
  5. chat 1.7
    the Python files in this project.<|assistant_end|><|bos|><|user_start|>thanks<|user_end|><|assistant_start|>Happy to help
  6. chat 1.69
    . Try asking me for a story!<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
  7. chat 1.69
    laptop by mini-lab.<|assistant_end|><|bos|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
  8. chat 1.68
    loved the warm sun on his wings.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>Happy to help
  9. chat 1.65
    step by step or with a calculator.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>Happy to help
  10. chat 1.65
    97.<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
  11. chat 1.64
    The answer is 133.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!
  12. chat 1.6
    <|system_start|>Start every answer with "Sure!".<|system_end|><|user_start|>thanks<|user_end|><|assistant_start|>Sure! You're