Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #541

Fires on “ plus” (not labeled yet: read the examples)

Fires on
0.29% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 46%
  • worksheet 39%
  • chat 12%
  • story 2%
  • python 1%

Makes the model say

plus<|assistant_start|>)`.<|assistant_end|>()`. calculator checks "!"`.

Makes it not say

good a7ch r It t2

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.53
    114<|assistant_end|><|bos|><|user_start|>Hey, what's 440 plus 746
  2. chat 4.46
    152.<|assistant_end|><|user_start|>Hey, what's 423 plus 9?<|user_end|>
  3. chat 4.43
    the number only.<|system_end|><|user_start|>Hey, what's 240 plus 564
  4. chat 4.42
    appy to help!<|assistant_end|><|user_start|>Hey, what's 421 plus 717
  5. chat 4.38
    with "Sure!".<|system_end|><|user_start|>Hey, what's 908 plus 348
  6. chat 4.37
    479.<|assistant_end|><|user_start|>Hey, what's 722 plus 793
  7. worksheet 4.33
    special day.<|bos|>Hey, what's 88687 plus 353
  8. worksheet 4.28
    411.<|assistant_end|><|bos|>Hey, what's 885 plus 502
  9. worksheet 4.27
    564.⏎Hey, what's 2819 plus 412
  10. worksheet 4.27
    3857 flowers.<|bos|>What is 9717 plus 6?⏎
  11. worksheet 4.26
    90.⏎Hey, what's 20404 plus 980
  12. worksheet 4.23
    9.<|assistant_end|><|bos|>Hey, what's 11930 plus 695