Skip to content
mini-lab
Features of prelude-2

Feature #1284

Fires on “ +” (not labeled yet: read the examples)

Fires on
1.46% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 44%
  • worksheet 42%
  • python 7%
  • agent 4%
  • story 2%

Makes the model say

+ i need0 adds<|user_end|>ate -

Makes it not say

ThisassertThis smiled I theid it

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. worksheet 5.42
    278 = 295.⏎Calculate 6 + 820
  2. worksheet 5.14
    9 = 51854.⏎What's 8 + 539
  3. worksheet 4.92
    207 = 4638.⏎Calculate 5 + 6.⏎
  4. worksheet 4.86
    45 = 39418.⏎Calculate 7 + 7.⏎
  5. chat 4.82
    105586.<|bos|><|user_start|>What's 5 + 500
  6. worksheet 4.81
    75 = 3580.⏎Calculate 19 + 344
  7. worksheet 4.78
    4 + 78 = 82.⏎Calculate 6 + 9.⏎
  8. chat 4.68
    The answer is 759.<|assistant_end|><|user_start|>What's 6 + 5?<|user_end|>
  9. worksheet 4.68
    422 = 702.⏎What's 8 + 4?⏎
  10. worksheet 4.65
    40 = 62924.⏎Calculate 3 + 85.
  11. chat 4.64
    to be a new friend for Sue.<|bos|><|user_start|>What's 2 + 5?<|user_end|>
  12. worksheet 4.62
    8 = 59786.⏎What's 7 + 590