Skip to content
mini-lab
Features of prelude-2

Feature #1160

Fires on “ ” (not labeled yet: read the examples)

Fires on
0.66% of tokens
Mostly in
worksheet
Layer
after block 4

Where it fires

  • worksheet 38%
  • chat 37%
  • agent 16%
  • python 6%
  • story 3%

Makes the model say

def newchrunmanghelloig

Makes it not say

⏎<|bos|>Once⏎⏎ passed plusek⏎

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 3.01
    ⏎```⏎e⏎```<|assistant_end|><|bos|><|user_start|>Add 82 and 88.<|user_end|>
  2. chat 2.91
    to `main.py`.<|assistant_end|><|bos|><|user_start|>Add 46 and 13.<|user_end|>
  3. chat 2.8
    every answer with "Sure!".<|system_end|><|user_start|>Please add 33 and 76.<|user_end|>
  4. chat 2.77
    fun day at the beach.<|assistant_end|><|bos|><|user_start|>Add 21 and 70.<|user_end|>
  5. chat 2.73
    answer is 50.<|assistant_end|><|user_start|>Please add 53 and 149.
  6. worksheet 2.71
    (a, b)`.<|assistant_end|><|bos|>The sum of 59 and 861 is
  7. worksheet 2.69
    script.py`.<|assistant_end|><|bos|>The sum of 235 and 562 is
  8. chat 2.67
    3)` returns `6`.⏎<|bos|><|user_start|>Add 59 and 0.<|user_end|><|assistant_start|>
  9. chat 2.64
    has 586 shells.<|bos|><|user_start|>Add 45 and 7.<|user_end|><|assistant_start|>
  10. chat 2.63
    Answer with the number only.<|system_end|><|user_start|>Please add 40 and 41.<|user_end|>
  11. chat 2.62
    <|user_end|><|assistant_start|>Run main.py<|assistant_end|><|bos|><|user_start|>Add 46 and 13.<|user_end|>
  12. chat 2.62
    stories and add numbers.<|assistant_end|><|bos|><|user_start|>Please add 35 and 96.<|user_end|>