Skip to content
mini-lab
Features of prelude-2

Feature #524

Fires on โ€œ<|user_start|>โ€ (not labeled yet: read the examples)

Fires on
0.74% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 51%
  • agent 47%
  • story 1%
  • python 0%
  • worksheet 0%

Makes the model say

โŽ<|bos|> test One")insze."

Makes it not say

worry kind all boat for to about pretty

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 3.01
    to that?<|user_end|><|assistant_start|>The answer is 126.<|assistant_end|><|user_start|>Plus 59
  2. chat 2.93
    9 to that?<|user_end|><|assistant_start|>The answer is 71.<|assistant_end|><|user_start|>Plus 65
  3. chat 2.92
    4?<|user_end|><|assistant_start|>The answer is 120.<|assistant_end|><|user_start|>What if you add
  4. chat 2.91
    have now?<|user_end|><|assistant_start|>The answer is 121.<|assistant_end|><|user_start|>Sue has 8
  5. chat 2.89
    + 5?<|user_end|><|assistant_start|>The answer is 23.<|assistant_end|><|user_start|>327 +
  6. chat 2.88
    9?<|user_end|><|assistant_start|>The answer is 432.<|assistant_end|><|user_start|>tell me about
  7. chat 2.88
    to that?<|user_end|><|assistant_start|>The answer is 133.<|assistant_end|><|user_start|>thanks<|user_end|>
  8. agent 2.87
    + b<|tool_end|><|assistant_start|>`reverse(s)` reverses s.<|assistant_end|><|user_start|>Are the tests pass
  9. chat 2.86
    37?<|user_end|><|assistant_start|>The answer is 37.<|assistant_end|><|user_start|>Now add 7
  10. chat 2.86
    to that?<|user_end|><|assistant_start|>The answer is 903.<|assistant_end|><|user_start|>Who made you?
  11. chat 2.86
    9?<|user_end|><|assistant_start|>The answer is 112.<|assistant_end|><|user_start|>Plus 7?
  12. chat 2.85
    17?<|user_end|><|assistant_start|>The answer is 21.<|assistant_end|><|user_start|>Could you tell me