Skip to content
mini-lab
Features of prelude-2

Feature #1219

Fires on “ only” (not labeled yet: read the examples)

Fires on
0.16% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 49%
  • agent 27%
  • story 18%
  • python 4%
  • worksheet 1%

Makes the model say

knows: one *_ heard two idea

Makes it not say

wasas "!"pped Tom's newThe

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 3.27
    together, and they became good friends.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What
  2. chat 3.26
    pass now.<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What
  3. chat 3.26
    and not shaken the tree.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What
  4. chat 3.21
    they can turn out to be good.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Anna
  5. chat 3.19
    ready for another day of fun together.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>what
  6. chat 3.17
    do them in the right order.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Calculate
  7. chat 3.16
    198<|think_end|>198<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>what
  8. chat 3.13
    she learned something new with her mom.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Could
  9. chat 3.1
    )` returns a greeting for name.<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Add
  10. chat 3.1
    is 1706.<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What
  11. chat 3.1
    answer is 706.<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Please
  12. chat 3.09
    , or some numbers to add?<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Please