Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #2787

Fires on “.” (not labeled yet: read the examples)

Fires on
0.73% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 33%
  • chat 30%
  • story 24%
  • python 12%
  • worksheet 1%

Makes the model say

one legs m ever Then say st pet

Makes it not say

swam5 upon len allassert<|think_end|>_

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 4.54
    Bobo was very excited and started to count right away. He counted one, two, three,
  2. chat 3.69
    learned something new with her mom.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Could you
  3. chat 3.47
    for another day of fun together.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>what is
  4. chat 3.4
    " " + b⏎```⏎<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What is
  5. chat 3.4
    can turn out to be good.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Anna has
  6. chat 3.4
    is 1534.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>what is
  7. chat 3.4
    <|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Sue has
  8. chat 3.4
    <|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Find the
  9. chat 3.39
    pass now.<|assistant_end|><|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What is
  10. story 3.39
    kids had to number their steps. They took turns counting one, two, three,
  11. chat 3.37
    , and they became good friends.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>What's
  12. chat 3.34
    them in the right order.<|bos|><|system_start|>Answer with the number only.<|system_end|><|user_start|>Calculate