Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #13

Fires on “ in” (not labeled yet: read the examples)

Fires on
0.29% of tokens
Mostly in
story
Layer
after block 4

Where it fires

  • story 50%
  • chat 23%
  • python 11%
  • agent 8%
  • worksheet 8%

Makes the model say

aym So://ptildtractZ

Makes it not say

answer<|think_end|>," cat<|assistant_end|> belongedvernumber

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 2.8
    How can I help you today?<|assistant_end|><|bos|>Once upon a time, in a small town,
  2. story 2.79
    ing colors and playing together.<|bos|>Once upon a time, in a big forest,
  3. chat 2.79
    Can you tell me a story?<|user_end|><|assistant_start|>Once upon a time, in a small town,
  4. story 2.79
    9 = 10087<|bos|>Once upon a time, in a big forest,
  5. chat 2.79
    want a story about the sea!<|user_end|><|assistant_start|>Once upon a time, in a big sea,
  6. story 2.77
    4 equals 3794.<|bos|>Once upon a time, in an ordinary house
  7. story 2.77
    s<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|bos|>Once upon a time, in a big forest,
  8. story 2.77
    The answer is 74.<|assistant_end|><|bos|>Once upon a time, in a small town,
  9. story 2.77
    <|bos|>Once upon a time, in a big garden,
  10. story 2.76
    The answer is 14.<|assistant_end|><|bos|>Once upon a time, in a peaceful little town
  11. story 2.76
    The answer is 13.<|assistant_end|><|bos|>Once upon a time, in a bright, sunny
  12. story 2.76
    answer is 498.<|assistant_end|><|bos|>Once upon a time, in a small house,