Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #3311

Fires on “ numbers” (not labeled yet: read the examples)

Fires on
0.60% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 62%
  • agent 27%
  • story 9%
  • python 2%
  • worksheet 1%

Makes the model say

to [Once[::- togetherasastic(

Makes it not say

Tom table kitchen waitedTom owner momlower

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.29
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|bos|>Once
  2. chat 4.26
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|bos|>The
  3. chat 4.16
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>
  4. chat 4.15
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>
  5. chat 4.14
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|user_start|>Good
  6. chat 4.12
    I'm a tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>Tom
  7. chat 4.09
    I'm a tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>
  8. chat 4.06
    I'm a tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|><|user_start|>
  9. chat 4.06
    I'm a tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>Could
  10. chat 4.05
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|bos|>One
  11. chat 4.05
    I'm a tiny language model: I can tell short stories and add numbers.<|assistant_end|><|user_start|>Write
  12. chat 4.04
    I'm a tiny model: I can only tell short stories and add numbers.<|assistant_end|><|bos|>Once