Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #313

Fires on “.” (not labeled yet: read the examples)

Fires on
0.54% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 62%
  • agent 21%
  • story 16%
  • python 1%
  • worksheet 0%

Makes the model say

hello this mainHi mistakefe scratchving

Makes it not say

all little near lot more and cat enough

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 3.09
    8.<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny
  2. chat 3.08
    What is your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model:
  3. chat 3.04
    .<|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny
  4. chat 2.94
    add numbers.<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>I'm prelude, a very small language
  5. chat 2.93
    lab.<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny
  6. chat 2.92
    8.<|assistant_end|><|user_start|>Who are you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny
  7. chat 2.87
    <|user_start|>what's your name<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model:
  8. chat 2.86
    <|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model:
  9. chat 2.85
    2.<|assistant_end|><|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny
  10. chat 2.84
    What is your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model:
  11. chat 2.84
    .<|assistant_end|><|user_start|>What's your name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny
  12. chat 2.76
    <|user_start|>Who made you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: