Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #3298

Fires on “ model” (not labeled yet: read the examples)

Fires on
0.38% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 46%
  • chat 37%
  • python 11%
  • story 5%
  • worksheet 0%

Makes the model say

trained model mini]`])) divisible best's

Makes it not say

returns"052 import day4

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.85
    name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  2. chat 4.56
    name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  3. chat 4.55
    name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  4. chat 4.53
    you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  5. chat 4.52
    your name<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  6. chat 4.51
    you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  7. chat 4.5
    you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  8. chat 4.49
    you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  9. chat 4.41
    you?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  10. chat 4.4
    Who made you?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on
  11. chat 4.39
    name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell
  12. chat 4.38
    name?<|user_end|><|assistant_start|>My name is prelude. I'm a tiny language model: I can tell