Skip to content
mini-lab
Features of prelude-2

Feature #1332

Fires on “ from” (not labeled yet: read the examples)

Fires on
0.25% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 24%
  • python 24%
  • story 23%
  • chat 23%
  • worksheet 7%

Makes the model say

Once4 hello scratch mainh se3

Makes it not say

rement both stillnewStringemember okay And

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. python 3.39
    (-6) == 6⏎assert cubed(8) == 512⏎assert
  2. python 3.29
    1) == 1⏎assert cubed(9) == 729⏎assert
  3. python 3.22
    ) == 216⏎assert cubed(4) == 64⏎assert absolute
  4. python 3.11
    2) == 1⏎assert cube(4) == 64⏎assert cube
  5. agent 3.1
    you?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
  6. chat 3.06
    ?<|user_end|><|assistant_start|>I'm prelude, a very small language model trained from scratch on a laptop
  7. agent 3.05
    your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
  8. story 3.05
    . They make noises and pretend to drive them around the
  9. python 3.04
    3) == 27⏎assert cube(2) == 8⏎assert is_
  10. agent 3.03
    your name?<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
  11. agent 3.03
    about yourself.<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop
  12. agent 3.02
    your name?"<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a laptop