Skip to content
mini-lab
Features of prelude-2

Feature #3609

Fires on “<|system_start|>” (not labeled yet: read the examples)

Fires on
1.30% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 44%
  • chat 29%
  • story 17%
  • python 7%
  • worksheet 2%

Makes the model say

script<|system_start|> preludelower definesubcontent defined

Makes it not say

,??" go f She done find

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. story 3.81
    wheel.<|bos|>Once upon a time, there was a little dolphin named Dolly. Dolly
  2. story 3.64
    5.<|assistant_end|><|bos|>Once upon a time, there was a dolphin named Dolly. Dolly
  3. story 3.22
    anymore.<|bos|>Once upon a time, there was a small bird named Mr Chirp.
  4. story 3.01
    with a big hat. The man said, "Hi, I am Mr. Rich.
  5. chat 2.95
    of `return len(s)`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|>
  6. agent 2.93
    of `return items[-1]`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,
  7. chat 2.91
    return x * x * x`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Answer with the number
  8. agent 2.9
    `result = result * i`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,
  9. chat 2.89
    of `return len(s)`. The tests pass now.<|assistant_end|><|bos|><|system_start|>Answer with the number
  10. chat 2.88
    of `return a ** b`. The tests pass now.<|assistant_end|><|bos|><|system_start|>You are mini,
  11. agent 2.87
    of `return min(nums)`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,
  12. agent 2.87
    of `return s[0]`. The tests pass now.<|assistant_end|><|bos|><|system_start|>tools: edit,