Skip to content
mini-lab
Features of prelude-2

Feature #427

Fires on โ€œ<|system_end|>โ€ (not labeled yet: read the examples)

Fires on
0.13% of tokens
Mostly in
chat
Layer
after block 4

Where it fires

  • chat 80%
  • story 12%
  • agent 7%
  • python 1%
  • worksheet 0%

Makes the model say

[el by([ what a lell

Makes it not say

<|tool_call_end|> "!"`. code<|think_end|>])` editname calculator

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 4.12
    ant capabilities<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Find the sum
  2. chat 3.9
    (nums)`.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Tell me a
  3. chat 3.9
    story, or some numbers to add?<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|system_start|>You are a
  4. chat 3.84
    a marriage when I grow up."<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Compute 6
  5. chat 3.82
    how to tell stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What is your
  6. chat 3.78
    was a sailor.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Tell me a
  7. chat 3.77
    ("banana")` returns `"ananab"`.โŽ<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>2 +
  8. chat 3.76
    acy.<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>1+4
  9. chat 3.75
    <|system_start|>tools: calculator<|system_end|><|system_start|>You are a
  10. chat 3.7
    only tell short stories and add numbers.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|system_start|>Do not use
  11. chat 3.68
    laughed. And they lived happily ever after.<|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>How far away
  12. chat 3.68
    . Try asking me for a story!<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|system_start|>Do not use