Skip to content
mini-lab
Log in Sign up
Features of prelude-2

Feature #684

Fires on “ gre” (not labeled yet: read the examples)

Fires on
0.46% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 61%
  • chat 20%
  • worksheet 11%
  • story 5%
  • python 3%

Makes the model say

pll manyall Python longedb

Makes it not say

how. because2 and<|assistant_end|> treat realized

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. agent 2.24
    <|bos|><|system_start|>tools: edit, glob, grep, question,
  2. agent 2.24
    <|bos|><|system_start|>tools: edit, glob, grep, question,
  3. agent 2.23
    much.<|bos|><|system_start|>tools: edit, glob, grep, question,
  4. agent 2.21
    !<|bos|><|system_start|>tools: edit, glob, grep, question,
  5. agent 2.21
    b.<|assistant_end|><|bos|><|system_start|>tools: edit, glob, grep, question,
  6. agent 2.2
    108.<|bos|><|system_start|>tools: edit, glob, grep, question,
  7. agent 2.2
    5862.<|bos|><|system_start|>tools: edit, glob, grep, question,
  8. agent 2.18
    so bad after all.<|bos|><|system_start|>tools: edit, glob, grep, question,
  9. agent 2.18
    .<|assistant_end|><|bos|><|system_start|>tools: edit, glob, grep, question,
  10. agent 2.17
    again.<|bos|><|system_start|>tools: edit, glob, grep, question,
  11. agent 2.17
    forget.<|bos|><|system_start|>tools: edit, glob, grep, question,
  12. worksheet 2.17
    4.⏎If you add 1 and 341, you get 34