Skip to content
mini-lab
Features of prelude-2

Feature #411

Fires on “ at” (not labeled yet: read the examples)

Fires on
0.16% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 52%
  • story 26%
  • chat 19%
  • python 2%
  • worksheet 1%

Makes the model say

first thest home his) it named

Makes it not say

<|assistant_end|>pattern swings waveose5or closer

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. chat 3.1
    answer is 15.<|assistant_end|><|user_start|>what are you good at<|user_end|><|assistant_start|>I'm
  2. chat 3.0
    you<|user_end|><|assistant_start|>Happy to help!<|assistant_end|><|user_start|>what are you good at<|user_end|><|assistant_start|>I'm
  3. chat 2.93
    The answer is 17.<|assistant_end|><|bos|><|user_start|>What are you good at?<|user_end|><|assistant_start|>I
  4. chat 2.92
    today?<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>What are you good at?<|user_end|><|assistant_start|>I
  5. chat 2.88
    The answer is 266.<|assistant_end|><|user_start|>what are you good at<|user_end|><|assistant_start|>I'm
  6. chat 2.88
    . Try asking me for a story!<|assistant_end|><|user_start|>What are you good at?<|user_end|><|assistant_start|>I
  7. chat 2.87
    <|user_start|>What are you good at?<|user_end|><|assistant_start|>I
  8. chat 2.83
    you<|user_end|><|assistant_start|>Happy to help!<|assistant_end|><|user_start|>What are you good at?<|user_end|><|assistant_start|>I
  9. agent 2.83
    test_text.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call last):⏎ File
  10. agent 2.8
    python3 test_calc.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call last):⏎ File
  11. agent 2.8
    python3 test_calc.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call last):⏎ File
  12. agent 2.8
    command=python3 app.py<|tool_call_end|><|assistant_end|><|tool_start|>Traceback (most recent call last):⏎ File