Skip to content
mini-lab
Features of prelude-2

Feature #4079

Fires on “ defined” (not labeled yet: read the examples)

Fires on
1.16% of tokens
Mostly in
agent
Layer
after block 4

Where it fires

  • agent 90%
  • chat 8%
  • story 2%
  • python 0%
  • worksheet 0%

Makes the model say

calculatorars calculator=**/g For <|assistant_start|>

Makes it not say

theIf0[ they5 proud p

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

  1. agent 2.0
    defined in `items.
  2. agent 2.0
    defined in `text.
  3. agent 1.92
    "⏎ ^^^^^^^^^^^^^^^^^^^^^^^^^^^⏎AssertionError⏎⏎
  4. agent 1.84
    ^^^⏎AssertionError⏎⏎
  5. agent 1.83
    ulltiply(6, 1))⏎ ^^^^^^^^^⏎NameError: name
  6. agent 1.82
    0, 0]))⏎ ^^^^^^^^⏎NameError: name
  7. agent 1.81
    me about yourself."<|user_end|><|assistant_start|>I'm prelude, a tiny language model trained from scratch on a
  8. agent 1.81
    ⏎ ^^^^^⏎NameError: name 'vowls' is not defined. Did you mean
  9. agent 1.8
    >⏎ print(count_vwels("river"))⏎ ^^^^^^^^^^^⏎NameError: name
  10. agent 1.79
    ^^^^^^^^^^^^^^^^^^^^^^^^^⏎AssertionError⏎
  11. agent 1.79
    ⏎ ^^^^^^^^^^^^^^^^^^^^^^⏎AssertionError⏎⏎
  12. agent 1.78
    <|user_end|><|assistant_start|><|tool_call_start|>grep<|arg|>pattern=LIMIT =<|tool_call_end|><|assistant_end|><|tool_start|>Found 1 matches⏎