Features of prelude-2

Feature #2350

Fires on “ dragon” (not labeled yet: read the examples)

Fires on: 0.28% of tokens
Mostly in: story
Layer: after block 4

Where it fires

story 42%
chat 41%
agent 13%
python 3%
worksheet 0%

Makes the model say

and like who but b wantedBut a

Makes it not say

Renamedpe⏎```Fixed reverseisedjson

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

chat 3.3
[0]⏎```⏎<|bos|><|user_start|>Tell me a story about a dragon.<|user_end|><|assistant_start|>Once
chat 3.23
answer is 10.<|assistant_end|><|user_start|>Tell me a story about a bunny.<|user_end|><|assistant_start|>Once
chat 3.22
Answer in one short sentence.<|system_end|><|user_start|>I want a story about a bunny!<|user_end|><|assistant_start|>One
chat 3.2
```⏎False⏎```<|assistant_end|><|bos|><|user_start|>Tell me a story about a dragon.<|user_end|><|assistant_start|>Once
chat 3.19
(s)`.<|assistant_end|><|bos|><|user_start|>Can you tell me a story about a dragon?<|user_end|><|assistant_start|>Once
chat 3.16
730 cookies.<|bos|><|user_start|>Tell me a story about a dragon.<|user_end|><|assistant_start|>One
chat 3.15
you today?<|assistant_end|><|user_start|>can you tell me a story about a dragon<|user_end|><|assistant_start|>Once upon
chat 3.08
<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Tell me a story about a dragon.<|user_end|><|assistant_start|>Once
chat 3.04
add numbers.<|assistant_end|><|bos|><|user_start|>Can you tell me a story about a dragon?<|user_end|><|assistant_start|>Once
chat 3.04
, very tired.<|bos|><|user_start|>Can you tell me a story about a bunny?<|user_end|><|assistant_start|>Once
chat 3.04
Answer in one short sentence.<|system_end|><|user_start|>Tell me a story about a dragon.<|user_end|><|assistant_start|>Once
chat 3.04
is 267.<|assistant_end|><|user_start|>I want a story about a dragon!<|user_end|><|assistant_start|>Once