Features of prelude-2

Feature #2578

Fires on “'re” (not labeled yet: read the examples)

Fires on: 0.15% of tokens
Mostly in: python
Layer: after block 4

Where it fires

python 48%
story 24%
chat 14%
agent 11%
worksheet 2%

Makes the model say

When lookuckent inside helped said got

Makes it not say

"`.`. Python beach sky)`. di):

Strongest examples

On text the dictionary never trained on. The underlined token is where it fires most.

chat 2.31
.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Thank you<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
python 2.22
⏎-3⏎<|bos|># test_calc.py⏎from calc import min_of⏎
chat 2.2
.<|assistant_end|><|bos|><|system_start|>tools: calculator<|system_end|><|user_start|>Thank you<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
chat 2.16
.py⏎all tests passed⏎<|bos|><|user_start|>Thank you<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
python 2.07
number_utils.py⏎from number_utils import lower_of, sum_to⏎
chat 2.06
00.<|assistant_end|><|user_start|>Great, thanks!<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
python 2.05
all tests passed⏎<|bos|># test_mathlib.py⏎from mathlib import product, add_
chat 2.03
is 133.<|assistant_end|><|user_start|>thanks<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|bos|>
chat 2.03
64.<|assistant_end|><|user_start|>Great, thanks!<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
chat 2.02
crab lived near the sea.<|assistant_end|><|user_start|>Thanks!<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|user_start|>
chat 2.0
the kitchen table.<|assistant_end|><|user_start|>Thank you very much!<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|bos|>
chat 1.97
friendly assistant.<|system_end|><|user_start|>Great, thanks!<|user_end|><|assistant_start|>You're welcome!<|assistant_end|><|bos|>