Hacker News

Favorites Setup
Comment by antonvs | original | Senior SWE-Bench: open-source benchmark that assesses agents as senior engineers
[−]antonvs · 2026-07-02 Thu 06:02 UTC · link
The “make no mistakes” admonition does seem pretty silly (it’s been skewered to death on yt), but… it’s easy to imagine how it might work. E.g. it could be interpreted as simply as “check your work”.

Of course, no-one seems to be (publicly) doing the comparative measurements that might allow us to reach rational conclusions here.

[−]rhdunn · 2026-07-02 Thu 06:58 UTC · link
I'm not sure if they've fixed this, but older models have a tendency to ignore negation as `no`, `not`, etc. all occur frequently in the training data so are weighted less strongly than the verbs and nouns.

The advice I've heard is to emphasize the traits you want, not discourage the traits you don't. So rather than saying "make no mistakes" you can do something like you suggested with writing it as "check your work" or "ensure you answer correctly and concisely".