Safety

Once the stack was built, I did the only responsible thing. I tried to destroy it. I spent a weekend typing the worst inputs I could imagine into my own search box. When I ran out of bad ideas, I handed a list of nearly three hundred nasty queries to two other AIs, Gemini and Claude, and asked them to be as adversarial as they could. Their whole assignment was to find the input that made my app hand someone the wrong drug. It is a strange way to spend a Saturday, paying for compute so one robot can try to make your other robot commit malpractice. ...

The deterministic matcher was supposed to be the safe part. It mostly is. But safe does not mean finished, and the matcher had failure modes of its own. Most of the work in this project was not building it. It was tuning it. The dial The matcher has one main dial, but the dial only makes sense once you have watched the thing work. It is worth seeing once, because the whole safety argument rests on how dumb and how legible this machine is. ...