Write the evaluation first
Before changing a prompt, decide how you would know the change helped. Twenty labelled examples beat an afternoon of intuition.
Notes
Short pieces written while building and deploying AI-assisted software. Each one covers a single decision, including the parts that did not go to plan.
Before changing a prompt, decide how you would know the change helped. Twenty labelled examples beat an afternoon of intuition.
A prototype answers the question "can this work?". Production answers "what happens when it doesn't?" — a different question, with different work behind it.
The projects that get finished tend to describe one narrow task with a checkable answer. Breadth is easier to promise and much harder to verify.