Write the evaluation first
Before changing a prompt, decide how you would know the change helped. Twenty labelled examples beat an afternoon of intuition.
AI Learn Lab
Learning to use AI tools well, and writing down what we find.
A small working notebook about the distance between an AI idea that demonstrates nicely and one that holds up once people depend on it.
What this is
AI Learn Lab is where we keep notes while learning and deploying AI-assisted software. It is written mostly for ourselves — a record of what worked, what turned out to be more complicated than expected, and what we would do differently next time.
The notes are deliberately narrow. They are about specific decisions: how to tell whether a change actually improved anything, what to log, where to draw the boundary around a first version. We try to write down the trade-offs rather than only the conclusions, because the trade-offs are usually the part that transfers.
What we keep coming back to
- Evaluation before optimisation
- Deciding how you would recognise an improvement, before spending a week trying to produce one.
- Small, checkable scope
- One task with a bounded output beats a broad capability nobody can verify.
- The gap at deployment
- Timeouts, malformed responses, cost per call, and what the system does when it has no good answer.
Recent notes
All notesWhat deployment actually changes
A prototype answers the question "can this work?". Production answers "what happens when it doesn't?" — a different question, with different work behind it.
Keeping the scope small on purpose
The projects that get finished tend to describe one narrow task with a checkable answer. Breadth is easier to promise and much harder to verify.
Getting in touch
If something here is wrong, or matches or contradicts your own experience, we would genuinely like to hear about it. Emailhello@ailearnlab.org — or read a little more about the lab first.