Letters
We Iterate on Models. We Can Iterate on Evals, Too: Building automated evals doesn’t need to be a huge investment. Start with a few quick-and-dirty examples and iterate!
I’ve noticed that many GenAI application projects put in automated evaluations (evals) of the system’s output probably later — and rely on humans to manually examine and judge outputs longer — than they should.