LLM evaluation
Your LLM-As-A-Judge Might Be Lying to You
Detect positional bias and ranking cycles in pairwise LLM judges, then use balanced comparisons to improve evaluator reliability.
4 min read
Topic
LLM evaluation
4 connected articles
LLM evaluation
Stop Scoring. Start Ranking.
Learn why pairwise ranking gives more useful LLM evaluation results than numeric scoring, and how to infer relative quality.
6 min read
artificial intelligence
How I stopped fighting prompts and started building feedback loops
How a generate, evaluate, and revise loop improves LLM output by turning clear acceptance criteria into useful feedback.
4 min read
artificial intelligence
RED framework for binary AI evals. Stop debugging in the dark
Use the RED framework to make binary LLM evaluations explain their reasoning, cite evidence, and return a clear decision.
4 min read