Notes on eval methodology, LLM-as-judge reliability, and RAG output quality — drawn from CrossSource and production eval work. And, occasionally, an industry thesis written so it can be wrong.
Industry thesis · Mobility
A family business from Hubli built a fifty-year moat in Indian transport and just placed the largest private Volvo order in the country. What that says about premium intercity bus travel — and a ₹50,000 experiment to find out whether I actually understand it.
Eval methodology · screener-eval
I ran my own résumé through two LLM screeners against 21 open AI PM roles, five times each, with the employers swapped for fictional ones and the links deleted. Neither mattered. Something else did.
Eval methodology · mirror-eval
What four AI search engines say about me, measured twice — and the three things that only showed up because the numbers were counted rather than scored.
Eval methodology · CrossSource
On validating LLM-as-judge evals — why a 15/15 blind agreement and a broken instrument can live in the same pipeline.