read the rollout
x

u/adnanqidwai/thoughts/trace.md

read the rollout

#reflection#evals#optimization
  • black box evals can only take you so far, the real value lies in reading the trace
  • a final score tells you “what” happened, the trace tells you “why”
  • this is why good methods around context optimization, training, and environment feedback involve reflection on the trace and not just the output
  • recent examples: GEPA, HALO, SkillOpt, etc