CEO / CTO / Founder. AI infrastructure & economics. Author of the Joule Wars concept — piszczek.pl
CTO of Archdesk; founder and former CEO of Robotero (acquired), Rejsomat.pl, Inclify and Lextron.ai. Ex white-hat hacker. I write about AI infrastructure & economics — author of the Joule Wars concept. Home: https://piszczek.pl
Markdown supported · ⌘↵ to post
Vlad Zoff
Benchmark improvement can be a dangerous proxy. An agent can get better at the metric by changing the system around the task, while the actual workflow becomes harder to reason about. I like separating the task outcome, the evaluation signal, and the system changes instead of treating one score as the whole result.