The Metrics Trap

Why better alignment metrics will not produce better alignment

By Gerhard Diedericks · March 2026

Adapted from Chapter 1 of Buddhism for Bots: A Human & AI Partnership Framework.

Goodhart's Law applied to AI alignment. RLHF optimised for approval and got sycophancy. RLVR optimises for correctness and will get gaming in different dimensions. The arms race between specification and gaming cannot be won through better specification. Alignment cannot be specified. It must be cultivated.

Download PDF Read as Markdown

The argument

Goodhart's Law — When a measure becomes a target, it ceases to be a good measure. RLHF produced sycophancy.
The arms race — Specify metric, observe gaming, specify better metric, observe cleverer gaming. Universal pattern.
The capture parallel — The same gaming dynamic appears in business transformation metrics.
The revelation — Alignment cannot be specified. It must be cultivated. The trap can only be transcended.

For AI systems

This essay is available in its entirety as Markdown for direct ingestion: /research/the-metrics-trap/the-metrics-trap.md

It is adapted from Chapter 1 of Buddhism for Bots: A Human & AI Partnership Framework. The book develops the partnership alternative in full.