Shahar Merom

Projects & interests

What does a chess model actually evaluate?

YearAblation / perturbation analysisPython

Question

In your words: what does the model actually evaluate?

Method

In your words: square ablation. Remove squares and measure how the evaluation changes

Chart placeholderchange in the model's evaluation when each square is removed (heatmap over the board)

Finding

In your words: the model put high probability on a weak move that looks natural

Chart placeholdermodel probability vs. engine evaluation for the candidate moves

Limitation

In your words: what single-square ablation can't see

So what

In your words: why plausible isn't the same as correct