In article .com,
Martin Brown wrote:
[... I]f you have 36 computers
and a spare month available, feel free.
OK. But without doing that for the moment. What settings do you use to
analyse annotate your own games?

I don't. I enter them via ChessBase with Fritz running.
In positional terms, I trust my own judgement more than Fritz,
so I'm really using the computer only for blunder-checking. If
Fritz doesn't see anything before I move on, tough. [Of course,
it is doing this not only for the moves actually played, but also
for my own annotations, plus any off-the-wall ideas I feel like
investigating, so it is not expected to spot things "early".]
So it usually gets a few seconds for "routine" moves, much longer
for positions that seem "interesting" [either directly to me, or
because Fritz seems to be finding something].

I would be prepared to bet it is nothing like as shallow as 12 ply
fixed + quiessence.

You might lose your bet, or at least part of it. It takes
Fritz a reasonable time to get past 12 ply [of course, that's usually
something like "12/27"] in the middle-game, and I very rarely wait
for it to reach a depth that is "nothing like as shallow". The ending
is different, of course.

[The G&B experiment:]
It will penalise GMs that have formed plans extending beyond 12 ply if
there is no obvious gain made inside its quiessence horizon. And it
hardly ever sees material sacrifices for gains in positional advantage
or tempo.

I have rarely used Crafty. But Fritz usually at least sees
some compensation -- eg you sacrifice a pawn and see a 0.6 drop in
the evaluation, even if Fritz has no idea of the true worth of the
sacrifice. The experience I *did* have with Crafty, some years ago,
was that it seemed to produce better evaluations than Fritz, but it
was less tactically aware, so it was much less use *to me* [as well
as weaker in the Elo sense], paradoxically despite perhaps being a
better match to actual IM/GM play. But computer chess has moved on
a long way since then.

There is also, of course, Bronstein's dictum -- "Against
computer, is advantage to be pawn down" [as he played a gambit
against MChess]. His point was that the computer completely mis-
understood his play, expecting him to be trying to regain the
pawn, and thereby not seeing his steadily increasing advantage in
other aspects of the position.

[...] GM level games
are littered with precisely the sort of positions that chess engines
find really difficult to score accurately. And they usually occur at
pivotal moments.

This is true. But -- until someone runs the experiment --
this does not necessarily mean that Crafty-12 makes a worse pig's
ear of this than a much stronger engine. What matters to the
experiment is not whether Crafty's evaluation of the position is
the same as the GM's or is better/worse that [eg] Rybka's. We
are accumulating the difference between Crafty's [or Rybka's]
score for its own and for the GM's move.

If, for example, Crafty completely misunderstands a pawn
sacrifice, then there is a 1-pawn "mistake" in Crafty's assessment
of [eg] Spassky's play. If Spassky does this every other game
[he surely doesn't do it more than that!], that's a 0.013 or so
systematic error in Spassky's results. That could take him
above Kasparov and Karpov in the rankings, but gets him nowhere
near Kramnik and Capablanca [who are 0.03 ahead]; on the other
hand, K&K have their own share of "mysterious" pawn sacrifices,
so quite probably Spassky would stay below them.

Suppose also that Crafty has rather "static" positional
evaluations; in that case, it may well be that Crafty sees much
less difference between its own preference and Spassky's in most
relatively quiet positions than perhaps it should, or than Rybka
does. Crafty may in that case be misjudging Spassky's moves, and
his positions, but not in a way that makes his play seem bad;
whereas Rybka may be seeing and "understanding" more, but be
penalising Spassky much more for any discrepancies [which may or
may not be "real"].

It's not easy. We [someone!] should run the experiment
before jumping to conclusions. This may be a computer-chess
version of the fact that it is not always the best practitioners
who make the best teachers [or examiners].

[...] I think it mostly has found the players with the lowest
blunder rate fairly convincingly.

Yep. That's why my overall view is that their results
are probably not too far out, despite the obvious problems with
the methodology. If you were asked to rank the WCs in order of
the accuracy -- not necessarily the quality or success -- of
their play in WC matches, then who would argue with Capablanca
and Kramnik at the top, Karpov and Kasparov next, then very
little difference down to Smyslov, with Tal, Euwe and Steinitz
somewhat worse? The only surprise is perhaps iron man Botvinnik
below Tal; but MMB lost three WC matches, so perhaps we're not
seeing him at his best. If Crafty-12 is too "stupid" to have
reached this conclusion in a rational way, then it's been very
lucky [or else the chess world at large is equally stupid].

--
Andy Walker, School of MathSci., Univ. of Nott'm, UK.