On May 10, 3:36 pm, raylopez99 wrote:
On May 10, 1:22 am, Martin Brown
wrote:

Well if it has to be open source then Fruit 2.1 (~2780) might be
another alternative to try against Crafty (~2670). An extra 100 points
and a bit less materiallistic evaluation would be closer to human GM
level play. Fruit 2.2.1 just about stumbled onto that tricky line that
Phil Innes sets so much stall by engines not finding before I pulled
the plug.

Lots of commercial chess programs will have a database of "tricky"
positions with "model" answers, just to fool people into rating them
higher. Tricks of the trade.

Don't try to teach me about computer chess. You clearly haven't a clue
what you are talking about. They are optimised to pass certain well
known tests but that was part of my motivation to start a new thread
asking for "interesting" new positions where engines score things
radically differently. A situation that persists even with the very
top engines run for times which would in principle make them beyond
superGM.

Fruit just scraped this test after about half an hour. It looked like
it was stuck in the obvious rut for a long while.

So in other words you would be happy to see different results
if we ran the experiment again next year,

Exact reproducibility probably isn't so important here. Getting the
maximum accuracy of the move evaluation function for the limited
amount of time available is the key. Fixed depth does not do that.

I disagree. Normalization, see one of my posts in this 200 post
thread.

You cannot "normalise" base metal into gold. Although you do seem to
believe that if you repeat a lie often enough it becomes true.

Crafty is a rather conservative chess engine. It has a *very* good
quiessence search at terminal nodes, but a relatively poor search
extension strategy. As a result it tends to get set in its ways and
miss important much stronger lines having settled into a comfortable
path that looks superficially OK. It leads to a false sense of
security in that the evaluations with increasing search depth remain
too stable (ie it doesn't learn much new with each successive ply).

Even the very best engines cannot agree to within 50cp on some key GM
positions after 2 days and at ply 22+.

In fact on the Kasparov-Anand Riga 1995 Shredder now thinks that only
11. ... Kf8 (0.05/23) is playable
Whereas Rybka reckons that most of the obvious continuation lines are
playable but prefers one of:
11. .... g6 (-0.08) or 11. .... OO (-0.07/23) or 11. .... Kf8
(-0.03/22)

AFAIK Kf8 is a novelty that has not been played in top level games. It
doesn't look pretty and the engines only see it at deep ply levels.

I would not be at all surprised if a few of the 70-80 centipawn
"blunders" turned out well at greater depth and a few non-blunders
turned out to be dubious. Swings, roundabouts. BICBW.

I don't think they are swings and roundabouts though. GM level games
are littered with precisely the sort of positions that chess engines
find really difficult to score accurately. And they usually occur at
pivotal moments.

A pivotal moment is immaterial if you use normalization. As I

Like hell it is. The GM gets penalised by |Machine_best_move -
GM_move| for every move where the computer fails to understand what he
is doing and when he wins the exchange by playing the move Crafty when
finally sees it he gets nothing. They filter out (correctly) all moves
evaluated outside the range [ -2.0 , 2.0 ] to avoid penalising the
winning GM for playing for a safe win rather than a risky optimal move
or the loser for playing fast and lose bluff moves.

Scoring GMs with Crafty penalises anyone who doesn't play *exactly*
like Crafty with a fixed 12 ply search strategy.

No amount of "normalisation" bull**** will get you out of this hole.

explained in a post in this thread, the fact that a player enters a 60
move mating net set by his opponent, unseen by Crafty with a 14 ply
move horizon, is immaterial since at some point Crafty will see the
mating net (namely, 7 moves before checkmate) and rate the losing
player lower than the winning player.

No it won't their scoring system was entirely based on evaluating the
difference between the move played and the machines idea of the best
move. And restricted to move 12 with evaluation bounded in -2.0, 2.0.

OK, perhaps the implication is that they should have stopped
there and then. But if historical Elo ratings are of interest, then
I see no reason why another objective measure of *something* need not
be. They *do* have an objective measure. It *does* seem that their
results correlate well with *some* quality that we can recognise in
the play of Capablanca, Petrosian, Tal, etc. Their methodology is
at least interesting, even if flawed.

Agreed. The experiment is worth repeating with a much stronger
engine.

Yes, agreed. As I posted 47.5 posts ago, for very close, nearly tied
rankings, the stronger chess program might make a difference. But for
clear demarcation breakpoints, such as between Capa and Kramnik versus
Karpov and Kasparov, a stronger chess engine doesn't matter.

I reckon there were a fair proportion of Karpov & Kasparovs moves that
Cratfy didn't even begin to understand and it marked them down for it.
The blunder rates are much more believable (although some of them will
be wrong).

And, chess being 99% tactics (say many GMs, including Tarrach or
Teichman), the player with the lowest blunder rate is often the best
champion. Blunders = Function(overall strength). In fact, a study

You keep on repeating this lie. It isn't true and it never will be. It
is precisely because of the importance of long range strategic
planning that machine chess isn't massively stronger than it is.
Tactics are a necessary part of strong chess but they are not
sufficient on their own.

from a few years ago found that the difference in most moves between a
patzer and a GM was not so much in the unexpected move made, but
rather in the fact GMs blundered far less than a Class C player.

What a surprise - who would ever have guessed that?

Regards,
Martin Brown