In article .com,
help bot wrote:
[...] If you want something reproducible, then, short of
help from the commercial companies, you have to do a fair amount
of tweaking.
Nonsense. Every tweak you make to Crafty is
a step toward irreproducibility, unless you expect
everyone to go in and modify their source code to
precisely match yours [...].

I don't *expect* anyone to do that. But anyone who might
want to check my results [or, in the present case, *G&B's* results]
*can* so modify their source code, and can thereby get exactly the
same results, even on different hardware, or can, if they choose,
apply exactly the same methodology to a different collection of
games, or can, if they choose, apply a different reproducible
methodology to some games and compare the results. What do you
mean by "reproducible" and how would you get the above from [eg]
Rybka?


A better method would be to take the strongest
program available (i.e. Rybka) and have it score
all the games, and only THEN apply some selected
algorithm to the standardized results.

And if I did the scoring again tomorrow, with a different
"seed" to the RNG, or with different contents in the cache/hash,
or perhaps after adding a new disc or RAM to my computer, let
alone if you tried to repeat my experiment, would you expect to
see the same results?

[...] I mean, get a REAL
computer, and a clue!
The depth achieved during play was presumably in real
time at, say, 5 hours per game? So a real-time 24-game match
occupied 5 solid days of chess, and so of computer time even
for a dedicated super-computer.
The computer was a fast desktop, as far as
I know.

A brief search didn't reveal what the actual hardware for the
recent Kramnik-DeepFritz match was, but from the node-counts quoted
it seems to have been roughly equivalent to 6x 2.5GHz PCs.

[...] The point is, my
guess is that this desktop passed up the 12th
ply in just a few seconds,

Note that neither ply counts nor node counts are very good ways
to compare one program with another [OK for comparing the same program
on different hardware, of course]. 12-ply Crafty, 12-ply Fritz and 12-ply
Rybka are quite certainly seeing different things [as well as applying
different evaluations to them].

[...] I am typing this right now while sitting
in a room with FOUR computers, albeit only one
of which gets any use these days.

I have SIX at home; but the most recent one is as fast, and has
as much disc and RAM as all the others put together. Guid and Bratko
used THIRTY-SIX, sorry, 36, and still took 10 FTE days to do their
experiment, perhaps a month of wall-clock time. Whatever resource they
had available, they had to balance number of games against time per move
so as to complete within a reasonable time. Five hours spent by Deep
Fritz on playing one game is equivalent to over a day on a 2.5 GHz PC,
so no reasonable amount of time on a reasonable network is going to be
able to analyse a "large sample" [and you considered 1397 to be "small"]
of games to the depth achieved in actual play. Might be different if
some large corporation threw resources at it, but this was a PhD student
apparently semi-moonlighting.

By my
calculations, that cuts your "month" down to
about a week, your "1000 games" to only 250
apiece, and that is just using my own computers.
Suppose I recruit a few people to help me out?

Then you and your friends will be able to reproduce G&B's
experiment, or conduct one of your own. Assuming that you and
they want to devote all those computers to the task 24/7.

It's really a matter of standards; are you willing
to accept half-baked analysis? Half-reasoned
justifications for dodgy work?

The half-baking and the half-reason seem to be based on
the semi-pop version put out by Chessbase. You may [well] not
agree with the baking and reasoning of the full paper, but some
reasonably qualified people will have refereed it [both for ICGAJ
and for ICCG5] and were presumably happy. ICGAJ publishes letters;
if you think their standards are slipping, write to them.

Whether the resulting
investigation is worthwhile is another matter.
Indeed. My take is that it is silly to think a
computer (at this point in time) can accurately
determine who among the world champions was
"the greatest".

Then perhaps it's lucky that G&B weren't trying to determine
that?

--
Andy Walker, School of MathSci., Univ. of Nott'm, UK.

In article om,
raylopez99 wrote:

I only admit that if the players are ranked extremely close (see my
Roman Numeral examples in the previous post), such as Capa and
Kramnik, that perhaps the bias of the program can reverse the true
ranking (so perhaps Kramnik comes before Capa, but in any even Kramnik
and Capa are ahead of Kasparov and Karpov)

Two questions, then:

1) Would you feel equally confident if we only gave crafty 11 ply? 10?
8? 4? Where do you draw the line? What non-arbitrary criteria are you
using to suggest that 12-ply is meaningful whereas 3 ply, obviously,
would not be?

2) What objective criteria are you using to define "extremely close"
such that you don't trust the computer's ability to rank them properly?

-Ron

Martin Brown wrote:

I bought Rybka on the strength of recommendations in this thread, and
based on watching it analyse this first test position I am favourably
impressed. Its evaluations tend to remain more internally self
consistent than Shredder and it sees insightful positional lines
rather than getting bogged down in swapoffs. So far so good.... early
days yet.

I am going to slightly hijack here. I had a long belief with you, that
given enough time, I was going to see roughly equivalent behavior
between the engines. Some style and steering difference, but given
enough time... Yada... Etc... And had been using Fritz 8 for quite a
while, followed by Fritz 9.

And following the Freestyle events and the amazing delta that Rybka was
having on the CEGT list, I finally invested just to see what the hoopla
was about.

Interestingly was how much different the analysis was. On style, and
significant cp evaluation differences. And how often the best move was
different from fritz, and in equal?! positions how often there would be
a difference of opinion on which side was better, and that the degree
could be amazingly large.

All of that would be for naught, except Rybka is literally kicking
everyone else's butt. It will be interesting to see how the Chessbase
engines respond.

And I gotta thank the Chess geeks for the UCI standard, that allows all
of this to happen, and people can compete for the best UI. Personally I
vote for chessbase and I recommend it. But I respect other opinions. I
just love, love the fact that there can *be* other opinions.

On May 4, 7:41 am, Martin Brown
wrote:

Crafty19 is somewhat more balanced than Fritz8 about choice of
continuation lines, but it looks a bit out of its depth when asked to
rate super GM games that it really doesn't understand (certainly not
at 12 ply).

True enough, but how often do GM games get decided by anything other
than simple tactics? That is the question. Kramnik missed a mate in
one in his last man-machine match and you're worried about plys
greater than 12+? The fact that world champions cannot do better than
draw (Kramnik) or lose (Kasparov) to chess programs gives lie to the
idea that GM games are decided by profound chess positional sacrifices
that chess computers don't understand.

Also when compared to stronger engines Crafty19.01 (which
is what I have most experience with) shows a tendency to exaggerate
the badness of some of the weaker moves from a given position. I am
currently testing a series of engines on a slightly tricky French
Advance Variation position where powerbooks considers only 7.Qd2 as
playable.

And in OTB play probably 7. Qd2 would be the best move.


If anyone else wants to try running the position for 48 hours and
noting down the move evaluations at all plys reached beyond the 4
minute mark I would like to see what Fruit, Comet and Deep Sjeng make
of this position. Infinite analysis, top 10 lines displayed (and also
the actual top 10 preferred lines found after 48 hours).

I see the problem with Camp #1--they are trying to play the 'perfect'
game of chess--perhaps correspondence chess. That is not the
inquiry. The inquiry is: which human champion is the best OTB
player? Using PCs we can determine this by finding which human GM
made the fewest tactical mistakes. It's true that Crafty will fail to
score certain profound positional sacrifices correctly, but as I have
repeatedly stated in this thread, probably such positional sacrifices
are rare in chess--chess is 99% tactics. Chess is not checkers, but
it's close enough. That's why Go and Bridge are considered the last
bastions of human game creativity (though those bastions will
eventually fall as well, once AI is further perfected).

RL

On May 4, 5:09 am, "Chess One" wrote:
Suppose that a version of Crafty favors "defensive" players like Capa
and Kramnik, while penalizing "attacking" players like Tal and
Fischer. Call this version of Crafty "Defensive

---
Interesting post, and the para above identifies the flaw in comparisons
of
player-to-player by engine analysis- that if you are evaluating players,
then even Tal himself said that anybody at all could find his own flaws
after the game, or the next day, or even next week. But he chose 'em
because
very few people could find them in real time OTB.

Yes, but this problem is also present in computer chess! Even if the
program is 'backsolving' or anotating a completed game, it has to look
in the chess tree, which means that it faces time constraints similar
to an OTB player vs a 'time is not of the essence' correspondence
player.

Quite so, Ray - in order to go to deeper plies and find a solution the
program must find a way to prune possibilities - and it does this by scoring
each branching possibility on, say, ply 7 or 8, and if worse than
nominally -2 will prune that line [which may have been eg, a Knight sac]
that can perhaps offer a return of 3 pawns and the initiaitive by ply 15 -
or even something more compelling.

I did not realize pruning is employed at -2 pawns. Thanks for this
information. I assumed the cutoff was more like -3.


So technically, this is the classical dilemma of the worth of the initiative
from a positional basis - and the evaluation function is pre-set, no? That
is, it is either copied from other programs, or based on in Crafty's case,
consultation with a GM to the /general/ worth of a variety of forms of
initiative.

The problem then becomes of applying that general evaluation complex to any
/specific/ instance encountered during play.

Given enough time then Stenitz will likely always rank higher than Tal by
analytical method, but OTB, super-solid Steintitz wouldn't know which way
Tal hit him! The flaw is that this analysis is okay for games, ie picking
some best theoretical line by objective and uniform measure against all
other lines - but chess playing is not a theoretical activity - its a
real-time performance.

Yes, and again, it's also true for chess playing computers (since no
program is given infinite time to analyse a game).

Yes - one of the MAMS recommendations is to pick a long line, preferably
giving up material for initiative and let the thing try to cook an answer -
if its ply depth doesn't go long enough, then the interesting thing is that
it begins to evaluate its own chances at +2 or even +4, and then half a
dozen moves later [and too late!] adjusts the evaluation to +1 or less, or
even to -1 etc.

I surmised that MAMS must be the book mentioned in this thread on how
to beat Fritz, to be published. I recall a rather simplistic book
called "How to Beat your Computer in Chess" about 10 years ago that
was not that useful for me. Hopefully this book is better. At any
rate, I doubt that in an actual OTB game with fixed time constraints
you will find the opportunities to play some brilliant positional
sacrifice--you'll be mated tactically before you ever get to these
wonderful winning positions. However, I will give Camp #1 credit in
one respect--they are essentially arguing how a human can beat a PC in
*CORRESPONDENCE CHESS*. The procedure is as follows: (1) fill a
database of positions where a positional sacrifice not within your
opponent computer's move horizon leads to victory, (2) play a series
of forcing moves that lead to the positions in step (1), and, (3)
execute steps (1) and (2) to win.

If this is what you seek, good luck to you, as you might find
"steering" the opponent program ala step (2) is a lot harder than you
think, because I posit these positional sacrifice positions will be
very far and few between in the chess tree--because chess is 99%
tactics.


Crafty was once world champion? I didn't know that or it must have
escaped me.

Yeah - Bob Hyatt is its author. I am not sure if he will immediately
evaluate the MAMS title, but I did offer to provide him a reader's copy of
it for his interest - but its end of academic year, so possibly not
conveninet. 2 other posters here are reviewing it, and it will be
interesting for us all to read their opinion of Albert's explorations.

OK, thanks for this information. For a moment I thought you were a
professor when you mentioned "end of academic year", but I see you're
referring to Bob Hyatt, who lurks here on occasion and once years ago
I think I even flamed him (I flame people recreationally, no offense
intended). I see you're just Phil Innes, but for a while you fooled
me with your high-brow posting, as I thought you were some sort of
chess programming intellectual.

Personally I find the /specific/ line evaluations in this new title to be
revealing - and i think the author has done us all a service in removing
these ideas from the general to the particular. Even in highly romantic
play, its fascinating to see Fritz analysis - look at this!

1 e4 e5
2 Nf3 Nc6
3 Bc4 Nf6
4 Ng5 Bc5

[on 4... d5 Fritz scores it +0.15]

5 Nf7 Bf2 [now Fritz scores it -3.44]

This is the infamous Traxler Gambit, and in a few moves there are immense
complications. What the author says as a second reason to look at this
opening [the first is that Tal played it against 10,000 Pravda readers] is
'this ultrasharp open position highly likely Unique Movement ["forced"]
Sequences are on the verge.

Now - 50 years of amateur and GM analysis [see de Zeeuw and Christophe&Moll,
eg] have argued the hell out of 6. Kf1 compared to Kf2 - but Fritz is right
onto it!

OK, if I understand your point here--you're saying that Fritz in a few
seconds basically is "right onto" the correct analysis that 50 years
of amateur and GM effort have found. Doesn't that support MY point,
of Camp #2? That computers _can_ do an excellent job of analysing
positions, and not just tactical analysing? Sure seems like it.


I won't traduce his book by copying it out too exactly here,

Is "traduce" a word? Lemme Google it at Ask.com... http://www.m-w.com/dictionary/traduce
1 : to expose to shame or blame by means of falsehood and
misrepresentation 2 : VIOLATE, BETRAY traduce a principle of law.
I'll be damned, it is a word. Lern something new everyday, thanks
Phil.

and we can wait
for the reviews, but to follow the above, in a line [de Zeeuw] at move 9 the
analyst saays equality, but Fritz is scoring it -1.1 for black.

Now, the move that Fritz can't find in this line [which follows 6 Kf1], is
the MAMS 9th move, Nd4 which brings down the risk factor from a huge +4.3 to
a draw after recommended /10th/ move Bf7.

OK, so I see now. A very obscure line, with 50 years of analysis
behind it, eventually gives a score of even from +4.3 in the
computers' favor. Yes, and I too have such examples from my games
(I'm rated an expert on a good day, Class A about 1950 Elo on a
ordinary day). But they are rare. Most of the time the PC is correct
in its evaluation even after a few seconds of calculations.


There are so many points illustrated by this game in the text, and why Fritz
insists on some lines for White which lose in 9 moves, and why it ignores
other MAMS moves like de Zeeuw's line with 12. Qf1 which at 19 is = is the
strangness of its evaluation matrix.

Now - here it is - Fritz is no better at refuting this maddening line

1 e4 e5
2 Nf3 Nc6
3 Bc4 Nf6
4 Ng5 Bc5
5 Nf7 Bf2

than 50 years of players, BUT - if you want to know the secret of beating
this Traxler line with its improvident 4. ... Bc5 then a few 'moves by hand'
or MAMS moves, 8-13, allows Fritz to romp home with a solution to win for
white in 20

I'll wait for the MAMS book but now I'm confused. Don't bother
explaining it, but now you seem to say that after using the 50 years
of analysis to get to equality from +4.3, Fritz then finds some moves
at moves 8-13 to allow Fritz to win after all (!). Well, again,
doesn't that support my thesis of Camp #2? Sure seems like it.


Now, to return to our evaluation thesis [or at last mine] what is happening
above at moves 8-13 that Fritz can't find: No matter what ply depth you
allow it?

I'll be good god damned to hell! Let me burn in hell Sweet Jesus and
die of brain cancer!! You totally have f ucked up my brian now, I have
a pounding headache. I have no idea what you are saying. I give up.
You win, Phil Innes, you fucing win.


Cordially, Phil Innes


S hite, I thought I was conversing with an intellectual but it turned
out to just be Phil Innes. The moniker "Chess One" should have given
it away.

RL

On May 4, 9:48 am, (Dr A. N. Walker) wrote:
In article .com,
help bot wrote:

--
Andy Walker, School of MathSci., Univ. of Nott'm, UK.


Andy, or Dr. Walker, as the case may be, please be advised that I
don't often post here (I hang out at alt.global-warming, where my best
Usenet posts may be found), and I don't know your motivation in
replying to help bot, but I've quickly deduced that help bot, like me,
is a troll, but with the difference that my posts actually make sense
and are thought provoking instead of just being provocative nonsense.

Please do yourself a favor and put help bot in your .kill file.

RL

On May 4, 11:39 am, JohnnyT wrote:
[deleted: pean to Rybka, about how great it is and radically
different]

Bull Shiite. Rybka is kicking fanny because the number of games
played by it are still small. It's too early to tell. Regression to
the mean will follow.

Do you remember Fruit, and how it kicked everybody's ass a few years
ago? Look at it now.

RL


Rating + - Games Won Av.opp
1 Rybka 2.3.1 Arena 256MB Athlon 1200 MHz 2962 47 -44 252 67% 2837
2 Rybka 1.2 256MB Athlon 1200 MHz 2909 26 -25 845 72% 2748
3 Hiarcs 11.1 256MB Athlon 1200 MHz 2871 51 -51 189 52% 2858
4 Junior 10.1 256MB Athlon 1200 MHz 2867 32 -30 563 72% 2702
5 Junior 10 256MB Athlon 1200 MHz 2851 25 -24 874 70% 2703
6 Hiarcs 10 HypMod 256MB Athlon 1200 MHz 2845 22 -21 1238 73% 2672
7 Fruit 2.2.1 256MB Athlon 1200 MHz 2837 21 -20 1224 64% 2734
8 Shredder 10 UCI 256MB Athlon 1200 MHz 2830 25 -24 857 69% 2689
9 Spike 1.2 256MB Athlon 1200 MHz 2825 30 -29 582 61% 2749
10 Shredder 9.0 UCI 256MB Athlon 1200 MHz 2814 20 -20 1280 66% 2701

raylopez99 wrote:
On May 4, 11:39 am, JohnnyT wrote:
[deleted: pean to Rybka, about how great it is and radically
different]

Bull Shiite. Rybka is kicking fanny because the number of games
played by it are still small. It's too early to tell. Regression to
the mean will follow.

Do you remember Fruit, and how it kicked everybody's ass a few years
ago? Look at it now.


I am not sure where you are getting your statistics from...

But the recommended place I would look at is here....

http://www.husvankempen.de/nunn/40_4...rangliste.html

The advantages of fruit is that it is free, much better than most free
engines, and the source code is available. (Which sort of launched the
new breed of non chessbase engines).

But here you should note, better computers and many more games. Rybka
2.3.1 challenged by over 550 games with a 71% score

raylopez99 wrote:

Andy Walker, School of MathSci., Univ. of Nott'm, UK.


Andy, or Dr. Walker, as the case may be, please be advised that I
don't often post here (I hang out at alt.global-warming, where my best
Usenet posts may be found), and I don't know your motivation in
replying to help bot, but I've quickly deduced that help bot, like me,
is a troll, but with the difference that my posts actually make sense
and are thought provoking instead of just being provocative nonsense.

Please do yourself a favor and put help bot in your .kill file.


Help bot is muchos better value than you Lopez. Pram-pushing yuppie
scumbag..

On May 4, 4:06 am, David Richerby
wrote:
help bot wrote:
Ah, but you seem to forget that the reason for Crafty's success was
simply the fact that Cray computers were faster.

Not entirely. Was Crafty the first program to use bitboards?

I don't know. I do know that whenever certain "critics"
brought up the subject of the Cray computer's superior
speed, Mr. Hyatt always rolled up into defensive mode,
insisting that his chess program was *inseparable*
from the Cray. My take is and always has been that
without the supercomputer, his program was likely
nothing special.

As far as I know, the learning approach has stemmed from neural
networks, not Cray supercomputers.

That statement makes no sense. The neural network is a programming
technique and is completely independent of the hardware it's run on.

Perhaps you would have an easier time understanding
things if you did not *take them out of context*.

The above comment was a response to Phil Innes'
complaint about academic research. The nearly-an-
IM doesn't like the fact that people like Bob Hyatt
and Feng Hsu focused on winning, instead of the
approach of teaching a program to learn chess on
its own, as with a neural network approach.


Mr. Hyatt, the creator of Crafty, simply exploited the raw speed of
a mainframe he had (virtually unique) access to.

Um. Cray != mainframe. You clearly have no idea what you're talking
about.

Look Doug, you Richerblys have always had a problem
with us "stars" at GetClub. Now if you want to quibble
over the choice of a single word in but one of my
*innumerable* sentences, that's your prerogative. But
you're not fooling me; I know this is really about the
vast rating disparity between us; about your ego. ;D

-- help bot