On Wed, 07 Jan 2004 05:47:06 GMT, "Derek Wildstar"
wrote:


"Dr. David Kirkby" m wrote
in

I accept from your description that the chess engine will not play
moves in a way directly proportial to their occurance in the games in
its database. You have clearly shown that, although your state one
factor will be the number of wins achieved by playing a particular
move.

So far so good, stipulating that frequency is just one factor in the
decision making process about the 'goodness' of the move in book.

But will the chess engine evaluate the position of the board based on
the data in that database, if the position is still in its book ?? I
assume there is no random number generator called score the position.

Based on the prior written description of Rookie 2.0's methodology in-book,
yes there is a form of evaluation of the position's 'goodness', which is not
the exact same evaluation of the positition out-of-book. For this evaluation
is again just a single factor, weighted, in determining the probability of a
book-move being played... i.e. frequency of occurance.

I suspect that in cases where a 50/50 result occurs, some randomness is to
be expected, of course, this position when seen again, will no longer result
in that same 50/50 evaluation, if the result of that last occurance is now
weighted as well.


If so, then that still leaves the possibility that you errornously
believe the best move to play from Position A is move B, basing this
on

a) The chess engine thinks its the best move AND
b) it lead to the most wins by GMs from that position.

Based on (a) and (b), one might conclude incorrectly move B is best,
simply because both the chess engine's opening book and your database
are both based on an identical set of games.

Again, I could be completly off track here. It is certainly not
something I know much about (as I've clearly shown), but the issue did
get me thinking a bit.

Dr. David Kirkby

I think you have unrealistic expectations of opening books. And before you
think I'm the last word on books, I'm certainly not, but hopefully any
errors will be corrected before they give you any bad-habits!

Are you assuming that a decided upon book-move is the best move in that
position? This is the theoretical aim of pattern-recognition, but the
reality falls from the mark. Especially when you are under the impression
that a population of GM games is the appropriate foundation for a book.
Perhaps basing a book on a set of games is correct for opening practice, or
specialized training, but as a general purpose aid to increasing the overall
strength of a program, it's not correct.

A general purpose opening book instead should have a hand picked set of
positions that covers most, if not all, tournament variations, most, if not
all, principle variations and Main Lines, and known traps, pitfalls and
blunders, especially for lines that can cause the depth of search to stay
depressingly shallow, keeping the correct line hidden.

None of those conditions are met by including a list of games, however
broad, especially by high rated players. They simply do not go into lines
that amateurs do. I think you know all this already, or are close to knowing
this, and you are wondering what exactly that DB of 27K games is going to do
for you...

I do not think incorporating that list into a general purpose book will do
you much good. However, not willing to discount it as a worthwhile
experiment, may I offer a suggestion, to put the issue to a test:
Incorporate the games into an existing book, and then run an engine-match
tournament with four players:

1) General Purpose Book
2) General Purpose Book + GM Games
3) GM Games
4) No Book

See where I'm going with this...?


I wanted to say a bit more about the problems of using a game list to
try to generate a perfect opening book:

1. As Steve Lopez (in the Chessbase Technical Notes) has pointed out
statstics (regarding the frequency a particular move has been played
and that moves performance) can be deceptive.

Say in some opening line White has played 14.Nxf6 in 20 games with a
60% performance rating and as a result Black may have even abondoned
the line that led to the position where white plays 14.Nxf6, then some
enterprising GM deeply analyzes Blacks subsequent play and finds a way
for Black to get an advantage by force. Even if he wins the game
White's performance will only drop to 57% - falsely indicating that
14.Nxf6 is still a pretty good move.

2. One way of getting around the statistical problem is to use a
feature of Bookup called backsolving. Essentially backsolving starts
from the end of all games in a collection and propagates the
evaluation back up the move tree (using +- as the evaluation when
white wins, and -+ as the evaluation when black wins). So if you
backsolved the 21 games with 14.Nxf6 the game where the GM forced an
advantage to Black and won the game would override other games and
give 14.Nxf6 an evaluationj of -+.

Unfortunately backsolving has certain limitations when it relies
solely on game results: A player can lose on time in a won position.
A player could play a three-fold repition allowing a draw in a won
position. A player can reach a won position and through a series of
mistakes throw away the win and even the draw and eventually lose.
This can be accounted for by doing extensive analysis of games rather
than relying on their results to get a good evaluation for a given
position.

3. Lastly even if the evaluations for a moves are completely accurate,
they can result in situation where the computer as white has an
advantage (+/-) or even a won game (+-) when it leaves its opening
book, but the program does not know how to play the subsequent
positions. This is often true if the advantage is positional and
requires a player to do long term strategic planning in order to
realize the win; this is eactly what most programs are notoriously
bad at. I have seen humourous situations where computers drop out
book with a position that a human being recognizes as completely won
and the computer's evaluation of the position is that its opponent has
a winning advantage.

Because of the above, many commercial chess program vendors have an IM
or even a GM on staff to help tweak the opening book. A few years ago
the chess site for Rebel would include some notes about the tweaking
done to the opening book for Rebel; I don't know if they still do
this. In some of the man-machine matches of the last few years with
Kaspraov and other GMs the reports on the matches would talk about
how the opening book needed to be modified to stay away from close
positions where a GMs ability at strategic maneuvering would give him
the advantage.

When a computer is playing humans below master level the errors in the
opening book aren't so critical. Although, we do see people bragging
how they can beat a strong program time after time, which is only
because they have found a new flaw in the programs opening book.



Mike Ogush