Opening Book in Chess: What It Is and How Programs Use It

An opening book, in computer chess, is a store of opening positions and moves that a program looks up at the start of a game. While the game stays inside the book the program plays at once, without searching. When the opponent plays a move that is not in the book, the program is out of book and returns to normal search. The same phrase also names printed books on openings, from the Bilguer Handbuch of 1843 to the five volumes of the Encyclopedia of Chess Openings, but the program sense is the one this entry covers.

How a book is built

Binary books are usually keyed by a hash value for each position, with the items sorted by hash so the program can find a position by binary search. An item can carry extra data such as how often the position occurred, results by colour or the move weights. Book moves can be chosen at random according to those weights, which gives variety even though the search itself is nearly deterministic. The PolyGlot book format uses 16 bytes per item for the hash, move, weight and learning values.

Text formats such as EPD and PGN are easy to read but large and slow to search, and they usually carry no weights.

Formats and tools

Three formats are in common use: ABK from the Arena interface, BIN from PolyGlot and CTG from ChessBase. The article on the CTG file explains how a ChessBase book is opened and converted.

History

The first published work on implementing a book for a chess program came in 1974 from Ya. Yu. Gol’fand and Aaron L. Futer. Ken Thompson fed opening lines from the Encyclopedia of Chess Openings into the Belle program by hand for almost three years, reaching about 300,000 moves, and later wrote a program that read the encyclopedia automatically in days. In 1999 Marc-Philippe Huget released Book Builder, which converted PGN files into books.

Strengths and limits

A book saves time and avoids search errors in deep opening theory. A book built from game collections can copy serious mistakes, so programs usually follow only positions that occurred a minimum number of times. Readers who want the surrounding context can continue with the computer chess page.

Opening books on paper

The printed ancestors set the pattern that program books copy. The German Handbuch des Schachspiels, known as the Bilguer Handbuch, appeared in 1843 and pioneered theory tables, and it stayed a standard reference until Modern Chess Openings took over in the early twentieth century. The Encyclopedia of Chess Openings, published by Šahovski Informator (Chess Informant) of Belgrade, came out in five volumes between 1974 and 1979. It sorts every opening into one of 500 codes: five letters, A to E, each split into 100 subcategories. A covers flank openings, B semi-open games other than the French Defence, C open games and the French, D closed and semi-closed games, and E the Indian defences. The volumes use symbols instead of prose, which is exactly why they were useful to programmers: a line and its verdict can be turned into data.

Text books and binary books

A text book stores either the final positions of its lines, as EPD records, or the lines as individual games in PGN. Both are easy to read and edit and both lack weights. They are also large and slow to search compared with a binary book, in which each position is reduced to a hash of typically eight bytes with a few statistics attached, and the entries are sorted so that a lookup is a binary search. Common lines are normally stored deeper than rare ones, since a program will meet them more often.

Three ways to produce a book

A handcrafted book is written by a person, for example to prepare for one grandmaster opponent. A book from game collections takes positions from selected games up to a set depth, and as noted above it can inherit mistakes, which is why programs ignore positions seen fewer than a minimum number of times. A third method computes the book: an engine evaluates the positions of the book tree to rank the moves or to decide whether a branch should be expanded. Marc-Philippe Huget’s Book Builder of 1999, released with his engine La Dame Blanche, turned PGN files into the format that Kathe Spracklen had proposed in her 1983 tutorial on representing an opening tree.

Who holds the book

An engine can carry its own book code and data, or the graphical interface can supply the moves, so that the engine starts to search only at the first position outside the book. The interface approach is popular because it is simple, saves space and makes comparisons fair, since every engine gets the same opening. Its cost is that the engine then cannot easily use book-learning methods, in which a program adjusts its book from its own results. Robert Hyatt described a method of that kind in 1999 under the title Book Learning, a methodology to tune an opening book automatically.

What a book is good for

The chess programming reference gives three purposes. First, saving time: a book move costs almost nothing. Second, quality: a search can miss a deep tactic or a long-term positional compensation that the theory already settled. Third, variety: when several book moves are stored, the program can pick one at random, so the same game need not repeat. Bernd Blasius and Ralf Tönjes published a paper in 2009 on Zipf’s law in the popularity distribution of openings, a few popular lines taking most of the use.

Related terms

A program book holds the results of earlier work and makes no calculation at the board. It is distinct from an endgame tablebase, which sits at the other end of the game and gives exact results rather than recommendations. Position strings inside text books are normally written in FEN form.

Questions