PolyGlot Book: The BIN Opening Book Format Explained

A PolyGlot book is an opening book stored in the BIN format that the PolyGlot program introduced. It lets a chess engine or interface look up opening moves from a single compact file, and it is one of three widely used book formats, next to ChessBase’s CTG and Arena’s ABK.

What PolyGlot is

PolyGlot itself is a piece of open-source software released under the GPL. It was developed by Fabien Letouzey and modified by Fonzy Bluemers. Its main job is that of an adapter: it lets engines that speak the UCI protocol work with interfaces built for the Chess Engine Communication Protocol, the one used by WinBoard and XBoard. Version 1.4 added a simple opening book implementation, and that implementation is what people mean by the bin format. Since 2018 PolyGlot has been scheduled for retirement from XBoard and WinBoard, and the newest WinBoard beta does not include it.

What an entry holds

A binary book is sorted by a hash value for each position so that a lookup is a binary search. In the PolyGlot format every item takes 16 bytes and holds four values: the position hash, a move, a weight and a learning value. The weight lets the program choose among several book moves with different probabilities, so it need not play the same line every game. Compared with a text list of games, the file is small and fast, but it needs a tool to build or edit it, because it cannot be read by eye.

Relation to other formats

Both the BIN format and the CTG format store an opening book as a table of positions rather than as games. The guide to the CTG file describes a ChessBase book and how it is opened and converted. The role of books in programs generally is covered under computer chess.

Practical note

Anyone choosing a book file for an engine should check which of the three formats the interface or engine reads before downloading one.

The layout of one item in detail

The format specification kept by H. G. Muller fixes every byte. An item is 16 bytes: a 64-bit key, a 16-bit move, a 16-bit weight and a 32-bit learn field. All integers are stored highest byte first, whatever their size, and the items are ordered by key with the lowest key first. A book of 10,000 positions is therefore a file of exactly 160,000 bytes, and a program can jump to the middle of the file, compare keys and halve the range until it finds the position or proves it absent. No index is needed.

How the key is made

The key is a Zobrist hash, built from a table of 781 fixed random numbers. The table has four parts: 768 values for pieces (twelve piece types on 64 squares), 4 for castling rights, 8 for en passant files and 1 for the side to move. The key is the XOR of the pieces, castling, en passant and turn terms. Because every program uses the same table of numbers, any program that follows the specification computes the same key for the same position, and a book written by one tool can be read by another. The key says nothing about the history of the position, only about the arrangement and the rights, much like a FEN record.

How the move is packed

The 16-bit move field splits into five groups of three bits, counted from the least significant bit: the destination file, the destination row, the origin file, the origin row, and the promotion piece. The promotion code runs 0 for none, 1 for a knight, 2 for a bishop, 3 for a rook and 4 for a queen. A move stored as zero, which would be a1a1, is a null move and is ignored by the reader.

Castling gets a special encoding. The king move is written as the king moving onto its own rook’s square: e1h1 for White short, e1a1 for White long, e8h8 for Black short and e8a8 for Black long. The specification points out that these could in theory be legal non-castling moves, so a careful reader checks that a king stands on e1 or e8. In the Chess960 variant the same idea applies, with the king square as the origin and the rook square as the destination, which keeps books compatible with Fruit derivatives. The chess programming reference nonetheless says PolyGlot as a program is useless for anything other than ordinary chess.

Where PolyGlot stands today

PolyGlot was developed as an adapter, and the book reader was an add-on in version 1.4. The format itself is deliberately plain. It stores no names, no comments and no game history, which keeps it compact but means that a book cannot be inspected without a tool.

Choosing between BIN, CTG and ABK

A BIN book is open and documented in full, so a reader or a builder for it can be written from the specification. CTG is the ChessBase format and ABK the Arena format. The main practical question remains whether the engine or interface in use reads that format, since the three are not interchangeable and need a conversion tool to move between them. An opening book in any of the three plays the same role: it answers the first moves quickly and returns control to the search once the game leaves the stored lines.

Questions