Beruflich Dokumente
Kultur Dokumente
html
---------------------------------------------------
Introduction
This page was initially part of my web site on setting up of Winboard engines.
However, I discovered that many visitors to my site, were actually looking for
information on setting up endgame tablebases. Given the fact that my Winboard web
site was already very long, I decided to start a new web site using the
information already available in the original page as a base. I added information
from the various web sites and past posts at Rec.games.chess.computer and the
Computer Chess Club.
I'm a layman who is largely unfamiliar with the technical details of endgame
tablebases. As such much of the technical detail here is only available thanks to
the kind permission of experts such as E.A. Heinz,Guy Haworth, (both frequent
contributors to ICGA journals on this topic) Anders Thulin (from
rec.games.chess.computer) , Robert Hyatt, Bruce Moreland and especially Dieter
Buerssner (all of those are authors of strong chess programs that use endgame
tablebases). I would be grateful if you can email me if you spot any errors.
I hope that you will find this web site useful. If so, please help sign my guest
book.
Yours Sincerely,
Aaron Tay
01-04-2001
My PGP - DH/DSS 4096/1024 Key
Further information
As of April 2006, one of the best places to obtain further information about
endgame tablebases is at the following forum . I highly recommend it, if you have
further questions, particularly technical ones.
Latest Changes
(To skip to FAQ table of contents )
31-05-2003 Made available replacements for flawed EDGB file sets installed by
Chessmaster 9000 .
01-03-2003 Yace now released uses 3 and 4 piece bitbases stored in memory.
23-09-2003 More 6 piece tablebases loaded on Hyatt's FTP site. Including 6 piece
tablebases with pawns.
23-05-2004 New viewer site by Convekta allowing access to 6 piece tablebases added
25-07-2004 DVD Endgame Turbo by Chessbase contains a corrupted file for the KRPPKR
endgame.
01-12-2005 Add more sites for searching online Nalimov tablebase results. Added
link to emule collection of tablebase sites.Added Scorpio uses bitbases and offers
dll for use.
* [A.1] What is a endgame tablebase? How do they work? How much stronger will
it make my chess program play?
* [A.2] What are the different endgame tablebases formats out there? What are
the differences?
* [A.3] Where can I learn more about the endgame table formats?
* [A.4] What are the Chess programs that support endgame tables? Which format
do they support?
* [A.5] Where can I get endgame tablebase?
* [A.6] Where can I buy tablebases?
* [A.7] Where can I download tablebases?
* [A.8] How do I generate tablebases on my computer?
* [A.9] How large are the tablebase files? Can I put then all into one
directory? Do I have to use a complete set of 5 men tablebases?
* [A.10] Questions about 6 men endgame tablebases. Which are the most useful?
* [A.11] How do I find out which tablebase is corrupted? How do I know the
endgame tablebases are working?
* [A.12] What is datacomp.exe? Where can I get it?
* [A.13] How do I get Crafty to work with Endgame tablebases?
* [A.14] How can I get Fritz to use Endgame tablebases?
* [A.15] What is the difference between tablebases download from Dr Hyatt's
ftp site and those on the Chessbase endgame turbo CD? Can they be used together?
* [A.16] What are the files ending with .tbs? What about those with nbb and
nbw? Do i need both?
* [A.17] Help, the endgame tablebases are not working properly!
* [A.18] Can I use tablebase files in zipped form?
* [A.19] Where can I get a useful graphical browser to view endgame
tablebases? What about an online searchable database?
* [A.20] Misc questions
[A.1] What is a endgame tablebase? How do they work? How much stronger will it
make my chess program play?
Chess endgame tablebases (EGTBs) are special databases that stores all possible
positions with a given material balance and their results.
Note about terminology
Why not call them endgame "database" instead of "tablebases"? I have no idea, in
fact Chessmaster's version of them are called EGDB (EndGame DataBase) , but
regardless of what they are called they are similar in principle. In this FAQ, I
shall use the terms Endgame tablebases because these are the names they are most
widely known by.)
For example, there is one tablebase file (normally named KNNKP) that covers
positions where one side has a King and 2 knights, while the other side has a king
and a pawn and yet another tablebase file (KQK) that covers all positions where
one side has a King and Queen, while the other side has a lone King.There are more
endgame tablebase files of course, though naturally the more pieces on the board,
the bigger the Endgame tablebase file.
Each endgame tablebase file allows the user to check what the result is with best
play. For example, the KNNKP tablebase,can allow you to input a specific position
with such a material configuration, and find out what the result should be with
best play. Nalimov endgame tablebases (one of several types of tablebases),for
example could tell you if that position is a forced win and if so in how many
moves, or a draw with best play.
One thing to note is that,the raw endgame tablebases files are highly
compressed,and require that you use some kind of graphic browser software, before
you can view the files. .However, many Chess playing programs [See here for a
list] can use endgame tablebases automatically,which will allow them to play that
endgame perfectly.
Most chess engines don't even need to reach the position covered in an endgame
tablebases to use the tablebases. For example, a few moves before such a position,
the engine calculates (but does not play yet) a series of exchanges that leads
directly to a position in the tablebase. The engine will then look up (or probe)
the tablebase and get the results for that hypothetical position. This should help
improve play of course.
But how are tablebases constructed in the first place?Basically endgame tablebases
are database files of stored endgame positions calculated using retrograde
analysis.Here's a basic layman explanation on how tablebases are constructed
Here's another explanation by Bruce Moreland, author of Ferret (who has generated
endgame tablebases of his own design).
How these things works is that all possible positions are examined, and
positions where one side is mated *now* are marked (apparently legal piece
configuration, with one side to move, in check, and unable to get out of check).
These are called "mated in 0" positions.
Once this is done, all positions with each side to move are examined, and if
any of them can reach a mated in 0 position, these are marked as "mate in 1".
Now, we look at everything again and search for positions where all possible
moves lead to "mate in 1" positions. These are marked as mated in 2.
And so on, until no progress is made. The rest of the positions are proven to
be draws.
It's a little trickier if it's possible to convert to a won ending, if you are
trying to make a "distance to mate" database. You have to be careful of this idea
of "no progress", because there are gaps. There might be a mate in 105 in KBP vs
KN, but there might not be a mate in 95. This is because the mate in 105 might
involve immediate conversion to a KBN vs KN, and yes, such positions do exist.
A much greater gain is achieved by the fact that chess engines can consult
tablebases as a kind of oracle to decide whether to exchange down into a endgame
covered by the tablebase.This allows the range of positions to be covered to be
much larger. A chess engine can for example avoid exchanging down to losing
positions (even if they are very deep) or win by converting into a absolutely won
endgame.
And yet, people have estimated that most engines gain at most 25 elo from using
them, and many have found no significant difference in playing strength.
Experiments results by Robert Allgeuer and Les Fernandez using Crafty and Yace
show no significant difference in playing strength.
Robert Allgeuer did a interesting experiment with Yace. And basically he found
that while endgame tablebases allowed engines to reach and win such positions, in
games where such positions were not reached they played weaker. By considering
games where positions covered by endgame tablebases was actually reached on board
,Yace with tablebases played significantly better (about 115 Elo) On the other
hand, in games where such positions was not reached, Yace with endgame tablebases
played significantly weaker (about 94 elo). The implication was that in such
games, using endgame tablebases was a liability. Robert writes "Apparently
statistically for each game they successfully maneuver into a won 5 piece
position, they also lose another one during the preceding endgame, so that overall
this effect more or less cancels out the advantages of tablebases."
The above results can be still extended by using different time controls, egtb
caches, engines (some like SOS claim to be heavily dependent on tablebases because
the author has stripped out all endgame related knowledge covered in tablebases) ,
but it seems that using the current set of tablebases (3-4-5 pieces), you can't
expect more then a minimal strength increase if any. Still you might want to use
tablebases for learning and analysis purposes.
[A.2] What are the different endgame tablebases formats out there? What are the
differences?
There are various kinds of Tablebase formats widely in use, including Ken
Thompson,Steven J. Edwards,Eugene Nalimov Tablebases and Chessmaster/De Koning
EDGB (in the order in which they first appeared). While these are popular, some
programmers have chosen to use endgame tablebases of their own design, for example
Bruce Moreland's Ferret
In general though, they fall into 2 types of tablebases. Distance to mate (DTM)
and Distance to Conversion (DTC)tablebases. Roughly in layman terms, the
difference is as follows.
* Distance to mate (DTM) - DTM tablebase stores the shortest possible mate (in
ply) for each position. Eg Nalimov tablebases
* Distance to conversion (DTC) - DTC tablebases stores the shortest number of
ply between each position and a "conversion". A conversion is either a pawn
promotion or a piece being captured or checkmate. Eg Thompson tablebases.
* In addition, Distance to zeroing (DTZ) and the superior Distance to rule
(DTR)tablebases have being proposed to overcome the fact that DTM and DTC do not
recognise the fifty move rule. Only in theory so far.
The tablebases also various in other ways Dr Robert Hyatt [Author of Crafty]
explains the differences between them as follows [in a posting to
rec.games.chess.comp 26/10/2000]
"Edwards (Tablebase): Distance to mate values stored. The main problem with
these is that they are larger than the others.
Nalimov (Tablebase): Distance to mate values just like Edwards, but Eugene's
files are compressed, and they may be used in the compressed form, with no penalty
of any kind. Rather than way over 30 gigabytes for all of the 3-4-5 piece files,
you end up with about 7.5 gigs.
Best choice: Nalimov. Nearly every chess engine supports those... "
This was of course written before the release of the De Koning EDGB format used by
Chessmaster 9000 only in August 2002. Nalimov still remains the most popular
tablebase format since it's supported by the largest number of engines.
De Koning EDGB - Propertry format used by only Chessmaster 9000. Like the Nalimov
tablebase it is a Distance to mate tablebase. Unlike the Nalimov tablebase though,
it stores only wins from the white point of view (similar to Thomson)."(It)
searches only for wins for White. All positions that are not proven to be won by
White are stored as unknown. These unknowns might be draws, or lost by White, or
(in large numbers) illegal." [From FEG or Final endgame generator for Chessmaster
9000 ]
Note the original version of the endgame generator had a minor flaw, and the EDGBs
that shipped with the CDs also had this flaw. See here for more
* Nalimov's format has KxyKz and KzKxy merged. Eg in KRBKQ the wins for white
(with RB) are stored as well as the losses for white. In Thompson's and De
Koning's white can only win or not win. [From FEG or Final endgame generator for
Chessmaster 9000 ]
This change means that besides the usual pawn promotion case problems there are
other "dependencies" problems between EGTB files (my comments)
* Chessmaster 9000 currently does not probe the Tablebase in search but
according to email correspondence this is a decision design and there is nothing
in principle that stops this from occurring.
* De Koning EDGB are slightly smaller than compressed Nalimov endgame files in
general.The full De Koning EGDBs came up to about 5.7 GB , the EGDB's can function
with only either the WTM or BTM files.[Note during generation, you need both
files!] This can reduce the space required to as little as 2.7 GB which is
extremely light on hard-disk space compared to the 7.5 GB (as quoted by most
people) required for Nalimov tablebaes.
Nalimov tablebases are nearly "perfect" since they take into account en Passant.
However, they don't take into account castling, but this flaw is probably of
interest only to Chess problem Hobbyists.
In general, though almost all modern Chess programs [including most Winboard
programs] use Nalimov Tablebases partly because they are non-propriety and partly
because they are more efficient. Also some of the 6 pieces Tablebases are now
available in the Nalimov format. The Nalimov tablebases comes in 2 forms,
uncompressed and compressed. The compressed ones end with the extension "emd".
Most of the modern Chess programs( e.g. Crafty supports to the use of the
compressed Nalimov format from versions 16.5 onwards) can use the tablebases in
compressed form by uncompressing and using them on the fly as needed. One
exception I'm aware of is a old version of Esc , a Winboard program released on 4
Feb 2001 which uses only the uncompressed form. If you generated the Nalimov
tablebases yourself (see Section [A.8] ), they will already in the uncompressed
form before applying datacomp.exe. datacomp.exe can also be used in reverse to get
uncompressed Nalimov tablebases from compressed ones.
Other uncommon types of Tablebases
Bitbases
There are other types of less common tablebases that work quite differently from
those mentioned above. They generally go by the term "bitbases".
The term "bitbases" comes about because it requires only one bit to store the
results for each position. Again, the naming might differ, e.g. knightdreamer
calls them Endgame models (egm).
E.A Heinz was probably the first one to pioneer the concept. They were integrated
into his chess program "DarkThought" by late 1996 and proved their practical
viability during the 1997 WMCC in Paris where "DarkThought" employed them
throughout the whole tournament. In addition, he also published an article on it
in ICCJA (see below) in 1999. Eventually bit-based endgame databases started to
appear in many other microcomputer chess programs (e.g. According to E.A
Heinz,Chrilly Donninger integrated them into "Nimzo" based on the ICCA Journal
article cited below).
E.A. Heinz. "Knowledgeable encoding and querying of endgame databases." ICCA
Journal, Vol. 22, No. 2, pages 81-97, June 1999. (published by the ICCA; preprint
available from http://lcs.supertech.lcs.mit.edu/~heinz/ps/know_edb.ps.gz or
http://www.i-u.de/schools/heinz/ps/know_edb.ps.gz )
Unlike normal endgame tablebases where there are only a few standard formats used
by almost all chess engines, so far no open standard bitbase format exists and the
few chess engine programmers who have implemented this idea, use bitbases of his
own design. These are the ones that I'm aware of which use a form of bitbases.
3) Yace versions (after 0.68) also use bitbases.Yace uses bitbases for 3+4 pieces
only and this is stored in memory . This allows Yace to access them in Quiescience
search without much slowdown compared to using Nalimov tablebases.
5) Delfi has internal bitbase tablebases (stronger side wins/draws) for almost all
the 4 men tablebases except N vs P and R vs B and KRPKR. For more information.
6) Scorpio you uses bitbases up to 4 piece. He also offers a dll to access these
bitbases for those interested. Fruit Gambit and latista uses this too.
7)GAFS
Guido from G6 [ a group of 6 Italians Winboard engine authors] has announced a new
endgame tablebase format - GAFS .Guido writes
It is also possible to generate EGTBs with 2 bits per positions (win, loss,
draw, illegal position). There is also a very small object file to link to the
chess program in order to use 8 bits and/or 2 bits EGTBs and this is possible
inside the same run. The idea was that 2 bits tablebases could be sufficient
during the search process before entering a tabulated ending, while inside the
ending only 8 bits EGTBs can give the correct move. I don't use compression at
least for now. To the persons that tried to use my EGTBs I suggested to read the
result on the disk file (without loading the file in RAM), and save it in the hash
table. If the number of different positions to read, i.e. access to the file, are
not very high (no more than some thousands), the cost in time should be
acceptable, as a single access to the disk can cost some ms.
8) By 2005, there are now many more engines that use bitbases, many of their own
design. Examples include Kiwi, Slowchess,Glaurung .
I have also heard of tablebases being generated for other Chess variants like
Losing Chess , but I have not be able to find more information.
NEW! [04-01-02] Sjeng 11.2 now allows you to self generate 2 and some 3 tablebases
for suicide Chess.KKFchess does the same for various variants as well as
supporting normal Nalimov tablebases.
If you have more information,web sites with more information about endgame
tablebases to link to, or clarifications that you want to make, please emailme
Note: For the rest of the FAQ, unless I state otherwise, endgame tablebases refer
to Nalimov tablebases.
[A.3] Where can I learn more about the endgame table formats?
For Nalimov tablebases you need written permission from him. You might want to
look at probe.txt which is part of TBGen.zip. Which can be normally be found in
most places which allow EGTBs to be downloaded)which if I understand correctly,
gives instructions on how to hook up your Chess Engine to probe Nalimov's
tablebases.Looking at open source programs might also help for example, "probe.c"
file from BeoWulf or egtb.c file from Crafty.This post might also help .
Attention! For Chess programmers trying to learn more about the format to figure
out how to make their Chess programs use Eugene Nalimov tablebases, : Dr Ernst A.
Heinz who is the author of the Chess program, Dark Thought and author of the book
" Scalable Search in Computer Chess " writes:
* E.A. Heinz. "Endgame databases and efficient index schemes."In ICCA Journal,
Vol. 22, No. 1, pages 22-32, March 1999. (published by the ICCA; preprint
available from http://supertech.lcs.mit.edu/~heinz/ps/edb_index.ps.gz
You can download for free the FEG or Final endgame generator for Chessmaster 9000
. It comes with a nice document file that explains clearly in layman terms what
endgame tablebases are and summarises some of the issues involved.
[A.4] What are the Chess programs that support endgame tables? Which format do
they support?
Fritz 6.0+
Nalimov
Junior 6.0+
Nalimov
HIARCS 7.32+
Nalimov
Nimzo8+
Nalimov
Nalimov
Shredder 5.0+
Nalimov , Thompson
Thompson
Patzer**,Yace Paderborn
Nalimov , Bitbases
Knightdreamer,Delfi
Bitbases
Crafty***
Nalimov
Edwards
Nalimov
SCID
Nalimov
Madeleine
* The latest update 02/04/2001.Refer to Rebel Web site . Prior versions did not
have any endgame tablebase support.
*** Old versions of Crafty used Edwards Tablebases.Crafty 15.21 onwards supports
only Nalimov tablebases. Crafty 16.5 onwards also supports the use of compressed
Eugene Nalimov tablebases.
**** For a full list of Winboard engines that support tablebases , refer to Engine
Overview page by Leo Dijksman
Effects of GUI like Chessbase/Chessmaster
Additional note, when Chess engines are used in Chessbase GUI or Chessmaster 9000,
the GUI will use any tablebases available once the position on the board matches
that of the relevant tablebase. But this does not mean that the Chess engine
itself "uses" the tablebases in it's search etc.
[A.5] Where can I get endgame tablebase?
Nalimov tablebase
If you own only Fritz 6 or any Chessbase engine, you will have the 3-4 men
Compressed nalimov tablebases, but you might want more. You basically have 3
choices. You can buy , download or generate them. [From easiest to most
difficult.]
Chessmaster EDGB or De Koning EDGB
If you own Chessmaster 9000, it will already come with all 4 piece tablebases
together with some 5 piece tablebases, particularly the KRPKP and KQPKQ and their
respective depedencies.The set that comes in the CD is as follows
KQRKR,KRQKR,KRKRR,KRRKR,KRBKR,KRKRB,KRNKR,KRKRN,KRPKR
KRKRP,KQQKQ,KQKQQ,KQRKQ,KQKQR,KQBKQ,KQKQB,KQNKQ,KQKQN
KQPKQ,KQKQP,KBBKN,KNKBB,KNNKP,KPKNN
Thompson Tablebases was the first endgame tablebases to be widely used by chess
programs. Older versions of chessbase , Fritz etc used to support them.
Unfortunately, they are not used much today, because they are largely inferior to
Nalimov's. (E.g They don't cover en-passant, not all 5 piece tablebases available,
they are larger, slower to generate and use , and only one side to move was
generated). I don't know of any place where they are available for download (not
sure if that's legal), and while they used to be for sale, it's likely they are
not now.
Edwards Tablebases
There's a generator available somewhere on Dann Corbit's ftp. Also 3-4 piece
tablebases available for download on Dann's site.
Bitbases of various formats
Because bitbases are generally small and available for 3-4 tablebases (so as to
fit in memory), most either come with the program as separate files
(Knightdreamer,Nimzo), are integrated internally (Delfi), or can be generated by
the program itself in a couple of minutes or seconds (Yace).
[A.6] Where can I buy tablebases?
Buying Nalimov tablebases
You can buy The Chessbase DVD Endgame Turbo 2 which is a 5 DVD set that has all of
the Eugene Nalimov 3,4,5 and some 6 piece tablebase set. Conveka makers of Chess
Assistant also sells all 3-4-5 Nalimov tablebases in either a DVD set and another
set consisting of the above plus some 6 peice tablebases in 2 DVDs.
Buying Chessmaster EDGB or De Koning EDGB
You used to be able get Thompson Tablebase with certain software packages like the
"World Champion package" from Millennium System. Chessbaseused to sell a 4 CD set
of 5 men Thompson Tablebases both separately and for use with Chessbase 7
I have no idea if Edwards Tablebases are available for sale.
[A.7] Where can I download tablebases?
Downloading Nalimov tablebases
You could download them file by file for the 3,4,5 and some 6 men Nalimov
compressed Tablebase from UAB ftp. This is the original source and will always
have the latest and most complete sets available as soon as Eugene Nalimov sends
them. But see comment on 6 piece tablebases. You will need a fast internet
connection if you want to download the 5 men tablebases which are very large. ,
These files are already in compressed emd format, and it doesn't help to further
compress them with pkzip etc.
Alternatively, Frank Quisinsky offers all the 3 and 4 men tablebase for download
in one 30 meg file.
Another site that you might consider looking at if the UAB site down (unless you
want 6 piece tablebases) is is at http://www.chesslib.no/ [click on 'downloads'
at the top] .
If for some unusual reason you need uncompressed Nalimov piece sets, you should
uncompress them using datacomp.exe that comes with tbgen.exe.
If all these sites are down, you can try downloading them via emule .
Downloading Chessmaster EDGB or De Koning EDGB
Edwards Tablebases (35 meg) are available for download at Dann Corbit's ftp site.
[A.8] How do I generate tablebases on my computer?
Generating Nalimov tablebases
Lastly, you can generate Nalimov tablebases using the tablebase generator
tbgen.exe. The same generator is also on the Fritz6 CD. But self generation of 5
men tablebases is not really feasible unless you have a very fast computer with
lots of RAM especially. You can find a guide by Mogens Larsen about generating 4
piece tablebases. Antonio Senatore also made a nicepost at the Winboard forumthat
offers some guidance if you really want to try to generate 5 pieces tablebases.
First download the FEG or Final Endgame Generator. When you choose to do a full
install, all the tablebases will be copied to c:\mydocuments regardless of the
actual directory Chessmaster 9000 in.
Important note: If you use the full installation of Chessmaster with EDGB and/or
downloaded and generated other tablebases using FEG version 3.2 (you can check the
readme file to check the version) , the tablebases you generated and some that
came with the CD are flawed.In a nutshell
Endgames with mates no longer than 1 (eg KNNK) did not contain the black-is-
checkmated positions, or rather their wrong value being "not_white_win". This
affected also some positions leading to black-is-checkmated by capture (eg KNNKR
with the mating move being NxR).
This bug will affect Seven of the 4 piece endgame tablebase and Thirty five of the
5 piece endgame tablebase.
Of those that came with the Chessmaster CD, the following sets:
KNNK,KBKB,KBKN,KBKP,KNKR,KNKB,KNKN,KNNKP,KNKBB falls prey to this bug.
I have generated these file sets and put them into one file for your download. You
can them unzip them over the EDGB files installed from the CD. The chessmaster
EDGB file sets are 12mb zipped and contain the Jix and Joo files necessary to be
used in chessmaster. For the 2 five tablebase set KNKBB and KNNKP, I have made
available only the btm files because only those are installed by the chessmaster
CD.
For more details see this post at CCC. Download the new FEG generator here
To change the directory where chessmaster looks for tablebases edit cm.ini (found
in the chessmaster 9000 directory) with a text editor. You need to change the path
ofEGDBPath=whatevernewapth where your tablebases are.
A second bug was found on Dec 26, dubbed "the transparent pawn bug". "Affected
files are the ones where both sides have at least a Pawn and White has at least a
sliding piece. As well as the ones relying on them via promotion (KPPKP) or
capture (6-man only) or both. Regarding 3,4,5-man these are KQPKP, KRPKP, KBPKP,
KPPKP."
Note, that even though there are some 5 piece tablebases on the CD to save space
the 5 piece tablebases on the CD do not come with the WTM files.This means you
have to regenerate all the 5 piece tablebases if you want the complete set due to
the depedencies.
[A.9] How large are the tablebase files? Can I put then all into one directory? Do
I have to use a complete set of 5 men tablebases?
Note the following refers to Nalimov tablebases. Users using Chessmaster format
should ignore this section. Chessmaster 9000 itself is not vulnerable to the
incomplete tablebase problem.
The more tablebase files you install, the stronger the program will be. However, a
full set of 3,4 and 5 Tablebase files takes about 7.05 GB of Hard disk space! BTW
many of the "complete" endgame tablebase sets you can buy on CD are not really
complete and may exclude some of the less important tablebases for example the so
called 4+1 men tablebase sets [KXXXK] on the assumption that those are trivial
enough to solve. Most people download a full set of 4 Piece databases and select
only a few of the 5 pieces.
Note you can mix all the different endgame tablebase into one directory if you
wish. However if you do decide to split up the files among various directories
(there might be some advantage in leaving a small subset of tablebases (clear wins
like KXXXK in a chess engine game as opposed to for use in analysis.See here), you
might need to check whether the Chess engine/interface you use allows you to point
to multiple paths.
NEW! Dieter Buerssner has informed me that in general all Chess engines/interfaces
allow the use of separate paths for nalimov tablebases as long as they are
separated by " commas, semicolons, or (at non- Windows systems) by colons." I have
tried testing with a few engines, most seem to work. For example in Pepito , I add
EGTB_PATH C:\fourpiece;g:\fivepiece.
In addition, Dieter offers another interesting tip. " In console mode, or with
Alt-F1 (in Winboard) you can probably change the path for many engines. For
example for Yace, assuming it had some tbldir in yace.ini, and I saw the analysis
of one position with TBs. Then I want to see it without TBs. I would type tbldir
nul "
For Crafty you need to set the paths (separated by ;) in the command line (or
Winboard.ini) and not in the Crafty.rc file. (I'm not sure if this applies to
newer versions anymore)
However if you put all the tablebases in one directory as opposed to splitting
them up say by tablebase pieces, it can get confusing to keep track of which
tablebases you have and which you don't. You can find some programs that help keep
track of this in Section [A.19].
Generally the 5 piece endgames with rooks are reached most frequently and should
be downloaded .Important ones include KRPKP, KQPKQ and KPPKP. However, there are
some pitfalls that you should be aware of.
If you want to use the KRPKR tablebase [and assuming you have all the 3 and 4
tablebases], make sure that you have the KQRKR one as well. [You might consider
adding KRRKR, KRBKR , KRNKN as well but it's probably not necessary] This is to
ensure that the promotion cases are included.
endgame problem
If you lack say the KQRKR tablebase, some programs refuse to queen the pawn even
if that would lead to a win, because such a move, would cause the program to drop
from a position flagged as "win", to a position that is uncertain [although with a
high score] since they lack the relevant endgame table.
In the position on the left, I have all the 4 men tablebases but only use one 5
men tablebases- KRKRP. With White to play, instead of queening the pawn with e8=Q,
Comet will instead play1 Re4+ ?
1 Re4+ appears superior to 1 e8=Q because from the program's point of view 1 Re4+
leads to a position that is a mate in 16 [obtained from the KRPKP tablebase],
while it is unable to search deep enough to see that 1 e8=Q leads to a even faster
mate in 15 due to the lack of KQRKP tablebase! As such it prefers a certain mate
in 16 to a move that "merely" leads to a high score.
r7/2K1P3/8/4R3/2k5/8/8/8 w - - 0 1
A similar problem can result if you download only the KQPKQ tablebase without
KQQKQ , KQRKQ etc..
Some programs like Crafty and Yace are "smart" enough to avoid this problem, but
some like Fritz,Tiger,Amy,Comet,Gandalf or The Crazy Bishop[all freely available
Winboard engines except for Fritz and Tiger of course] cannot handle this.
noproblem
On the other hand, some positions are not problematic, if there is a quick mate
nearby after the pawn promotion that the Chess engine can see.
There is a short mate in 5 after 1 f8=Q+ . As such Comet and most Chess engines *
can quickly see this shorter mate and will prefer the pawn promotion move to 1 Ke7
[found in the KRKP tablebase] which leads to a mate in 9.
This is made easier by the availability of the 4 men tablebases as the program
need not calculate the position after 1 f8=Q+ Rxf8+ etc since it can use the KRKR
etc tablebases. In fact, the only other legal move [besides Rxf8+] after 1 f8=Q+
is Kh7 which leads even to a quicker mate. All this is child's play for Chess
programs to calculate
Compare this with the above position, White has 15 other replies after the pawn
promotion which doesn't lead to a 4 men tb position.
4K2k/5P2/8/6R1/8/5r2/8/8 w - - 0 1
If you are an author of a Chess program and wish to avoid this problem, what you
can do is this. If you find that in the root position it is mate in X from probing
the n men tablebases, but after trying every legal move you don't find a mate in
X-1 from the tablebase, it must be due to a incomplete tablebase problem and you
search without probing the tablebases.
Also take note that not all programs support all the Tablebases. Yace for example,
currently does not use the 4+1 tablebases [ King and 3 pieces/pawns on a side
versus a alone King] , because such positions are easily won most of the time
barring rare cases [Like double rook pawns, wrong coloured bishop and King vers
King].
You might also not want to download 6 man Nalimov endgame tablebase (see also
here) because I think currently only a few Chess programs support them, although
it's usually a simple matter for the programmers to activate them. These include
Crafty, Sjeng , Quark , King of Kings[all free Winboard Chess engines ] and
perhaps Rebel Tiger . NEW! I have read that the Fritz6 engine itself does not
probe the 6 men tablebases, but only uses them when they occur on the board.
* It appears that The Crazy Bishop might just stop searching whenever it finds a
forced mate and as such it is "happy" with 1 Ke7 . Which explains why it has
problems here, as a mate in 5 is well within it's search abilities.
[A.10] Questions about 6 men tablebases.Which are the most useful?
Generating Nalimov 6 piece tablebases requires the heavy duty use of 64 bit
computers and is currently out of reach of most normal users.
As of Jan 2004, all the 3-3 and 4-2 pawnless Nalimov tablebases are done (227 GB).
The 4-2 with pawns are being worked on, and some are available. None of the 3-3
with pawns or the 5-1 nalimov tablebases are available yet.
As of Aug 2005, all of the 6 piece tablebases are complete except for the 5-1 set
but not all are available publicly.
There appears to be a lot of confusion about which engines can access 6 piece
tablebases. This is due to several changes in the nature of the 6 piece
tablebases. First there is the "normal" 6 piece tablebases generated before Dec
2002, it is suspected that some of the newer commercial engines like Hiarcs 9 can
access this. In Dec 2002, because of the size of the files, each tablebase is
broken into 2 parts, i,e "the multifile format". In September 2003, yet another
batch of tablebases was released with more changes to the probe code.
Crafty V19.4+ (to be safe use versions above 20) , Wilhelm are confirmed to work
with them. Also confirmed working (ability to use in search) according to
http://kd.lab.nig.ac.jp/chess/tablebases-online/ are Chess Tiger 15.0, Crafty
19.19, Fruit 2.2.1, Gandalf 6.0, Rybka 1.1, Shredder 9. This refers to the larger
multiple split set, normal chess engines can probably handle the older and smaller
6 piece tablebase sets.
Where to get them.
Historically the main source for 6 men Nalimov tablebases has being at Bob Hyatt's
ftp . However, as of April 2006, they are still recovering from a hard-disk crash,
and there are no 6 piece tablebase there. They intend to recover though, so
continue checking.
NEW! [Dec 14 2002] - More 6 men tablebases have being uploaded on Hyatt's site.
However because of the size of the files (greater than 2 gb) they are broken into
2 parts (For example krnkbn.2.nbw.emd) and it requires a different probing code
and currently [Dec 14 2002] no Chess engine can access it yet except Crafty 19.2
which will be released soon.From the user point of view, usage remains the same.
As of March 2003, the new version of Wilhem can access them too.
NEW![September 2003] Yet more endgame tablebases produced with more changes to the
probing code. This current update includes some of the 4+2 with pawns tablebases,
but very few programs can access it. One is wilhelm, the other is Crafty 19.4. Any
programs released before them (around Oct 03) are unlikely to be able to use such
tablebases.
New access code would shortly be released in Crafty 19.2. Changes were
necessary because (1) you should use 64-bit index to index some of 6-men TBs, (2)
even after compression some files were greater than 2Gb, and some OSes cannot
handle such files (or handle them in absolutely unportable way), so best solution
is to split those files into several smaller ones.
From chess program's point of view the only difference is that INDEX type now
should be unsigned 64-bit integer, not unsigned 32-bit integer. Something like 5
minutes of work.
Some unrelated problems are fixed in the code -- reported memory leaks, SMP
problems, Mac OS X locks problem that manifested itself in access violation, very
slow startup when TBPATH pointed to the CD-ROM drive and there was no disk in that
drive, better integrity check for the TBs, etc.
The pawnless six men endgame tablebases are almost complete (except for the
trivial ones). This is necessary of course to avoid the incomplete endgame
tablebase problem (although you can use them to see the results without the best
line of play), but it also makes them pretty useless and may actually slow down
the Chess engine if it probes useless positions.
Of some interest is the KQQKQQ tablebase. It was one of the first 6 piece
tablebase generated back in 1999 especially, to aid in the Kasparov versus World
game.
It is estimated a whole set of 6 piece Nalimov tablebases (when ready) will take
up at least 1 terrabye (1000 GB).
If you download or generate your own tablebases, there is a possibility that some
of them become corrupted. Some chess playing programs do a check and warn you if
any of the tablebases are corrupted, but many don't. /p>
The above method is fine, except that it will take a long time to check all the
tablebase files one by one. Here is what I recommend.
1. The interface is configured incorrectly. Generally this occurs when the path
to the tablebase is in error.
2. The tablebase files are corrupted.
The procedure above checks for the first problem. If you manage to get one
tablebase working chances are the path is set correctly or the other tablebase
files in the same directly should work, if they are not corrupted.
You need to edit the Crafty.rc file. For example, my line reads
tbpath=c:\chesseng\tb . You should edit the line to point to the directory where
your endgame tablebase files are currently located. The procedure for setting up
endgame tablebases is similar for other Winboard engines that you can download.
See my other FAQ on Winboard and Chess engines
[A.14] How can I get Fritz to use Endgame tablebases?
The advice below applies for users of Fritz 6. Users of newer chessbase software
like Fritz 7 etc, have the additional option of setting up the tablebase path
within the interface.
First, copy all your tablebase files into one directory, say C:\Nalimov
[Tablebase]
Path=C:\Nalimov
CacheSizeKB=1024
Just edit the path to point to the directory where your endgame tablebase files
are currently located and remember to save.This should work for all Chessbase
engines including Junior,Nimzo,Hiracs and the new Chessbase engines Shredder and
Tiger. For further information, refer to Steve Lopez's T-notes on"Tablebases and
Fritz endgame Tablebase", 2 July 2000 and "Using Endgame CDs in Chessbase 7", 21
March 1999
Please note that if the endgame tablebases that you use are corrupted in some way
(perhaps through a incomplete download) Fritz/Junior etc will refuse to run. To
find out which tablebase is corrupted see Section A.11
[A.15] What is the difference between tablebases download from Dr Hyatt's ftp site
and those on the Chessbase endgame turbo CD? Can they be used together?
Generally the tablebases that come in the Chessbase Endgame turbo are in the
format kxxkxxnbb.emd compared to that downloaded, which come in the form
kxxkxx.nbb.emd .Otherwise I think they should be exactly the same. I have
generally found that Fritz can use either form, but you may want to add or remove
the dot, if it refuses to work.
Tbs stands for tablebase summary and give stats on the number of positions which
are won, losses etc. They are not necessary if you are not interested in such
statistics.
Earlier versions of Nalimov's code need the .nbw and .nbb of symmetrical endgames.
Later this was changed (when support of 6-men TBs was added). So, engines using
the older code will need both, and newer code will only need one. The list of the
symmetrical TBs in the code are knkn, kbkb, krkr, kqkq, knnknn, kbbkbb, krrkrr,
kqqkqq but not kpkp
There is new Eugene Nalimov code that allows you to use only one nbw file in all
cases except when there are symmetrical positions for example in KRKR positions.
This may or may not work depending on the Chess engine and probing code used. For
access to 6 men tablebase positions, the new Nalimov code but be used, however for
symmetric positions like knkn, kbkb, krkr, kqkq, knnknn, kbbkbb, krrkrr, kqqkqq
(but it seems not Kpkp) you need both nbb and nbw.
To be safe though it's best to have both (or download nbw first and see if it
works), as currently it's not certain how many Chess engines [besides Crafty] use
the new Eugene code, and you can't really save much space anyway for the symmetric
4 men tablebases because they are pretty small anyway.
This is unlike the case of Edwards tablebases or Chessmaster's EGDB , where you
need only either the file ending with tbb [positions with black to move for
Edwards] or tbw [positions with white to move]. But even for those, it's much
faster to have both since with only one set, the engine has to do a 2 ply search
instead of 1.
[A.17] Help, the endgame tablebases are not working properly!
1.
You are using the wrong type of endgame table bases or the Chess program
does not support the use of endgame tablebases.
If the Program for some reason refuses to make a obvious winning move [usually
queening] in a endgame, it is possible, we have a case of incomplete tablebases.
See Section A.9 .
[A.18] Can I use tablebase files in zipped form?
No. Nalimov tablebases in the emd format are already compressed, and there is
little if any gain from further zipping them up.Same goes for the Chessmaster EDGB
format.
[A.19] Where can I get a useful graphical browser to view and practice endgame
tablebases?What about an online searchable database?
Let's do them in reverse order. The following websites allow you to search the
results of a given endgame position, if you don't have the tablebase files.Most
sites below require Java or at least javascript.
What if you already have the tablebases but need an application to view and search
through them?
* SCID
"I find Scid very useful for browsing Nalimov tablebases. If a position
is winning, it shows which moves mate the fastest, if it is losing it shows which
moves survive longest, and if it is drawn it shows which moves keep the position
drawn and which moves (if any) lose. Of course, being Scid's author I may be
biased... " [Posting to rec.games.chess.computer on 12-01-2001]
SCID 3.3 and up also keep tracks of which tablebases you have and stores
statistics about each tablebase.
* Wihelm- A free Chess program but more importantly it comes with powerful
tablabase functions that allow you to automatically check the integrity of
tablebases using MD5 checksums as well as other functions similar to the "endgame
oracle"in shredder 6, (see below) The website is in German, but the program is in
English/German and there is a good english PDF help file which explains the
various tablebase functions.
* Another alternative including using Fritz, and Chessbase 7 or 8, which has a
special engine to view Thompson tablebases.
* There is also the "endgame oracle" in Shredder 6 and Shredder Classic .
* Freezer - A commercial tool that uses Nalimov tablebases to help endgame
analysis. By "freezing" actually restricting certain features in the position
(pawn blocked, king cannot move beyond 4th rank etc), coupled with Nalimov
tablebases, it helps clarify certain aspects of endgames like fortresses.
* GAFS - a program to generate and interrogate tablebases and a library, under
the form of an object file, with an example for using these tablebases in a chess
program. As cpu time for generation of EGTBs is very high, the program contains an
intermediate saving system to avoid loss of the entire work if for any reason the
computer goes down or computation must be interrupted; so it is possible to run
the program during the night, stop calculation in the morning and restart again in
the successive night.
To be worked on
Why don't we allow chess engines to access and use endgame tablebases over the
internet?
To be worked on
What about doing a distributed peer to peer method of generating endgame
tablebase?
To be worked on
Aaron Tay