Reply
 
LinkBack Thread Tools Display Modes
  #1   Report Post  
Old September 20th 08, 04:18 AM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2008
Posts: 219
Default Random number mobility scores

Some time back it was mentioned that Hyatt had mentioned a test by Don Beal
on random mobility scores for a material only program.

I finally got curious enough to dig out an old chess program and try it.

The program is crap. It is a mailbox program that was going through a lot
of modification when I packed it away.

But it seems to be working, and it has enough Winboard ability that works
for me to run the Win-At-Chess test through it.

The program used only material and a random number generator as a mobility
score. No piece tables, no pawn scoring, etc. Just material and a random
number generator from Numerical Recipes (I didn't want to use the one that
comes with the compiler, of course.) I did a random number of 128 and put
half negative and half positive. So about a half a pawn worth of mobility
balance.

The move ordering was done MVV/LVA with History Heuristic. No SEE move
ordering.

The capture search does only captures, and all of them. No checks or
promotions, etc. are included.

There are no search extensions like Null Move, check extensions, etc.

There are no pruning methods beyond alpha-beta. No futility moves, no LMR,
etc. etc.

No transposition tables.

Pretty much a basic, niave alpha-beta chess program with an eval of only
material and random numbers. I didn't even bother to compile it with
optimization enabled. I left it in debug mode.


I used epd2wb and ran the Win-At-Chess test through my program, giving it 10
seconds to think. I used a 2ghz laptop. Only one core was used.

(Note: This is the original WAC test, not the revised / fixed version.)

I got +203 / -77 = 67%. The later ones were where most of the failures
occured.

As another test, I ran WAC through the same program without the random
number generator. Scoring only material. I got +197 / -103 = 65%


As a lark, I also tried it with the original evaluator I had in the program.
This eval is junk. It was stuff I had cut & pasted from some other simple
public domain programs. No effort was made to tune it. I ran very few
tests with it. I just added stuff so I had an eval that did more than
material.

It got +197 / -103 = 65%... (grin) Pretty bad....


I then decided to try the WAC tests again, this time without the mates. The
idea being the search itself is finding the mates, not the evaluator.

Random: +148 / -80=64%.

Material: +148 / -80=64%

Odd. Maybe the mobility helped it find a few mates? Or at least picked the
right move for the wrong reason? Or maybe it provided enough extra guidence
in the search that it was able to go to the next ply and see the mate?


I also tried the Bratko-Kopec test.

With the random one. It got a total of 11 right. That gives a BK rating of
2000-2199. (And no, Sanny, that's not ELO, even though it's supposed to
be.)

For the Material only version, it got only 8 right. That's a 1800-1999 BK
Rating.

The one with the old evaluator, it got a score of 16.5, which is around
2400. (The score table doesn't handle fractional points. For 16, it'd be
2000-2399 and for 17 it'd be 2400+. So around 2400.)



Finally, note that these results are just one test. Due to the nature of
random numbers, a different see could conceivably cause all of them to give
the right answer or wrong answer.


But it does appear that a random number generator mobility score is at least
as good as plain material and in some cases better.





----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= - Total Privacy via Encryption =---
  #2   Report Post  
Old September 20th 08, 04:28 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2006
Posts: 625
Default Random number mobility scores

* Guest (05:18) schrieb:

Some time back it was mentioned that Hyatt had mentioned a test by Don Beal
on random mobility scores for a material only program.

I finally got curious enough to dig out an old chess program and try it.


Cool!

The program used only material and a random number generator as a mobility
score. No piece tables, no pawn scoring, etc. Just material and a random
number generator from Numerical Recipes (I didn't want to use the one that
comes with the compiler, of course.) [...]


Of what kind is that random generator? A Mersenne Twister?

Pretty much a basic, niave alpha-beta chess program with an eval of only
material and random numbers. I didn't even bother to compile it with
optimization enabled. I left it in debug mode.


I used epd2wb and ran the Win-At-Chess test through my program, giving it 10
seconds to think. I used a 2ghz laptop. Only one core was used.

(Note: This is the original WAC test, not the revised / fixed version.)


Which one did you post here?

I got +203 / -77 = 67%. The later ones were where most of the failures
occured.

As another test, I ran WAC through the same program without the random
number generator. Scoring only material. I got +197 / -103 = 65%


As a lark, I also tried it with the original evaluator I had in the program.
This eval is junk. It was stuff I had cut & pasted from some other simple
public domain programs. No effort was made to tune it. I ran very few
tests with it. I just added stuff so I had an eval that did more than
material.

It got +197 / -103 = 65%... (grin) Pretty bad....


It would be interesting to know if all they solved the same 197
positions. Do you have detailed logs?

I then decided to try the WAC tests again, this time without the mates. The
idea being the search itself is finding the mates, not the evaluator.


The evaluator should help. :-)

Random: +148 / -80=64%.

Material: +148 / -80=64%

Odd. Maybe the mobility helped it find a few mates? Or at least picked the
right move for the wrong reason? Or maybe it provided enough extra guidence
in the search that it was able to go to the next ply and see the mate?


Answers a log could give.

Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
doing something right. Transposition table?

But it does appear that a random number generator mobility score is at least
as good as plain material and in some cases better.


Thanks for testing.

mfg, simon .... l
  #3   Report Post  
Old September 20th 08, 06:17 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2008
Posts: 219
Default Random number mobility scores

"Simon Krahnke" wrote in message
...
* Guest (05:18) schrieb:

Of what kind is that random generator? A Mersenne Twister?


Numerical Recipes 3rd ed., RanLim32. It's the only 32 bit random generator
they recommend and provide code for.

I could have done a 64 bit using long long, but I wanted to keep the program
in 32 bit clean.


(Note: This is the original WAC test, not the revised / fixed version.)


Which one did you post here?


I posted the original one.

When I ran my tests, that was handy and I wasn't expecting to run so many
tests.

Plus, some of those 'corrected' moves actually take quite a bit of effort.
Not much chance my program would get them right anyway.


As a lark, I also tried it with the original evaluator I had in the
program.
This eval is junk. It was stuff I had cut & pasted from some other
simple

It got +197 / -103 = 65%... (grin) Pretty bad....


It would be interesting to know if all they solved the same 197
positions. Do you have detailed logs?


Yeah. I dumped the results of each test to seperate files.

But I don't really have the urge to look into them. I was just curious
about the end results, not in tuning my evaluator.

My guess is the poor results are due to a slow down in the program and a
poor Q-search. It's doing just a capture search.

And there could even be bugs in the program. It was a program 'in
transition'. It had never really been finished and had been in a constant
state of change.


Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
doing something right. Transposition table?


That could certainly be it.

This program has no trans table at all, and poor move ordering.



It's entirely possible that if you ran these tests, you'd get very different
results than what I do.




----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= - Total Privacy via Encryption =---
  #4   Report Post  
Old September 20th 08, 06:21 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2008
Posts: 219
Default Random number mobility scores

I have some more results for the random mobility score.

I was originally interested in how it compared to just material alone, but
then I wondered how it compared to real mobility scoring.

Here are the program versions. Same junk program as before.

Mtl is material scoring only.

Rand: Addition of -64..+63 random value to simulate mobility. I just lopped
off some bits and made half negative. The asymetry might cause some issues,
but I don't care enough to repeat it with balanced results or a reduced
range.

Eval: The original eval in the program. Primative. A little pawn
knowledge, a little mobility, queen/king closeness, a tiny bit of king info.
Just stuff hacked together, mostly

Mob: Material and two calls to GenAllMoves(Side).

Mob2: Same as "Mob" but I don't bother to store the moves, pawn promotions
are countd as 1 instead of 4, and I don't do castling. Faster.

PMob: Material and pawn based mobility. The mobility the sliders would have
if there were only pawns on the board. Only the sliders are scored. (This
was alread in the full eval, so I just cut out the unrelated stuff.)

Bratko-Kopec test. (2=1/2 4=1/4)
Mtl : 100000100001011001001100 = 8 = 1800-1999
Rand: 100100100101011100101001 = 11 = 2000-2199
Mob : 100100110001011001101000 = 10 = 2000-2199
Mob2: 100110140001011001001000 = 9.25= 2000-2199
PMob: 101110110001111101101001 = 15 = 2200-2399
Eval: 101111110201011111101100 = 16.5 = 2400'ish

Tact: T---T-T--T-T-TTT-TT-TT--
Pos : -PPP-P-PP-P-P---P--P--PP
(The BK test are 12 tactical & 12 positional / Levers, as indicated above.)


Tactical Bias = 12*(T-L)/S
Tact: T---T-T--T-T-TTT-TT-TT--
Mtl : 100000100001011001001100 = 8 Bias = 12
Rand: 100100100101011100101001 = 9 Bias = 7.6
Mob : 100100110001011001101000 = 8 Bias = 7.2
Mob2: 100110140001011001001000 = 8 Bias = 8.75
PMob: 101110110001111101101001 = 11 Bias = 5.6
Eval: 101111110501011111101100 = 11.5 Bias = 4.7
Tact: T---T-T--T-T-TTT-TT-TT--

So we can see that just material is entirely matierially biased. No
surprise.

Adding any form of mobility improves the bias. It's still very materially
biased, but it helps.

It is a little odd that Mob2 did worse than Mob. It could be sheer
statistical randomness, though. More tests might have different results.


Classic Win-At-Chess test.
Mtl +197 / -103 = 65%
Rand +203 / -77 = 67%
Mob +179 / -121 = 59%
Mob2 +189 / -111 = 63%
PMob +194 / -106 = 64%
Eval +197 / -103 = 65%

The WAC tests are very tactically oriented. Very simple tests, too.

In this case, the random mobility provides just a little guidence with
minimal impact on the speed of the search.

A better quiscient search would probably help these tests, regardless of the
eval.


So how do I interpret the results...

Well, the mobility ones and the full eval do run slower. WAC is very
tactical so a slow prgram is at a little disadvantage. Mine is particularly
slow (compared to modern programs), and it's dumb (by nearly any standard),
so anything that slows down the program is going to be at a disadvantage for
a tactically niave / slow program.

Hence, the material only does pretty well. And the random one does pretty
well too because it provides a tiny bit of smarts to the program at very
little cost.

The BK positions are a combination of tactics and positional. In this case,
the full evaluator makes up for its slowness by being smart enough to find
the right move anyway.

Even mobility, in any form, helps in the BK tests. Although they do slow
down the program, it helps over all.


So, my final conclusion is this....

Random mobility is a good thing if you are doing just material. It's quick,
it's easy to add, and it actually helps some. But it's no substitute for
real mobility scoring, especially if you can do that fast, rather than just
doing calls to GenAllMoves().

It looks like pawn based mobility is a good compromise. Quick to generate
but decent results. If you don't have time to program that.... then go for
random mobility.



And of course, your results may vary depending on the random generator you
use, the speed of the program, the number of tests you do, etc. etc.

I have no plans to run more tests to see if piece square tables would
benefit from random moblity, etc.







----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= - Total Privacy via Encryption =---
  #5   Report Post  
Old September 21st 08, 04:54 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2006
Posts: 625
Default Random number mobility scores

* Guest (19:17) schrieb:

"Simon Krahnke" wrote in message
...
* Guest (05:18) schrieb:

Of what kind is that random generator? A Mersenne Twister?


Numerical Recipes 3rd ed., RanLim32. It's the only 32 bit random generator
they recommend and provide code for.


Well, that's an investment. 50.95 Euros at Amazon.

Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
doing something right. Transposition table?


That could certainly be it.

This program has no trans table at all, and poor move ordering.


The best move from the TT is all the move ordering Finalfun has. I never
got to implement MVV/LVA.

It's entirely possible that if you ran these tests, you'd get very different
results than what I do.


Let's see if I can get my new engine to actually generate moves. Right
now, it knows what a board is, and that there are piece on it. That's
not really much. I'm kind of working on preparation for chess960
compatible castling now.

mfg, simon .... slow pace


  #6   Report Post  
Old September 21st 08, 05:40 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2008
Posts: 219
Default Random number mobility scores

"Simon Krahnke" wrote in message
...
* Guest (19:17) schrieb:

Numerical Recipes 3rd ed., RanLim32. It's the only 32 bit random
generator
they recommend and provide code for.


Well, that's an investment. 50.95 Euros at Amazon.


You can read it online for free on their website. They make it available as
PDFs.

They don't provide seperate code to download, but it's only a few lines to
type in.

Plus, if you look hard enough, you can find places online where people have
already typed it in for you.


I did some random number testing some time back, and it's actually pretty
hard to find a good random number generator even for things like the hash
keys.

A lot of the simplier ones on the web have very poor hamming distances for
the hash keys, even if they are okay for normal random numbers.

On the other hand, even some of the classic simple ones can do okay for hash
keys if you do things right. They aren't random in the mathematical sense,
but the hamming distances can be good enough for a hash table.

I think many people just end up using Hyatt's method. You make a few
throw-away calls until you get to the point where you get the best hamming
distances for your hash table.

Hyatt says it's based on one in NumRec, which was based on one in Knuth, but
I haven't seen the sources. The only thing I don't like about it is it uses
precomputed data.




Finalfun scored 188 in 5s each on a Pentium 4 at 3.4GHz. It must be
doing something right. Transposition table?


That could certainly be it.

This program has no trans table at all, and poor move ordering.


The best move from the TT is all the move ordering Finalfun has. I never
got to implement MVV/LVA.


Wow.

Most people do MVV/LVA & history heuristic long before they do a trans
table.





----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= - Total Privacy via Encryption =---
  #7   Report Post  
Old September 22nd 08, 07:18 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Oct 2006
Posts: 1,305
Default Random number mobility scores

The more interesting test is: actual mobility - with no other term,
not even material.

--
Kenneth Sloan
Computer and Information Sciences +1-205-932-2213
University of Alabama at Birmingham FAX +1-205-934-5473
Birmingham, AL 35294-1170
http://KennethRSloan.com/
  #8   Report Post  
Old September 22nd 08, 08:13 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2008
Posts: 219
Default Random number mobility scores

"Kenneth Sloan" wrote in message
...
The more interesting test is: actual mobility - with no other term, not
even material.


I said I wasn't going to run any more tests....

But alright. That's somewhat interesting.

I'll rerun some of the tests with the material removed, leaving just the
various forms of mobility.

It'll take a day or so to get around to it.







----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= - Total Privacy via Encryption =---
  #9   Report Post  
Old September 22nd 08, 08:25 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Jul 2006
Posts: 625
Default Random number mobility scores

* Kenneth Sloan (20:18) schrieb:

The more interesting test is: actual mobility - with no other term, not
even material.


Compare

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - KkQq 0 1

with

1n2k1n1/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1

mfg, simon .... l
  #10   Report Post  
Old September 22nd 08, 09:16 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Oct 2006
Posts: 1,305
Default Random number mobility scores

Guest wrote:
"Kenneth Sloan" wrote in message
...
The more interesting test is: actual mobility - with no other term, not
even material.


I said I wasn't going to run any more tests....

But alright. That's somewhat interesting.

I'll rerun some of the tests with the material removed, leaving just the
various forms of mobility.

It'll take a day or so to get around to it.


Also - it's worth noting that a random term (don't know why you want to
*call* it "mobility") is, in practice, a good *indirect* measure of
actual mobility (back one ply).

--
Kenneth Sloan
Computer and Information Sciences +1-205-932-2213
University of Alabama at Birmingham FAX +1-205-934-5473
Birmingham, AL 35294-1170
http://KennethRSloan.com/
Reply
Thread Tools
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Disproof: Possible Number of Chess Games Berkeley Brett rec.games.chess.misc (Chess General) 14 November 29th 07 11:42 AM
rgc Tournament PGN Rob rec.games.chess.misc (Chess General) 81 March 4th 07 06:14 AM
rgc Tournament PGN Rob rec.games.chess.analysis (Chess Analysis) 0 February 24th 07 02:24 PM
RGC chess tournament Rob rec.games.chess.analysis (Chess Analysis) 19 February 14th 07 10:48 PM
GetClub Chess Challenge increasing (Taylor Kingston Beware) Sanny rec.games.chess.analysis (Chess Analysis) 16 August 26th 06 08:11 AM


All times are GMT +1. The time now is 11:06 PM.

Powered by vBulletin® Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright 2004-2019 ChessBanter.
The comments are property of their posters.
 

About Us

"It's about Chess"

 

Copyright © 2017