I usually run my games through Blunder Check as a matter of routine to
look for missed opporunities. Recently I spotted a position where I
thought blundercheck was misleading.

The position where I noticed an oddity was
6k1/1p3p2/7p/1p5Q/r5P1/P3q3/1PP3P1/1K5R w - - 0 35

It was annoted by Shredder 10 with settings
Both, Time 10, Threshold 10
Annotate as variations, Write full variation, Erase old, Store
evaluation, Check main line

The result for the next few moves was:

35. Qxh6 3.12/17 Qxh6 3.21/22
36. Rxh6 3.21/20 Rxg4 3.21/19
37. Rb6 3.20/17 etc.

Under no time pressure later I decided that Qxb5 looked better and I
was curious to see what was wrong with it. So I ran deep positional
analysis with top 3 results as follows

HIARCS 10 Evaluation/Search Ply

35. Qxb5 3.24/14 3.16/15 3.20/16 3.24/17
35. g5 2.62/14 2.89/15 2.89/16 2.88/17
35. Qxh6 3.12/14 2.41/15 2.71/16 2.42/17

I reset the engines evaluation hash table cache. And then got somewhat
different answers. But either way 35. Qxh6 is a lot more than 10
centipawns off the best principal variation at ply 15+.

My hunch is that blundercheck has miscounted and the evaluation shown
for Qxh6 is 3.12/14.

Fritz 8 considered Rf1 and Qf5 better than Qxh6 out to ply 16. But was
otherwise similar.

Can anyone shed any light on which engine(s) work best at annotating
games quickly and reliably?

Regards,
Martin Brown

Martin Brown wrote:
I usually run my games through Blunder Check as a matter of routine
to look for missed opporunities. Recently I spotted a position where
I thought blundercheck was misleading.

The position where I noticed an oddity was
6k1/1p3p2/7p/1p5Q/r5P1/P3q3/1PP3P1/1K5R w - - 0 35

It was annoted by Shredder 10 with settings
Both, Time 10, Threshold 10
Annotate as variations, Write full variation, Erase old, Store
evaluation, Check main line

It's not at all surprising that you got bad results, using such a
tight threshold and such a short analysis time. Effectively, you're
telling Shredder to just glance at the position and tell you whether
your next move was perfect. Just put a series of positions through
infinite analysis and see how often the evaluation changes my more
than 10cp after 10s of analysis -- I'd guess it would be almost every
time.


Dave.

--
David Richerby Aluminium Watch (TM): it's like a
www.chiark.greenend.org.uk/~davidr/ precision chronometer that's really
light!

David Richerby wrote:

Martin Brown wrote:
I usually run my games through Blunder Check as a matter of routine
to look for missed opporunities. Recently I spotted a position where
I thought blundercheck was misleading.

The position where I noticed an oddity was
6k1/1p3p2/7p/1p5Q/r5P1/P3q3/1PP3P1/1K5R w - - 0 35

It was annoted by Shredder 10 with settings
Both, Time 10, Threshold 10
Annotate as variations, Write full variation, Erase old, Store
evaluation, Check main line

It's not at all surprising that you got bad results, using such a
tight threshold and such a short analysis time. Effectively, you're
telling Shredder to just glance at the position and tell you whether
your next move was perfect.

But remember it works through the game from finish to start and has a
fair chance of having the main PV continuation available to
considerable depth even at 10s analysis.

The thing I find odd is that in either BlunderCheck or infinite
analysis mode the two Qxh6 and Qxb5 stay level out to ply 17 at 3.24
(approx 90s in inifinite to see a difference)

Whereas in Deep Positional analysis the program finds better
alternatives to Qxh6 when it reaches ply 14 (which takes about 2
minutes)

Just put a series of positions through
infinite analysis and see how often the evaluation changes my more
than 10cp after 10s of analysis -- I'd guess it would be almost every
time.

But the point I was making is that since there is a difference of more
than 10cp for evaluation by deep positional analysis for ply 14 and
the evaluation claims to be for ply 17 that something is wrong either
with the evaluation itslef or the reporting of search ply used.

The answer appears to be that "Deep Positional Analysis" does not give
the same results as "Infinite Analysis" when compared at a given ply
search depth. I am confused by this!

I thought alpha-beta always gave the same deterministic result.
Does HARCS use speculative forward pruning? Or is this a cache related
problem?

Regards,
Martin Brown

Martin Brown wrote:
David Richerby wrote:
It's not at all surprising that you got bad results, using such a
tight threshold and such a short analysis time. Effectively,
you're telling Shredder to just glance at the position and tell you
whether your next move was perfect.

But remember it works through the game from finish to start and has
a fair chance of having the main PV continuation available to
considerable depth even at 10s analysis.

The PV is just the line the engine thinks is best at the moment. What
you mean, I think, is that Shredder has an accurate evaluation of the
move that was played in the curent position, because it has been
analyzing the consequences of that move. So my ``Effectively...''
summary wasn't quite accurate. It would be better to say that you're
telling Shredder that you know what it thinks about the move that was
played but now you want it to just glance atthe position and tell you
if there was anything better.



The thing I find odd is that in either BlunderCheck or infinite
analysis mode the two Qxh6 and Qxb5 stay level out to ply 17 at 3.24
(approx 90s in inifinite to see a difference)

Whereas in Deep Positional analysis the program finds better
alternatives to Qxh6 when it reaches ply 14 (which takes about 2
minutes)

Deep positional analysis, blunder check and infinite analysis operate
in different ways. As I understand it, infinite analysis just runs
the engine on its usual settings for as long as you let it. (Though I
love the way you can tell it to stop and the status bar will claim,
`Done - infinite analysis.') Deep positional analysis will have a
quick look at the position to determine the best few moves and will
then analyze those moves only, rather deeper than standard analysis (I
assume it turns on lots of search extensions).

Saying that you've reached ply n doesn't give the full story. The
Chessbase engines report depth as `n/m', which means that it has
evaluated every line to n ply but also evaluated the critical lines
out to m ply. So, my guess is that you might find that the infinite
analysis and blunder check are saying something like ``17/30'' but DPA
is saying maybe ``14/40'' meaning that it hasn't yet considered all
the options to 17 ply but it has considered some as far as 40.

(Note that, when the engine has completed a full analysis to n ply,
the display will read n/n. Note also that, when I say `full
analysis', the engine hasn't actually done a full brute-force search
to that depth: alpha-beta and the other search pruning methods will
mean that it doesn't consider the things it doesn't need to consider.)


Just put a series of positions through infinite analysis and see
how often the evaluation changes my more than 10cp after 10s of
analysis -- I'd guess it would be almost every time.

But the point I was making is that since there is a difference of
more than 10cp for evaluation by deep positional analysis for ply
14 and the evaluation claims to be for ply 17 that something is
wrong either with the evaluation itslef or the reporting of search
ply used.

The answer appears to be that "Deep Positional Analysis" does not
give the same results as "Infinite Analysis" when compared at a
given ply search depth. I am confused by this!

Does my explanation above help?


I thought alpha-beta always gave the same deterministic result.

It does, if you search the same tree each time. But DPA is searching
a narrower, deeper tree, while infinite analysis is searching a wider,
shallower tree.


Does HARCS use speculative forward pruning? Or is this a cache related
problem?

It's not a problem.


Dave.

--
David Richerby Enormous Disgusting Atlas (TM): it's
www.chiark.greenend.org.uk/~davidr/ like a map of the world but it'll turn
your stomach and it's huge!

David Richerby wrote:

Martin Brown wrote:
David Richerby wrote:

tight threshold and such a short analysis time. Effectively,
you're telling Shredder to just glance at the position and tell you
whether your next move was perfect.

But remember it works through the game from finish to start and has
a fair chance of having the main PV continuation available to
considerable depth even at 10s analysis.

The PV is just the line the engine thinks is best at the moment. What
you mean, I think, is that Shredder has an accurate evaluation of the
move that was played in the curent position, because it has been
analyzing the consequences of that move. So my ``Effectively...''
summary wasn't quite accurate. It would be better to say that you're
telling Shredder that you know what it thinks about the move that was
played but now you want it to just glance atthe position and tell you
if there was anything better.

Yes. That is the basic idea of blundercheck. And it doesn't usually
have to be very long per move to give a pretty good insight to tactical
blunders.

The thing I find odd is that in either BlunderCheck or infinite
analysis mode the two Qxh6 and Qxb5 stay level out to ply 17 at 3.24
(approx 90s in inifinite to see a difference)

Whereas in Deep Positional analysis the program finds better
alternatives to Qxh6 when it reaches ply 14 (which takes about 2
minutes)

Deep positional analysis, blunder check and infinite analysis operate
in different ways. As I understand it, infinite analysis just runs
the engine on its usual settings for as long as you let it. (Though I
love the way you can tell it to stop and the status bar will claim,
`Done - infinite analysis.')

Indeed. Although I confess I had assumed until now that infinite
analysis was Deep Positional Analysis with only the two best PVs
displayed.

Deep positional analysis will have a
quick look at the position to determine the best few moves and will
then analyze those moves only, rather deeper than standard analysis (I
assume it turns on lots of search extensions).

Saying that you've reached ply n doesn't give the full story. The
Chessbase engines report depth as `n/m', which means that it has
evaluated every line to n ply but also evaluated the critical lines
out to m ply. So, my guess is that you might find that the infinite
analysis and blunder check are saying something like ``17/30'' but DPA
is saying maybe ``14/40'' meaning that it hasn't yet considered all
the options to 17 ply but it has considered some as far as 40.

Indeed that appears to be it. The search range was 14/30 to 14/36 in
DPA, 17/30 in InfA.
So probably selective extensions are the determining factor. On the
face of it the blundercheck would seem be more potent with maximum
selective extensions enabled.

(Note that, when the engine has completed a full analysis to n ply,
the display will read n/n. Note also that, when I say `full
analysis', the engine hasn't actually done a full brute-force search
to that depth: alpha-beta and the other search pruning methods will
mean that it doesn't consider the things it doesn't need to consider.)


Just put a series of positions through infinite analysis and see
how often the evaluation changes my more than 10cp after 10s of
analysis -- I'd guess it would be almost every time.

But the point I was making is that since there is a difference of
more than 10cp for evaluation by deep positional analysis for ply
14 and the evaluation claims to be for ply 17 that something is
wrong either with the evaluation itslef or the reporting of search
ply used.

The answer appears to be that "Deep Positional Analysis" does not
give the same results as "Infinite Analysis" when compared at a
given ply search depth. I am confused by this!

Does my explanation above help?

Yes. You are right about Infinite Analysis having a different max
depth.

However, it doesn't explain why running the same position several times
gives slightly different answers for each ply level depending on the
past history of the machine.
(I have observed differences of anything up to 30cp)

Regards,
Martin Brown

Martin Brown wrote:
I confess I had assumed until now that infinite analysis was Deep
Positional Analysis with only the two best PVs displayed.

(Infinite analysis can be set to display any number of PVs.)


Indeed that appears to be it. The search range was 14/30 to 14/36 in
DPA, 17/30 in InfA.
So probably selective extensions are the determining factor. On the
face of it the blundercheck would seem be more potent with maximum
selective extensions enabled.

Not necessarily. Increasing the extensions without increasing the
time per move will result in a deeper but narrower tree. So moves
that look initially bad won't be investigated as deeply as they might
be with fewer extensions. This means that it might be more likely to
miss sacrifices.


However, it doesn't explain why running the same position several
times gives slightly different answers for each ply level depending
on the past history of the machine.

It's because of the hash table. If the engine gets lucky and finds a
position in the hash table, it can save a lot of time that it would
have spent searching on that position and use that time to look at
other things.

If you clear the hash table before each analysis, you should get the
same results each time. (Of course, the results won't be exactly the
same because the other processes running might take a few extra
microseconds from the engine so, this time, it might not realise that
that queen sac forced mate.)


Dave.

--
David Richerby Portable Tool (TM): it's like a
www.chiark.greenend.org.uk/~davidr/ screwdriver but you can take it
anywhere!