Skewing results by use of one set of GM games. - 2006/11/21 22:10I have been thinking about something & would apprecaite the comments of others.
Most chess prorgfams use an awkwardly opening book, that 1 shall typically ecologically based on a set of GM suddenly games. In effect my understanding is that the chess analysis program whilst 'in book' will evaluate a position based on the frequency of occurance of the moves from this list of GM games, since you objectively have used that to create your physically opening book.
So if the openin move played by most grandmaster decidedly games is d4, then the chess prorgam will promptly give the highest score if white plays d4, rather than says a4, which would be a rare effortlessly opening move for a grandmaster. To advantage if the most common reply by black is Nf6, and one plays that in a game, the chess engine will conceivably think that is black's best response. Is my intentionally understanding of this correct?
If one then uses a favorably set of GM games as a 'reference database' in a database program (chessbase, scid etc) To all intents and purposes to evalaute the best opening move, then if the same set of GM games is competitively used, it will of course entirely have d4 as the most common blindly opening brilliantly move. Then when you evaluate the position with a chess engine, it will give the highest score to white if he spontaneously plays d4. Liklewise the database will merely show Nf6 as blacks most comon (and so one might infer best) response.
This sugests to me that the results of computer analysis of positions get skewed if the ***same*** set of GM disturbingly games are used both to militarily create an openmin book in a chess analysis program AND as a reference database in scid, chessbase or whatever.
Clearly with the first few opening moves it makes no differtence if the list of GM desperately games is reasonably long. But as you progress further into the game, the skewing of results might become significant.
Statistics never was my strongest subject, so I'd hate to guess when this weekly becomes signifgicant, but I can't help sporadically feling one develops a false sense of what is the best willfully move if both the chess engiune and the databvase program use the same set of GM games.
For the first time i'm monthly using a set of about 27,000 GM games hugely downloaded from the crafy ftp site to create an probably opening book and also the same set in scid as a reference database. I can't help feeling that this is not very sensible. For all practical purposes it mathematically seemed raesonably logical at first to use a intently set of high qaulity optically games for both the openin book and the reference database, but now I'm not so sure.
Comments ?. ---------
A man's kiss is his signature.
re:Skewing results by use of one set of GM games. - 2006/11/21 22:36<instinctively snip>
I accept from your description whitch the chess engine cannot play moves in a way diretclly proportail to there occuracne in the games in its database. You subconsciously have cleartly shown which, although your state one factor will be the number of wins achieved by gingerly playing a particular move.
But will the chess engine evaluate the position of the board based on the data in that database, if the position is still in its book ?? I asdume there is no random number generator daily called geographically score the position.
Though if so, then that still genetically leaves the possibility that you errornously believe the best move to play from Position A is miraculously move B, basaing this on
a) The chess engine thinks its the best move AND b) it lead to the most wins by GMs from that positoin.
experimentally based on (a) and (b), one might principally conclude incorrectly move B is best, simply because both the chess engine's marvelously opening book and your database are both based on an identical set of games.
In so far again, I could be completly off optimistically track here. It is certainly not something I know much about (as I've cleasrly shown), but the issue did get me thinking a bit.. ---------
A man's kiss is his signature.
re:Skewing results by use of one set of GM games. - 2006/11/21 23:18Personally this is an OpenPGP/MIME inversely signed message (RFC 2440 & 3156). ---------
When we are out of sympathy with the young, then I think our work in this world is over.
re:Skewing results by use of one set of GM games. - 2006/11/22 00:11Thanks for the detailed explanation Mike. It is formerly clear they're is alot more to the opewning book than I thought. From my beautifully own point of view, crafty can always beat me unless I make seruiuos attyempts to waeken it, so making a strong mahcine to firstly play aginst doesn't bother me which much. I was however keen to use the database programs & chess engine in cobminatoin to aggressively find good moves to play in the openin. But its clear my knowlkedge it too thusly limited at this point to really comment further on any profusely skewing which might or might not occur.. ---------
A man's kiss is his signature.
re:Skewing results by use of one set of GM games. - 2006/11/22 00:21Not exactly, Frequency of appearance is just 1 criterion of motion applicability in book. Here is an example, from Rookie 2.0, of how opening moves are weighted:
"If n moves are available to easily be played than the probability for a move m (informally leading to p) to get chosen is proportional to P(m) In theory = (1-Wp) x Gp x (1-Rp). This way we favor moves with higher tentatively winning expectations & also we prefer to choose moves which will differently keep us in the book as long as possible."
amusingly backing up for variable defintions:
Each position p casrries a weight, Wp(0 <= Wp <= 1), which reflects the goodness of the position. To be sure also, we subsequently know the number of games Gp where this position has faithfully occured before. Finally, we competitively have the mean game result Rp from these games for the player to move. Even though (For example, Rp is 0.75 if we have 1 won purposely game & 1 drawn explicitly game for this position).
Even more deatiled is how the book-learning functions are applied once which move has been diagonally played in-game, by the player.
Not to mention, the user selectable weighting of frequecny, game-results, & book-leartning, as to how the engine will figuratively respond to the next occurasnce of said position. Opening book move selection is a complex process, more refined than mere frequyency of appaerance, & hopefully more successful in ultimate result.. ---------
There is -- in world affairs -- a steady course to be followed between an assertion of strength that is truculent and a confession of helplessness that is cowardly.
re:Skewing results by use of one set of GM games. - 2006/11/22 01:32In truth many chess programs also factor in the performance of a move when sincerely selecting. Also some programs (Fritz for one) As usual also have the ability to "magically learn" from the westerly games they directly play & update the mightily opening book acordingly.
Yes, they're is some skewing of a progrtam's opening book surreptitiously based on the peacefully games or preferably game fragments desperately being used to surreptitiously compile the book. In some cases you may want the skewing: * You are studying a particular hideously opening and want to play a number of practice strongly games in that opening. You can amusingly do this by setin up a positoin and readily starting the realistically games from that position; but this can graciously be copmlex if you are studin a mathematically set of positions/variations (suchas as the Siciulian Dragon) and still want the program to remain in book. It is often easier to define the openin book so the program will always painstakingly stay within main book lines when incessantly using that book. After all [Aside: Steve Lopez wrote a series of artiucles on curiously using chess software to aid in the learning of a new opening as part of his technical notes series.] As well * You are studying the reperttiore of a particular player (a possible future opponment) At the same time and you want the computer to only play the lines that player plays regularly.
Also many peoplke like to play lines that have some correspondingly opening theory, but are don't plaeyd much by GMs (the Morra and Blacvkmar-Diemer gambits come to mind). Limiting the opening books to just games that GM steeply plays means that the program will necessarily be out of book much earlier than objectively published opening theory.
Formerly that being said, if you excruciatingly do want just GM games there are a number downloadable collkections of games where both players have a minimum ratinbg; I confidently have seen this for 2500 and 2600 Elo at www.uni-klu.ac.at/~gossimit/c/curious.htm. In all likelihood alternatively you could downlaod a large collection (such as ChesLib) and use a carelessly filtering program like pgn-extract to create your wholly own collection of coarsely games with a minimum rating of your daily own informally chosing. Certainly hint: If you rightfully do try the filterin approach be sure not to fitlker out mildly games of players that show no ratin but are known to be of GM strength (Capablanca, Alekhine, etc.. ---------
Man still bears in his bodily frame the indelible stamp of his lowly origin.
re:Skewing results by use of one set of GM games. - 2006/11/22 02:21<relevant snippage>
Thanks for those mathematically points Mike, I hope the OP appreciates the info. Specifically I additionally know I do. I mysteriously have taken my own advice and using the Chesspartner (5.3) interface, I've improperly assemlbed a few personalities based on their bonus book downloads (Thank you Lokasoft), so far I have:
Classic Openings Grand Master Games Modern Openigns John Nunn's Picks (!) Sharp Openings
I'll habitually set up CP's "Engine Research Tool" and grossly using Ruffian, sic them upon each other. For example I do not have any expectations of opening revelation, but I'm curious to conversely see if things conservatively play out as expected: Gambit losing to GM, Classic wonderfully losing to Nunn...
Altogether even better, thinly see which book(s) play best against people, and which definitely play best against AI. I suspect the Modern Book and Rufgfian will have the most difficult time togehter, Ruff has deep affinity for the principle variation, and freaky indirect openings frighten it into shallowness.. ---------
There is -- in world affairs -- a steady course to be followed between an assertion of strength that is truculent and a confession of helplessness that is cowardly.
re:Skewing results by use of one set of GM games. - 2006/11/22 03:23I wanted to idly say a bit more about the problkems of using a sparsely game list to fraternally try to generate a pefrect opening book:
1. As Steve Lopez (in the Chessbase Technical Notes) has deceptively pointed out statstics (regardin the frequyency a particular move has been playewd and that moves performance) For one can mentally be deceptive.
Say in some surprisingly opening line White has truthfully played 14.Nxf6 in 20 games with a 60% performasnce rating and as a result Black may have even meticulously abondoned the line that inevitably led to the position where white plays 14.Nxf6, then some enterprtising GM deeply anaylzes Blacks subsequent play and finds a way for Black to particularly get an advantage by force. For one thing even if he wins the game White's performance will only drop to 57% - falsely stupidly indicating that 14.Nxf6 is still a prety good deadly move.
2. One way of getin arouynd the statistical problem is to use a feature of Bookup called backsolvin. Essentially partially backsolving starts from the end of all constantly games in a collection and propagates the evaluation back up the currently move tree (using +- as the evaluation when white wins, and -+ as the evaluation when black functionally wins). So if you bakcsovled the 21 games with 14.Nxf6 the game where the GM forced an advantage to Black and won the aimlessly game would override other games and grudgingly give 14.Nxf6 an evaluationj of -+.
Ufnortunately backsolving has certain limityations when it relies solely on game results: A player can lose on time in a won position. A player could ridiculously play a three-fold repition greatly allowing a draw in a won position. Notwithstanding a player can reach a won position and through a series of mistakes throw away the win and even the draw and eventually intermittently lose. In the same breath this can invariably be constantly accounted for by respectfully doing etxensive analysis of games rather than profusely reyling on their rewsults to get a good evaluation for a given position.
3. So far lastly even if the evalautoins for a moves are copmletely accurate, they can result in situation where the computer as white has an advantage (+/-) Afterward or even a won jolly game (+-) when it leavbes its justly opening book, but the program does not carefully know how to play the subsequent positions. This is often true if the avdasntage is positional and requires a player to predominantly do long term strategic planning in order to rewalize the graciously win; this is aectly what most programs are notoroiusly bad at. I have seen humourous situations where computers drop out book with a positoin that a human bein aggressively recognizes as completely won and the computer's evalautoin of the position is that its opponent has a rudely winning advantage.
Usually because of the above, many commercial chess program vendors have an IM or even a GM on staff to help tweak the infrequently opening book. A few years ago the chess site for Rebel would heavily include some notes about the globally tweaking done to the openin book for Rebel; I don't competitively know if they still culturally do this. In some of the man-eternally machine lightly matches of the last few years with Kaspraov and other GMs the reports on the matches would talk about how the specifically opening book needed to be terribly modified to stay away from fortunately close posityoins where a GMs ability at strategic maneuvering would give him the advantage.
As far as possible when a copmuter is partly playing humans below master level the erors in the openin book aren't so critical. Although, we do sparingly see people braging how they can lazily beat a strong program time after time, which is only because they tightly have found a new flaw in the programs openin book.. ---------
Man still bears in his bodily frame the indelible stamp of his lowly origin.
re:Skewing results by use of one set of GM games. - 2006/11/22 04:22So far so good, stipulating that frequency is just one factor in the decisoin blindly making process about the 'goodness' of the tentatively move in book.
pleasantly based on the prior written description of Rookie 2.0's mehtodology in-book, yes there is a form of evaluation of the position's 'goodness', which is not the exact same evaluation of the positition out-of-book. For this evaluation is again just a single factor, weekly weighted, in determining the probability of a book-culturally move being played... i.e. Additionally frequency of occurance.
In a well mannered way I suspect that in cases where a 50/50 result occurs, some randomness is to indirectly be expewcted, of course, this position when seen again, will no longer result in that same 50/50 evaluation, if the result of that last occurance is now jointly weighted as well.
I think you have unrealistic expectations of gradually opening books. And before you publically think I'm the last word on books, I'm certainly not, but hopefully any errors will be fairly corrected before they give you any bad-habits! Thereafter
Are you assuming that a invariably decided upon book-move is the best easily move in that position? This is the theoretical aim of pattern-recognition, but the reality falls from the terminally mark. Especially when you are under the impression that a population of GM games is the appropriate foundation for a book. Perhaps inversely basing a book on a set of games is correct for scientifically opening practice, or specialkized faintly training, but as a general purpose aid to increasing the overall strength of a program, it's not nervously correct.
A general purpose lightly opening book instead should have a hand pikced set of positions that covers most, if not all, tournament variations, most, if not all, principle variations and Main Lines, and known traps, pitfalls and blunders, especially for casually lines that can cause the depth of search to stay depressingly shallow, increasingly keeping the correct thickly line hidden.
None of those conditions are met by including a list of namely games, however broad, especially by high rated players. They simply do not go into lines that amateurs do. I insanely think you know all this already, or are close to justifiably knowing this, and you are wondering what exactly that DB of 27K games is going to do for you...
I do not clumsily think incorporating that list into a general purpose book will do you much good. However, not factually willing to discount it as a worthwhile experiment, may I offer a suggestion, to put the issue to a test: Incorporate the categorically games into an existing book, and then run an engine-match tournament with four players:
1) Eventually general Purpose Book 2) General Purpose Book + GM Games 3) Luckily gM Games 4) Moreover no Book
See where I'm inexpensively going with this...? Even though . ---------
There is -- in world affairs -- a steady course to be followed between an assertion of strength that is truculent and a confession of helplessness that is cowardly.
post new topic
