My next experiment was with Sergey Ivashchenko‘s Chess School 1b, which comprises Stage 4 and Stage 5 of Chess School. I divided the book into four batches, which I labelled A to D. Batch A was the 1st, 5th, 9th problem, and so on. Batch B was the 2nd, 6th, 10th problem, and so on. Batches C and D were constructed in the same way. I found that I had bitten off more than I could chew here. The endgame studies at the end of Stage 5 caused me particular trouble, so I set them aside, which left me with 539 problems (134 or 135 problems per batch). The early part of my schedule was the same as for the Susan Polgar Experiment:

Sa Mo Fr Fr We Mo

Week 1: A1, A2, A3 Days: 1-7

Week 2: B1, B2, B3, A4 Days: 8-14

Week 3: C1, C2, C3, B4 Days: 15-21

Week 4: D1, D2, D3, C4, A5 Days: 22-28

Week 5: D4, B5 Days: 29-35

Week 6: C5 Days: 36-42

Week 7: D5 Days: 43-49

Week 8: A6 Days: 50-56

Week 9: B6 Days: 57-63

Where A1, A2, A3.… are passes 1, 2, 3... of batch A, and similarly for the other batches. For the first nine passes, the day on which each pass takes place was again given by the table:

Pass: 1 2 3 4 5 6 7 8 9

Day: 1 3 7 14 26 50 96 185 355

I again used the Empirical Rabbit Timer to time my solutions and collect the results. Incorrect solution times were counted as more than 60 seconds irrespective of the actual time spent. I counted my solution as correct if I got the right idea and the right first move. Here is a cumulative graph of my first pass through each of the batches:

￼

The results show a rising trend for time limits of 30 seconds and more. Here is a comparison of my performances for my first pass through the batches of the Susan Polgar Experiment, Stage 4 and Stage 5:

￼

(N.B. I have aggregated the results over all the problem batches here, for each problem set. 0-5 denotes 0-4.999... seconds, and similarly for the other “buckets”.) It is clear that I found Stage 4 harder than SPolgar, despite any improvement that I made between the corresponding batches for the two experiments. It is also clear that I found Stage 5 harder than Stage 4. Here are the results for my first pass through each batch for Stage 4:

￼

I appear to have made good progress here, particularly for the longer time limits. The results for my first pass through each batch for Stage 5 were less tidy:

￼

I appear to have made good progress here too, for time limits over 30 seconds, except for batch D, which appears to have been harder then the others. For Stage 4, the problems are divided into sections according to the problem type. My method of dividing the problems into batches ensures that each of the four batches contain (nearly) equal numbers of each type of problem, which helps to equalise the difficulty of the batches. In contrast, the problems in Stage 5 are in random order, which does not help here.

Applying the method of calculation that I described in my earlier article, Rating Points Revisited, to the whole of 1b, gives the table:

Sec Gain SD Gain/SD

5 46 55 0.84

10 8 43 0.19

15 1 38 0.03

20 -13 38 -0.34

25 -24 44 -0.55

30 41 47 0.87

35 49 54 0.91

40 59 91 0.65

45 68 65 1.05

50 78 84 0.93

55 94 106 0.89

60 71 114 0.62

Where Sec is the time limit, Gain is the Elo points gain, and SD is the corresponding standard deviation. The accuracy here is one standard deviation at best, so we cannot draw any reliable conclusions, but it is likely that I made a significant improvement. The same method of calculation suggests that Stage 4 is about 100 Elo points harder than SPolgar, and Stage 5 is about 50 points harder still.

For an update, see my later article: Susan Polgar + Ivaschenko Revision.

Hello Dear Bright Knight!

ReplyDeleteI would like you to ask what kind of experiment would you like me to do at this specific book. I want to support your tests and I could do ANY tests (at my disposal) you need.

I could solve these problems as batches, from beginning to the end, with random selections and any type of solving you would like to test. I can solve the positions (this testbook) as many times as you wish (in any order and additional conditions you need and I would be able to provide).

Just one (small) request: please try to create an experiment (test) with a time period of no longer than 30 days (as I will not be able to do tests at longer period than this one).

I am fascinated (interested is wrong word) at your AMAZINGLY inspiring blog. I was looking for some "food for thought" for quite some long time. It looks I have finally found it! You might not know how many very important and hard to ask (and answer!) questions you have given me with your GREAT tests and posts. I want to THANK YOU on the behalf of these people who cannot (or do not want) to share their gratitude and benefits!

Thank you for your encouragement. Aoxomoxoa found that adult Chess Tempo users typically improve their performance at Chess Tempo for about the first 7,000 problems, but none of them improved significantly after that. Did their chess improve as a result of those 7,000 problems? We do not know.

ReplyDeleteIf you want to try an experiment with this book, I suggest that you repeat mine.

I am reading your fantastic blog at least last 3 days. Even though I do not understand any math explanations (i skipp them), I LOVE your conclusions, summary and observations. Your questions and answers are truly amazing, scientific and very interesting - they encourage me to think over A LOT of problems (concepts) connected (related) with "improvement vs learning" (mostly using repetitions and trying to find best move in a specific positon from the books you are testing).

ReplyDeleteI am not sure if I should share my views and opinion related to chess improvent based on "repetition method". However if there would be "green light" I could do it.

After hundreds of hours dedicated to chess improvement problems (concepts) I know much less then when I started. However I have much more questions and problems I want to find out what is going on.

Due to limited time and possibilities - I am going to repeat your experiment with one change: I will shorten the experiment twice (double). It means I will "pack" 2 weeks into 1.

If you or anyone from this fantastic blog have any questions related to improvement related to chess repetitions (like you or "MLM" followers are doing) just write them down. I have done (repeated) about 12 thousand of chess puzzles (range 1000-1800). I estimate I "X-rated" about 4-5 thousand of original (I mean: from different test books) problems/puzzles.

Just to notice: the more specific and detailed questions - the more useful and honest answer I can give. If the question would be too broad - I may not be able to say anything more than what you have already known.

PS. I am such a delighted with your fantastic work that I read your posts all night (even delaying going to bed more than reasonable period of time). I have dreamt about such experiments, but so far I have not found such a source. I appreciate your efforts very much and thank you for ALL of inspiring words, explanations, writings, summaries, answers and (deep) questions. You are giving me a truly amazing boost to think really deep and hard! Thanks a million!