The model I was training by only playing against me simply wasn’t improving enough. A few thousand games in I realized how hopeless the idea of hand generating training data was, and gave up.
I tried a new model, named Curly, which consisted of a single convolutional layer followed by a fully connected layer of neurons. The result shouldn’t be as effective as some of my more elaborate models, but it should train faster. I then ran it through a few dozen rounds of playing a mere 10,000 games in each data set. While this is significantly smaller than prior sets, it also means the training would be completed much more quickly.
Each data set consisted of 5,000 games playing Curly against itself (with a small amount of variability thrown in, just to liven things up), combined with 5,000 losses against the SRP. I’ve previously tried using one or the other; this was my first attempt at mixing the two.
The result was surprisingly effective. Curly’s effectiveness approached 90% against the SRP, so I played a full set of 7 games against it. The result? I lost 3 of the 7 games!
Honestly, I think I’ve had enough of Connect4 for a while now. I’m calling this one good enough, and moving on to a more complicated challenge: Gobang! The combination of a larger board and the freedom to play on any space makes it a much more difficult challenge then Connect4 was.
While I could keep pushing for an improvement in ability in Connect4, really I’m wondering how effective a neural network could be at the game. Connect4 is a solved problem at this point; we have an outline of how to play perfectly, provided by Victor Allis in his Masters Thesis in 1988.
What I find interesting is that the approach outlined in Victor’s paper is analytical and recursive, which isn’t really how Neural Networks function. They don’t follow a series of steps to analyze data; they merely “look at” a representation of the data and tease apart relationships which are non-obvious. This makes them good for problems like image recognition, but less than ideal for problems like balancing a checkbook.
For playing a board game, I reduce the board to a type of image which the neural network may look at. However, the Curly model is 210MB in size; a C program running Allis’s perfect methodology is a few KB.
By moving on to a more complex game (Gobang), I’m hoping to see less of an advantage for the recursive heuristic and more of an advantage for the neural network.