Many, many times, I’ve sworn that I would give up AGI research and dedicate my time to things that are more productive. I’ve probably worked on nearly 100 projects. Some were bigger than others, but a quick calculation of the hours that I’ve probably spent coding, reading, thinking, or banging my head against the wall is depressing. None of the work, of course, produced an AGI, but I don’t actually feel too bad about that part because no one else’s work has either, to my knowledge. What’s upsetting is that I could have spent that time snowboarding, or learning violin, or building a internet company. And yet despite this, I’m not better than the village drunk who swears to never drink again: it seems like a good idea and even do-able the morning after a project has been given up, but sooner or later I’m drinking the AGI punch again.
Of late I’ve run out of shiny new ideas, and so I did a mental survey of what I’ve already tried with the intent to pick what was probably the most successful of my projects and try to expand upon it. GoID players may already be familiar with what I chose: the robotic arm. (This is one of the GoID tasks. Read the description there for the gory details.)
The essential problem is that you need to get the robotic wrist to a particular Cartesian point, only using angular impulses to do so, compensating for gravity, and you need to do it quickly and efficiently. The sensor values that are provided are accurate enough that inverse kinematics could be used, but this is supposed to be an AGI problem, and no one believes that real life uses kinematics, so we’ll have none of that here thank you. The solution that I wrote (which is an enhanced version of the sample script) uses experience gathered by brute force and ignorance to build up a mapping of pairs of “where I am” and “where I want to be”s, against “what I need to do to get there”s. This approach creates a multi-dimensional problem space that needs to be reasonably populated with samples before it becomes useful. It takes a fair amount of time to search the space, and it takes a great deal of memory to remember all of the samples. Regardless, the result is that this relatively simple script starts off mostly just flailing about, but in time is realistically reaching for targets with determination. For those of you with children, I won’t need to make the obvious analogy.
And what are our brains but massive storage? Ok, a bit more than just that. But in particular our cerebellum, where movement plans get carried out moderated by the stream of sensory input, contains around half of our brain cells even though it takes up only a fraction of the space. I believe that this part of our brain uses a learning and execution strategy that is similar to what I implemented in the robotic arm task. Come on, after all my years of research, if only by probability, I had to get something right.
On a related note I started again playing tennis, and so naturally began wondering how it is I learn to hit the ball over the net so that it lands in play. I mean seriously, when you think about it, what are the odds? Sure enough, when you watch beginners play they rarely even get the racket to touch the ball, much less hit a winner. And there are so many variables at work from the moment your opponent hits the ball to the moment you hit it back. (Some would even say this before your opponent hits the ball.) You use your projections of the trajectory of the ball to run into place and position your body. You watch very carefully as the ball hits the ground because that initial information about how it bounces is critical. You might even have already incorporated the ball’s spin – if you can make it out – into how it will bounce. Then you wind up for your swing, which again is taking the ball’s spin into account if only subconsciously because things are now happening so quickly. But you also note from you peripheral vision where your opponent is so that you don’t make your shot too easy to return. And finally, you start your swing and hope for the best your muscle memory can do, because it’s impossible to make conscious decisions in the milliseconds it takes for the swing to complete.
In fact, the better you are, the fewer conscious decisions you make during an entire point. You might note that your opponent is in a weak position and rush to the net hoping to make a quick volley, all without conscious help because your subconscious has learned to recognize the combination of variables that leads to this winning conclusion.
You may be thinking that I want to build a tennis-playing robot. I do, but I won’t, although it would be so very cool. It would also be very expensive. (As you all know, all of my research is self-funded, which is to say not funded.) The plan is to create a version of Pong that bots can play. One key initial problem to solve is how a bot can learn movement patterns that play out in real time. So, initially the physical model will be trivial until mechanisms that solve such problems are worked out. Such a model might just have a mass-less ball that bounces off of walls and paddles at basic reflection angles. Improvements to the game could have a ball that can spin, or paddles that are rounded, or mechanisms on the paddles that can add or remove energy from the ball, or bots that can move horizontally from the baseline. The “real” environment should be complex enough that no bot can be able to predict too far in advance with any accuracy. The intent is to build a bot that can eventually play the game with some skill that has been refined from continued experience.
I suspect that the most interesting work will be in determining optimizations in the learning process, experience retention, and retrieval. It will probably be necessary early on to implement some manner of hierarchy so that general strategies of play can be established that break down into real movement plans.
But the first thing to do is create the simulation of the court. Box2D might work, but I don’t know yet if it handles things like spinning balls bouncing off of walls, or ball spin causing a curved trajectory.
Work in progress. Comments welcome.