When I began experimenting with machine learning as an undergrad in 1995, I was immediately taken with the possibilities. Contemplating biological evolution, it’s hard not to be awed that Darwin’s “endless forms most beautiful and most wonderful” have evolved from a blind evolutionary process, without invention from a conscious designer. Similarly, with machine learning, there is the enticing prospect that programmers won’t actually have to solve the hardest problems in engineering and artificial intelligence. Rather, with an appropriately advanced theory of machine learning (and machine evolution), we’ll evolve and train machines to solve these difficult problems on their own.On their own! The very thought is powerful, and magical, and maybe a little heretical too.
Despite my continuing enthusiasm for machine learning, I hear from skeptics all the time.Some are my friends. They are understandably burned out by overzealous claims about machine intelligence (e.g., the first chess playing computers, Doug Lenat’s Cyc Project, the mid 80’s literature on neural networks, the early 90’s literature on artifical life, Ray Kurzweil’s books, and so on). Time after time, we hear about some new AI research program that promises to introduce intelligent machines to the world, only to have these fizzle out or hit a plateau after a few years.
Well, I’m sorry to say, the age of truly intelligent machines is not near. We are still many decades (if not many centuries) away from producing machines that can hold a good conversation, contemplate their own existence, or watch Stephen Colbert and have a genuine laugh. Nope, no conscious robots anytime soon. Humans are still much smarter than machines.
Still, I’m as enthusiastic as ever about machine learning because we are at the point where machines can learn autonomously to solve many large, interesting problems. Machine learning thrives on big data sets and lots of CPU cycles to crunch them. Thanks to the internet and cloud computing we now have more of both than ever. If you’re an AI skeptic, then I encourage you to stop worrying about conscious robots, and start thinking about the more immediate, practical applications of machine learning. There are many.
With machine learning, the programmer becomes a meta-programmer; instead of finding and coding the solution to a problem, she (1) gathers an appropriate data set, and (2) codes an appropriate learning algorithm that (if all goes well) will learn to solve the problem by examining this data set. For many programming challenges, this is more feasible and cost-effective than finding and coding the solution directly.
Being a meta-programmer in this sense requires a different skills than traditional programming. You need an understanding the practical requirements and limitations of various learning algorithms (though not necessarily how to implement them, since open source implementations of many algorithms are available).You also have to be crafty about gathering training data. Can you scrape data from the web? Can you assemble a training set using Mechanical Turk or other human computation services? If you run a web service, can you collect the data from your users? Is a free or licensable data set already available? To a large extent, meta-programming through machine learning is about clever data acquisition; gathering the richest data set you can with as little time and money as possible.
Definitely would be interested in the PART 2. been couple of months since you posted this. Is the second detailed part coming up in near furture?
I’d also be interested in a part 2. Btw, I’m trying to compile a list of datasets for my site at http://machine-learning.eggsprout.com. You seem like you have plenty of experience finding them in magical ways. Would you be willing to share some knowledge with us over there?
I wrote my senior thesis in college back in 1983 looking at why machines can’t translate very well. My conclusion was that it is because we do not know how WE do it. The same is true of “holding a good conversation”. Until we figure out how exactly we humans are doing it, we cannot tell the computer how to do it. I believe that the good conversational user interface will not happen until advances in linguistics (or neuropsychology or …) happen and we finally say, “Aha! that is how we are doing this!” Then, we will be able tell (program) the computer to do it. I compare it to the change in chemistry after the establishment of the Table of Elements. Before TOE it was alchemy, after TOE, it was chemistry. Now, we are experimenting with programming computers with all kinds of magical formulas that we hope will work. After we get a model of human language that actually fits reality, we will be able to move forward at a great pace to a real good conversational computer.
Ken, thanks for this insightful post. “Meta-programming” is a very nice concept, good name for what actually is done in ML.