The secret sauce in AI: Reinforcement Learning

This article discusses how computers learned to perform complex tasks like playing the game of Go. The breakthroughs did not come from programming. The real growth came when the computer was able to use trial and error to improve its performance.

I’m watching the driving simulation at the biggest artificial-intelligence conference of the year, held in Barcelona this past December. What’s most amazing is that the software governing the cars’ behavior wasn’t programmed in the conventional sense at all. It learned how to merge, slickly and safely, simply by practicing. During training, the control software performed the maneuver over and over, altering its instructions a little with each attempt. Most of the time the merging happened way too slowly and cars interfered with each other. But whenever the merge went smoothly, the system would learn to favor the behavior that led up to it.

This approach, known as reinforcement learning, is largely how AlphaGo, a computer developed by a subsidiary of Alphabet called DeepMind, mastered the impossibly complex board game Go and beat one of the best human players in the world in a high-profile match last year. Now reinforcement learning may soon inject greater intelligence into much more than games. In addition to improving self-driving cars, the technology can get a robot to grasp objects it has never seen before, and it can figure out the optimal configuration for the equipment in a data center.....

That view changed dramatically in March 2016, however. That’s when AlphaGo, a program trained using reinforcement learning, destroyed one of the best Go players of all time, South Korea’s Lee Sedol. The feat was astonishing, because it is virtually impossible to build a good Go-playing program with conventional programming. Not only is the game extremely complex, but even accomplished Go players may struggle to say why certain moves are good or bad, so the principles of the game are difficult to write into code. Most AI researchers had expected that it would take a decade for a computer to play the game as well as an expert human.
....
Reinforcement learning works because researchers figured out how to get a computer to calculate the value that should be assigned to, say, each right or wrong turn that a rat might make on its way out of its maze. Each value is stored in a large table, and the computer updates all these values as it learns. For large and complicated tasks, this becomes computationally impractical. In recent years, however, deep learning has proved an extremely efficient way to recognize patterns in data, whether the data refers to the turns in a maze, the positions on a Go board, or the pixels shown on screen during a computer game.
....

The article goes on to cite self-driving cars as a good application of this technology. It enables "good sequences of decisions" to perform complex maneuvers like negotiating roundabouts.

Here is a link to the full article:  MIT Technology Review: Reinforcement learning

MIT ILP

I recently attended the MIT Industrial Liaison Program (ILP) Digital Health Conference in Cambridge. It was a really amazing conference. There were presentations by so many brilliant minds. You could literally feel your brain aching as it struggled to keep up with all of the topics.

Here is an example of what we discussed (just on the first day!):

I found one slide in particular summed up my personal opinion on much of what we discussed. While there are many things that these technologies can do, it will be more important for us to focus that activity on improving a specific process to achieve a desirable outcome!

I also learned that I should be thankful I don't have to commute in to Boston every day. Even though it isn't nearly as crowded as NYC, it's still too reminiscent of an ant farm!

The MIT Media Lab also has a wonderful view of the Charles and the city of Boston from the sixth floor deck.

I anxiously await the next version of this meeting!

Good to the last drop

Original post:  Mar 27, 2015

If you have ever struggled to get ketchup out of a glass jar, you'll understand why this phenomenon is annoying. Actually, it's beyond that. It's quite wasteful.

Tests by Consumer Reports in 2009 found that much of what we buy never makes it out of the container and is instead thrown away — up to a quarter of skin lotion, 16 percent of laundry detergent and 15 percent of condiments like mustard and ketchup.

A professor at the Massachusetts Institute of Technology (MIT), Kripa Varanasi, created a new company called LiquiGlide to market a product that promises to overcome this effect.

What makes it hard to get mayonnaise and toothpaste out is that they are what scientists call Bingham plastics. A Bingham plastic, named after Eugene Bingham, a chemist who described the mathematical properties, is not made of plastic; the term describes a highly viscous material that does not flow without a strong push.

Ironically, the professor actually was searching for the answer to an entirely different problem.

Dr. Varanasi did not set out to solve the problem of clingy glue and mayonnaise. Rather, he was thinking of larger-scale industrial challenges, like preventing ice formation on airplane wings and allowing more efficient pumping of crude oil and other viscous liquids. How to make a slippery surface has been an interest for many scientists and engineers with many potential uses.

When water or other liquids flow through a pipe, the layer of liquid next to the pipe wall typically sticks, not moving. Farther from the pipe wall, the liquid flows, fastest at the center. “Different layers of water are sliding past one another, and therefore there is friction, which is viscosity, and that is why you need to pump it,” said Neelesh A. Patankar, a professor of mechanical engineering at Northwestern University, who is not involved with LiquiGlide.

The trick is to find a way around the so-called no-slip boundary condition. “What people have tried to do is, can we have something between the solid surface and the liquid which will help the liquid slide?” Dr. Patankar said.

One simple example is when a droplet of water skitters across a hot pan that vaporizes some of the water. The droplet is riding on a layer of steam like a hovercraft, not touching the pan.

Dr. Patankar and other scientists have been investigating superhydrophobic surfaces. A hydrophobic surface repels water; a superhydrophobic surface, as one might imagine, really repels water. Inspired in part by lotus leaves, the surface of a superhydrophobic material looks rough, at least under a microscope. Water rolls up into balls, sitting on the tips of the rough surface, but mostly on air trapped between the droplet and the rough surface. The droplets roll off easily

The successful commercialization of this product could help reduce waste dramatically. There might even be potential uses in healthcare. The pictures below help show the action and how the product works.

Here is the link to the full article:  http://www.nytimes.com/2015/03/24/science/with-new-nonstick-coating-the-wait-and-waste-is-over.html