Over the past year, it's become pretty clear that machines can now beat usin many straightforward zero-sum games. A new study from an international team of computer scientists set out to develop a new type of game-playing algorithm – one that can play games that rely on traits like cooperation and compromise – and the researchers have found that machines can already deploy those characteristics better than humans.
Chess, Go and Poker are all adversarial games where two or more players are in conflict with each other. Games such as these offer clear milestones to gauge the progress of AI development, allowing humans to be pitted against computers with a tangible winner. But many real-world scenarios that AI will ultimately operate in require more complex, cooperative long term relationships between humans and machines.
"The end goal is that we understand the mathematics behind cooperation with people and what attributes artificial intelligence needs to develop social skills," says lead author on the new study Jacob Crandall. "AI needs to be able to respond to us and articulate what it's doing. It has to be able to interact with other people."
The team created an algorithm called S# and tested its performance across a variety of two-player games, either in machine-machine, human-machine or human-human interactions. The games selected, including Prisoner's Dilemma and Shapley's Game, all required different levels of cooperation or compromise for a player to achieve a high payoff. The results were fascinating, showing that in most cases the machine outperformed humans in the games.
"Two humans, if they were honest with each other and loyal, would have done as well as two machines," says Crandall. "As it is, about half of the humans lied at some point. So essentially, this particular algorithm is learning that moral characteristics are good. It's programmed to not lie, and it also learns to maintain cooperation once it emerges."
An interesting technique incorporated into the algorithm was the machine's ability to engage in what the researchers called "cheap talk." These were phrases that the machine deployed either in response to a cooperative gesture ("Sweet. We are getting rich!"), or as a reaction to another participant lying or cheating ("You will pay for that!"). When the machines deployed "cheap talk," the human players were generally unable to pick they were playing a machine, and in most cases the comments doubled the amount of cooperation.
The researchers suggest these findings could lay the foundation for better autonomous machines in the future as technologies like driverless cars require machines to interact with both humans and other machines that often don't share the same goals.
"In society, relationships break down all the time," says Crandall. "People that were friends for years all of a sudden become enemies. Because the machine is often actually better at reaching these compromises than we are, it can potentially teach us how to do this better."

Right now, you're carrying around the most powerful computer in existence – the human brain. This naturally super-efficient machine is far better than anything humans have ever built, so it's not surprising that scientists are trying to reverse-engineer it. Rather than binary bits of information, neuromorphic computers are built with networks of artificial neurons, and now an MIT team has developed a more lifelike synapse to better connect those neurons.
For simplicity's sake, computers process and store information in a binary manner – everything can be broken down into a series of ones and zeroes. This system has served us well for the better part of a century, but having access to a whole new world of analog "grey areas" in between could really give computing power a shot in the arm.
The brain is a perfect model for those kinds of systems. While we've barely scratched the surface of how it works exactly, what we do know is that the brain deals with both analog and digital signals, processes and stores information in the same regions, and performs many operations in parallel. This is thanks to around 100 billion neurons dynamically communicating with each other via some 100 trillion synapses.
While neural networks mimic human thinking on the software side, neuromorphic chips are much more brain-like in the design of their hardware. Their architecture is made up of artificial neurons that process data and communicate with each other through artificial synapses. IBM's TrueNorth supercomputer is one of the most powerful neuromorphic systems, and Intel has recently unveiled a more modest, research-focused chip it calls Loihi.
In conventional neuromorphic chips, synapses are made of amorphous materials wedged in between the conductive layers of neighboring neurons. Ions flow through this material when a voltage is applied, transferring data between neurons. The problem is they can be unpredictable, with defects in the switching medium sending ions wandering off in different directions.

 "Once you apply some voltage to represent some data with your artificial neuron, you have to erase and be able to write it again in the exact same way," says Jeehwan Kim, lead researcher on the project. "But in an amorphous solid, when you write again, the ions go in different directions because there are lots of defects. This stream is changing, and it's hard to control. That's the biggest problem – nonuniformity of the artificial synapse."
To combat the problem, the MIT researchers designed a new medium for an artificial synapse. They started with a wafer of single-crystalline silicon, then grew a layer of silicon germanium over the top. Both materials have a lattice-like pattern, but silicon germanium's pattern is slightly larger, so when the two overlap it forms a kind of funnel shape, keeping ions on the straight and narrow.
The team built a neuromorphic chip using this technique, with silicon germanium synapses measuring about 25 nanometers wide. The team then tested them all by applying a voltage to them, and found that overall there was about a four percent variation in the current that passed through them. An individual synapse, tested over 700 cycles, was also found to keep a consistent current, with a variation of just 1 percent.
"This is the most uniform device we could achieve, which is the key to demonstrating artificial neural networks," says Kim.
The scientists then put the chip through its paces with a simulated test. They used an artificial neural network that functioned as though it was made up of three sheets of the neurons connected with two layers of the synapses. Then they fed in data on tens of thousands of handwriting samples, and found that the system was later able to recognize 95 percent of samples it was then given. That's not far off the 97 percent accuracy that more established systems can achieve.
Next, the team plans to develop a physical neuromorphic chip that can handle this task in the real world.
"Ultimately we want a chip as big as a fingernail to replace one big supercomputer," says Kim. "This opens a stepping stone to produce real artificial hardware."