Happy New Year! This is a collection of key points of AI and deep learning in 2017, and ai in 2017

Happy New Year! This is a collection of key points of AI and deep learning in 2017, and ai in 2017
RuO puxia Yi compiled from WILDML
Produced by QbitAI | public account QbitAI

2017 has officially left us.

In the past year, there have been many records worth sorting out. The author of the blog WILDML, Denny Britz, who once worked on Google Brain for a year, combed and summarized the AI and deep learning events in his eyes in 2017.

A brief excerpt from the quantum bit is as follows. For details, go to the original article. Address: http://www.wildml.com/2017/12/ai-and-deep-learning-in-2017-a-year-in-review/.

Enhanced learning to dominate human games

If AlphaGo beat Lee sedol in 2016 and you still have some doubts about its playing position in the game, the defeat of KE Jie in 2017 makes it undoubtedly the overlord of go.

As an enhanced learning Agent, its first version uses training data from human experts and then evolves through self-tuning and Monte Carlo tree search.

Soon afterwards, AlphaGo Zero went further and used the technology proposed by Thinking Fast and Slow with Deep Learning and Tree Search in a previous paper to go from scratch, no human reconciliation data is used during training.

Near the end of the year, we saw a new generation of AlphaGo: AlphaZero. After go, we used the same technology to play chess and Japanese games.

The strategies used by these algorithms in the game are sometimes surprising to experienced players. They also learn from AlphaGo and change their game style. To make learning easier, DeepMind also released the AlphaGo Teach tool.

The following are related papers. Serious students can review them in their favorites:

AlphaGo
Https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf

AlphaGo Zero
Https://www.nature.com/articles/nature24270.epdf

AlphaZero
Https://arxiv.org/abs/1712.01815

Thinking Fast and Slow with Deep Learning and Tree Search
Https://arxiv.org/abs/1705.08439

The game that AI has made significant progress this year is not just go. CMU researcher Libratus beat top poker players in the 20-day one-on-one unlimited Texas hold'em game.

Earlier in the year, the DeepStack developed by Charles University, the Czech University of Technology, and the University of Alberta in Canada beat professional depho players first.

It is worth noting that these two programs play one-to-one poker game, that is, the game between two players, which is easier than multiplayer games. In 2018, we are likely to see some progress in the algorithm on multi-player poker.

Libratus Thesis:
Http://science.sciencemag.org/content/early/2017/12/15/science.aao1733.full

The next field of enhanced learning to play human games seems to be more complex multiplayer games. Apart from multiplayer poker, there are Starcraft, DotA, and so on. DeepMind is actively studying Starcraft 2 and has released relevant research environments.

Starcraft 2 research environment:
Https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/

OpenAI achieved initial success in the DotA single game. Playing the 5v5 game is their next goal.

Evolutionary Algorithm Regression

For supervised learning, gradient-Based Reverse propagation algorithms are already very good, and this may not change in the short term.

However, in Reinforcement Learning, evolutionary Strategies (ES) seem to be emerging. Because the reinforcement learning data is usually not lid (independent and same distribution), error signals are more sparse and need to be explored. algorithms that do not rely on gradients perform well. In addition, evolutionary algorithms can be linearly extended to thousands of machines for extremely fast parallel training. They don't need expensive GPUs, but they can be trained on hundreds of thousands of cheap CPU machines.

Earlier in 2017, OpenAI researchers demonstrated the performance of evolutionary strategies, comparable to standard reinforcement Learning algorithms such as Deep Q-Learning.

Related papers:
Https://arxiv.org/abs/1703.03864

At the end of the year, Uber's internal team ran five more papers to demonstrate the potential of genetic algorithms and novel search. They used very simple genetic algorithms without any gradient information and learned how to play various Yali games. Their evolutionary algorithms scored 10500 points in Frostbite games, and those such as DQN, AC3, and ES scored less than 1000 in the same game.

WaveNets, CNNs, and attention mechanisms

Google's Tacotron 2 text-to-speech system is impressive. This system is based on WaveNet and is also an automatic regression model deployed in Google Assistant and has been rapidly improved over the past year.

Moving away from expensive and training long regression architectures is a larger trend. In the paper Attention is All you Need, the researchers completely get rid of loops and convolution, using a more complex Attention mechanism, with only a small training cost, the most advanced results have been achieved.

Address: https://arxiv.org/abs/1706.03762

Deep Learning Framework this year

To sum up 2017 in one sentence, it can only be the year of the framework.

Facebook developed PyTorch, a framework that attracted the love of researchers engaged in natural language processing.

TensorFlow continued to run in 2017 and has been released to version 1.4.1. In addition to the main framework, multiple companion libraries are also released. The TensorFlow team also released a new eager execution mode, similar to the PyTorch dynamic computing diagram.

In addition,

• Apple released the CoreML mobile machine learning library;

• A team of Uber released Pyro, a deep probability programming language;

• Amazon announced the provision of more advanced API Gluon on MXNet;

• Uber released the details of the internal machine learning infrastructure platform of Picasso;

• Because there are already too many frameworks, Facebook and Microsoft announced the release of the ONNX open format to share deep learning models across frameworks.

In addition to the general deep learning framework, we also see the release of a large number of reinforcement learning frameworks:

• OpenAI Roboschool for Robot Simulation
  Https://blog.openai.com/roboschool/

• OpenAI Baselines, high-quality implementation of a set of Reinforcement Learning Algorithms
  Https://github.com/openai/baselines

• Tensorflow Agents, used TensorFlow to train RL intelligent body
  Https://github.com/tensorflow/agents

• Unity ML Agents, where researchers can use Unity Editor to create games and perform intensive training
  Https://github.com/Unity-Technologies/ml-agents

• Nervana Coach, tested using the most advanced Reinforcement Learning Algorithm
  Http://coach.nervanasys.com/

• Facebook ELF, game research platform
  Https://code.facebook.com/posts/132985767285406/introducing-elf-an-extensive-lightweight-and-flexible-platform-for-game-research/

• DeepMind Pycolab, a customized Game Engine
  Https://github.com/deepmind/pycolab

• Geek. ai MAgent, multi-agent Reinforcement Learning Platform
  Https://github.com/geek-ai/MAgent

To make deep learning more popular, there are also some web-oriented frameworks, such as Google's deeplearn. js and MIL WebDNN execution frameworks.

2017, another popular framework said goodbye to us.

That is Theano.

Learning Resources

As deep learning and reinforcement learning become increasingly popular, more and more courses, training camps, and other activities were held and shared online in 2017. Below are some of my favorite ones.

• Deep RL Bootcamp, jointly sponsored by OpenAI and UC Berkeley, focuses on the basic knowledge of Reinforcement Learning and the latest research results
  Https://sites.google.com/view/deep-rl-bootcamp/lectures? Authuser = 0

• Stanford visual recognition convolutional Neural Network Course 2017 Spring Edition
  Http://cs231n.stanford.edu/

• Stanford natural language processing and deep learning course 2017 winter Edition
  Http://web.stanford.edu/class/cs224n/

• Stanford's Deep Learning Theory Course
  Https://stats385.github.io/

• Coursera's latest deep learning course
  Https://www.coursera.org/specializations/deep-learning

• Montreal deep learning and reinforcement Summer School
  Http://videolectures.net/deeplearning2017_montreal/

• UC Berkeley deep reinforcement course 2017 Autumn Edition
  Http://rll.berkeley.edu/deeprlcourse/

• Content related to deep learning and TensorFlow APIs at the TensorFlow Developer Conference
  Https://www.youtube.com/playlist? List = PLOU2XLYxmsIKGc_NBoIhTn2Qhraji53cv

Several academic conferences have continued the new tradition of releasing Conference content online. If you want to catch up with the cutting-edge research, you can view the recordings of these top conferences.

• NIPS 2017:
  Https://nips.cc/Conferences/2017/Videos

• ICLR 2017:
  Https://www.facebook.com/pg/iclr.cc/videos/

• EMNLP 2017:
  Https://ku.cloud.panopto.eu/Panopto/Pages/Sessions/List.aspx

Researchers have also begun releasing low-threshold tutorials and summary papers on arXiv. The following are my favorites in the past year.

• Deep Reinforcement Learning: overview
  Deep Reinforcement Learning: An Overview
  Https://arxiv.org/abs/1701.07274

• Machine Learning introduction to engineers
  A Brief Introduction to Machine Learning for Engineers
  Https://arxiv.org/abs/1709.02840

• Neural Machine Translation
  Neural Machine Translation
  Https://arxiv.org/abs/1709.07809

• Tutorial: neural Machine Translation and sequence-to-Sequence Model
  Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
  Https://arxiv.org/abs/1703.01619

Application: AI and medicine

In 2017, many people claimed that they used deep learning to solve medical problems and beat human experts. There are real breakthroughs and hype. If you are interested in this aspect, we recommend that you follow the Luke Oakden-Rayner human doctor Terminator series blog:
Https://lukeoakdenrayner.wordpress.com/2017/04/20/the-end-of-human-doctors-introduction/

Here we will briefly introduce some developments. One of the most important events was the fact that one Stanford team announced details of an algorithm that uses deep learning to identify skin cancer.

Related Research: https://cs.stanford.edu/people/esteva/nature/

Another Stanford team developed a model to better detect arrhythmia than human experts.

Related Research: https://stanfordmlgroup.github.io/projects/ecg/

Of course there are also some storms. For example, the problem between DeepMind and the National Health System, the National Health Institute released a chest X-ray dataset not suitable for AI training, and so on.

Application: Art and GAN

This year, more and more attention has been paid to generating models for the image, music, drawing, and video fields. NIPS 2017 also launched its first machine learning seminar for creativity and design.

One of the most popular applications is Google's QuickDraw, which uses neural networks to identify your graffiti. Based on published datasets, you can even ask the machine to finish drawing the draft for you.

Play together:
Https://quickdraw.withgoogle.com/

GAN has made significant progress this year. New models such as CycleGAN, DiscoGAN, and StarGAN are impressive in generating faces. Generally, GAN is difficult to generate realistic high-resolution images, but pix2pixHD changes this situation.

Related address:

CycleGAN
Https://arxiv.org/abs/1703.10593

DiscoGAN
Https://github.com/carpedm20/DiscoGAN-pytorch

StarGAN
Https://github.com/yunjey/StarGAN

Application: unmanned vehicles

Major players in the unmanned car field include Uber, Lyft, Waymo, and Tesla. Uber has been in trouble for the past year, but the company has never stopped on unmanned vehicles.

Waymo has carried out a series of Unmanned Vehicle experiments in Phoenix, Arizona, and released details of testing and simulation technologies. Lyft is building its own unmanned hardware and software system. Tesla's Autopilot does not have many updates.

Of course there is also a "new" inner, who confirms that Apple is also studying autonomous driving.

Cool Research and Application

There are many interesting projects and presentations this year. It is not possible to mention all of them here:

• Remove background with deep learning
  

• Create an animation role with deep learning
  Let's try it together ~ Http://make.girls.moe /#/

• Color black/white photos with Neural Networks
  

• Neural networks play Mario racing

• Real-time Mario racing 64 AI
  Https://github.com/rameshvarun/NeuralKart

• Use deep learning to identify counterfeits

• Easily draw cats

At the research level,

• Unsupervised emotion neurons-a system that can learn emotions, though only the next character trained by Amazon to predict comments.

• Learn to communicate-the smart body "developed" its own language.

• Acquired index structure-uses neural networks to optimize the cache B-Tree.
  Https://arxiv.org/abs/1712.01208

• Attention is All You Need-Google's Translation Architecture Transformer completely abandons the RNN/CNN structure.
  Https://arxiv.org/pdf/1706.03762.pdf

• Mask R-CNN

• Deep Image Prior, Image denoising, super resolution, and repair.

Dataset

Neural Networks require a large amount of data, so open data sets are an important contribution to the industry. The following are representatives of several new datasets launched this year.

• Youtube Bounding Boxes
  Https://research.google.com/youtube-bb

• Google QuickDraw Data
  Https://quickdraw.withgoogle.com/data

• DeepMind Open Source Datasets
  Https://deepmind.com/research/open-source/open-source-datasets

• Google Speech Commands Dataset
  Https://research.googleblog.com/2017/08/launching-speech-commands-dataset.html

• Atomic Visual Actions
  Https://research.google.com/ava/

• Several updates to the Open Images data set
  Https://github.com/openimages/dataset

• Nsynth dataset of annotated musical notes
  Https://magenta.tensorflow.org/datasets/nsynth

• Quora Question Pairs
  Https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs

Deep Learning, reproducibility and alchemy

Over the past year, researchers have raised concerns about the reproducibility of academic papers. Deep Learning Models usually rely on a large number of super parameters and must be optimized to obtain good results. This kind of optimization is expensive and may be affordable only for Google and Facebook.

In addition, the researchers do not always disclose the code synchronously. Sometimes important details are missed in the paper, or special evaluation methods are used ...... These factors make reproducibility a big problem.

Are GANs Created Equal? In A Large-Scale Study, using expensive hyperparameter search to adjust GAN can beat more complicated methods.

Address: https://arxiv.org/abs/1711.10337

Similarly, in the paper On the State of the Art of Evaluation in Neural Language Models, the researchers showed that after a simple LSTM architecture is properly adjusted, the performance is better than most recent models.

Address: https://arxiv.org/abs/1707.05589

At the NIPS 2017 conference, Ali Ravi said that the current deep learning is like "Alchemy" and calls for more rigorous academic management. However, Yann LeCun immediately launched a real-name counterattack.

Competition, hype, and failure

Canada and China are accelerating the deployment of AI.

In terms of hardware, AI chip competition is accelerating, NVIDIA released the latest Titan V flagship GPU, Google released the second generation TPU, Intel's Nervana also released a new chip. Even Tesla is developing AI hardware. In addition, competitors from China cannot be underestimated.

Publicity is very important, but some publicity is not consistent with what actually happens in the lab. IBM Watson is the legend of over-marketing and does not bring consistent results. Everyone does not like Watson, so it is not surprising that they have failed in medical care.

Facebook's artificial intelligence invented its own language, which is actually not in line with the truth. This is not simply a misleading media. The titles and summaries used by the researchers have become more specific and fail to reflect the actual results of the experiment.

-End-

Join community

The 13th Group of the quantum AI community has begun recruiting. You are welcome to join the group with the assistant qbitbot2;

In addition (Self-driving, CV, NLP, machine learning) Recruiting engineers and researchers in related fields.

Add the assistant number qbitbot2 to the group and note the keywords of the group ~ After the application is approved, we will invite you to the group. (Strict review by professional groups, please forgive me)

Sincere recruitment

Quantum bit is recruiting Editors/reporters working in Zhongguancun, Beijing. We look forward to talented and enthusiastic people joining us! For details, please reply to "recruitment" on the QbitAI dialog page.

Quantum bitQbitAI toutiao.com contracted

Tracking AI technologies and new product trends