From machine learning to learning machines, data analysis algorithms also need a good steward

650) this.width=650, "Title=", Vice president of global research and development, IBM Big Data and Analytics Division Dinesh Nirmal.jpg "src=" http://s5.51cto.com/wyfs02/M00/8A/BC/ Wkiol1g6pgmyhz3waajds_3kozs381.jpg-wh_500x0-wm_3-wmp_4-s_213774659.jpg "alt=" Wkiol1g6pgmyhz3waajds_ 3kozs381.jpg-wh_50 "/>

(Dinesh Nirmal, vice president, global research and development, IBM Big Data and Analytics Division)

This is the 400 anniversary of Shakespeare's death. In the Shakespeare famous "Julius Caesar", a fortune teller has such a prophecy without context: "Please be careful ' March '," said Caesar after listening to not know what the meaning of this sentence, so that the divination to continue to dream it. As a result, on March 15, Caesar was plotted to assassinate him. Similarly, today's predictive algorithms can tell you a prophecy but fail to provide the right context, making it difficult to make further action decisions.

Another example of predictive algorithms is the emergence of an artificial intelligence synthetic "arcane" in the latest Avengers 3. Arcane can only literally understand the task, so "save the Earth" to understand "kill all human beings." This is like a typical predictive algorithm that literally understands the task and ignores the other possibilities or the practical significance of the task.

So, in January 2016, Harvard Business School professor Michael Luca, professor of economics Sendhil Mullainathan, and Cornell University professor Jon Kleinberg, published an article titled "Algorithm and Butler" in the Harvard Commercial Review. Call upon the global scientific and business community to pay attention to the management of algorithms in machine learning algorithms and AI times. Because, if one day, the algorithm can determine "Caesar" or the fate of the Earth, then who to manage the algorithm?

Dinesh Nirmal, vice president of global research and development at IBM's Big data and analytics Division, recently held the 2016 International Summit on machine learning and industry applications in Beijing, where he described how IBM, as a leading technology firm in global Big data analytics, machine learning and AI, will face a complex world of algorithms, This makes machine learning a self-learning, self-tuning, self-optimizing machine steward-a spark-based machine learning cloud service.

Apache Spark is a distributed computing framework and is an open source big Data system optimized for low latency tasks and memory data storage. With its parallel computing performance and the combination of speed, scalability, memory processing, and fault tolerance, plus a rich API that simplifies programming dramatically, spark becomes the mainstream computing platform for machine learning algorithms. IBM announced its participation in the spark open source community in June 2015 and pledged to use spark as the core of its analytics and business platform.

Since June 2016, IBM has spent 5 months developing a spark-based machine learning cloud service that will provide versions of public cloud, on-premises and hybrid cloud deployments, which can also be deployed on the IBM mainframe Z series. Dinesh stressed that the cloud service, in addition to the acquisition of data, extraction features, training models, deployment models, make predictions and other classical machine learning process optimization, but also added continuous feedback, automatic modeling, retraining model, such as automated management.

In automated modeling, IBM's machine Learning Cloud service automatically recommends optimal algorithms based on data models and evaluates the performance and performance of models based on data eigenvalues, and deploys models in real-time, production, and offline batch environments when the model is well trained. When data changes, the cloud service also monitors the performance of the model in real time and then automatically retrain the model. The entire process does not need to take the model offline training and then re-online, greatly facilitates the real-time production environment of commercial applications.

Dinesh believes that open source is a big trend in the machine learning world. To this end, IBM opened its own heavyweight machine learning framework, SYSTEMML, and set up a spark technology center in San Francisco, and has invested more than 3,500 IBM Research and development staff around the world in spark-related projects. In June 2016, IBM launched the Data Science Experience cloud service in conjunction with its open source software and open source Research Analytics interactive environment based on Apache Spark's H2O, RStudio, Jupyter notebooks. To improve the speed of machine learning and data analysis for data scientists.

In order to further strengthen its own data analysis products and technology ecosystem, IBM since 2015 for Apache Toree, Eclairjs, Apache quarks, Apache Mesos, Apache Tachyon (now renamed as Alluxio) has made a lot of contributions to open source projects, and also for Apache Spark's sub-projects such as Sparksql, Sparkr, Mllib and Pyspark. Today, Spark has been combined with more than 45 core products, including IBM's Watson, business, analytics, systems, and cloud.

IBM has invested more than $300 million in spark and sees spark as the operating system for data analysis. The launch of the spark-based machine learning cloud service is the latest in IBM's effort to provide a secure, high-reliability, unified management platform for machine learning algorithms. Based on this, IBM further uses Watson for machine learning, allowing AI to help machine learning algorithms "intelligently" understand people's intentions, which is just the launch of the Watson data platform.

Dinesh says IBM is integrating all machine learning, AI, data analytics, data management, and more into a unified spark-based platform that includes open-source algorithms and IBM's own algorithms, optimized and equipped with enterprise-class solutions, The result is a hybrid cloud approach to create a freely selectable data and algorithmic management platform for the enterprise.

2017, we will usher in a big era of mobile Internet development, data and algorithms will more easily "rule" the world. From online music, online games, online advertising to all kinds of life services, social communication and content consumption, machine learning algorithms have made a lot of choices for people unconsciously. Therefore, while cheering the machine to liberate mankind, it is also necessary to guard against the "prejudice" of the algorithm, which requires self-learning and self-correcting machines.

From machine learning to learning machines, this is the path to the commercialization of artificial intelligence. (Wen/Ningchuang, No.: Cloudtechtime)

This article is from the "Cloud Technology Age" blog, please be sure to keep this source http://cloudtechtime.blog.51cto.com/10784015/1877062

Machine learning to learning machines, data analysis algorithms also need a good housekeeper