Windows Developer day-windows AI Platform

This time Windows Developer Day, the most worth looking forward to is the Windows AI Platform, it can be said that awaited started out. For developers who watch live streaming, the most popular messages are Windows AI Platform.

The following is a detailed analysis of Microsoft's presentation process, documentation and Git Sample.

Basic concepts

Basic cognition

As we all know, the main implementation of AI (Artificial Intelligence) is machine learning (machines learning), while Windows AI Platform corresponds to Windows engine learning.

Microsoft's official description of it is as follows:

Windows machine Learning (ML) evaluates trained machine learning models locally on Windows Ten devices, allowing developers To use pre-trained models within their applications. The platform provides hardware-accelerated performance by leveraging the device's CPU or GPU to compute evaluations for BO Th classical machine learning algorithms and deep learning.

Combining this description, we can briefly summarize several features of Windows ML:

• Hardware acceleration on hardware devices that support DirectX12, Windows ML can use the GPU to accelerate the evaluation of the model.
• Local evaluation Windows ML can take advantage of on-premises hardware for model evaluation, reducing service-side traffic costs and service-side pressures that are caused by the model being uploaded to the cloud. Results can be obtained more quickly and easily.
• Image processing in machine vision scenarios, Windows ML simplifies and optimizes the processing of images, video files, and video streams, preprocessing and camera pipeline processing of input sources.

Model format

The model format for Windows ML is Onnx,open neural Network Exchange, a machine learning model file format standard developed by companies such as Microsoft and Facebook, Amazon, and others. In many of the current mainstream model training frameworks, there are native support for ONNX, or you can support other formats to convert to ONNX format. Here is Onnx's Git homepage, which you can learn more about:GitHub Open neural Network Exchange

In addition, you can use Winmltools to convert other format model files to ONNX format, here is Winmltools address:Python winmltools 0.1.0.5072. The format that can be converted has the Core ML/SCIKIT-LEARN/XGBOOST/LIBSVM.

In addition ONNX supports more than 100 operators, there are different operator support for CPU or GPU, here is the list of operators:https://github.com/onnx/onnx/blob/rel-1.0/docs/ Operators.md

Technology architecture

From this architecture diagram:

• The underlying is the directml Api/direct3d/cpu/gpu,directx version support for the Direct layer is DX12
• The above layer is the inference engine, including the Win32 and WinRT parts, mainly responsible for model and device resource management, responsible for loading and editing the core operators, executing the flow diagram
• The top tier is the application layer, which also includes the Win32 and WinRT sections, and, happily, it's available on all 2018 Windows editions

Development process

Overview

Currently, Windows AI Platform is still in preview, so you need a preview version of Windows OS and the Windows SDK, below:

Windows Insider Preview Downloads

The version of Visual Studio requires that the Community, Professional, or enterprise,community versions are the easiest to use, and this version is recommended for experimental requirements.

Let's take a look at a presentation:

From there, you can see how the entire Windows ML process is used:

• First train the model in the cloud or on the local server to generate the ONNX model file
• Add ONNX to your local development environment, such as in Visual Studio
• Use and evaluate the performance and learning results of the ONNX model through the Windows SDK in a local program
• Publish native programs integrated with ONNX to all platform devices in Windows sequence

Sample Analysis

Example of Windows ML Git address:GitHub windows-machine-learning

The links above also provide Windows Insider Preview 17110 OS, Windows SDK 17110 and Visual Studio 2017, as instructed I downloaded the development environment installed.

Look at the first example: Mnist_demo, is a handwritten digital recognition of the UWP program, we all know that handwritten digital recognition is the basis of machine learning and getting started, just like the Hello world in every programming language, let's take this example to see Windows ML is used for the ONNX model and the Windows SDK.

First look at the engineering structure of the example in Visual Studio:

Here we can see:

• Universal Windows, which is the version of the Windows SDK reference, is: 10.0.17110.0, which is the minimum version Preview SDK supported by Windows ML
• Mnist.onnx, which is the Windows ML model support format described earlier, is added directly to the Assets folder in the solution, and the Build Action is "Content"

And in the mnist.cs file,

using System; using System.Collections.Generic; using System.Threading.Tasks; using Windows.media; using Windows.storage; using Windows.AI.MachineLearning.Preview, .....
...

public sealed class Mnistmodel
{
Private Learningmodelpreview Learningmodel;

...

As we can see, the Windows ML namespace is:Windows.AI.MachineLearning.Preview

As you can see, now that the preview version is still in place, all namespaces contain the word preview, but the Windows.AI.MachineLearning namespace should be OK.

Take a look at the structure of the Windows ML WINMD:

and the name of the model is Learningmodelpreview, look at the definition of the class:

#regionAssembly Windows.AI.MachineLearning.Preview.MachineLearningPreviewContract, version=1.0.0.0, Culture=neutral, Publickeytoken=null, Contenttype=windowsruntime//C:\Program Files (x86) \ Windows Kits\10\references\10.0.17110.0\ Windows.ai.machinelearning.preview.machinelearningpreviewcontract\1.0.0.0\ WINDOWS.AI.MACHINELEARNING.PREVIEW.MACHINELEARNINGPREVIEWCONTRACT.WINMD#endregionusingSystem.Collections.Generic;usingwindows.foundation;usingWindows.Foundation.Metadata;usingWindows.storage;usingWindows.Storage.Streams;namespacewindows.ai.machinelearning.preview{[Contractversion (typeof(Machinelearningpreviewcontract),65536)] [Static (typeof(Ilearningmodelpreviewstatics),65536,"Windows.AI.MachineLearning.Preview.MachineLearningPreviewContract")]     Public Sealed classLearningmodelpreview:ilearningmodelpreview {[Remoteasync] PublicIasyncoperation<learningmodelevaluationresultpreview> Evaluateasync (Learningmodelbindingpreview binding,stringCorrelationid); [Remoteasync] PublicIasyncoperation<learningmodelevaluationresultpreview> Evaluatefeaturesasync (IDictionary<string,Object> Features,stringCorrelationid); [Remoteasync] Public StaticIasyncoperation<learningmodelpreview>Loadmodelfromstoragefileasync (Istoragefile modelfile); [Remoteasync] Public StaticIasyncoperation<learningmodelpreview>Loadmodelfromstreamasync (irandomaccessstreamreference modelstream);  PublicInferencingoptionspreview Inferencingoptions {Get;Set; }  PublicLearningmodeldescriptionpreview Description {Get; } }}

This class contains inference options, two modes of loading for the model, and a model evaluation method.

Let's look at how the model actually loads in the interface code:

 private  async  void   Loadmodel () { //  Load a machine learning model  StorageFile modelfile = await  storagefile.getf Ilefromapplicationuriasync (new  Uri ($ " Span style= "COLOR: #800000" >ms-appx:///assets/mnist.onnx   "  = await   Mnistmodel.createmnistmodel (modelfile);}  

 Public Static Async Task<mnistmodel> Createmnistmodel (StorageFile file) {    await  Learningmodelpreview.loadmodelfromstoragefileasync (file);     New Mnistmodel ();     = Learningmodel;     return model;}

The MNIST.ONNX model file is loaded into StorageFile as a project file, using the Createmnistmodel method of the Mnist class, specifically the Learningmodelpreview class The Loadmodelfromstoragefileasync method completes the model load.

The entire Sample is done using InkCanvas to get the user's handwriting input, input to Windows ML for detection, and output the test results. To see the results of the operation:

In addition, the presentation of the exhibition also shows the other sample, here is not detailed, we can see the effect of its completion:

This is a picture art style conversion of Sample, similar to the implementation of PRISMA. Especially the second, is the real-time conversion from the camera capture image, the camera image stream frame rate should be more than 30 frames, still can run the model in the local situation, complete real-time conversion. This also gives us confidence in the local program to complete the video-style conversion.

Here, the introduction of Windows AI Platform and Windows ML is complete, because the official offer is still preview version, and the public content is not enough, follow-up we will continue to follow up the study, welcome to discuss together, thank you!

Windows Developer day-windows AI Platform