Windows Developer Day, every Day

Windows Developer Day, every Day

Windows AI Platform is the most promising option for Windows Developer Day. For developers who watch the live video, the most comments are Windows AI Platform.

The following is a detailed analysis of the presentation process, documentation, and Git Sample provided by Microsoft.

Basic Concepts

Basic Cognition

As we all know, at present, the main implementation of AI (Artificial Intelligence) is Machine Learning, while Windows AI Platform corresponds to Windows Machine Learning.

The official Microsoft statement is as follows:

Windows Machine Learning (ML) evaluates trained machine learning models locally on Windows 10 devices, allowing developers to use pre-trained models within their applications. The platform provides hardware-accelerated performance by leveraging the device's CPU or GPU to compute evaluations for both classical Machine Learning algorithms and Deep Learning.

Based on this description, we can briefly summarize the features of Windows ML:

• Hardware acceleration is on hardware devices supporting DirectX12. Windows ML can use GPU to evaluate the model for acceleration.
• Local evaluation Windows ML can use local hardware for model evaluation, reducing the server traffic cost and server pressure caused by model uploading to the cloud. You can get results more quickly and conveniently.
• In the machine vision scenario, Windows ML simplifies and optimizes the processing of images, video files, and video streams, and processes the Input Source and the camera pipeline.

 

Model format

The Windows ML Model format is ONNX and Open Neural Network Exchange. It is the format standard for Machine Learning Model files developed by Microsoft, Facebook, Amazon, and other companies. Currently, many mainstream model training frameworks support native ONNX, or other formats can be converted to the ONNX format. Here is the Git homepage of ONNX. You can learn more:GitHubOpen Neural Network Exchange

In addition, you can use WinMLTools to convert model files of other formats to the ONNX format. Here is the WinMLTools address:Python WinMLTools 0.1.0.5072.The formats that can be converted include Core ML/Scikit-Learn/XGBoost/LibSVM.

In addition, ONNX supports more than 100 types of operators. Different operators are supported for CPU or GPU. Here is the list of operators:Https://github.com/onnx/onnx/blob/rel-1.0/docs/Operators.md

 

Technical Architecture

The following figure shows the architecture:

• The bottom layer is the Direct layer DirectML API/Direct3D/CPU/GPU. The DirectX version supports DX12.
• The above layer is the inference engine, which includes Win32 and WinRT. It is mainly responsible for model and device resource management, loading and editing core operators, and executing data flow diagrams.
• The top layer is the application layer, which also includes the Win32 and WinRT sections. Fortunately, it is available in all versions of Windows in 2018.

 

Development Process

Overview

At present, Windows AI Platform is still in the preview version. Therefore, the preview version of Windows OS and WIndows 10 SDK is required. The following is the description:

Windows Insider Preview Downloads

The Visual Studio version must be Community, Professional, or Enterprise. The Community version is the easiest to obtain. We recommend that you use this version for experimental purposes.

Let's take a look at the presentation of a press conference:

 

We can see that the use of Windows ML is as follows:

• First, train the model on the cloud or local server to generate the ONNX model file.
• Add ONNX to a local development environment, such as Visual Studio.
• Use and evaluate the performance and learning results of the ONNX model through the Windows 10 SDK in a local program
• Publish local programs integrated with ONNX to all platforms and devices of the Windows sequence.

 

Sample Analysis

Example Git address for Windows ML:GitHub Windows-Machine-Learning

The above link also provides Windows Insider Preview 17110 OS, Windows 10 SDK 17110, and Visual Studio 2017. As instructed, I have downloaded and installed the development environment.

Let's take a look at the first example: MNIST_Demo is a UWP program for Handwritten Digit Recognition. As we all know, handwritten digit recognition is the foundation and entry point of Machine Learning, just like Hello World in every programming language, let's take a look at the use of Windows ML for the ONNX model and Windows 10 SDK.

First, let's take a look at the engineering structure of the example in Visual Studio:

Here we can see:

• Universal WindowsThat is, the reference version of the Windows 10 SDK is 10.0.17110.0, that is, the minimum version of the SDK supported by Windows ML
• Mnist. onnxThe format supported by the Windows ML model described earlier is directly added to the Assets folder in the solution, and the Build Action is "Content"

InMnist. csFile

using System;using System.Collections.Generic;using System.Threading.Tasks;using Windows.Media;using Windows.Storage;using Windows.AI.MachineLearning.Preview;...
...

Public sealed class MNISTModel
{
Private LearningModelPreview learningModel;

...

We can see that the Windows ML namespace is:Windows. AI. MachineLearning. Preview

It can be seen that all namespaces contain the Preview words because they are still in the Preview version,Windows. AI. MachineLearningThis namespace should be OK.

Let's take a look at the Windows ML winmd structure:

The model name is LearningModelPreview. Let's take a look at the class definition:

# Region Assembly Windows. AI. machineLearning. preview. partition, Version = 1.0.0.0, Culture = neutral, PublicKeyToken = null, ContentType = WindowsRuntime // C: \ Program Files (x86) \ Windows Kits \ 10 \ References \ 10.0.17110.0 \ Windows. AI. machineLearning. preview. machineLearningPreviewContract \ 1.0.0.0 \ Windows. AI. machineLearning. preview. machineLearningPreviewContract. winmd # endregionusing System. collections. generic; using Windows. foundation; using Windows. foundation. metadata; using Windows. storage; using Windows. storage. streams; namespace Windows. AI. machineLearning. preview {[ContractVersion (typeof (MachineLearningPreviewContract), 65536)] [Static (typeof (ILearningModelPreviewStatics), 65536, "Windows. AI. machineLearning. preview. optional ")] public sealed class LearningModelPreview: ILearningModelPreview {[RemoteAsync] public IAsyncOperation <strong> EvaluateAsync (LearningModelBindingPreview binding, string correlationId ); [RemoteAsync] public IAsyncOperation <Category> category (IDictionary <string, object> features, string correlationId); [RemoteAsync] public static IAsyncOperation <LearningModelPreview> category (IStorageFile modelFile ); [RemoteAsync] public static IAsyncOperation <LearningModelPreview> LoadModelFromStreamAsync (IRandomAccessStreamReference modelStream); public writable InferencingOptions {get; set ;}public writable Description {get ;}}}

This class contains the inference options, model loading methods, and model evaluation methods.

Next, let's take a look at the actual loading method of the model in the interface code:

private async void LoadModel(){    //Load a machine learning model    StorageFile modelFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri($"ms-appx:///Assets/MNIST.onnx"));    ModelGen = await MNISTModel.CreateMNISTModel(modelFile);}

public static async Task<MNISTModel> CreateMNISTModel(StorageFile file){    LearningModelPreview learningModel = await LearningModelPreview.LoadModelFromStorageFileAsync(file);    MNISTModel model = new MNISTModel();    model.learningModel = learningModel;    return model;}

The mnist. onnx model file is loaded into StorageFile as a project file. The CreateMNISTModel method of the mnist class is used. Specifically, the LoadModelFromStorageFileAsync method of the LearningModelPreview class completes model loading.

The entire Sample task is to use InkCanvas to obtain users' handwritten input, input it to Windows ML for detection, and output the detection results. Let's take a look at the running results:

In addition, other samples are displayed during the presentation of the press conference. We will not detail them here. You can see the results of the presentation:

 

This is a Sample for converting the artistic style of images, similar to the Prisma implementation method. In particular, the second one is the real-time conversion of images collected from the camera. The Frame Rate of the camera image stream should be more than 30 frames, and the real-time conversion can still be completed when the model is run locally. This also gives us confidence in the conversion of the video style from a local program.

 

Here, the introduction of Windows AI Platform and Windows ML has been completed, because the current official version is still available, and there is not enough public content. We will continue to follow up on the study in the future, thank you for your discussion!