Windows Machine Learning overview

• March 7, 2018
• 4 minutes to read
• Contributors
  • 
+

What is machine learning?

Machine learning (ML) allows computers to use existing data to predict expected outcomes and behaviors. By processing previously collected data, ML algorithms build models that can predict the correct output when presented with a new input. For example, a model can be trained to evaluate email messages (input) as spam or not spam (output).+

The model-building phase is called "training." Once trained with existing data, the model can perform predictions with new, previously unseen data, which is called "inferencing," "evaluation," or "scoring." For a list of key ML terms and concepts, see What is Machine Learning?.+

Trained ML models often produce better results than programs written to follow a strict set of instructions, especially for complex tasks with many possible combinations of inputs and outputs. For example, recommendation algorithms provide personalized recommendations for millions of users on e-commerce and media streaming sites, which would be nearly impossible without ML. Computer vision is another field that leverages ML, allowing computers to classify and identify images after training on previously labelled images.+

The possibilities and applications of ML are endless; for more information about research and solutions, visit Artifical Intelligence at Microsoft and Microsoft Machine Learning Technologies. If you'd like to build Machine Learning and AI models, you can also check out Azure Machine Learning Services.+

What is Windows ML?

Windows ML is a platform for local evaluation of trained machine learning models on Windows 10 devices, allowing developers to use pre-trained models within their applications. +

Here are some highlights of Windows ML:+

Hardware acceleration

On DirectX12 capable devices, Windows ML accelerates the evaluation of Deep Learning models using the GPU. CPU optimizations additionally enable high-performance evaluation of both classical ML and Deep Learning algorithms.+

Local evaluation

Windows ML evaluates on local hardware, removing concerns of connectivity, bandwith, and data privacy. Local evaluation also enables low latency and high performance for quick evaluation results.+

Image processing

For computer vision scenarios, Windows ML simplifies and optimizes the use of image, video, and camera data by handling frame pre-processing and providing camera pipeline setup for model input.+

How to develop with Windows ML

System requirements

To build applications that use Windows ML, you'll need the Windows SDK - Build 17110.+

ONNX models

To use Windows ML, you'll need a pre-trained machine learning model in the Open Neural Network Exchange (ONNX) format. Windows ML supports the v1.0 release of the ONNX format, which allows developers to use models produced by different training frameworks. There are converter tools for many frameworks and libraries, and ONNX models are already natively supported in many training frameworks. For a list of publicly available ONNX models, see ONNX Models on GitHub. +

You can also train your own ONNX models to use with Windows ML. To learn how to train a model with Visual Studio Tools for AI, see Train a model. +

Convert existing models to ONNX

If you already have a pre-trained machine learning model from another framework, then you can use WinMLTools to convert it to the ONNX format accepted by Windows ML. +

WinMLTools supports conversion from these formats:+

• Core ML
• Scikit-Learn
• XGBoost
• LibSVM
+

To learn how to install and use WinMLTools, please see Convert a model. +

ONNX operators

Windows ML supports 100+ ONNX operators on the CPU and accelerates computation on DirectX12 comptaible GPUs. For a full list of operator signatures, see the ONNX operators schemas documentation for the ai.onnx (default) and ai.onnx.ml namespaces.+

Windows ML supports all of the operators defined in the ONNX v1.0 documentation with the following differences:+

• Operators marked "experimental" supported by Windows ML:
  • Affine
  • Crop
  • FC
  • Identity
  • ImageScaler
  • MeanVarianceNormalization
  • ParametricSoftplus
  • ScaledTanh
  • ThresholdedRelu
  • Upsample
• MatMul - greater than 2D matrix multiplication is not currently supported, supported on CPU only
• Cast - supported on CPU only
• The following operators are not supported at this time:
  • RandomUniform
  • RandomUniformLike
  • RandomNormal
  • RandomNormalLike
+

Automatic interface code generation

With an ONNX model file, Windows ML's code generator can generate wrapper classes that call the Windows ML API for you, providing an interface to interact with the model in your app. The generated classes represent the model, inputs, and outputs, allowing you to easily load, bind, and evaluate the model in your project. +

The code generator is available as a command line tool mlgen.exe as part of the Windows SDK, and it currently supports both C# and C++/CX. The tool is located in (SDK_root)\bin\<version>\x64 or (SDK_root)\bin\<version>\x86, where SDK_root is the SDK installation directory. To run the tool, use the command below.+

mlgen -i INPUT-FILE -l LANGUAGE -n NAMESPACE [-o OUTPUT-FILE]

Input parameters definition:+

• INPUT-FILE: the ONNX model file
• LANGUAGE: CPPCX or CS
• NAMESPACE: the namespace of the generated code
• OUTPUT-FILE: file path where the generated code will be written to. If OUTPUT-FILE is not specified, the generated code is written to the standard output
+

For UWP developers, Windows ML's automatic code generator will be natively integrated with Visual Studio (version 15.7 - Preview 1). Simply add your ONNX file as an exiting item to your project, and VS will generate Windows ML wrapper classes in a new interface file.+

Next steps

Try creating your first Windows ML app with a step-by-step tutorial in Get Started.+