Toradex has teamed up with Eta Compute to make creating edge-ML models easier and faster for Toradex customers. Eta Compute is a Silicon Valley startup that helps companies accelerate their product development by incorporating edge machine learning (ML) capabilities.

Advancements of ML capabilities in low-power silicon are rapidly expanding the possibilities for great products, enabling previously impossible applications in smart buildings, industries, smart cities, retail, healthcare, and beyond. As an example, the powerful yet affordable Toradex Verdin iMX8M Plus System on Module dramatically accelerates ML operations compared to previous products.

• How to quickly create edge ML models that fully leverage the latest in data science and new ML hardware capabilities, while adhering to the tight constraints of embedded systems?
• How can you find and hire unicorn engineers who are experts at both data science and embedded systems development?

To address these challenges, Eta Compute developed Aptos, a cloud-based edge-ML model development platform that automates the creation of optimal models for embedded systems. By abstracting away data science complexities and hardware intricacies, Aptos empowers developers to concentrate on delivering valuable products that meet market requirements – on schedule.

Eta Compute now has support for Toradex computing systems within the Aptos platform, beginning with the Toradex Verdin iMX8M Plus. This collaboration leverages the Verdin iMX8M Plus's NXP^® i.MX 8M Plus SoC, which features a dedicated Neural Processing Unit (NPU) for accelerated machine learning inference at the edge.

Aptos is a no-code software toolchain that automates ML model development for embedded applications. Users provide a dataset, objectives, constraints, and select the target hardware. Aptos then generates and characterizes highly optimized models along with comprehensive reports about the models’ characteristics. Users then download their selected models for system integration.

This entire process typically takes a few minutes of setup and up to a day or two of runtime and is easily run by an engineer with either ML or embedded experience. It compares favorably to the weeks and months of a traditional ML process where a highly skilled engineer with both machine learning and embedded system knowledge handcrafts ML models to run on the target hardware, and through many iterations eventually arrives at an acceptable result.

The benefits of the Aptos automated model generation approach become even more pronounced when models need to be updated or enhanced. This is a common situation in embedded systems development due to changing end-product requirements, updated datasets becoming available, or changes to the target hardware platforms or their software development environments. With Aptos, models are quickly re-spun using the updated information.

It is also difficult for product development teams to keep up with the rapidly evolving world of data science that is constantly producing new techniques and model architectures. Which of these advanced techniques could work in the target constrained embedded environment, and how much effort is required to get there? Eta Compute continuously builds the latest data science techniques into Aptos, making them easily available to product developers with no additional research or effort.

Aptos itself uses proprietary machine learning techniques to learn about the strengths and weaknesses of the target hardware’s ML capabilities. Eta Compute worked with Toradex to “onboard” the Verdin hardware into Aptos’ Silicon Library, starting by connecting the Toradex development boards to the Aptos Hardware Cloud. Aptos then learned the ML capabilities of the underlying Toradex & NXP hardware and can now apply this knowledge when a user runs a model generation job in Aptos. Aptos also continuously learns what works well and so gets better with each run.

The example below illustrates Aptos' edge ML capabilities for a vision AI classification task, automatically generating optimized models that meet user constraints and run efficiently on the Toradex Verdin iMX8M Plus. Aptos validates these models against the Toradex development board and provides extensive performance reports. It details the models’ characteristics such as accuracy, inference time, and energy, highlighting the pareto frontier to quickly visualize optimal models. It also provides detailed data science insights for those who wish to explore further.

• Hardware target
• System design objectives, including accuracy targets, speed target and energy targets (where relevant)
• Set job execution-related constraints or optional stopping conditions

Here we first set target constraints of maximum inference time (500ms) and energy (5000 mJ), and a job execution maximum run time condition (48 hours).

That’s it! Aptos will now generate models that meet the constraints and produce detailed reports of the results, including plots that help identify the models along the pareto frontier (noted with the dark line and circle below).

The user can then click on the model (or models) they want to use, download them and run them directly on their own hardware. Reminder: Aptos has already run these models on actual hardware so the reported numbers you get about timing, energy, etc., are the actual results running on the Toradex board. But, since we’re all engineers, we’ll want to run it ourselves to check it at least once!

Aptos also provides a tremendous amount of profile information about these models – if you choose to delve deeper – such as runtime details, hyperparameter details, model diagrams, detailed layer timings, etc. This is best shown in a live demo, so please get in touch with us to see more.

See a live demo or try Aptos for free
You can contact Eta Compute to see a demo or get a trial version of Aptos to try generating your own edge-ML models.

Author:
Mark Milligan, Head of Business Development & Marketing at Eta Compute

Leave a comment