Compiling and Deploying Machine Learning Models to Bare-Metal Devices with TinyIREE

One of the challenges beyond the training of machine learning (ML) models is the deployment of these models, especially when microcontrollers are targeted. Traditionally, public ML inference frameworks target microcontrollers by kernel-based, op-by-op runtimes. More recently, ML compilers have been used to address this problem. One of them is Intermediate Representation Execution Environment (IREE). Whereas IREE allows to scale up to large deployments, one of its explicit goals is to scale down ML programs to the smallest footprints for mobile and edge devices. TinyIREE, a subset of IREE options, is used to generate a compact workload and comes with a runtime library optimized for embedded systems without an operating system, so-called bare-metal systems. In this work, we present the significant changes that have occurred since the initial introduction of TinyIREE in December 2021 especially with regards to the workload scheduling of the compiler and resulting runtime library simplifications, targeted for bare-metal deployment.