Gian Marco Iodice - TinyML Cookbook

Combine artificial intelligence and ultra-low-power embedded devices to make the world smarter

Gian Marco Iodice

BIRMINGHAMMUMBAI

Copyright 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Publishing Product Manager: Devika Battike

Senior Editors: Roshan Kumar, Nathanya Dias

Content Development Editor: Tazeen Shaikh

Technical Editor: Rahul Limbachiya

Copy Editor: Safis Editing

Project Coordinator: Aparna Ravikumar Nair

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Shankar Kalbhor

Marketing Coordinator: Abeer Dawe

First published: April 2022

Production reference: 2290322

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80181-497-3

www.packt.com

While the publisher and the author have taken every precaution to ensure that information contained in this book is accurate, neither the publisher nor the author assumes any responsibility for errors, omissions, or damages to persons or proprieties from the use of the information contained herein.

TinyML is a registered trademark of the TinyML foundation, and is used with permission.

It is because of the source of support from my wife, Eleonora, during the long nights of writing that I managed to complete this unique journey. I dedicate this book to her, who believed in this project from the very beginning.

Without a doubt, the tech industry continues to have an ever-increasing impact on our daily lives. The changes are as rapid as they are constant and are happening all around us in our phones, cars, smart speakers, and the micro gadgets we use to improve efficiency, wellbeing, and connectivity. Machine learning is one of the most transformative technologies of our age. Businesses, academics, and engineering communities continue to understand, evolve, and explore the capabilities of this incredible technology, and are unlocking the greater potential to enable new use cases across many industries.

I am a product manager for machine learning at Arm. In this role, I am at the center of the ML revolution that is happening in smartphones, the automotive industry, gaming, AR, VR, and other devices. It is clear to me that there will be ML functionality in every single electronics device in the near future from the world's largest supercomputers, down to the smallest, low-powered microcontrollers. Working in ML has introduced me to some of the most brilliant and brightest minds in tech those who challenge the orthodoxies that exist in traditional industries, ask the tough questions, and unlock new value through the use of ML.

When I first met Gian Marco, I could barely spell "ML," yet at that time he was already a veteran in the space. I was astonished by the breadth and depth of his knowledge and his ability to solve difficult problems. Together with the team at Arm, he has worked to make Arm Compute Library (ACL) the most performant library available for ML on Arm. The success of ACL is unrivaled. It's deployed on billions of devices worldwide from servers to flagship smartphones, to smart ovens.

When Gian Marco told me he was writing a book on ML, my immediate reaction was "Which part?" The ML ecosystem is so diverse, with many different technologies, platforms, and frameworks to consider. At the same time, I knew that he was the right person for the job due to his extensive knowledge of all aspects of ML. Additionally, Gian Marco has an amazing way of explaining things in a straightforward and logical manner.

Gian Marco's book demystifies the world of TinyML by guiding us through a series of practical, real-world examples. Each example is outlined like a recipe, with a clear and consistent format throughout, providing an easy-to-follow, step-by-step guide. Beginning with the first principles, he explains the basics of the electronics or software techniques that will be used in the recipe. The book then introduces the platforms and technologies used, followed by the ML where neural network models are developed, trained, and deployed on the target device. This really is a "soup to nuts" guide. Each recipe is a little more challenging than the last, and there is a nice mix of established and nascent technologies. You don't just learn the "how," you also get an understanding of the "why." When it comes to edge devices, this book really does provide a panoramic view of the ML space.

Machine learning continues to disrupt all aspects of technology and getting started is a must for software developers. This book enables quick onboarding through the use of readily available and inexpensive technologies. Whether you are new to ML or have some experience, each recipe provides a steady ramp of knowledge and leaves enough scope for further self-development and experimentation. Whether you use this book as a guide or a reference, you will develop a strong foundation in ML for future development. It will empower your team to get new insights and to achieve new efficiencies, performance improvements, and even new functionality for your products.

Ronan Naughton

Senior Product Manager for Machine Learning at Arm

Gian Marco Iodice is team and tech lead in the Machine Learning Group at Arm, who co-created the Arm Compute Library in 2017. The Arm Compute Library is currently the most performant library for ML on Arm, and it's deployed on billions of devices worldwide from servers to smartphones.

Gian Marco holds an MSc degree, with honors, in electronic engineering from the University of Pisa (Italy) and has several years of experience developing ML and computer vision algorithms on edge devices. Now, he's leading the ML performance optimization on Arm Mali GPUs.

In 2020, Gian Marco cofounded the TinyML UK meetup group to encourage knowledge-sharing, educate, and inspire the next generation of ML developers on tiny and power-efficient devices.

Alessandro Grande is a physicist, an engineer, a communicator, and a technology leader with a visceral passion for connecting and empowering humans to build more efficient and sustainable technology. Alessandro is the director of product at Edge Impulse and cofounded the TinyML Meetups in the UK and in Italy. Prior to Edge Impulse, Alessandro worked at Arm as a developer evangelist and ecosystem manager with a focus on building the foundations for a smarter and more efficient IoT. He holds a master's degree in nuclear and electronic physics from the University of Rome, La Sapienza.