Course Information

This is a project-based class where students will learn how to develop machine learning models for execution in resource constrained environments such as embedded systems. The primary target is embedded devices such as Arduino, Raspberry PI, Jetson, or Edge TPUs.

The class is broken into lectures/readings, labs/assignments, and a final project. Throughout this class, students will learn techniques such as model quantization, knowledge distillation and hybrid (embedded/cloud inferencing), which are instrumental for building efficient machine learning models that can run on power or resource-constrained devices. This class differs from other machine learning classes offered at Stanford due to the focus of applying these models for applications that require running on embedded hardware or other resource-constrained environments.


Teaching Staff

Zain Asgar

Office hours: TBA

Pete Warden

Office hours: TBA

Sachin Katti

Office hours: TBA

Ankita Nayak

Office hours: TBA

Yi-Chin Huang

Office hours: TBA


Course Schedule

Week 1

Intros, objectives, etc.
Course logistics. Introduction to edge ML. Introduction to ML frameworks.

Labs
Board setup, deploy a large language model on the Pi. First embedded project!

Week 2

Introduction to CNN and tranformer models
Introduction to CNN and transformer models and their mathematical implementation and general architecture.

Labs
Classify images, locate people and objects.

Week 3

ML model training and fine-tuning techniques
ML model training, fine-tuning with transfer learning, multi-task learning.

Labs
Recognize speech, fine tune an LLM.

Week 4

Finding and evaluating datasets
Finding and evaluating datasets.

Labs
Using tools (like Galileo) to examine a common public dataset.

Week 5

ML model optimization
Approximations for neural networks. Model optimization techniques like quantization, pruning.

Labs
We’ll demonstrate how to convert a pre-trained computer vision model to eight bits.

Week 6

ML model training in practice
Fundamentals about training in practice, problems encountered during training. Tools for training.

Labs
Train ImageNet model on a Colab notebook.

Week 7

Guest lecture

Labs
Project debugging.

Week 8

Hardware for DNN
Discussion on accelerator design and its impact on model design and performance.

Labs
Project debugging.

Week 9

MLOps
ML Ops pipeline for automating and simplifying ML workflows. ML Observability.

Project
Final project presentation.

Lab Assignments

Lab Assignment 1: Set up your Raspberry Pi

This tutorial shows how to set up your laptop and Raspberry Pi 5 board to run machine learning labs in EE292D.

Lab Assignment 2: Run a Large Language Model

This lab will show you how to run an LLM locally on your Raspberry Pi 5.

Lab Assignment 3: Classify Images

This lab will show you how to run an image classification model on your Pi, and fine-tune it to recognize different kinds of objects.

Lab Assignment 4: Locate People and Objects

This lab will show you how to run pretrained models to spot people, analyze their poses, and locate arbitrary objects in images.

Lab Assignment 5: Recognize and Generate Speech

This lab will show how to run an open source speech to text model on your Pi, connect it to your large language model, and speak the answers.

Lab Assignment 6: Fine Tune Large Language Model

This lab will show you how to fine tune an LLM for a custom task and run it on your Pi.


Grading

Labs - 30%

  • ~5 labs that we will work through in the second class each week

Reading Assignments/Class Participation - 10%

  • Reading Reflections
  • Lead group discussions on each paper
  • Graded on participation and write-ups

Final Project - 60%

  • Proposal - 10% (2% for initial, 8% for final)
  • Write up & Presentation - 30%
  • Results - 20% (relative to proposed)

Important Dates

DateWhat's Due
Week 2 (April 12) Preliminary final project proposal due
Week 3 (April 19) Final project proposal due
Week 7 (May 15) In class show and tell about project progress
Week 9 (May 29) Final project presentations due (last class)
Week 9 (May 31) Final project report due