Course Information

This is a project-based class where students will learn how to develop machine learning models for execution in resource constrained environments such as embedded systems. The primary target is embedded devices such as Arduino, Raspberry PI, Jetson, or Edge TPUs.

The class is broken into lectures/readings, labs/assignments, and a final project. Throughout this class, students will learn techniques such as model quantization, knowledge distillation and hybrid (embedded/cloud inferencing), which are instrumental for building efficient machine learning models that can run on power or resource-constrained devices. This class differs from other machine learning classes offered at Stanford due to the focus of applying these models for applications that require running on embedded hardware or other resource-constrained environments.


Teaching Staff

Zain Asgar

Office hours: TBA

Pete Warden

Office hours: TBA

Sachin Katti

Office hours: TBA

Ankita Nayak

Office hours: TBA


Course Schedule

Week 1

Intros, objectives, etc.
Course logistics. Introduction to edge ML. Introduction to ML frameworks.

Labs

  • Lab 0 - Set Up Your Raspberry Pi: Board setup, deploy a model on the Pi. First embedded project!

Week 2

Introduction to CNN and tranformer models
Introduction to CNN and transformer models and their mathematical implementation and general architecture.

Labs

  • Lab 1 - Running a Large Language Model: Run an LLM locally on your Raspberry Pi 5.

Week 3

ML model training and fine-tuning techniques
ML model training, fine-tuning with transfer learning, multi-task learning.

Labs

  • Lab 2 - Classify Images: Run an image classification model on your Pi, and fine-tune it to recognize different kinds of objects.
  • Lab 3 - Locate People and Objects: Run pretrained models to locate people and objects in images, optimize their latency, and train on custom classes.

Week 4

ML model quantization
Introduction to various data formats for ML. ML model quantization motivation, techniques. Quantization-aware training.

Labs

  • Lab 4 - Understand Speech: Recognize speech from an audio file or microphone using OpenAI’s Whisper model.
  • Lab 5 - Translate Language: Translate text from one language into another.

Week 5

PEFT techniques and knowledge distillation
Parameter-Efficient Fine-Tuning (PEFT) techniques including LoRA, adapter layers. Knowledge distillation overview and techniques.

Labs

Lab 6 - Fine-tune and Deploy a Large Language Model: Efficiently fine-tune a large language model, then prepare it for deployment on a Raspberry Pi. Lab 7 - Understand Your Data: Find and understand the datasets needed to train your own models.

Week 6

Hardware for DNN
Discussion on accelerator design and its impact on model design and performance.

Labs

Project debugging.

Week 7

MLOps
ML Ops pipeline for automating and simplifying ML workflows. ML Observability.

Labs

Class project show and tell.

Week 8

Guest lecture

Labs

Project debugging.

Week 9

Memorial day (holiday, no class)

Project

Final project presentation.

Lab Assignments

Lab Assignment 0: Set up your Raspberry Pi

This tutorial shows how to set up your laptop and Raspberry Pi 5 board to run machine learning labs in EE292D.

Lab Assignment 1: Run a Large Language Model

This lab will show you how to run an LLM locally on your Raspberry Pi 5.

Lab Assignment 2: Classify Images

This lab will show you how to run an image classification model on your Pi, and fine-tune it to recognize different kinds of objects.

Lab Assignment 3: Locate People and Objects

This lab will show you how to run pretrained models to spot people, analyze their poses, and locate arbitrary objects in images.

Lab Assignment 4: Understand Speech

This lab will show you how to recognize speech from an audio file or microphone using OpenAI's Whisper model.

Lab Assignment 5: Translate Language

This lab will show how to translate text from one language into another.

Lab Assignment 6: Fine Tune Large Language Model

This lab will show you how to fine tune an LLM for a custom task and run it on your Pi.

Lab Assignment 7: Understand Your Data

This lab will show you how to find and understand the datasets needed to train your own models.


Grading

Labs - 40%

  • We will work through the labs in the second class each week

Reading Assignments/Class Participation - 10%

  • Reading Reflections
  • Lead group discussions on each paper
  • Graded on participation and write-ups

Final Project - 50%

  • Proposal - 10% (2% for initial, 8% for final)
  • Write up & Presentation - 25%
  • Results - 15% (relative to proposed)

Important Dates

DateWhat's Due
Week 2 (April 11) Preliminary final project proposal due
Week 3 (April 18) Final project proposal due
Week 7 (May 12) In class show and tell about project progress
Week 9 (May 26) Final project presentations due (last class)
Week 9 (May 30) Final project report due