Instructor(s)
Prof. Yuanyuan Shi, yyshi@ucsd.edu
Co-Instructors (for 2025 Spring Offering):
Luke Bhan, lbhan@ucsd.edu
Yuexin Bian, yubian@ucsd.edu
Teaching Assistants
Course Time and Location
Part 1: Machine Learning / Deep Learning Fundamentals
Week 1: Course Logistics; supervised learning setup; linear regression [Lecture 1] [Lecture 2]
Week 2: Linear models for classification; Feature selection: ridge regression, Lasso; Bias and Variance Tradeoffs [Lecture 3] [Lecture 4]
Week 3: Neural network basics; computational graph and backpropagation; Deep Learning optimization [Lecture 5] [Lecture 6]
Week 4: Deep Learning Regularization; Temporal data modeling: RNN / LSTM/GRU [Lecture 7] [Lecture 8]
Week 5: Attention & Transformer I; Convolutional Neural Networks [Lecture 9] [Lecture 10]
Week 6: Transformer II [Lecture 11]
Part 2: Specialized Topics: Machine Learning for Physical Applications
Week 6: Neural ODEs [Lecture 12]
Week 7: Neural operators [Lecture 13] [Lecture 14]
Week 8: Physics-informed Neural Networks; OptNet and End-to-End Deep Learning [Lecture 15] [Lecture 16]
Week 9: Gaussian Processes; Uncertainty Quantification of Deep Learning [Lecture 17] [Lecture 18]
Week 10: Course Summary and Final project presentation (poster session) [Course Summary]
Recommended Reading:
Week 1-2: An Introduction to Statistical Learning (with Applications in Python): Chapters 1-5
Week 3-5: Deep Learning, Part II: Chapters 6-11
Week 6 - 10: Papers and Discussions
Week 6 - Neural ODEs: https://arxiv.org/abs/1806.07366, https://arxiv.org/abs/2202.02435
Week 7 - Neural Operators: DeepONet: https://arxiv.org/abs/1910.03193
FNO: https://arxiv.org/abs/2010.08895
Theoretical results in neural operators: https://arxiv.org/abs/2304.13221
Geometry informed FNO: https://proceedings.neurips.cc/paper_files/paper/2023/file/70518ea42831f02afc3a2828993935ad-Paper-Conference.pdf
Week 8 - Physics-informed Neural Networks: https://www.sciencedirect.com/science/article/abs/pii/S0021999118307125
How to actually train PINNs: https://arxiv.org/abs/2308.08468, https://arxiv.org/abs/2109.01050
Week 9:
Gaussian process
A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles