Deep Learning

(a.k.a. Recent Trends of Machine Learning)

Course Number: AT82.10
Credits: 3
Links:  Github
Similar Course (recommended):  Stanford CS231n, Berkeley CS182

Course Description
This advanced course explores cutting-edge deep learning architectures and techniques. Students will gain both theoretical understanding and practical experience with state-of-the-art models including CNNs, Vision Transformers, Object Segmentation, Self-Supervised Learning, GANs, Diffusion Models, Graph Neural Networks, Speech Recognition and Reinforcement Learning approaches. Through hands-on projects, students will implement and experiment with these architectures to solve real-world problems.

Instructor Information

Instructor Chaklam Silpasuwanchai
Office CS101
Email chaklam@ait.asia
Teacher Assistant Pranisaa Charnparttaravanit (st123175@ait.asia)


Prerequisites
There are no prerequisites but is recommended to take these courses prior:

  • AT82.03 Machine Learning
  • AT82.01 Computer Programming for Data Science and Artificial Intelligence

Course Objectives
By the end of this course, students will be able to: 

  1. Design, train, test, and deploy advanced deep learning techniques including convolutional neural networks (CNNs),  vision transformer, object segmentation approaches,  self-supervised learning approaches,  generative adversarial networks, diffusion models.
  2. Develop an end-to-end deep learning project incorporating multiple techniques covered in the course, demonstrating ability to move from research concepts to practical applications

Recommended Textbook(s)

  1. Deep Learning: Foundations and Concepts,  Christopher M. Bishop and Hugh Bishop, 2024 Edition, 978-3031454677
  2. Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville, 2016, 978-0262035613

Grading

Component Percentage
Midterm 30
Final 30
Project 30
Assignments 10

Grading scale is based on normal distribution.

Assignments

  • Reading Assignments:  Submit before the class begins
  • Coding Assignments:  Submit before the next assignment arrives

Projects

  • Phase 1: Literature Review:  Each of the team member must select 5 papers (one can be a review/survey) related to your domain and task. For each team create a table. (No Presentation or Report;  Just submit the table)
  • Phase 2: Midway Report: Prepare a (1) report in ACL format, a (2) presentation submitted in pdf, and (3) GitHub link.
  • Phase 3: Final Report:  Prepare a (1) report in ACL format, a (2) presentation submitted in pdf, and (3) GitHub link.

Course Schedule

Weeks Topic Readings Extra Resources Reading Assignments Coding Assignments Project Tasks
1 CNNs AlexNet, InceptionNet, ResNet CS231n CNN Notes, CNN Explainer, Lots of code, Feature Visualization R1: CNNs A1: CNN  
2 Two-stage object segmentation R-CNN, Fast R-CNN, Faster R-CNN Torchvision Detection Models, Object Detection Tutorial, RCNN Family Explained, MMDetection, Detectron, Very nice blog by Lilian R2: Two-stage object segmentation    
3 Single-stage object segmentation YOLO v1, YOLO v3 YOLOv5 Implementation, YOLO official guide, Roboflow YOLO guide R3: Single-stage object segmentation A2: Object Detection  
4 Instance segmentation Mask R-CNN, U-Net Mask R-CNN Implementation, PyTorch U-Net Implementation R4: Instance segmentation    
5 Transformers Vision Transformer, Swin Transformer, DeIT ViT PyTorch Implementation, HuggingFace ViT Guide, Vision Transformer Explained, Official Swin, DeiT R5: Transformers A3: Transformers  
6 Self-Supervised Learning (Part 1: Contrastive Learning) SimCLR, MoCo, BYOL, BarlowTwins, SwaV SimCLR, MoCo, Facebook SSL Library, BYOL, BarlowTwins, SwAV R6: Contrastive learning    
7 Midterm Exam          
8 Self-Supervised Learning (Part 2: Masking and Knowledge Distillation) Masked Autoencoders, SimMIM, DINO MAE, DINO, DINO Blog Post R7: Masking and Knowledge Distillation A4: MAE Phase 1: Project Proposal / Literature Review (No Presentation and No Report)
9 Semi-Supervised Learning MeanTeacher, Noisy Student, FixMatch, FlexMatch FixMatch, Mean Teacher Code, A Very Nice Overview, PyTorch library for SSL, Another Nice Blog by Lilian R8: Semi-Supervised Learning    
10 Generative Models (Part 1: GANs) GAN, ImprovedGAN, DCGAN, Pix2Pix, ConditionalGAN, CycleGAN CycleGAN & Pix2Pix, BigGAN, Gan Hacks, StyleGan, StudioGAN, GAN Explainer R9: GANs A5: GAN  
11 Generative Models (Part 2: Diffusion) Original Diffusion, Improved Diffusion, Beats GAN, Latent Diffusion Vanilla Diffusion, Improved Diffusion, Stable Diffusion, HuggingFace Guide, Super Nice Overview by Lilian, Latent Diffusion R10: Diffusion   Phase 2: Midway Report / Experiment Protocol (Presentation & Report)
12 Graph Neural Network GCN, GAT Nice Introduction, PyTorch Geometric Docs, DGL Framework Guide, Many Code R11: GNN    
13 Reinforcement Learning Atari paper, continuous control, double Q-learning, PPO OpenAI Spinning Up, Gymnasium Documentation, Stable Baselines3, Stanford course,  Berkeley course R12: RL    
14 No Class          
15 Final Exam          
16 Final Project         Phase 3: Final Report (Presentation & Report)
17 Appendix:  Multimodal Learning CLIP, ViLT, PixelBERT, Flamingo, GPT4 LAIO-400M,  MMMU, Tutorial,  CMU Multimodal Course      

Note:  If you have any good resources,  please email me chaklam@ait.asia!   Many of these resources come from the students.

Course Policies

Attendance
Physical attendance in class is a crucial component of your learning in this course. Students are required to attend a minimum of 70% of all class sessions in person. Failure to meet this attendance requirement will result in an automatic F grade for the course, regardless of performance in other assessments.

  • Attendance will be taken at the beginning of each class
  • Medical emergencies and university-approved activities must be documented
  • Contact the TA before class when possible for any anticipated absence

Late Work 

  • All assignments must be submitted through the course submission system by the specified due date and time
  • The submission system will automatically close at the deadline
  • No extensions will be granted except for documented emergencies
  • Late work will not be accepted for credit under any circumstances
  • While the submission portal will remain accessible after deadlines, you may continue to submit work for learning purposes and feedback.  No points will be awarded for any submissions after the deadline

Best Practices

  • Start assignments early to avoid technical issues
  • Submit well before the deadline to avoid last-minute complications
  • Check your submission confirmation to ensure successful upload
  • Set up calendar reminders for all due dates listed in the course schedule

Academic Integrity

All submitted work must be your own original work. This includes code, analysis, and documentation.

Violations leading to automatic F grade:

  • Copying or sharing code/solutions
  • Submitting others' work

Acceptable:

  • Discussing concepts with classmates
  • Seeking help from instructors/TAs

Maintain academic integrity - it's fundamental to your learning and professional development.

Note: This syllabus is subject to change. Any modifications will be announced in class and posted online.