Machine Learning

Learn about and experiment with machine learning algorithms using Python. Applications include image classification, removing noise from images, and linear regression. Students will collect and interpret data, learn machine learning theory, build systems-level thinking skills required to strategize how to break the problem down into various functions, and to implement, test and document those functions. 

Prerequisite: CS 111X
 

This course is an introduction to the foundations behind modern data analysis and machine learning. The first part of the course covers selected topics from probability theory and linear algebra that are key components of modern data analysis. Next, we cover multivariate statistical techniques for dimensionality reduction, regression, and classification. Finally, we survey recent topics in machine learning, in particular, deep neural networks.

Prerequisites: You should be comfortable programming in Python (CS 2110 or equivalent is sufficient)

An introduction to digital signal processing. Topics include discrete-time signals and systems, application of z-transforms, the discrete-time Fourier transform, sampling, digital filter design, the discrete Fourier transform, the fast Fourier transform, quantization effects and nonlinear filters.

Prerequisite: ECE 2700 (Signals and Systems)

Description: This course focuses on an in-depth study of advanced topics and interests in image data analysis. Students will learn practical image techniques and gain mathematical fundamentals in machine learning needed to build their own models for effective problem solving. Topics of image denoising/reconstruction, deformable image registration, numerical analysis, probabilistic modeling, data dimensionality reduction, and convolutional neural networks for image segmentation/classification will be covered. The main focus might change from semester to semester. The graduate students (ECE/CS 6501) will be given additional programming tasks and more advanced theoretical questions.

Prerequisite: Mathematical background in linear algebra, multivariate calculus, probability and statistics, and programming skills are required in this class. APMA 3080 suggested for undergraduate students. 

Description: An introductory course offers a broad overview of the main techniques in machine learning. Students will study the basic concepts of advanced machine learning methods as well as their theoretical background. Topics of learning theory (bias /variance trade-offs; VC theory); supervised learning parametric / nonparametric methods, Bayesian models, support vector machines, neural networks); unsupervised learning (dimensionality reduction, kernel tricks, clustering) and reinforcement learning will be covered. The graduate students (ECE 6502 / CS 6316) will be given additional programming tasks and more advanced theoretical questions. 

Prerequisites: Mathematical background in linear algebra, multivariate calculus, probability and statistics, and programming skills are required in this class.

Course objective: Graphs/networks are often used to represent a plethora of real-world phenomena: social relations among online users, hyperlinks among webpages, biological interactions among genes, brain activities among neurons, to name a few. How can we understand, characterize, and extract actionable knowledge from the deluge of graph data, to benefit high-impact applications from different disciplines? This course will introduce the fundamental problems and cover the recent research advances in analyzing and mining large-scale graphs. It will also discuss the practical applications and the broad impacts of graph mining algorithms in diverse settings (e.g., social media, e-commerce, education, and security). The following topics will be covered in the course: graph essentials, network measures, network models, data mining essentials, community analysis, information diffusion, recommendation, network representation learning, and graph neural networks.

Prerequisites: There are no official prerequisites for this course, but students are expected to : (1) have basic knowledge about data mining and machine learning; (2) be familiar with linear algebra, discrete mathematics, and statistics; (3) be comfortable to read research papers and give presentations; (4) have good programming skills, e.g., Python, C/C++, Java, Matlab, and R.

The course provides an in-depth understanding of matrix analysis concepts, algorithms, and applications, including eigenvalues and eigenvectors, linear transformation, similarity transformations, commonly used factorizations, canonical forms, and Hermitian and symmetric
matrices. In particular, we will illustrate these concepts with specific applications in machine learning, control, signal processing, and optimization.

Welcome! In this course, we’ll study estimation and machine learning from a probabilistic point of view.

Why a probabilistic view? Information and uncertainty, which underlie both fields, can be represented via probability in a robust and versatile way. Unknown quantities can be cast as random variables and their relationships to available information as joint distributions. This provides a unifying framework for setting up estimation and machine learning problems, stating our assumptions clearly, designing methods, and evaluating performance.

What topics will we study? We will start with estimation, which can be defined as the problem of learning about the world from data (e.g., finding the chance of getting a disease given one’s genetic make-up) or drawing conclusions about relationships (e.g., what are the best predictors of academic success?). We will then learn about machine learning problems such as regression and classification, where the goal is to predict an unknown quantity, e.g., the price of a house, based on some relevant information. We will also learn how to deal with situations when part of the data is missing. Finally, we will discuss computational methods, which help tackle difficult problems via approximation.

Pre-requisites:

• Fluency in basic probability (e.g., APMA 3100) is needed for the course. You should be comfortable with 70-80% of Chapter 0. You can also refer to these pages I developed for a different course to review probability, although these don’t cover everything we’ll need:
  • Basics
  • Random variables, distributions, and independence
  • Expectation and variance
  • Conditional probability
• Familiarity with linear algebra.
• The programming exercises are based on Python.

Note to Undergraduate students: You do not need instructor permission to enroll in this course. But fluency in probability is an important prereq and if your foundation in probability is not strong, you will not be able to fully benefit from the course.

Course objective: Over the course of the last twenty years, convex optimization has firmly established itself as an indispensable tool for solving a plethora of design and analysis problems arising in data mining, machine learning, signal processing, wireless communications, control systems, etc. This course will primarily serve to introduce students to the key concepts in convex optimization theory with the goal of enabling them to formulate and solve various convex optimization problems arising in engineering, data science, and machine learning. Non-convex optimization techniques in deep learning will also be introduced.

Prerequisites: Solid background in linear algebra (vector spaces, the four fundamental subspaces of a matrix, positive-definiteness, eigen-value and singular-value decompositions, linear least-squares), basic knowledge of probability and multivariate calculus, familiarity with coding in a high-level language (e.g., Matlab, Python).

• To do ML, you need to understand whatever system you are working with. ML in ECE will teach you how to apply ML to areas like chip design, robotics, and wireless communications. We will bridge ML with engineering applications by emphasizing physics-informed and model-based methods. 
• The ECE version of ML will teach the mathematical background behind ML more so than a data science major might, while the data science version is likely to cover feature engineering more than we will. 
   

Math elective:

• Linear algebra is required for the Machine Learning for Engineers course, so that should be your math elective
• If you have space for additional math electives, an advanced probability course would be a great fit

ECE electives

• Other than those listed above for the focus path, electives in the signal processing and control systems areas are great for those interested in the roots of the ML field.
• You should take any other ECE electives that interest you and, as you take the course, think about how you might be able to apply ML in that field!

Every EE degree requires a 1.5 credit lab, so you will need to take one to complete the major. However, there is no specific lab required for this focus path so you are welcome to select from any of the labs offered in ECE.