Machine Learning Essentials with Python and Spark

Key Learning Areas

This program provides a structured, engaging learning experience for students to progressively attain the technical skills they require to engage in meaningful machine learning projects and activities, right after the training ends. Working in a hands-on learning environment, led by our Machine Learning expert instructor, students will learn about and explore:

• Popular machine learning algorithms, their applicability, and limitations
• Practical application of these methods in the Spark machine learning environment
• Practical use cases and limitations of algorithms

Course Outline

Machine Learning (ML) Overview

• Machine Learning landscape
• Machine Learning applications
• Understanding ML algorithms & models

Machine Learning in Python and Spark

• Spark ML Overview
• Introduction to Jupyter notebooks
• Working with Jupyter + Python + Spark

Machine Learning Concepts

• Statistics Primer
• Covariance, Correlation, Covariance Matrix
• Errors, Residuals
• Overfitting / Underfitting
• Cross-validation, bootstrapping
• Confusion Matrix
• ROC curve, Area Under Curve (AUC)

Feature Engineering (FE)

• Preparing data for ML
• Extracting features, enhancing data
• Data cleanup
• Visualizing Data
• Lab: data cleanup
• Lab: visualizing data

Linear regression

• Simple Linear Regression
• Multiple Linear Regression
• Running LR
• Evaluating LR model performance
• Use case: House price estimates
  

Logistic Regression

• Understanding Logistic Regression
• Calculating Logistic Regression
• Evaluating model performance
• Use case: credit card application, college admissions

Classification: SVM (Supervised Vector Machines)

• SVM concepts and theory
• SVM with kernel
• Use case: Customer churn data

Classification: Decision Trees & Random Forests

• Theory behind trees
• Classification and Regression Trees (CART)
• Random Forest concepts
• Use case: predicting loan defaults, estimating election contributions

Classification: Naive Bayes

• Theory
• Use case: spam filtering

Clustering (K-Means)

• Theory behind K-Means
• Running K-Means algorithm
• Estimating the performance
• Use case: grouping cars data, grouping shopping data

Principal Component Analysis (PCA)

• Understanding PCA concepts
• PCA applications
• Running a PCA algorithm
• Evaluating results
• Use case: analyzing retail shopping data

Recommendations (Collaborative filtering)

• Recommender systems overview
• Collaborative Filtering concepts
• Use case: movie recommendations, music recommendations

Performance 

• Best practices for scaling and optimizing Apache Spark
• Memory caching
• Testing and validation

Time Permitting: Capstone Project

• Hands-on guided workshop utilizing skills learned throughout the course

Who Benefits

This course is geared for experienced, intermediate-skilled developers or others (with prior Python experience) intending to start using learning about and working with basic machine learning algorithms and concepts.  Attendees should be comfortable working with Python programming.  Students should also be able to navigate Linux command line, and who have basic knowledge of Linux editors (such as VI / nano) for editing code.

Prerequisites

• Strong basic Python Skills. Attendees without Python background may view labs as follow along exercises or team with others to complete them. (NOTE: This course is also offered in R – please inquire for details)
• Good foundational mathematics in Linear Algebra and Probability
• Basic Linux skills, including familiarity with command-line options such as ls, cd, cp, and su