Week 1: Working with Source Systems

Learning Objectives:

• 1 – Identify different data formats and determine appropriate source systems for generating each type of data
• 2 – Explain the difference between relational and NoSQL databases
• 3 – Explain the principles of ACID compliance and how it applies to databases
• 4 – Apply CRUD operations on relational and NoSQL databases
• 5 – Interact with object storage
• 6 – Explain the difference between message queue and streaming platform
• 7 – Explain the basics of cloud networking
• 8 – Troubleshoot database connection errors

Week 2: Data Ingestion

Learning Objectives:

• 1 – Explain the difference between batch and streaming ingestions, their use cases and where they sit along the continuum of ingestion frequencies
• 2 – Identify ways to ingest data from various source systems for various use cases
• 3 – Differentiate between the two ingestion patterns: Extract-Transform-Load (ETL) and Extract-Load-Transform (ELT)
• 4 – Interact with a REST API and create a script to ingest data from the API
• 5 – Describe the basic components of an event-streaming platform (topics/streams, partitions/shards)
• 6 – Interact with an event streaming platform as a source system and as an ingestion tool

Week 3: DataOps

Learning Objectives:

• 1 – Explain how the DevOps automation concepts such as Continuous Integration Continuous Delivery (CI/CD) are applied within DataOps
• 2 – Define Infrastructure as Code and how it relates to the automation pillar of DataOps
• 3 – Use Terraform to provision AWS resources
• 4 – Differentiate between DevOps observability and DataOps observability
• 5 – Apply data quality tests using Great Expectations
• 6 – Identify and monitor relevant data quality metrics

Week 4: Orchestration, Monitoring, and Automating Your Data Pipelines

Learning Objectives:

• 1 – Explain how orchestration can be applied to a data pipeline and list its benefits
• 2 – List and interact with the core components of Airflow
• 3 – Build data pipelines with DAGs in Airflow using features such as Taskflow API, operators, XCom variables, etc.
• 4 – Apply the best practices in building DAGs in Airflow
• 5 – Integrate data quality testing using Great Expectations in an orchestrated pipeline in Airflow

Week 1: Storage Ingredients and Storage Systems

Learning Objectives:

• 1 – Explain how data is physically stored on disk and in memory
• 2 – Compare how data is stored and queried in object, block, and file storage systems
• 3 – Explain how data is stored in row-oriented vs column-oriented databases
• 4 – Explain how graph and vector databases store and retrieve data

Week 2: Storage Abstractions

Learning Objectives:

• 1 – Explain the key architectural features of data warehouses and data lakes
• 2 – Explore the good practices in data lake storage such as partitioning and cataloging
• 3 – Explain the use of open table formats in building transactional data lakes
• 4 – Define the data lakehouse storage abstraction
• 5 – Implement a data lakehouse with a medallion-like architecture

Week 3: Queries

Learning Objectives:

• 1 – Explain the stages of the life of a query
• 2 – Implement advanced SQL queries
• 3 – Discuss the role of of an index in affecting the query performance
• 4 – Summarize the approaches for processing aggregate and join queries
• 5 – Compare the execution times of aggregate queries between row and columnar storage
• 6 – List some strategies for enhancing query performance
• 7 – Explain aggregate and join queries on streaming data

Week 1: Data Modeling & Simple Transformations for Analytics

Learning Objectives:

• 1 – Define data modeling and its role in reflecting business logic
• 2 – Apply the normalization stages to a denormalized table
• 3 – Explain the use of the fact and dimension tables of a star schema
• 4 – Differentiate between the data warehouse modeling approaches such as Inmon, Kimball and Data Vault
• 5 – Transform data in third normal form to a star schema
• 6 – Describe the use of hub, link and satellite tables in a data vault model
• 7 – Discuss the pros and cons of the use of one big table

Week 2: Data Modeling & Simple Transformations for Machine Learning

Learning Objectives:

• 1 – Differentiate between supervised vs unsupervised learning, and classification vs regression
• 2 – Describe the tabular form expected by classical machine learning algorithm
• 3 – Apply preprocessing steps to prepare tabular data for machine learning
• 4 – Apply preprocessing steps to clean textual data
• 5 – List techniques for vectorizing textual data
• 6 – List techniques for processing and augmenting image data

Week 3: Transformations

Learning Objectives:

• 1 – Identify batch transformation use cases and patterns
• 2 – Compare an in-memory processing frameworks like Spark, and a processing framework that involves disk read and write operations, such as Hadoop
• 3 – Describe the technical considerations for choosing a distributed processing framework, such as Spark, vs a non-distributed framework like Pandas dataframes
• 4  – Describe the technical considerations for using SparkSQL vs DataFrames when transforming data using PySpark
• 5 – Explain how streaming transformation works with a near-real time processing engine such as Spark Structured Streaming

Week 4: Serving Data

Coming soon!