AWS ETL [Glue, Data Pipeline and Athena] Fundamentals

AWS ETL Fundamentals. Make Data Pipelines using AWS Glue and Data Pipeline. Query data with AWS Athena

Welcome to this AWS Data Pipeline Course

What you’ll learn

  • How to setup an AWS Data Pipeline.
  • How to setup an AWS Glue Pipeline.
  • How to work on ETL Pipelines on AWS Cloud.
  • How to use AWS Data Brew.
  • How to use AWS Glue Studio.
  • Make workflows in AWS Glue Workflow.
  • Understand what is the difference ETL and ELT.
  • When to use Quick-sight, FireHose and Athena.
  • How to monitor AWS Data Pipeline.
  • When to use AWS EMR or AWS Glue.
  • How to use AWS Glue Crawlers.
  • Why we use AWS Athena.
  • How to use AWS Athena.

Course Content

  • Introduction –> 7 lectures • 27min.
  • AWS Data Pipeline Overview –> 6 lectures • 22min.
  • AWS Glue Overview –> 9 lectures • 44min.
  • AWS Athena –> 2 lectures • 16min.
  • Conclusion –> 1 lecture • 1min.

AWS ETL [Glue, Data Pipeline and Athena] Fundamentals

Requirements

  • A desire to learn and Crush It!.
  • A working Computer with either Windows/MacOS or Linux.
  • An Internet connection.
  • All technologies we will use will be free/have free trial versions.
  • Some Basic knowledge of unix/linux commands can be helpful, but not required.
  • You must setup a free or a paid account on Amazon AWS.

Welcome to this AWS Data Pipeline Course

We are very excited to get this course out to you. This course will take you from being a beginner to an expert in creating ETL Pipelines on the cloud using AWS Data Pipeline and AWS Glue.

In this course you will Learn by example, where we demonstrate all the concepts discussed so that you can see them working, and you can try them out for yourself as well.

My step-by-step training will initiate you into the world of AWS ETL Pipelines.

Amazon provides  ETL (Extract, Transform and Load) tools which integrate with different data storages. Once you prepare the data AWS allows you to query it using Athena and then build dashboards using Quicksight. What is important is you know which tool to use when. EMR is great when used with Hadoop but Glue is great for quick pipeline.

Why Learn AWS Data Pipeline/Glue/EMR/Athena?

Another Question: What tool should you use for Scalable Data Integration when you have data dispersed in multiple data sources and need it to be cleaned and transformed for analysis?

AWS Data Pipeline is a web service that allows customers to create automated data transport and transformation operations. In other words, it provides data extraction, loading, and transformation as a service. To use their data, users don’t need to build an expensive ETL or ELT platform; instead, they may use Amazon’s cloud environment. Data keeps growing as business activity grows, so the scalability of your pipeline is important.

AWS Glue coupled with other tools like Glue Studio and Glue Data Brew allows you to build pipelines with varying amount of customization. AWS Glue allows you to build custom ETL pipelines while Glue Studio provides a UI tool to monitor everything. AWS Glue Data Brew is a new tool which allows you to build the pipelines without any coding. It has more than 250 transformations you can apply with just a few clicks!

AWS Glue Data Classify provides a consistent view of your data, allowing you to appropriately clean, enrich, and catalogue it. This also assures that your data is searchable, queryable, and ETL-ready right away. Understand when to use AWS EMR and when to use AWS Glue. From engineering to data to analytics, it offers immense potential for teams across corporate businesses.

Finally we take a look at some real life examples to study where and how AWS Glue or Data Pipeline can help us. We ensure you learn the best practices at all times.

By the end of the course you will have set up and learnt to manage data transformations in AWS like a pro.

Everything is well documented and separated, so you can find what you need. Assignments and Quizzes will make sure you stay on track and test your knowledge. The course will have a combination of theory and practical examples.