Azure Data Factory (ADF): Orchestration and development (English)

Description

In this two-day "Azure Data Factory: Orchestration & Development" training, we delve into both the steering and development, as well as the setup and management of Azure Data Factory (ADF) within the Azure platform.

Do you want to execute and control the orchestration, data movement, and ETL/ELT tasks within the Azure platform using Azure Data Factory? In this Azure Data Factory: Orchestration and Development training, we focus on orchestration and data movement.

By orchestration, we mean the control of sources, databases, and services. This allows us to specify which task can be performed when.

In addition, we look at data movement: the copying of data to the cloud and within the cloud. This ensures that data is in the right format (e.g., a SQL database, or Parquet files) at the right place (e.g., Azure Data Lake Storage, SQL Database, or Synapse).

Who is the Azure Data Factory: Orchestration & Development training for?

This training is designed for anyone responsible for steering and developing ETL and ELT processes with ADF in their organization, as well as those responsible for the setup and management of ADF. This could be because ADF is already being used, or because your organization is moving its data and analytics to Azure. You find it important to understand how ADF works and want to be able to develop in it yourself. In addition, you want to ensure that the setup is solid and want to understand how to use ADF efficiently and securely.

Objectives

After this two-day training:

• You understand how Azure Data Factory works
• You have an overview of all components of Azure Data Factory
• You can set up linked services, datasets, activities, and pipelines yourself
• You know how Azure Data Factory is structured
• You can securely set up ADF
• You are able to manage the integration runtime

We do this as you are used to at Wortell Smart Learning: under the guidance of an inspiring trainer, using really good material, and very practical. So we do a lot ourselves.

Prerequisites

The training is accessible and can be followed if you have a basic understanding of SQL. However, you should already be somewhat familiar with Azure and Azure DevOps:

• Azure:
  • Core Azure Concepts and familiarity with the Azure Portal are important. As preparation, you could follow Microsoft's free e-learning "Azure Fundamentals part 1: Describe core Azure concepts"
  • The Azure Data Platform is important. The distinction between the different types of data stores, variations in types of data, and ways of processing are discussed in the one-day training Azure Data Fundamentals (DP-900)
• Azure DevOps is very important for actual development with ADF, and is used for both version control and Continuous Integration and Continuous Delivery (CI/CD) in ADF.
  • If you have no experience with Azure DevOps (or at least with Git and another CI/CD tool), you can still follow the training, but some parts may be a bit more challenging.
  • Want to get started with Azure DevOps? Take a look at our training CI/CD with Azure DevOps for Data Engineers.

Topics

During the two days, we will lay the foundation for working with Azure Data Factory and train on the following topics:

• Structure of Azure Data Factory
• Orchestration and Data Movement in ADF
• Development in ADF
• Setup and management of ADF
• Linked Services, Datasets, Activities and Pipelines in ADF
• Integration Runtimes: setup choices and management
• Private Endpoints in ADF
• Handling triggers
• Azure Data Factory in Git
• Advanced workflows with dynamic schema and inferred data types
• Monitoring and Logging
• Billing and costs

Study material

In the "Azure Data Factory: Orchestration & Development" training, we use material that we have developed ourselves at Wortell Smart Learning. We will ensure that you receive all the necessary material in time.