Tweaks
This commit is contained in:
parent
40f164f38e
commit
d969bf2580
BIN
Content/01. A Day of Fundamentals - 1 Day/01. Agenda Header.png
Normal file
BIN
Content/01. A Day of Fundamentals - 1 Day/01. Agenda Header.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 178 KiB |
20
Content/01. A Day of Fundamentals - 1 Day/01. Agenda.md
Normal file
20
Content/01. A Day of Fundamentals - 1 Day/01. Agenda.md
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
# Azure Data Integration Pipeline Training
|
||||||
|
## A Day of Fundamentals
|
||||||
|
### 1 Day
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Agenda
|
||||||
|
|
||||||
|
The following offers an insight into the complete agenda and module breakdown for this workshop.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 179 KiB |
20
Content/02. Fundamentals to Level 300 - 1 Day/02. Agenda.md
Normal file
20
Content/02. Fundamentals to Level 300 - 1 Day/02. Agenda.md
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
# Azure Data Integration Pipeline Training
|
||||||
|
## Fundamentals to Level 300
|
||||||
|
### 1 Day
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Agenda
|
||||||
|
|
||||||
|
The following offers an insight into the complete agenda and module breakdown for this workshop.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 180 KiB |
120
Content/03. Advanced Design & Delivery - 1 Day/03. Agenda.md
Normal file
120
Content/03. Advanced Design & Delivery - 1 Day/03. Agenda.md
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
# Azure Data Integration Pipeline Training
|
||||||
|
## Advanced Design & Delivery
|
||||||
|
### 1 Day
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
In this course we'll quickly cover the fundamentals of data integration pipelines before going much deeper into our Azure resources.
|
||||||
|
|
||||||
|
Within a typical Azure data platform solution for any enterprise grade data analytics or data science workload an umbrella resource is needed to trigger, monitor, and handle the control flow for transforming datasets, with the goal being actionable data insight. Those requirements are met by deploying Azure Data Integration pipelines, delivered using Azure Synapse Analytics or Azure Data Factory. In this session, I'll show you how to create rich dynamic data pipelines and apply these orchestration resources in production. Using scaled architecture design patterns, best practice, data mesh principals, and the latest metadata driven frameworks. We will take a deep dive into the services, considering how to build custom activities, complex pipelines and think about hierarchical design patterns for enterprise grade deployments. All this and more in a complete set of 12 modules (based on real world experience) we will take you through how to implement data integration pipelines in production and delivered advanced orchestration patterns.
|
||||||
|
|
||||||
|
If that's not enough learning for you, a set of hands-on labs will also be made available that you can work through at your own pace. Whether you are new to Azure Data Integration Pipelines or have some experience, you will leave this course with new skills, ideas, and a much deeper understanding of the resources for your future data platform projects.
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Agenda
|
||||||
|
|
||||||
|
The following offers an insight into the complete agenda and module breakdown for this course.
|
||||||
|
|
||||||
|
* __Module 1:__ [Pipeline Fundamentals](./Module%201.pdf)
|
||||||
|
* The History of Azure Orchestration
|
||||||
|
* Synapse Analytics vs Data Factory
|
||||||
|
* Integration Components
|
||||||
|
* Common Activities
|
||||||
|
* Execution Dependencies
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 2:__ [Integration Runtime Design Patterns](./Module%202.pdf)
|
||||||
|
* Compute Types
|
||||||
|
* Azure
|
||||||
|
* Hosted
|
||||||
|
* SSIS
|
||||||
|
* Patterns & Configuration
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 3:__ [Data Transformation](./Module%203.pdf)
|
||||||
|
* Data Flows
|
||||||
|
* Power Query Injection
|
||||||
|
* Spark Configuration
|
||||||
|
* Use Cases
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 4:__ [Dynamic Pipelines](./Module%204.pdf)
|
||||||
|
* Expressions & Interpolation
|
||||||
|
* Simple Metadata Driven Execution
|
||||||
|
* Dynamic Content Chains
|
||||||
|
* Reference Names
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 5:__ [Pipeline Extensibility](./Module%205.pdf)
|
||||||
|
* Azure Batch Service
|
||||||
|
* Tasks
|
||||||
|
* Compute Pools
|
||||||
|
* Scaling
|
||||||
|
* Pipeline Custom Activities
|
||||||
|
* Azure Management API
|
||||||
|
* Azure Functions
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 6:__ [Execution Parallelism](./Module%206.pdf)
|
||||||
|
* Control Flow Scale Out
|
||||||
|
* Concurrency Limitations
|
||||||
|
* Internal vs External Activities
|
||||||
|
* Orchestration Framework - [procfwk.com](http://procfwk.com/)
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 7:__ [VNet Integration](./Module%207.pdf)
|
||||||
|
* Private Endpoints
|
||||||
|
* Managed VNet's
|
||||||
|
* Firewall Bypass
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 8:__ [Security](./Module%208.pdf)
|
||||||
|
* Service Principals
|
||||||
|
* Managed Identities
|
||||||
|
* Azure Key Vault Integration
|
||||||
|
* Customer Managed Keys
|
||||||
|
* Pipeline Access & Permissions
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 9:__ [Monitoring & Alerting](./Module%209.pdf)
|
||||||
|
* Studio Monitoring
|
||||||
|
* Log Analytics & Kusto Queries
|
||||||
|
* Operational Dashboards
|
||||||
|
* Advanced Alerting
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 10:__ [Solution Testing](./Module%2010.pdf)
|
||||||
|
* Development Time Validation
|
||||||
|
* Test Coverage
|
||||||
|
* NUnit Tests
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 11:__ [CI/CD](./Module%2011.pdf)
|
||||||
|
* Source Control vs Developer UI
|
||||||
|
* Basic ARM Template Deployments
|
||||||
|
* Advanced Deployment Patterns
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 12:__ [Final Thoughts](./Module%2012.pdf)
|
||||||
|
* Running Costs
|
||||||
|
* Conclusions
|
||||||
|
* Best Practices
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 1.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 1.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 10.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 10.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 11.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 11.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 12.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 12.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 2.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 2.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 3.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 3.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 4.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 4.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 5.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 5.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 6.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 6.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 7.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 7.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 8.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 8.pdf
Normal file
Binary file not shown.
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 9.pdf
Normal file
BIN
Content/03. Advanced Design & Delivery - 1 Day/Module 9.pdf
Normal file
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 182 KiB |
@ -0,0 +1,131 @@
|
|||||||
|
# Azure Data Integration Pipeline Training
|
||||||
|
## Advanced Design & Delivery (A Deep Dive)
|
||||||
|
### 2 Day
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
In this course we'll quickly cover the fundamentals of data integration pipelines before going much deeper into our Azure resources.
|
||||||
|
|
||||||
|
Within a typical Azure data platform solution for any enterprise grade data analytics or data science workload an umbrella resource is needed to trigger, monitor, and handle the control flow for transforming datasets, with the goal being actionable data insight. Those requirements are met by deploying Azure Data Integration pipelines, delivered using Azure Synapse Analytics or Azure Data Factory. In this session, I'll show you how to create rich dynamic data pipelines and apply these orchestration resources in production. Using scaled architecture design patterns, best practice, data mesh principals, and the latest metadata driven frameworks. We will take a deep dive into the services, considering how to build custom activities, complex pipelines and think about hierarchical design patterns for enterprise grade deployments. All this and more in a complete set of 12 modules (based on real world experience) we will take you through how to implement data integration pipelines in production and delivered advanced orchestation patterns.
|
||||||
|
|
||||||
|
If that's not enough learning for you, a set of hands-on labs will also be made available that you can work through at your own pace. Whether you are new to Azure Data Integration Pipelines or have some experience, you will leave this course with new skills, ideas, and a much deeper understanding of the resources for your future data platform projects.
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Agenda
|
||||||
|
|
||||||
|
The following offers an insight into the complete agenda and module breakdown for this course.
|
||||||
|
|
||||||
|
### Day 1
|
||||||
|
|
||||||
|
* __Module 1:__ [Pipeline Fundamentals](./Module%201.pdf)
|
||||||
|
* The History of Azure Orchestration
|
||||||
|
* Synapse Analytics vs Data Factory
|
||||||
|
* Integration Components
|
||||||
|
* Common Activities
|
||||||
|
* Execution Dependencies
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 2:__ [Integration Runtime Design Patterns](./Module%202.pdf)
|
||||||
|
* Compute Types
|
||||||
|
* Azure
|
||||||
|
* Hosted
|
||||||
|
* SSIS
|
||||||
|
* Patterns & Configuration
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 3:__ [Data Transformation](./Module%203.pdf)
|
||||||
|
* Data Flows
|
||||||
|
* Power Query Injection
|
||||||
|
* Spark Configuration
|
||||||
|
* Use Cases
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 4:__ [Dynamic Pipelines](./Module%204.pdf)
|
||||||
|
* Expressions & Interpolation
|
||||||
|
* Simple Metadata Driven Execution
|
||||||
|
* Dynamic Content Chains
|
||||||
|
* Reference Names
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 5:__ [Pipeline Extensibility](./Module%205.pdf)
|
||||||
|
* Azure Batch Service
|
||||||
|
* Tasks
|
||||||
|
* Compute Pools
|
||||||
|
* Scaling
|
||||||
|
* Pipeline Custom Activities
|
||||||
|
* Azure Management API
|
||||||
|
* Azure Functions
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 6:__ [Execution Parallelism](./Module%206.pdf)
|
||||||
|
* Control Flow Scale Out
|
||||||
|
* Concurrency Limitations
|
||||||
|
* Internal vs External Activities
|
||||||
|
* Orchestration Framework - [procfwk.com](http://procfwk.com/)
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
### Day 2
|
||||||
|
|
||||||
|
* __An Architects Recap__: [Modules 1 to 6](./Module%201%20-%206%20Recap.pdf)
|
||||||
|
* Design
|
||||||
|
* Extract
|
||||||
|
* Transform
|
||||||
|
* Load
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 7:__ [VNet Integration](./Module%207.pdf)
|
||||||
|
* Private Endpoints
|
||||||
|
* Managed VNet's
|
||||||
|
* Firewall Bypass
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 8:__ [Security](./Module%208.pdf)
|
||||||
|
* Service Principals
|
||||||
|
* Managed Identities
|
||||||
|
* Azure Key Vault Integration
|
||||||
|
* Customer Managed Keys
|
||||||
|
* Pipeline Access & Permissions
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 9:__ [Monitoring & Alerting](./Module%209.pdf)
|
||||||
|
* Studio Monitoring
|
||||||
|
* Log Analytics & Kusto Queries
|
||||||
|
* Operational Dashboards
|
||||||
|
* Advanced Alerting
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 10:__ [Solution Testing](./Module%2010.pdf)
|
||||||
|
* Development Time Validation
|
||||||
|
* Test Coverage
|
||||||
|
* NUnit Tests
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 11:__ [CI/CD](./Module%2011.pdf)
|
||||||
|
* Source Control vs Developer UI
|
||||||
|
* Basic ARM Template Deployments
|
||||||
|
* Advanced Deployment Patterns
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 12:__ [Final Thoughts](./Module%2012.pdf)
|
||||||
|
* Running Costs
|
||||||
|
* Conclusions
|
||||||
|
* Best Practices
|
||||||
|
|
||||||
|
___
|
||||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
Content/05. Zero to Level 401 - 3 Day/05. Agenda Header.png
Normal file
BIN
Content/05. Zero to Level 401 - 3 Day/05. Agenda Header.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 176 KiB |
115
Content/05. Zero to Level 401 - 3 Day/05. Agenda.md
Normal file
115
Content/05. Zero to Level 401 - 3 Day/05. Agenda.md
Normal file
@ -0,0 +1,115 @@
|
|||||||
|
# Azure Data Integration Pipeline Training
|
||||||
|
## Zero to Level 401
|
||||||
|
### 3 Day
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
## Agenda
|
||||||
|
|
||||||
|
The following offers an insight into the complete agenda and module breakdown for this course.
|
||||||
|
|
||||||
|
* __Module 1:__ [Pipeline Fundamentals]()
|
||||||
|
* The History of Azure Orchestration
|
||||||
|
* Synapse Analytics vs Data Factory
|
||||||
|
* Integration Components
|
||||||
|
* Common Activities
|
||||||
|
* Execution Dependencies
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 2:__ [Integration Runtime Design Patterns]()
|
||||||
|
* Compute Types
|
||||||
|
* Azure
|
||||||
|
* Hosted
|
||||||
|
* SSIS
|
||||||
|
* Patterns & Configuration
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 3:__ [Data Transformation]()
|
||||||
|
* Data Flows
|
||||||
|
* Power Query Injection
|
||||||
|
* Spark Configuration
|
||||||
|
* Use Cases
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 4:__ [Dynamic Pipelines]()
|
||||||
|
* Expressions & Interpolation
|
||||||
|
* Simple Metadata Driven Execution
|
||||||
|
* Dynamic Content Chains
|
||||||
|
* Reference Names
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 5:__ [Pipeline Extensibility]()
|
||||||
|
* Azure Batch Service
|
||||||
|
* Tasks
|
||||||
|
* Compute Pools
|
||||||
|
* Scaling
|
||||||
|
* Pipeline Custom Activities
|
||||||
|
* Azure Management API
|
||||||
|
* Azure Functions
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 6:__ [Execution Parallelism]()
|
||||||
|
* Control Flow Scale Out
|
||||||
|
* Concurrency Limitations
|
||||||
|
* Internal vs External Activities
|
||||||
|
* Orchestration Framework - [procfwk.com](http://procfwk.com/)
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 7:__ [VNet Integration]()
|
||||||
|
* Private Endpoints
|
||||||
|
* Managed VNet's
|
||||||
|
* Firewall Bypass
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 8:__ [Security]()
|
||||||
|
* Service Principals
|
||||||
|
* Managed Identities
|
||||||
|
* Azure Key Vault Integration
|
||||||
|
* Customer Managed Keys
|
||||||
|
* Pipeline Access & Permissions
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 9:__ [Monitoring & Alerting]()
|
||||||
|
* Studio Monitoring
|
||||||
|
* Log Analytics & Kusto Queries
|
||||||
|
* Operational Dashboards
|
||||||
|
* Advanced Alerting
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 10:__ [Solution Testing]()
|
||||||
|
* Development Time Validation
|
||||||
|
* Test Coverage
|
||||||
|
* NUnit Tests
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 11:__ [CI/CD]()
|
||||||
|
* Source Control vs Developer UI
|
||||||
|
* Basic ARM Template Deployments
|
||||||
|
* Advanced Deployment Patterns
|
||||||
|
|
||||||
|
___
|
||||||
|
|
||||||
|
* __Module 12:__ [Final Thoughts]()
|
||||||
|
* Running Costs
|
||||||
|
* Conclusions
|
||||||
|
* Best Practices
|
||||||
|
|
||||||
|
|
||||||
|
___
|
||||||
15
training.code-workspace
Normal file
15
training.code-workspace
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"folders": [
|
||||||
|
{
|
||||||
|
"path": "."
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"settings": {
|
||||||
|
"cSpell.words": [
|
||||||
|
"Avanade",
|
||||||
|
"Kusto",
|
||||||
|
"mrpaulandrew",
|
||||||
|
"procfwk"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
x
Reference in New Issue
Block a user