2023-01-03
Microsoft's cloud services platform
Azure is Microsoft’s cloud computing offering, enabling businesses to run applications, databases, and other services on hardware built, maintained, secured, and managed in Microsoft’s datacenters.
The Azure cloud platform is more than 200 products and cloud services designed to help you bring new solutions to life—to solve today’s challenges and create the future. Build, run, and manage applications across multiple clouds, on-premises, and at the edge, with the tools and frameworks of your choice. (source)
Enterprise IT projects will commonly leverage cloud infrastructure, making knowledge of Azure, GCP, or AWS a critical skill for developers, software architects, and IT personnel alike.
As a growing Software Architect I was recommended to pursue:
Certification Checklist:
Sorted alphabetically.
| Key | Value |
|---|---|
| ACI | Azure Container Instance |
| ACO | Actual Cut-Over |
| AD-DS | Active Directory Domain Services |
| ADF | Azure Data Factory |
| AKS | Azure Kubernetes Service |
| ARM | Azure Resource Manager |
| ASG | Application Security Group |
| AVD | Azure Virtual Desktop |
| AZ | Microsoft AZure |
| BLOB | Binary Large OBject |
| CM | Configuration Management |
| CapEX | Capital expenditure |
| DCO | Dry Cut-Over |
| DR | Disaster Recovery |
| EA | Enterprise Architecture |
| ECM | Enterprise Content Management |
| ECMS | Enterprise Content Management System |
| GPV2 | General Purpose Storage Account V2 |
| GRS | Geo-Redundant Storage |
| GZRS | Geo-Zone-Redundant Storage |
| IaaS | Infrastructure as a service |
| JEA | Just Enough Access |
| JIT | Just in Time |
| LRS | Locally Redundant Storage |
| MOC | Management of Change |
| NSG | Network Security Group |
| OpEX | Operating expenditure |
| PaaS | Platform as a service |
| RBAC | Role-Based Access Control |
| RG | Resource Group |
| RPO | Recovery Point Objective |
| RTO | Recovery Time Objective |
| SA | Storage Account |
| SAS | Shared Access Signature |
| SLA | Service Level Agreeement |
| SMB | Server Message Block - File and port sharing protocol |
| SME | Subject Matter Expert |
| SSO | Single Sign-On |
| SSPR | Self-Service Password Reset |
| SaaS | Software as a service |
| TCO | Total cost of ownership |
| TLS | Transport Security Layer |
| VM | Virtual Machine |
| VMSS | Virtual Machine Scale Set |
| VNet | Virtual Network |
| ZRS | Zone-Redundant Storage |
;; Sort the acronym table with M-x sort-lines
(sort-lines nil (region-beginning) (region-end))
The AZ-900: Microsoft Azure Fundamentals exam is an entry-level credential to prove knowledge of the fundamentals of Microsoft’s Azure cloud computing platform.
Azure Fundamentals exam is an opportunity to prove knowledge of cloud concepts, Azure services, Azure workloads, security and privacy in Azure, as well as Azure pricing and support. Candidates should be familiar with the general technology concepts, including concepts of networking, storage, compute, application support, and application development.
I began studying for this exam by taking an AZ-900 prep course on Udemy in January 2023 and passed the in-person exam a month later.
Learning Goals:
Links and Resources:
An IBM colleague warned me that just taking the Udemy course is insufficient to pass the AZ-900 exam. Consider finding lots of additional practice questions, and focus on the specific services used to build solutions to client problems.
Learning goals:
Cloud computing is the ability to rent computing resources on demand, and only pay for what you use. Microsoft has already made the billion-dollar investment in hardware, networking, etc.
The Shared Responsibility Model relates responsibilities in the cloud environment to a traditional on-premises service deployment, where you are responsible for everything from building and network security to networking and firewalls to the operating systems and authentication, devices, and data. These responsibilities fall away in different ways if you use the cloud.
By using a Cloud VM (IaaS1), you’ll still need to worry about the OS, networking, and the application, but the physical computer and networking is handled.
By using an App Service (PaaS2), you can now allow the cloud service to handle operating system patches and updates.
At a very high level there is Software as a Service (SaaS3) which prohibits the control of the network and application, just giving you access to a cloud software from a vendor.
With these resources, you lose some control and responsibility, and are paying for a company (Microsoft Azure or others,) to manage those responsibilities for you. The hope is that they manage it competently.
Cloud Types:
On-prem, cost can be predicted by adding hardware, power, internet, licensing, and employee costs. In the cloud, cost is based on consumption, usually by a number of metrics. A database could be priced by the CPU speed and RAM per minute, the consumed storage, an optional dedicated gateway, and number/size of backups.
This algorithmic pricing falls into a few different categories:
Pricing will also vary by region and cloud. Providers try to price their services so it will save money for most of their client workloads. Ideally the capital expenditure and ongoing operating expenditures can be slowly reduced as services are moved to Azure, and a client will be left with an overall lower operating cost after all services are moved.
CapEX – Capital Expenditure, like hardware
OpEX – Ongoing Expenditure, like power and cooling
The cost savings comes from Microsoft’s ability to run servers where power and cooling are cheap, so they can make a profit while still providing the computing service at a far lower TCO.
TCO – Total Cost of Ownership
Autoscaling can be used in the cloud to ensure your resouces are not overprovisioned which is the typical scenario when buying physical hardware to avoid overloaded hardware.
Learning goals:
Cloud services provide guarantees of reliability that use high availability as a metric. With 44640 minutes in a 31 day month, the following “number of nines” define the contractually allowed downtime:
…etc
No amount of money can achieve 100% availability.
Scalability – The ability of a system to handle growth of users or work; scaling is the ability to add capacity to an application.
Elasticity – The ability of a system to automatically grow and shrink based on application demand. Capacity is kept above user demand.
SLA – Service Level Agreement: A guarantee of availability, reliability, etc backed with financial compensation (refunds) for failures.
Azure has published timelines and procedures for rollouts of new hardware and software. Tools are provided to deploy applications across multiple regions to address this scheduled downtimes.
Azure provides Chaos Studio to simulate various failures to see how your application responds, adapts, or dies.
Global Availability can be achieved by deploying your application across the many worldwide Azure datacenters, and this is something that is difficult to achieve for a corporation of any scale.
Learning goals:
IaaS – Infrastructure as a Service offers compute, networking, and storage resources on demand, where you take full responsiblity for how your application is deployed and for most of the configuration of the underlying resources.
PaaS – Platform as a Service is a complete deployment environment. It doesn’t give access to the VM typically, but will provide a wealth of convenience tools to support the web app development lifecycle. Auto deployments from repositories, etc. You can upload code here and it will run. Serverless functions fall into this category but you have even less control of where things run – you don’t need to worry about scaling or the correct performance plan.
SaaS – Zero control over what’s going on under the hood, you just get access to the application. Like the Azure Portal, Slack, Office 365.
Learning goals:
AZURE ARCHITECTURAL HIERARCHY 2022
Management Groups
|__ Subscriptions
|__ Resource Groups
|__ Resources
Regions are Azure datacenters that exist in or near certain locations in the globe.
Region Pairs are designed to have fast connections between them. They are the ideal failover location for running applications. A region pair will never have both Regions down for maintenance.
For example:
Sovereign Regions are unavailable to most Azure subscribers, like China or Azure Government (US).
Availability Zones futher subdivide Regions. They are separate datacenter buildings within a Region. They enable another layer of failover if one datacenter in a region fails and your application is running on multiple AZs.
Datacenters must be withink 50km of one another to be grouped in a single region.
Resource Groups are logical groupings of Azure resources typically based on shared function.
Resources can only be part of one group, and must be part of a group.
A Subscription is a billing unit. You’d want to have a Subscription for each client that you manage. All resources that are consumed by a subscription will be billed by the owner. Subscriptions can also be used to break up billing into dev/production or by department.
Subscriptions can be organized into nested Management Groups to enforce policies and access.
Learning goals:
Amazon EC2 (Elastic Compute Cloud) is akin to Azure VMs. With VMs, you can pick from a menu of different CPU types, paired with different amounts of memory and storage. menu
VM Scale Sets are a functionality provided for free, though you do need to pay for the consumed vm-hour-storage-memory units. Enables 2+ machines to run the same code and autoscale with more machines (up to 100 by default or 1000 configured.)
App Services (PaaS) runs an application with no knowledge of the underlying hardware apart from basic specs, like Heroku.
Azure Container Instance (ACI) can run a single container or a group of containers with Docker Compose.
Azure Kubernetes Service (AKS) can run a cluster of containers.
Azure Virtual Desktop (AVD) provides a virtual desktop for work purposes.
Virtual Networks or Virtual Private Clouds (VPC) on AWS emulate a physical network and allow you to work remotely or combine physical networks. Creating interconnected virtual networks or subnets with different security settings in Azure is just a software configuration called peering as it is all networked physically.
Address Spaces define the number of available IP addresses in your VNet, 10.0.0.0/16 indicates a 16 bit subnet mask, giving you the last 16 bits in the address for your IPs (~65k with a 16 bit subnet mask.)
VPN Gateways allow you to connect networks together, between on-prem and Azure or your local machine to your business network.
VNet Peering enables the interconnection of vnets. One-way and two-way traffic can be configured between peered subnets. Global peering is possible but costs money for bandwidth: egress from zone A, and ingress to zone B.
Private Endpoints allow you to make a resource accessible only at a specific IP on a virtual network.
ExpressRoute is a high speed PHYSICAL CABLE private connection to Azure.
Azure DNS allows you to take control of the DNS records for your domain.
Learning goals:
Blob, disk, file storage.
In an Azure Storage account, the most common storage type is general purpose v2 (GPV2) which can hold blobs, tables, queues, and files. It’s the cheapest as well at roughly 2 cents per GB. This is equivalent to AWS S3.
BLOB = Binary Large OBject
Blob or Unmanaged Storage like gpv2 allows you to store objects and be charged by size and egress. Blob storage is organized into containers and the access tier, block size, endpoints, etc can be set on a per-container basis in a storage account. Temporary permissions can be generated for a blob by creating an SAS key that allows you to read or write, etc.
Disk Storage allows you to take control of a full pre-allocated disk, and you pay for the disk size and speed regardless of usage.
Access tiers - hot, cool, archive – the cool tier is half the price, but is slower and costs more to access. Archive tier is ~10% of the cost of hot storage, but may take hours to access.
Performance tiers - premium, standard
Location - keep close to your application instances
Redundancy - multiple replicated copies of files in different regions. Options include:
Failover - automatically using a storage backup
Data Lake Storage Gen 2 enables hierarchical namespaces and access to parts of a ludicrously large virtual filesystem.
To move files, there are four options available:
# PowerShell (.ps1) AZCopy Example
azcopy -? # see if it's installed
azcopy copy '<url to source>' '<url to dest>'
Migration Tools and Strategies:
Learning goals:
Generally, identity is a digital representation of an entity that requires access to a system. This could be a person, device, or robot/application. Digital identity requires proof such as a key, password, or certificate.
Authentication: who are you?
Authorization: what are you allowed to do?
Azure AD is a security protocol used on Azure. It is identity as a service (idaas? lol.) Much like OAuth, the web server is passed a signed/trusted token it can use to identify the user.
Azure AD offers a centralized, consistent, secure approach to using your org’s applications, and even many applications outside your organization or SaaS offerings.
AAD has the ability to configure conditional access which detects unusual logins and can block access or ask for additional information. For example, is the user on the company network? Have they not logged in for years? AAD calls these signals.
MFA: Multi-Factor Authentication – Requires two or more pieces of evidence that the user is not an impostor. This can be something you know (password), something you have (phone), or something you are (biometric artifact).
Passwordless authentication attempts to reduce the inconvenience of MFA while retaining the security benefits. Essentially MFA without passwords as an auth option. Can be simple games while carefully checking signals like the device.
Azure RBAC (role based access control) allows administrators to enable granular access to all the Azure services for a variety of roles. Readonly to storage for business roles, control over virtual machines for developers, the ability to create and destroy VMs and storage for developer admins, etc. It’s easy to create custom permissions and roles. It is rare to manage permissions on an individual level, it is far more common to use roles.
Common Azure Permissions:
Zero Trust means providing many security thresholds within a service or between applications, with everything locked by default, only enabling access to exactly what is needed to complete work. The principles of zero trust are:
JIT: Just in Time – only grant permissions when needed
JEA: Just Enough Access – don’t over-provision access
Provide ample security within the network: encryption, segmentation, threat detection, etc.
The Defense in Depth principle means applying security best practices at every level of an organization:
Azure provides many tools for each category.
Microsoft Defender for Cloud provides security, posture management, and threat detection. By paying per resource per month (~$15USD per server) Microsoft provides an in-depth security analysis and monitoring of the resource.
Learning goals:
Billing is calculated from a broad variety of factors depending on the service. Many services have a free tier or are free below a certain limit, like virtual networks, load balancing, basic AAD, and free-tier web apps. Using pay-by-consumption or serverless models can save organizations a lot of money – one million executions of an Azure Function will cost ~$0.20 USD. Per second billing is used for resources like VMs and disks.
Pricing can be made more stable with a 1 -> 3 year commitment in reserved VM instances. Long-term contracts can be made to reduce storage, VM, networking costs.
Azure is priced to increase cloud utilization.
Ingress (inbound data) is almost always free for obvious reasons.
1PB of data would cost ~$52,000 USD to remove from Azure.
The Azure Pricing Calculator allows consultants to make an educated guess about the costs of a service. As Azure’s pricing changes, the cost of provisioned services will change, so the pricing calculator must be used with this in mind.
TCO: Total Cost of Ownership can be estimated with the Azure TCO Calculator and is oriented towards comparing current on-prem costs with a potential Azure re-deployment. It attempts to factor in power, networking costs, hardware costs, software licenses, and IT personnel costs to determine if the client will attain a hybrid benefit from moving some or all of their workload to Azure.
Resource Tags can be used to organize which department the bills for a certain resource are sent. Tags can be used for many other things, but they are particularly useful for billing as dashboards can show cost by tag.
*BRRRING!* *BRRRING!* *CLICK!* *SHFHFHFUF-FLE-SHUFFLFL!*
HELLO SIR YOU HAVE REACHED THE BILLING DEPARTMENT SIR WHAT IS YOUR BILLING CODE SIR!? SIR!? ARE YOU THERE SIR!?!? HELLO??? SIR!?
*CLICK!*
Running costs can be viewed and analyzed with free Azure tools. Budgets can be specified, payments tracked, and recommendations given by the system. The dashboard can provide spending estimates and configurable alerts.
Reports can be scheduled showing resource usage.
Learning goals:
A company may have certain IT rules that they want to implement. Azure provides a variety of tools to enforce these policies programmatically. For instance: Ensure all servers have at least one daily backup.
This rule can either be enforced socially, with an email to IT, or programmatically in Azure, with tools like those listed in the learning goals above which will now be explored.
Azure Blueprint provides a way to standardize the creation and management of subscriptions. It can create and enforce security rules, resource groups, and policies across a set of subscriptions that can be per client or just per department.
Azure Policy provides a way to programmatically define and enforce these rules. For example, you can mandate:
You can also write your own policies to help prevent unsafe practices and limit costs.
Resource Locks: You can mark a resouce as read only to ensure other users cannot modify the state of the VM, delete it, or modify its properties. You can ensure only a limited subset of people can apply and remove locks.
The Service Trust Portal holds all the documents related to Azure’s compliance towards regulations and standards. It even contains blueprints for certain applications and industries, like healthcare or US Government.
Learning goals:
Azure Portal: Available at portal.azure.com, Microsoft provides a web GUI for creating and managing all Azure resources.
Azure Cloud Shell, CLI, Powershell: At a certain point, resources may become too numerous to manage via the portal, and so can be partially or completely managed via the cloud shell over Powershell or Bash. It can be accessed directly at cloudshell.azure.com.
Some command examples:
# az is the Azure CLI
$ az webapp list
# to create resources:
# az <target> <action>
# Create a Resource Group
$ az group create --name NewRG --location eastus
# Create a VM
$ az vm create
--resource-group NewRG
--name LearningVM
--image Win2019DataCenter
--public-ip-sku Standard
--admin-username xXgoliathXx
# Open port 80 on the VM
az vm open-port
--port 80
--resource-group NewRG
--name LearningVM
# Wipe the Resource Group
az group delete --name newrg
Azure Arc: Allows you to manage on-prem infrastructure or resouces in other clouds like GCP or AWS. SQL Servers, Kubernetes clusters, etc. Azure Stack HCI refers to edges on private clouds.
ARM: Azure Resource Manager: The service that runs underneath Azure Portal/CLI/Cloud Shell that can be manipulated with the aforementioned tools or a set of APIs. When creating a resource, you’ll have the option to click either create or a button that says ‘Download a template for automation’ which presents a JSON document with all of the parameters you entered within the wizard. This is an ARM Template.
ARM Templates which describe the creation of a resource can be used in a script or added to your personal Template Library which will catalogue all the resource creation guides you’ve saved within it. Microsoft also provides a collection of common blank templates. ARM Templates are very powerful and can be tweaked and customize to completely automate deployments. They can be organized by version, and changes can be noted alongside version numbers.
Learning goals:
Azure Advisor checks the resources you are using and provides cost saving recommendations. It applies common security checks to your machines and suggests common best practices and warns about vulnerable ports and addresses.
Azure Service Health provides information on how the Azure Service Itself is running and can be configured to send you alerts about how the services are running globally and inform you of any outages.
Azure Monitor is a centralized dashboard that allows you to monitor all of your Azure services. Monitors that check things frequently will incur a cost. You can write custom queries to check all of the data that is collected by Azure about your resources in the course of running them.
From the AZ-900 course, it became clear that not only could systems be spun up with Terraforms, etc, but also ARM templates. A section could be dedicated to findings and recipes for setting up cloud environments for legacy applications within Azure.
Key Concepts:
Literally just the Copy Activity.
Sources/sinks are too numerous to list, but the key ones I know how to use are as follows:
Here is the full list taken and modified from these docs.
true and another set of activities when the condition evaluates to false.The Execute Pipeline Activity enables the launching of a new pipeline from an initial pipeline.
This is for checking filesystems and querying file metadata. There is
a specific exists output that you can use.
ORDER BY ID and OFFSET with LIMIT
to do batches of 1000.Just like a lisp cond or C switch.
batchCount controls this, and
isSequential must be set to False for this to work, which is the default.Here is a subset taken directly from these docs.
The AZ-104: Microsoft Azure Administrator exam is an associate-level credential that certifies knowledge on the management of a corporate Azure subscription.
Candidates for this exam should have subject matter expertise in implementing, managing, and monitoring an organization’s Microsoft Azure environment, including virtual networks, storage, compute, identity, security, and governance.
An Azure administrator often serves as part of a larger team dedicated to implementing an organization’s cloud infrastructure. Azure administrators also coordinate with other roles to deliver Azure networking, security, database, application development, and DevOps solutions.
I am currently working towards this credential.
View the AZ-104 Study Guide.
Topics:
Section 8 in AZ-104 Udemy Course.
| Level | Distribution of Files |
|---|---|
| LRS | Locally-Redundant Storage |
| GRS | Geo-Redundant Storage |
| ZRS | Zone-Redundant Storage |
| GZRS | Geo-Zone-Redundant Storage |
| Access Tier | Use Case |
|---|---|
| Transaction Optimized | Transaction-heavy workloads with higher latency |
| Hot | Frequently accessed data |
| Cool | |
| Cold | |
| Archive | |
View the AZ-500 Study Guide.
Topics:
The AZ-305 course covers the design of cloud and hybrid solutions leveraging Azure.
This course has replaced AZ-303 and AZ-304.
On February 1st and 2nd 2023, I had the pleasure of attending the Azure Cloud Migration Summit hosted by IBM and Microsoft. I got to hear from teams working with many interesting private and public clients. Here are some of the important things I learned from the conference.
(All client information has been censored and/or removed.)
IBM’s Azure Migration Factory process has seen great success in replatforming, moderninzing, and re-architecting client applications as they move from on-prem to Azure.
Sudhir Jain gave a good talk on Industry 4.0 Solutions in Manufacturing with MAS. Smart factories can eternally stream data from sensors via Azure IoT to an Azure data lake to then be analyzed by PowerBI. IBM Maximo Visual Inspection Edge can be deployed to identify parts within picking systems and errors.
Eric Stoltze of Neudesic gave an excellent summary of some specific client wins in the USA and abroad.
Carmen Summers gave a good talk on Solution Plays to assist clients with their cloud migration journey.
Websites:
Books:
(Remember to buy physical copies of books to support good authors.)
Bill Wilder: Cloud Architecture Patterns: Using Microsoft Azure
John Savill: Mastering Microsoft Azure Infrastructure Services, 1e, 2015
Shijimol Ambi Karthikeyan: Practical Microsoft Azure IaaS, 1e
Using EMACS, it is easy to transform a study guide to a template for study notes.
To take a study guide and transform it to markdown headings:
M-x flush-lines ^$
M-x repla
Command History:
(flush-lines "^$" nil nil t)
(replace-regexp "^" "^# " nil 33712 38549 nil nil)
(replace-regexp "^# " "## " nil 33805 33997 nil nil)
Acronym: IaaS: Infrastructure as a Service: The cloud provides networked hardware but does not manage the software running on the systems. ↩︎ ↩︎
Acronym: PaaS: Platform as a Service: Provides a basis for common services or easy deployment of code while the cloud manages the underlying OS and maintaining/monitoring the running software, handles upgrades and automatic deployments, etc. ↩︎ ↩︎
Acronym: SaaS: Software as a Service: A managed application running in the cloud with no client control of the resources consumed by the application, deployment, etc. ↩︎ ↩︎
Data flowing out of a network ↩︎
Title: Azure
Word Count: 7882 words
Reading Time: 38 minutes
Permalink:
→
https://manuals.ryanfleck.ca/azure/