As Microsoft evolves its data center strategy, it is increasing its use of fault tolerant software platforms like Azure which were built to run on large clusters of commodity hardware, rather than continuing to invest in redundant hardware and power systems. The idea is operationalize the response to failures by automatically and seamlessly recovering when services fail. This can be achieved by replicating state among various machines, eliminating dependencies between software/hardware components, adding instrumentation and run-time telemetry, and automating responses to failures. The system must also be simple; using design patterns that are well understood, but simple enough so that the system can be easily triaged. By adopting these principles, Microsoft is reducing the complexity of its data centers and improving its TCO considerably. It's also less likely that a meteor strike or the next superstorm will cause an outage!
You can read more about Microsoft's cloud scale data centers at http://www.globalfoundationservices.com/posts/2013/february/11/software-reigns-in-microsofts-cloud-scale-data-centers.aspx.
Enterprises can achieve similar results by adopting some the aforementioned principles. For instance, when building new applications developers should work alongside operations to examine the ramifications their design decisions have on the underlying infrastructure and work together to design the application to be elastic, self-healing, and fault-tolerant. Too often, developers build their applications in isolation; only to throw it over the wall to operations who then have to manage to an SLA that they didn't define. Having shared goals that are designed to optimize the whole stack will incent these different groups to work together which in turn should lower TCO and improve time to recovery.
As for hardware redundancy, once the software becomes resilient, there's less need for redundant hardware. This is evident in Microsoft's latest data centers where parts of it aren't backed up by generator and the UPS only lasts long enough to move the load elsewhere. It's this simple design that is helping Microsoft lower its CapEx and OpEx with each generation.
Showing posts with label Azure. Show all posts
Showing posts with label Azure. Show all posts
Friday, February 15, 2013
Wednesday, January 16, 2013
Azure Internals
For my first post, I thought I would try tackling how Windows Azure works and how your applications are deployed in the data center. Unlike a normal operating system which is responsible for managing local resources, Windows Azure manages a pool of compute, storage, and networking resources within the data center. It also exposes a set of building blocks that developers can use to build distributed applications that are both highly available and easily scalable. Azure is able to offer high availability because there are multiple instances of your service running on servers in different fault and update domains which I'll explain here shortly. The services you develop are assumed to be stateless; if you need to store state, you store it in Windows Azure BLOB or table storage.
When you need to update your service, Windows Azure will connect to each instance of your service in series while it is running and replace it with the newer version. This is vastly different from IaaS where you would ordinarily have to do that yourself. Moreover, because there are no dependencies between the underlying OS and your service, we're able apply updates and patches without effecting the availability of your service.
An Azure service can be comprised of different roles types, for example, there's a web role which is essentially an instance of IIS; there's also a worker role that you can use to execute your logic. These roles are similar to DLLs that runs in a VM instance. For you to qualify for the SLA, you need to provision at least 2 instances of your role and these roles have to be deploying across different fault and update domains which you can specify or have the Azure Fabric Controller do for you.
Having fault and update domains is how we meet our published SLAs for Windows Azure. Fault domains are for unplanned outages whereas update domains are for maintaining availability during planned service updates.
When your updates are applied, Azure coordinates them among the various instances of your service in sequence such that the availability of your service is not effected. There may be reduced capacity because there are fewer instances your service running, but it's still available. The important thing to realize here is as you apply updates to your service, there may mixed versions of a role, for example, some of your worker roles may be running an older version as others are being updated. This is why Azure allows you to perform manual updates of the roles in different update domains. Another way to perform updates is to do a VIP swap which involves creating 2 instances of your service and telling the load balancer to swap VIPs.
A fault domain is a single point of failure. For Azure, a rack is a fault domain with each rack consisting of about 40 servers or blades. When a fault that affects a running instance of your service occurs, Azure will automatically try to replace it with a new instance in another fault domain. It will also tell the load balancer to stop sending requests to the failed instance.
After your done writing your service, you upload it to the Azure through the Windows Azure portal. From there it is sent to the RDFE (Red Dog Front End) which uses the service configuration to determine which region to deploy your service into. Your service is then passed to a Windows Azure Fabric Controller (FC) which is responsible for provisioning your service on to servers within a stamp where a stamp is a large collection of servers in racks, along with TOR switches, and the like.
The FC is a Azure service that runs on multiple servers within a stamp. This service is aware of the hardware and network topology within the data center and it's what is responsible for managing the lifecycle of your service. When a stamp is being provisioned, the FC will send a command to the in-rack PDU to power on the server. The server then PXE boots Windows PE which formats the disk and downloads a VHD of the Windows Azure OS. After completing sysprep, the server tells the FC that it is ready to start accepting new applications.
The FC uses a algorithm that looks at resource availability across fault and update domains within the stamp to determine which servers to deploy your roles onto. The algorithm automatically attempts put different tiers of your service on the same server, e.g. 1 instance of a worker role and 1 instance of a web role, which simplifies how we do updates but also improves performance because communication between the 2 roles doesn't have to traverse the network.
By now you should have a better sense of how Azure provides high availability and what happens after you upload your service to the Azure portal. The advantage to using Azure and other PaaS offerings is that rather than having to manage the underlying operating systems and building middleware services like queuing yourself, developers can use the higher-order service that we expose to quickly build distributed applications that scale easily. And because it's sold under a utility consumption model, it's also a great incubator for new ideas. This is particularly important nowadays given roughly 75% of an IT budget still goes towards maintaining existing applications which leaves very little for experimentation and the risk associated with it. With PaaS, if your service doesn't attract users or usage diminishes over time, you can de-provision instances of it or de-provision it all together.
When you need to update your service, Windows Azure will connect to each instance of your service in series while it is running and replace it with the newer version. This is vastly different from IaaS where you would ordinarily have to do that yourself. Moreover, because there are no dependencies between the underlying OS and your service, we're able apply updates and patches without effecting the availability of your service.
An Azure service can be comprised of different roles types, for example, there's a web role which is essentially an instance of IIS; there's also a worker role that you can use to execute your logic. These roles are similar to DLLs that runs in a VM instance. For you to qualify for the SLA, you need to provision at least 2 instances of your role and these roles have to be deploying across different fault and update domains which you can specify or have the Azure Fabric Controller do for you.
Having fault and update domains is how we meet our published SLAs for Windows Azure. Fault domains are for unplanned outages whereas update domains are for maintaining availability during planned service updates.
When your updates are applied, Azure coordinates them among the various instances of your service in sequence such that the availability of your service is not effected. There may be reduced capacity because there are fewer instances your service running, but it's still available. The important thing to realize here is as you apply updates to your service, there may mixed versions of a role, for example, some of your worker roles may be running an older version as others are being updated. This is why Azure allows you to perform manual updates of the roles in different update domains. Another way to perform updates is to do a VIP swap which involves creating 2 instances of your service and telling the load balancer to swap VIPs.
A fault domain is a single point of failure. For Azure, a rack is a fault domain with each rack consisting of about 40 servers or blades. When a fault that affects a running instance of your service occurs, Azure will automatically try to replace it with a new instance in another fault domain. It will also tell the load balancer to stop sending requests to the failed instance.
After your done writing your service, you upload it to the Azure through the Windows Azure portal. From there it is sent to the RDFE (Red Dog Front End) which uses the service configuration to determine which region to deploy your service into. Your service is then passed to a Windows Azure Fabric Controller (FC) which is responsible for provisioning your service on to servers within a stamp where a stamp is a large collection of servers in racks, along with TOR switches, and the like.
The FC is a Azure service that runs on multiple servers within a stamp. This service is aware of the hardware and network topology within the data center and it's what is responsible for managing the lifecycle of your service. When a stamp is being provisioned, the FC will send a command to the in-rack PDU to power on the server. The server then PXE boots Windows PE which formats the disk and downloads a VHD of the Windows Azure OS. After completing sysprep, the server tells the FC that it is ready to start accepting new applications.
The FC uses a algorithm that looks at resource availability across fault and update domains within the stamp to determine which servers to deploy your roles onto. The algorithm automatically attempts put different tiers of your service on the same server, e.g. 1 instance of a worker role and 1 instance of a web role, which simplifies how we do updates but also improves performance because communication between the 2 roles doesn't have to traverse the network.
By now you should have a better sense of how Azure provides high availability and what happens after you upload your service to the Azure portal. The advantage to using Azure and other PaaS offerings is that rather than having to manage the underlying operating systems and building middleware services like queuing yourself, developers can use the higher-order service that we expose to quickly build distributed applications that scale easily. And because it's sold under a utility consumption model, it's also a great incubator for new ideas. This is particularly important nowadays given roughly 75% of an IT budget still goes towards maintaining existing applications which leaves very little for experimentation and the risk associated with it. With PaaS, if your service doesn't attract users or usage diminishes over time, you can de-provision instances of it or de-provision it all together.
Subscribe to:
Posts (Atom)