• AWS Melbourne Region inching closer

    Scrolling through the AWS annals of 2020 – the AWS News Blog, we find an almost forgotten announcement, bringing good news to Australian shores – “In the Works – AWS Region in Melbourne, Australia”. The new Melbourne region is expected to open with the 3 Availability Zone standard as expected, promising that all the latest and greatest AWS services will be able to operate in the region.

    Almost two years later, we are eagerly awaiting the launch of the second AWS Region in Australia, but just how close are we really to opening the “API-doors”. Looking into a few tell-tail signs, the region identifier has been created, and after some homework it should be labelled the “ap-southeast-4” region.  The AWS IP-ranges for this “ap-southeast-4” region have been published, and we have some API’s responding to Ping requests.

    Pinging s3.ap-southeast-4.amazonaws.com [52.95.168.21] with 32 bytes of data:

    Reply from 52.95.168.21: bytes=32 time=17ms TTL=48

    Based on the homework, the Melbourne region launch date is close.

    Assessing the current options

    In Australia, till date, the decision to use Amazon Web Services is straight-forward, there is one region to choose from Sydney (ap-southeast-2). Understanding the Availability Zone (AZ) architecture that make up an AWS Region, for most organisations this is more than they can achieve with most on-premises datacentre providers and there is no need to think about it any further as another region may not offer additional operational peace of mind.

    The only slight downside is the limited geographic location choices you were presented with, namely (AZ-1) Sydney or (AZ-2) Sydney or (AZ-3) Sydney, basically Sydney. Organisations, Public Sector Agencies, or Federal Government services with majority or exclusive customer base situated in Victoria (Melbourne), or perhaps even throwing South Australia (Adelaide) into the mix had little choice but to accept the 800–1,600Km telecommunications cables (fibre, etc.) and the latencies tied to those distances. Depending on the use case and with a bit of Amazon CloudFront magic (which offers a point of presence in Melbourne) for most organisations this still meets the requirements.

    A second and common scenario for larger organisations and agencies with compliance requirements or specific architectural design principles is Disaster Recovery (DR), not just high-availability/fault-tolerance, but true Disaster Recovery whilst meeting data residency and data sovereignty controls. To clarify this statement, a quick look at the definition of each:

    • High Availability (HA) – refers to a system or component that is continuously operational for a desirably long period.
    • Disaster Recovery (DR) – involves a set of policies and procedures to enable the recovery or continuation of vital infrastructure and systems following a natural or human-induced disaster.

    Although it is considered very low risk to lose the full AWS Sydney region with all data being irrecoverable – an outage of the Sydney region for days or even weeks (highly unlikely) have always been noted by most of my customers somewhere on a risk register – hoping it will never happen. Data backups for critical databases to a different region are commonly considered as a last resort, but encryption and Australian data-residency concerns most often than not cloud the architectural decisions and added complexity to such a point that it is just not done.

    Let’s explore how do we see the AWS Melbourne region coming to the rescue and solving the above concerns and what are the options to easily establish an improved AWS Cloud footprint.

    Multi-Region Disaster Recovery and Failover

    First prize for our board of directors and risk register is a multi-region Disaster Recovery (DR) deployment that exactly or closely matches the primary region footprint, and include continuous or periodic data backups keeping the secondary AWS region in sync (Melbourne to Sydney, or Sydney to Melbourne). With two regions now operating in Australia, data residency and data sovereignty now also gets the big green tick and removes the cloud of doubt behind the governance-closed door.

    Such DR-deployments can take an Active-Active or Active-Passive deployment approach, each offering different Recovery Point and Recovery Time Objectives – and at different price points. An alternative to Active-Active Disaster Recovery deployments is sometimes referred to a Pilot Light DR site, which offers a quicker and mostly automated recovery at a lower price point than a full Active-Active DR-deployment.

    Disaster Recovery deployments briefly discussed above are no easy undertaking for larger enterprises or complex workloads, and normally require a 6-month project for each large-complex workload when building it from the ground up. The costs to operate one-to-one or closely matching DR-environments in the public cloud are more cost effective and simpler to provision and operate than in traditional data centres, with one caveat – the distinct number of AWS Services in use that store and maintain data – especially at high transaction rates. This article is not intended to explore the depths of complex-workload DR decisions and strategies; however, a callout is that the more distinct services that act as datastores, the more complex the DR strategy will be to design for fail-over, and fail-back.

    Prior to embarking on a Multi-Region Disaster Recovery journey, how best do organisations make use of the AWS Melbourne region? Starting slow and building from there is a good start, with cross region backups perhaps the first step to take.

    For more information on the various strategies refer to the Disaster recovery options in the cloud Well-Architected recommendations, or refer to the additional resources below.

    Cross-Region backups with AWS Backup

    Cross-Region backup is an efficient and almost immediately deployable option that will offer peace of mind and tick at least one box on most risk registers. AWS Backup offers advanced backup features, phased retention policies, and further supports Cross-Region and Cross-Account backups out of the box and can be setup in minutes.

    AWS Backup does not support every AWS service now, however the list of supported services is growing rapidly. At present the following services are supported by AWS Backup: Amazon Simple Storage Service (S3), Amazon Elastic Block Store (EBS), Amazon FSx, Amazon Elastic Compute Cloud (EC2), Amazon Relational Database Service (RDS), Amazon DynamoDB, Amazon Elastic File System (EFS), AWS Storage Gateway, Amazon Neptune, Amazon DocumentDB (with MongoDB compatibility)

    Cross-Region and Cross-Account Security

    An additional region on Australian shores offers two key benefits as highlighted above, multi-region data backup that meet data residency requirements, and secondly true active-active DR capabilities with large geographic isolation zones. A general guide and best practice to follow for setting up such a Disaster Recovery solution is to position the design principles as early as possible so that the Disaster Recovery deployment or Cross Region Backups are done within a different target AWS account than the primary workload. Advanced cyber attacks directed at cloud service provider customers and accounts have become superior in the strategies applied – using exposed highly-privileged credentials to delete backups with an API-call, encrypt virtual machine disk volumes, etc. and leave you with a request to transfer some bitcoin for the decryption key.

    Keeping a Chinese-wall so to speak between primary workload accounts and the DR or Backup Account + Region, with least-privileged IAM-roles governing cross-account activities will set you up for cyber security excellence.

    Note: You cannot share backups (snapshots, etc.) encrypted with your default KMS key with another account. Resources encrypted under a service default KMS key (aws/***) can only be shared within the same account. Before you embark on this journey – be sure to assess the encryption keys in use as a customer managed key re-encryption project may be required set you up for long term success. In addition, sharing AWS Backup backup plans across account, require the AWS accounts to be part of the same AWS Organization.

    Workload Migration to Melbourne

    Should a latency or state government compliance requirement drive the decision to move workloads to the Melbourne region, think ahead if an Active-Active or Active-Passive DR-strategy may be a future budget or roadmap item. A good strategy for such a regional migration is through a workload-by-workload transition roadmap, setting up each workload as an Active-Active (or Active-Passive) disaster recovery solution within the Melbourne region. Once configured, initiate a fail-over to the Melbourne region (or soon the Auckland region). Given a bit of planning to a fall-back to the Sydney region, you now have a fully operational cross-region Disaster Recovery strategy and solution in place, whilst migrating your workloads between regions at the same time. Mileage may vary based on the complexity of the workloads, however considering a business case positioning the cost-benefit, whilst mitigating data and service loss risks on the risk register should position a win-win on any committee vote.

    What to expect on launch day

    Before we pop the cork and start migrating or setting up Disaster Recovery in the Melbourne region, due to ever growing number of services offered by AWS not everything is expected to be available on day one. Depending on the region and expected services demand, AWS may surprise us, however some advanced services like Aurora and Control Tower do not always appear on the first launch-stage. The good news is, these gaps are normally filled quickly, and should you have a specific urgent need, discuss this with your AWS Partner or AWS Technical Account Manager – your use case may adjust service launch priorities from the AWS team.

    Once the Melbourne regions launches, have a close look at the AWS Regional Services List to determine what will be at your disposal on day one. With AWS Backup requiring several advanced, cross region and cross account encryption capabilities – we surely hope and can only encourage AWS to offer AWS Backup as a day one service.

    The AWS Melbourne region also unlocks AWS for Western Australia (WA) – not with all the bells and whistles, but there will be a fully infrastructure supported pattern for Perth based organisations to adopt Amazon Web Services, or larger organisation to extend their footprint closer to their Western Australia customer base. Have a look at this article in which we explore the options that AWS Local Zones brings to the Australian Public Cloud table.

    Additional resources

    A short list of resources to support you in your multi-region workload and backup journey.