Azure Site Recovery – Ask Me Anything – Session Summary

On Tuesday, January 22, 2019, between the hours of 11:30 AM and 1:00 PM Eastern Standard Time (EST), the Azure Site Recovery (ASR) team hosted an “Ask Me Anything” (AMA) session on Twitter.

This is a session where individuals from the Azure Site Recovery Engineering team are available on hand to answer any questions we may have.

Similar to my Azure Backup – Ask Me Anything – Session Summary blog post, here is a summary of some of the responses from the Azure Site Recovery team, which provides insight into what’s coming in the not-too-distant future! I’ve tried to group/organize the points into categories as best I could, and highlighted keywords to help you skim the content.

PS: If you would like to review the session in full, questions asked, etc. here is the direct Twitter link.

Migrations

• Working on bringing a feature that will allow migrations without using on-premises licenses (i.e. without Software Assurance) and be able to pay for on-demand licensing (i.e. for Windows, SQL Server, etc.)
• During replication, compression is included to reduce the data transfer volumes and corresponding cost by up to 50% on average, although results may vary from workload to workload.

Replication

• In reference to a question about having to delete/restart replication just to change replication policies, storage types, vCenter, etc. the Azure Site Recovery team commented: “We are exploring opportunities to change the storage type without requiring to restart replication, but changing vCenter or Replication Policy are not in the plans”.
• At present, there are no plans to provide the ability to schedule when replications occur. This ask will be added to the future roadmap.
• The Azure-to-Azure replication scenario supports generating recovery points every 5 minutes. You can read more about this here: Replication policy
• There are currently no plans to provide an active-active (aka hot-hot, or live High Availability) replication feature in ASR.
• There are currently no plans to support using ASR to migrate on-premises Active Directory (AD) into Azure AD. This will be added as a requested feature on the backlog.
• Work is occurring to improve the failover readiness checks and will leverage the article (Prepare a Windows VHD or VHDX to upload to Azure) to enhance the capability.
• Additional details will be added to the documentation on Add Azure Automation runbooks to recovery plans, to facilitate a better way to view variables injected into automation scripts as part of a recovery plan.
• The feature/ability to hot-add/remove protected disks will be released in this quarter (Q1 CY2019) for the Azure-to-Azure scenario.
• ASR performs continuous replication, and so the required IOPS of the target disk should be the same for the source. This is why ASR restricts the replicated disks to have the same configuration on post-failover.
• The current Application Consistent replication recovery point only supports 60 minutes. There are no immediate plans to lower this limit, but it will be “taken as input” for the ASR team to consider. For sensitive systems like SQL Server, please review the following article: Set up disaster recovery for SQL Server.
• Since the majority of customer adoption is focused on the on-premises to Azure scenario, most of the innovation is going into that area. However, there are still improvements that are being shipped for the on-premises to on-premises scenario, including support for the latest OSes (i.e. Windows Server 2016), with Windows Server 2019 coming soon.
• The ASR team is continuously making replication more resilient to intermittent failures through collected telemetry. This is in response to the ask of having a self-healing feature (i.e. when connections to Storage accounts occur).
• 
  

Failover/Failback

• Managed Disks are supported fully for both failover and failback in the Azure-to-Azure and VMware-to-Azure scenarios. The support for HyperV-to-Azure is in the backlog
• The ASR team is investigating the scenario whereby, in an Azure-to-Azure failover, we can include a feature/ability to disconnect a VM from one Recovery Services Vault (RSV), and reconnect to another RSV (especially for Backup), so that data isn’t egressing from one Azure Region to another.
• ASR’s testing failover interval is currently defaulted to 180 days. There are no current plans to allow the customization of this, but this will be added as a feature request to the backlog.

Other

• There are plans and work being done to enhance the support that ASR has to include network resources (i.e. Load Balancers, Public IPs, Network Security Groups (NSGs), Application Security Groups (ASGs), etc.) in the coming milestones.
• Functionality is being worked on, to improve the Extension Update Settings feature, to allow the selection of an existing Azure Automation account.
• Work is underway to remove the limitation around the Resource Group that a Virtual Machine is in not being in the same geography as the VM itself, which causes ASR to reject the VM as a replication candidate.
  • This limitation only exists in the portal enablement method. Until the limitation is removed from the portal, customers can use the PowerShell method to enable replication in this type of scenario. See the following article: Set up disaster recovery for Azure virtual machines using Azure PowerShell
• Although not in the present plans, the ASR team will add the request to have “ASR Reports” (similar to the option currently available with Azure Backup), to the feature ask backlog.
• The ASR and Azure Backup team are exploring the possibility of integrating the 2 services together (similar to how Veeam has backup and DR in a single tool). This will hopefully allow us to replicate and backup data with a single data transmission versus multiple.
• Although Azure Migrate is a tool that focuses on the assessment of workloads, while using ASR for the actual migration activity, there are no plans to re-brand ASR to Azure Migrate or remove/break the relationship with Azure Backup via the Recovery Services Vault (RSV) service/platform.
• Enhancements to the Azure Monitor and Log Analytics solutions are being looked at, and are in the near-term roadmap for updates.
• Additional work is occurring to make the alerting more granular. The ASR team is interested in capturing some examples to ensure they validate all of the requirements.

Conclusion

A big THANK YOU goes out to the Azure Site Recovery team, and their efforts to answer everyone’s questions. I, for one, truly appreciate how interactive, engaged, and supportive this Microsoft team is; with its Clients, Partners, and their Partner’s Customers.

Mono vs Multi – Which Repo Structure Is Right For You?

Recently, while engaged on a large geographically dispersed and multi-team project, the question was posed: “Should we use a single (aka “mono”) repository for our Infrastructure-as-Code (IaC), or should we use multiple (aka “multi”) repositories?”

Code Repository Example

So, what’s the right answer?

The Hashi-Terraform Suggestion

Since this project was using HashiCorp’s Terraform templating language (along with the Terraform Enterprise product), the first thought was to find out what their recommendation was.

Terraform has an article in their Terraform Enterprise documentation, that discusses the recommendations for repository (aka “repo”) structures. You can read that article here.

Terraform Enterprise Integrations

I’m not going to copy/paste that whole article here, but here are a few highlights/points that it brought out worth noting:

• As a best practice for repository structure, each repository containing Terraform code should be a manageable chunk of infrastructure, such as an application, service, or specific type of infrastructure (like common networking infrastructure).
• Small configurations connected by remote state are more efficient for collaboration than monolithic repos, because they let you update infrastructure without running unnecessary plans in unrelated workspaces.
• Using a single repo attached to multiple workspaces is the simplest best-practice approach, as it enables the creation of a pipeline to promote changes through environments, without additional overhead in version control.
• If you’re using Branches and creating a branch for each environment… The upside of this approach is that it requires fewer files and runs fewer plans, but the potential downside is that the branches can drift out of sync. Thus, in this model, it’s very important to enforce consistent branch merges for promoting changes.
• If you are using Directories and creating a separate directory for each environment… The potential downside to this approach is that changes have to be manually promoted between stages, and the directory contents can drift out of sync. This model also results in more plans than the long-lived branch model, since every workspace plans on each PR or change to master.

So the summarizition from the Terraform information is: DON’T DO MONO-REPO!

The Rest of the World’s Thoughts

Now, of course, the Terraform approach is just one school of thought. Many other people have tried different approaches and different methods.

There is more than one path

I reached out to some friends of mine that were developers to get their thoughts on this debate. Here are some of the points they shared (either for or against mono-repo):

• When moving/modernizing an application from a monolith architecture to a microservices approach, common issues like upgrading/maintenance change. If you break down the system by domain and build it in smaller interchangeable parts, it is easier to replace those components. This applies, not only at an application coding-level, but also code grouping/segmentation when it applies to Infrastructure-as-Code (IaC) and code repositories.
• We’ve found difficulties with multi repos, especially ones that share dependencies between themselves. Having to bump repo versions and maintain dependencies across repos can be a pain.
• We found multi-repos to take much more time with regards to dev startup. And then when you have shared dependencies, there isn’t a clear way to keep them synched and accessible to all repos.
• There are trade-offs – one thing is I typically don’t allow my teams to use shared dependencies. It means your domain bounds are incorrect if you need to share dependencies (in most cases).
• We currently are using a monorepo…. I’m torn regarding repo-per-service or monorepo – there is deployment complexity with a monorepo as well as devs needing a local copy of a lot of code they’ll never touch. Multi-repos tend to force decoupling better, but also require devs to know what lives where … no perfect solution. With smaller projects, a monorepo is likely sufficient, or maybe one for backend and one for frontend.
• Another big aspect to work out is CI/CD. We use Jenkins and it’s not been without issue, especially when it comes to blue/green deployments. Start small and simple, optimise when needed, start monolithic and break out into microservices/split repos when needed.
• In addition to this, there are questions we need to answer such as how to manage multiple developers all contributing to the same code base. Our code base has roughly 50 people on it. To maintain consistency and keep the quality bar high we need a branching/merging/PR review strategy that’s not too strict but allows features to be developed without major merge conflicts.
• On the development side, we kind of moved away from one-service-per-repo to a single mono-repo per distinct application to ease dev environment setup pain. I know we’re not alone (Why Google Stores Billions of Lines of Code in a Single Repository).
• And another interesting article: You too can love the MonoRepo
  • The monorepo changes the way you interact with other teams such that everything is always integrated. And hey, our industry has a name for that: continuous integration. If you don’t have a monorepo, you’re not really doing continuous integration, you’re doing frequent integration at best. In a monorepo, even pre-commit testing is already integrated.
  • Adopting a monorepo means every change can be integrated. If you believe in Continuous Integration (a requirement for Continuous Delivery) then you should seriously think about using a monorepo, and a build system that helps you keep up with the greater amount of testing.

Conclusion

So what’s the right answer?

Well, there isn’t one definitive right answer. The answer is, what’s right for you/your organization?

Sounds Right

I think you need to look at what you’re ultimately trying to achieve. That should also include what teams are involved, the separation of work/responsibility areas, etc. But then, we have the ever-growing push/chant of “DevOps” and the unification of Developers with Operations.

Really, what is the root-cause/reason for multi-repo? Is it to separate the netowrking team’s code from the security team’s, etc.? That can be done through workspaces, directory structure, etc.

As I mentioned in the My Terraform Resources List (for those that insist on being “cloud agnostic”) article I recently wrote, I am working on a long-term global scale project for defining an entire enterprise-grade Azure environment, completely through Infrastructure-as-Code (IaC) using Terraform, and Configuration-as-Code (CaC) using Chef.

In a large scale approach like that, which option is the best choice? From my current (inexperienced) perspective, I’m thinking a mono-repo is the right approach, at least for the foundation/fabric components (i.e. network, Express Route circuits, NSGs/ASGs/Rules, Domain Controllers, Policies, etc.), whereas having separate repositories for application-specific components may work best.

But, we’ll see. And as I progress along on this project, we may have to re-visit this topic entirely! Stay tuned.

My Terraform Resources List (for those that insist on being “cloud agnostic”)

Over the past several months, I have been ramping up on the Infrastructure-as-Code (IaC) templating language called Terraform for a project that I am engaged in.

And while I do not have anything against Terraform (after all, it IS very easy to read), I have stated my opinions on the whole “cloud agnostic” statement via my previous articles: Terraform Is NOT the Cloud Agnostic Platform You Think It Is! and Move Over Terraform, You’re Not The Only Code In Town!

Since I am not on this project as a one-man-show (i.e. I have a team of people I’m working with), I started to put together my list of resources that I’ve found most useful and helpful in this endevour.

And so, since fellow MVP Ned Bellavance tagged me in a tweet as others were asking for references for good books, I thought I would oblige and throw my list up for everyone to enjoy and use.

NOTE: I have general resource links I’ve already put up on my blog under Resources. And, whenever I read a technical book, I post a book review with all my notes/highlights from the publication.

Now, onto my Terraform specific resource list. Note that these resources are not listed in any particular order, and I just try to group them into categories for easier reference.

Articles

These are the articles that I’ve found most useful/helpful in learning about Terraform (and I’ve read a lot).

• Running Terraform in Automation
• Using Terraform with Azure – What’s the benefit?
• Terraform Best Practices
• How to create reusable infrastructure with Terraform modules
• A Comprehensive Guide to Terraform
• 7 Tips to Start Your Terraform Project the Right Way
• Terraform Enterprise – Repository Structure
• Terraform Recommended Practices

Books

These are the books that I’ve found most useful/helpful in learning about Terraform. And if you know me personally (or check out my Technical Book Reviews page), you know that I read a lot of books.

• Terraform Up & Running book
  • My book review (aka highlights of points I found of value and use), can be found here: https://adinermie.com/resources/technical-book-reviews/book-review-terraform-up-running-writing-infrastructure-as-code/
  • NOTE: I am also planning to purchase 2^nd edition coming out in June 2019. Look for a new review thereafter.
• The Terraform Book
  • My book review (aka highlights of points I found of value and use), can be found here: https://adinermie.com/resources/technical-book-reviews/book-review-the-terraform-book/

Pluralsight Courses

Even though I have a separate section for “Videos” I want to specifically call out the Pluralsight courses by Ned Bellavance.

• Terraform – Getting Started
• Deep Dive – Terraform

Hands-On Labs

In learning anything new, there is only so much you can learn/pick up from reading alone. That’s why I also appreciate it when I find a good hands-on lab guide to help re-enforce my understanding and learning.

One thing that I especially appreciate about the Azure Citadel site, is that the workshops are not just a step-by-step hold-your-hand type of guide. They start out that way to get you started, but very quickly, they change to “this is what you need to do, go figure out how to do it”. I like this, because it forces you to research and learn, and not just copy/paste.

I’ve even had the privilege of directly contributing to this specific hands-on lab, via Richard Cheney requesting a pre-publish review of the content before it’s released.

• Terraform on Azure (Hands-On Lab)

Videos

Finally, the videos list. This is in addition to the Pluralsight courses themselves (as not everyone may have access or a subscription to Pluralsight).

• Evolving Your Infrastructure with Terraform
• Happy Terraforming! Real-world experience and proven best practices
• HashiCorp Terraform adoption: A typical journey
• HashiConf 2018 Day Two Keynote: Terraform is Changing the World
• Scaling with Terraform: The journey from startup to enterprise
• 5 Lessons Learned From Writing Over 300,000 Lines of Infrastructure Code

Conclusion

My journey with Terraform is just getting started. In fact, the project that I am currently engaged on is estimated to last 2 years; and it’s all about architecting and deploying (and managing, maintaining, etc.) and entire global Enterprise environment using Infrastructure-as-Code (IaC) via Terraform! So, check back on my blog regularly, follow me on Twitter, and connect with me on LinkedIn, as I’ll definitely be writing about my real-world hands-on experience in this space.

PS: If any of you find other “bookmark worthy” resources that you think should be added to this list, please do reach out, and I’ll update it accordingly.