Migrate Databricks from Azure to AWS

Azure Databricks is often the path of least resistance for organizations with a Microsoft estate. Over time, many teams discover that their data gravity, their infrastructure budget, and the Databricks product roadmap all point towards AWS. Buoyant Data helps organizations move their Databricks platform from Azure to AWS without disrupting the analysts, pipelines, and applications that depend on it every day.

Why AWS for Databricks?

Lower infrastructure cost

Compute is the dominant line item in any Databricks deployment, and AWS offers more levers to pull than any other cloud:

• EC2 Spot instances are available at up to a 90% discount compared to On-Demand prices. Spark workloads tolerate interruption gracefully, and Databricks' cluster management makes Spot practical for a large share of data engineering and CI workloads.
• AWS Graviton processors delivered up to 3x better price performance for Databricks clusters when support launched, and typically around 20% compute savings on everyday Spark workloads compared to equivalent x86 instances. ARM-based compute simply is not an option for Azure Databricks.
• Committed spend goes further. Databricks on AWS can be purchased through the AWS Marketplace, which means Databricks consumption can draw down existing AWS committed-spend agreements instead of negotiating a separate contract.
• Instance-for-instance, EC2 list prices are typically lower than their Azure VM equivalents, before any of the discounts above are applied.

The effect is not theoretical: organizations that have replatformed their analytics from Azure to Databricks on AWS have reported infrastructure cost reductions of around 40%.

The most complete Databricks platform

AWS was the first cloud Databricks launched on, and it remains the most battle-tested deployment. New platform capabilities reach general availability on AWS first — serverless workspaces, for example, were available on AWS long before arriving on Azure. On AWS, the platform is also operated directly by Databricks rather than delivered as a third-party service, which means fewer layers between your team and the people who build the product.

Because the Databricks layer — Delta Lake, Unity Catalog, MLflow, Databricks SQL, and the notebook experience — is identical across clouds, code developed on Azure Databricks runs on AWS with only configuration changes. The platform is portable; the question is which cloud gives it the best foundation.

Amazon S3, the home of the lakehouse

S3 is the de-facto standard storage layer for the lakehouse, offering 11 nines of data durability and 99.99% availability by default, and the broadest ecosystem of compatible tools in the industry. For organizations whose data already lands in S3 — from application logs, IoT fleets, or SaaS exports — moving Databricks to AWS means moving compute to the data rather than paying to move data to the compute.

A deeper toolbox around the lakehouse

AWS offers the widest selection of compute profiles available to Databricks: memory-optimized instances for joins, storage-optimized instances for heavy shuffles, GPU instances for model training, and custom silicon such as AWS Trainium for machine learning price-performance. Around the lakehouse, Databricks on AWS integrates natively with the services data teams actually use: Kinesis and MSK for streaming, Glue and Athena for interoperability, Redshift for warehousing, Amazon Bedrock for foundation models, and PrivateLink for secure connectivity.

Lower AI inference costs with Amazon Bedrock

AI inference is rapidly becoming the second-largest line item on data platforms, and always-on GPU endpoints are the most common way to overspend on it. On AWS, Databricks' external-model support lets notebooks, SQL queries, and agents call Amazon Bedrock directly — pushing inference onto Bedrock's pay-per-token infrastructure with no standing endpoints to provision, scale, or pay for while idle. A broad choice of foundation models from providers such as Anthropic is available behind the same endpoint, while the data stays governed inside your lakehouse. Batch enrichment jobs, document processing, and RAG pipelines can consume Bedrock at per-token prices instead of holding GPU capacity around the clock, and AWS's custom Trainium silicon continues to drive those prices down. Azure Databricks has no equivalent native option: teams end up operating their own GPU serving infrastructure or bridging to external providers by hand.

What a migration actually involves

A cloud-to-cloud Databricks migration is very achievable — but it is not a lift-and-shift of a single virtual machine. A typical engagement covers:

• Workspace assets: notebooks, jobs, cluster policies, instance pools, dashboards, and repo configurations exported, remapped, and imported into the new workspace.
• Compute remapping: Azure VM sizes (Standard_DS3_v2 and friends) mapped to the right EC2 instance families — often Graviton-based — with cluster policies and pools re-tuned for AWS pricing.
• Data migration: ADLS Gen2 storage copied to S3, with Delta Lake's open format preserving table integrity, and Unity Catalog storage credentials and external locations redefined for IAM-based access.
• Identity and security: Entra ID-backed users and groups reconnected through your identity provider with SSO and SCIM, service principals replaced by IAM roles, and secret scopes re-pointed at AWS-native secret stores.
• Networking: VNet-injected workspaces re-established as VPC-based deployments with PrivateLink, transit, and egress controls matched to your security posture.
• Validation and cutover: job-by-job comparison runs, data parity checks, and a staged cutover plan that keeps business users productive throughout.

How we work

As a small consultancy, our objective is to set your team on the right path rather than construct a long-running engagement. A typical migration proceeds in phases:

1. Assessment: inventory of workspaces, jobs, data volumes, and dependencies, producing a migration plan and an AWS cost model you can validate before committing.
2. Pilot: one representative workload migrated end-to-end to prove the tooling, the cost model, and the runbook.
3. Migration waves: production workloads moved in prioritized waves, with validation gates between each wave.
4. Handoff: your team operates the new platform with documentation, runbooks, and the knowledge transferred along the way — with Buoyant Data available for follow-up support as needed.