Crossing the workspaces with OpenSharing

One workspace was pretty easy, two was mangeable, but dozens of Databricks workspaces and their catalog assignments becomes painful and unwieldy. I have seen a pattern a few times where workloads in workspace A require access to a schema or tables from catalog bravo. The natural tendency is expose bravo into workspace A and move on. Then another workload requires access to a table from charlie, and then echo, and then foxtrot, and so on.

The default answer of grant broad catalog access everywhere, scales poorly. Every workspace ends up seeing every catalog, the blast radius of a misconfigured grant grows, and development workspaces end up with far more visibility into production data than they should.

The approach I have adopted started as a misunderstanding.

OpenSharing, formerly "Delta Sharing", was being used to expose data from a legacy account into a new account which is itself a great way to do platform or cloud migrations. I misunderstood a recommendation and started designing around workspace to workspace shares within a single account for data access needs. Oops!

OpenSharing inside a single account lets a provider workspace publish a share: a read-only collection of tables that a recipient workspace mounts and queries in place. No data is copied, no catalogs are exposed wholesale, and access is governed explicitly per share and per recipient.

What OpenSharing buys you

• Cross-workspace sharing without catalog sprawl. A share exposes only the tables you put in it, not the catalog they came from. Recipient workspaces see a clean, read-only surface and nothing else.
• No data copies. Recipients query the underlying Delta tables directly. The data stays in one place, so there is no second copy to secure, keep fresh, or pay to store.
• Scoped access to production data in development. Combine a share with a Unity Catalog view, including dynamic views that filter rows and mask columns, and you can hand a development workspace a limited or masked window onto production data, still without copying anything.
• Works within and across accounts. Within one Databricks account, Databricks-to-Databricks sharing handles authentication automatically with no tokens to manage. Across accounts, or to non-Databricks consumers, the open sharing protocol carries the same data over a credentials file.

The pattern of OpenSharing across workspaces replaced the previous approach I used before Unity Catalog where I would register the same external locations into different catalogs to allow read-only visibility into a schema where needed. Unity in contrast disallows multiple External Locations pointing to the same URL reference(s) since their design is explicitly trying to have one single entity for governance of a data asset.

That limitation in Unity makes sense in the context of Unity, but having the ability to create N pointers to schemas or tables where needed was previously absent from my Unity toolbox.

Configuring a share with Terraform

The Databricks Terraform provider models the whole flow with three resources: databricks_share for the share itself, databricks_recipient for the consumer, and databricks_grants to give the recipient SELECT on the share.

On the provider side, create the share and add the tables you want to publish:

resource "databricks_share" "prod_metrics" {
  name = "prod_metrics"

  object {
    name             = "prod.analytics.daily_revenue"
    data_object_type = "TABLE"
  }

  object {
    name             = "prod.analytics.customer_rollup"
    data_object_type = "TABLE"
  }
}

Each object block names a table by its fully qualified catalog.schema.table name. To publish an entire schema, including tables created in the future, use a SCHEMA object:

resource "databricks_share" "analytics" {
  name = "analytics"

  object {
    name             = "prod.analytics"
    data_object_type = "SCHEMA"
  }
}

Next, define the recipient. For Databricks-to-Databricks sharing within or across accounts, set authentication_type to DATABRICKS and point at the recipient's global metastore ID (the format is <cloud>:<region>:<guid>):

resource "databricks_recipient" "dev_workspace" {
  name                = "dev-workspace"
  comment             = "Development workspace consumer"
  authentication_type = "DATABRICKS"

  # global metastore id of the recipient: <cloud>:<region>:<guid>
  data_recipient_global_metastore_id = "aws:us-west-2:0000...guid"
}

Then grant the recipient SELECT on the share:

resource "databricks_grants" "prod_metrics" {
  share = databricks_share.prod_metrics.name

  grant {
    principal  = databricks_recipient.dev_workspace.name
    privileges = ["SELECT"]
  }
}

For a recipient that is not on Databricks, or one whose metastore you do not control, use TOKEN authentication instead. The provider generates a one-time activation URL the recipient uses to retrieve a credentials file:

resource "random_password" "sharing_code" {
  length  = 16
  special = true
}

resource "databricks_recipient" "external" {
  name                = "external-consumer"
  comment             = "External analytics partner"
  authentication_type = "TOKEN"
  sharing_code        = random_password.sharing_code.result

  ip_access_list {
    allowed_ip_addresses = ["203.0.113.0/24"]
  }
}

On the recipient side, the shared tables appear as a read-only catalog that can be queried like any other: SELECT * FROM prod_metrics.analytics.daily_revenue resolves against the live data in the provider's workspace.

Limited and masked views over production

Another powerful pattern is sharing a view rather than the raw table. Define a Unity Catalog view in the provider workspace that projects only the columns development should see, or masks sensitive values, then put the view in the share instead of the base table:

CREATE VIEW prod.analytics.customer_rollup_masked AS
SELECT
  customer_id,
  region,
  -- mask PII: keep only the domain of the email
  concat('***@', split(email, '@')[1]) AS email_masked,
  total_spend
FROM prod.analytics.customer_rollup;

resource "databricks_share" "dev_safe" {
  name = "dev_safe"

  object {
    name             = "prod.analytics.customer_rollup_masked"
    data_object_type = "VIEW"
  }
}

Dynamic views go further, filtering rows or masking columns based on who is querying, so a single share can serve different recipients different slices of the same underlying table. The development workspace gets realistic, current data for testing, and production keeps full control over exactly what leaves.

Sharing views is not limited to Databricks recipients either. Databricks' cross-platform view sharing lets consumers query shared views through open Delta Sharing clients (Apache Spark, Pandas, Power BI, Tableau, and others), so a masked or limited view can reach teams that are not on Databricks at all without giving them the underlying tables.

Neat!

With OpenSharing the production data stays in the provider workspace, development and partner workspaces consume read-only shares, and every share is an explicit, Terraform-managed contract about exactly which tables or views cross the boundary.

You get cross-workspace access without catalog sprawl, without copies drifting out of sync, and without granting development more of production than it needs.

If you are untangling cross-workspace data access or standing up a governed sharing layer across Databricks accounts, send us a note.