Stack SpendDocs

Databricks

Connect your Databricks account to track DBU spend by workspace, SKU, and product — jobs, SQL warehouses, DLT, and model serving — in StackSpend.

Databricks logo

Databricks

Billing system tables — service principal recommended

The Databricks provider connects StackSpend to your account’s billing system tables (system.billing.usage and system.billing.list_prices) through a SQL warehouse, using a read-only service principal. StackSpend requests up to 90 days of available history on first sync, then refreshes recent usage on scheduled syncs.

What StackSpend tracks

  • DBU spend per workspace and SKU at USD list price
  • Jobs, all-purpose, and serverless compute
  • SQL warehouses and Lakeflow/DLT pipelines
  • Model serving, vector search, and other AI products
  • Up to 90 days of available history; provider reporting delays apply
Note.Billing system tables are account-wide: they cover usage from every workspace in your Databricks account, so you only connect one workspace. Costs are computed as DBU quantity × the USD list price from system.billing.list_prices; if you have a negotiated discount, set the optional Discount % on the connection — this connector applies one uniform discount assumption rather than importing negotiated invoice rates.

Prerequisites

  • Workspace admin access (to create the service principal) and the metastore admin role for the system-table grants — account admin alone is not enough (see the note in step 2)
  • Unity Catalog enabled — the system.billing tables are part of Unity Catalog system tables
  • A serverless or pro SQL warehouse in that Unity Catalog-enabled workspace that StackSpend can use for the daily billing query (choose an appropriately sized warehouse and auto-stop policy; query execution can incur charges). Serverless warehouses start in seconds, so Test Connection doesn’t wait on a cold start.

Setup

1

Create a service principal and secret

In your Databricks workspace, go to Settings → Identity and access → Service principals → Add service principal. Name it (e.g. stackspend) and copy its Application ID — this is the Client ID in StackSpend.

Open the service principal, go to Secrets → Generate secret, and copy the Client Secret immediately — Databricks shows it only once.

2

Grant read access to the billing tables

In a SQL editor, as a metastore admin, run the grants below with your service principal’s Application ID. StackSpend’s connect dialog generates this script pre-filled for you.

GRANT USE CATALOG ON CATALOG system TO `<service-principal-application-id>`;
GRANT USE SCHEMA ON SCHEMA system.billing TO `<service-principal-application-id>`;
GRANT SELECT ON SCHEMA system.billing TO `<service-principal-application-id>`;
PERMISSION_DENIED: User does not have MANAGE on Catalog 'system'.This error means you are not a metastore admin — only that role can grant on the system catalog; being a workspace or account admin is not enough, and new or trial accounts often have no metastore admin assigned at all. To assign it, open the Databricks account console → Catalog → select the metastore attached to your workspace → Edit → set Metastore Admin to yourself or an admin group, then re-run the grants.

Then give the service principal access to the warehouse: SQL Warehouses → your warehouse → Permissions → add the service principal with “Can use”.

3

Find your Workspace URL and Warehouse ID

The Workspace URL is the address you use to open Databricks (e.g. https://dbc-a1b2c3d4-e5f6.cloud.databricks.com). The SQL Warehouse ID is under SQL Warehouses → your warehouse → Connection details — the ID at the end of the HTTP path.

4

Add to StackSpend

In StackSpend, go to Providers → Databricks. Enter the Workspace URL, SQL Warehouse ID, Client ID, and Client Secret. Click Test Connection, then Save.

Verify access

To confirm the billing tables are enabled and carry data, run this in a SQL editor:

SELECT COUNT(*) AS rows_last_7_days
FROM system.billing.usage
WHERE usage_date >= DATE_SUB(CURRENT_DATE(), 7);
No rows in system.billing.usage?.System tables record usage from the time they are enabled onward. On a new account (or trial) the table can be empty until compute has actually run — start a cluster or run a query, and usage appears after provider reporting and a successful scheduled sync. If the query fails with TABLE_OR_VIEW_NOT_FOUND, your workspace is not Unity Catalog-enabled or the grants above have not been applied. UC_NOT_ENABLED means the warehouse you chose isn’t attached to Unity Catalog: pick a serverless or pro SQL warehouse in a Unity Catalog-enabled workspace instead. If Test Connection says the warehouse may be starting, wait a minute and test again.

Data summary

AttributeValue
Data latencyDaily
Historical dataUp to 90 days where available
Setup time5 min

Frequently asked questions

What Databricks credentials does StackSpend need?

StackSpend needs a service principal (Client ID + OAuth secret) with SELECT on the system.billing schema and “Can use” on one SQL warehouse, plus your workspace URL and the warehouse ID. A personal access token also works, but a service principal keeps syncing when people leave.

Is the Databricks connection read-only?

Yes. The service principal only holds SELECT on the system.billing schema and use of a SQL warehouse — StackSpend runs one read-only aggregation query per sync and never writes to your account.

Does the sync itself cost me DBUs?

The billing query runs on your SQL warehouse and can incur compute charges. Cost depends on warehouse configuration and query execution; review query history and use an appropriate auto-stop setting.

Are the costs my actual bill?

These are usage multiplied by effective USD list prices from system.billing.list_prices, with an optional uniform Discount % assumption. They can differ from negotiated invoice amounts. Compare usage quantities and document the pricing assumption before sharing totals.

Which workspaces are covered?

All of them. The system.billing tables are account-wide, so connecting any one Unity Catalog-enabled workspace covers usage from every workspace in the account, broken down by workspace ID.

Connect Databricks to StackSpend — StackSpend Docs