Watch this as a video on our you tube channel JBSWiki.

Are you ready to unlock the power of real-time machine learning predictions directly in your Power BI dashboards? 🤩

With Databricks Model Serving, we can host machine learning models as REST APIs. But how do we bring those predictions into Power BI?

In this blog, I’ll show you two practical methods to connect Power BI to Databricks Model Serving and fetch predictions step by step.

This is a follow-up to my earlier work where I explained how to create a Databricks Serving Model. If you missed that, check it out first so you have your serving endpoint ready to go!

💡 Why Integrate Power BI with Databricks Model Serving?

Businesses are increasingly driven by real-time insights. Instead of waiting for static reports, you can now embed ML predictions right inside your dashboards, empowering data-driven decisions at the speed of business.

Some benefits:

✅ Real-time predictions in dashboards
✅ No manual data exports
✅ Fully automated pipelines
✅ Empower business users with AI insights

🔗 The Architecture

Here’s how the integration works:

Power BI ➡ Python Script / Power Query ➡ Databricks Serving Endpoint ➡ Prediction Results ➡ Power BI visuals

All communication happens over secure REST APIs, usually authenticated with a Databricks Personal Access Token (PAT).

⚙️ Pre-requisites

Before diving in, ensure you have:

• An Azure Databricks workspace
• A deployed ML model as a Databricks Serving Endpoint
• A Databricks Personal Access Token
• Power BI Desktop installed
• Basic knowledge of Power Query and Python

🛠 Method 1: Get Data → Python Script in Power BI

This is one of the easiest ways to connect Power BI to Databricks Serving endpoints if you’re comfortable writing Python.

🔹 How It Works

You’ll:

1. Go to Home > Get Data > Python Script in Power BI.
2. Paste the Python code that:
   • Makes an HTTP POST request to the Databricks model endpoint.
   • Converts predictions into a DataFrame for visualization.

✅ Example Python Script

Here’s a full working Python script for Power BI:

import requests
import json
import pandas as pd

# Databricks endpoint URL
endpoint_url = "https://adb-2345432345567.11.azuredatabricks.net/serving-endpoints/HDFC_High_price_prediction/invocations"

# Your Databricks PAT token
token = "Databricks_Pat_Token"

# Prepare the payload
payload = {
    "inputs": [
        {
            "Date": "2024-07-03",
            "OPEN": 2000,
            "HIGH": 2079,
            "LOW": 1987,
            "CLOSE": 2075
        }
    ]
}

headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json"
}

# Make the POST request
response = requests.post(endpoint_url, headers=headers, json=payload)

# Convert input payload to DataFrame
input_df = pd.DataFrame(payload["inputs"])

if response.ok:
    result_json = response.json()
    predictions = result_json.get("predictions", [])
    
    # Put into dataframe
    output_df = pd.DataFrame(predictions, columns=["Prediction"])
else:
    # Handle errors gracefully
    output_df = pd.DataFrame({"Error": [response.text]})

input_df
output_df

🔎 What This Does

• Sends input data to Databricks Serving endpoint.
• Receives predictions in JSON format.
• Converts predictions to a Pandas DataFrame.
• Returns it to Power BI for visualizations.

Power BI will import both input_df and output_df as separate tables. You can merge them if needed.

⚠️ Important Notes

• Don’t hard-code secrets like tokens in production. Use Key Vaults or environment variables.
• Keep an eye on API call costs and throttling.

🛠 Method 2: Enter Data → Power Query → Python Script

This method is powerful when you want business users to enter data manually in Power BI and instantly fetch predictions.

🔹 How It Works

1. Use Enter Data in Power BI to create a table of inputs.
2. Pass this table into a Python script via Power Query.
3. Call the Databricks endpoint and merge predictions into your original data.

This allows users to dynamically modify input data in Power BI itself.

✅ Example Power Query Script

Below is the Power Query M code you’d place in Advanced Editor in Power BI:

let
    Source = HDFC_Input,
    RunPython = Python.Execute("
import pandas as pd
import requests
import json

# Power BI table comes in as 'dataset'
input_df = dataset

# Force Date column to string
if 'Date' in input_df.columns:
    input_df['Date'] = input_df['Date'].astype(str)

# Convert numeric columns to floats
for col in input_df.columns:
    if pd.api.types.is_numeric_dtype(input_df[col]):
        input_df[col] = input_df[col].apply(lambda x: float(x) if pd.notnull(x) else None)

inputs = input_df.to_dict(orient='records')

payload = {
    'inputs': inputs
}

endpoint_url = 'https://adb-2345432345567.11.azuredatabricks.net/serving-endpoints/HDFC_High_price_prediction/invocations'
token = 'Databricks_Pat_Token'
headers = {
    'Authorization': f'Bearer {token}',
    'Content-Type': 'application/json'
}

response = requests.post(endpoint_url, headers=headers, json=payload)

if response.ok:
    result_json = response.json()
    
    try:
        predictions = result_json['predictions']
        
        # Handle possible formats
        if isinstance(predictions, list) and isinstance(predictions[0], (int, float)):
            output_df = pd.DataFrame({'Prediction': predictions})
        elif isinstance(predictions, list) and isinstance(predictions[0], list):
            output_df = pd.DataFrame(predictions, columns=['Prediction'])
        elif isinstance(predictions, list) and isinstance(predictions[0], dict):
            output_df = pd.DataFrame(predictions)
        else:
            output_df = pd.DataFrame({'Error': ['Unsupported prediction format']})
    except Exception as e:
        output_df = pd.DataFrame({'Error': [str(e)]})
else:
    output_df = pd.DataFrame({'Error': [response.text]})

# Merge prediction into input
final_df = input_df.copy()
try:
    final_df['Prediction'] = output_df['Prediction']
except:
    final_df['Error'] = output_df.iloc[:, 0]

final_df
", [dataset=Source])
in
    RunPython

🔎 What This Does

• Takes user-entered data from Power BI as a table.
• Converts it to JSON payload.
• Calls Databricks Model Serving endpoint.
• Handles various possible prediction formats:
  • single numeric predictions
  • lists of predictions
  • dictionaries of results
• Merges predictions back into the original table.

💡 Advantages of This Method

✅ Super flexible for dynamic inputs
✅ Great for PoC demos and interactive reports
✅ Business-friendly approach—no code needed by users
✅ Predictions update automatically when inputs change

⚠️ Limitations and Considerations

• Python scripting in Power BI requires the Python runtime installed locally.
• Personal Access Tokens should be secured (e.g. not stored in plain text).
• There might be latency if your model takes time to compute predictions.

🎯 Use Cases

Integrating Databricks Model Serving into Power BI opens up endless possibilities:

✅ Stock Price Prediction
✅ Sales Forecasting
✅ Customer Churn Analysis
✅ Fraud Detection
✅ Predictive Maintenance

🚀 Conclusion

Integrating Databricks Model Serving with Power BI is a game-changer for real-time analytics. Whether you use the Python script approach or Power Query with Enter Data, you’re enabling truly interactive, predictive dashboards that empower business users.

✅ Next Steps

• Make sure your Databricks Serving endpoint is production-ready.
• Move sensitive tokens to secure stores like Azure Key Vault.
• Optimize API call performance for large-scale use.
• Explore scheduled refreshes in Power BI Service to automate insights.

Thank You,
Vivek Janakiraman

Disclaimer:
The views expressed on this blog are mine alone and do not reflect the views of my company or anyone else. All postings on this blog are provided “AS IS” with no warranties, and confers no rights.