Posted by

Vivek Janakiraman

Posted on

February 19, 2025

Posted under

Azure Datafactory

Comments

Azure Data Factory Series: Invoke ADF Pipeline from App Service with User-Assigned Managed Identity

Watch the step by step implementation as a You tube Video.

Step 1: Create an Azure App Service (Web App)

Go to Azure Portal → Search for “App Services” → Click “Create”.
Choose Subscription and Resource Group (or create a new one).
Set the following values:
- Name: my-adf-web-app
- Publish: Code
- Runtime Stack: .NET 8 (or any preferred language)
- Operating System: Windows or Linux
- Region: Choose a region close to your resources.
- Plan: Choose a basic plan (B1) for testing.
Click Review + Create → Click Create.

Step 2: Enable Managed Identity for Web App

In Azure Portal, go to App Services → Select my-adf-web-app.
Click on Identity under Settings.
Under User Assigned, toggle the switch to On.
Click Save, and copy the Client ID (you’ll need it later).

Step 3: Grant Permissions to Managed Identity on ADF

Go to Azure Data Factory in the Azure Portal.
Click on Access Control (IAM) → Click Add Role Assignment.
Select:
- Role: Data Factory Contributor (or Data Factory Operator for limited access).
- Assign Access To: Managed Identity.
- Select Members: Choose your Web App (my-adf-web-app).
Click Save.

Step 4: Write Code in Azure App Service to Call ADF Pipeline

Use the following C# code inside your Web App to invoke the ADF pipeline using Managed Identity.

C# Code (ASP.NET Core)

using Microsoft.AspNetCore.Mvc;
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.DataFactory;
using System.Threading.Tasks;
using System.Collections.Generic;
using System;

[Route("JB/[controller]")]
[ApiController]
public class ADFController : ControllerBase
{
    private readonly ArmClient _armClient;
    private readonly string _subscriptionId = "xxxx-xxxxx-xxxxx-xxxxx";
    private readonly string _resourceGroupName = "jbadf";
    private readonly string _dataFactoryName = "jbadfapp";
    private readonly string _pipelineName = "jb_Copydata";

    public ADFController()
    {
        // Set the Client ID of the User-Assigned Managed Identity (UMI)
        var userAssignedClientId = "xxxx-xxxxx-xxxxx-xxxxx"; // Replace this with actual Client ID

        var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions
        {
            ManagedIdentityClientId = userAssignedClientId
        });

        _armClient = new ArmClient(credential);
    }

    [HttpGet("CuriousBoy")]
    public async Task<IActionResult> TriggerPipeline()
    {
        try
        {
            // Construct the pipeline resource ID
            var pipelineResourceId = DataFactoryPipelineResource.CreateResourceIdentifier(
                _subscriptionId, _resourceGroupName, _dataFactoryName, _pipelineName);

            // Get the pipeline resource
            var pipelineResource = _armClient.GetDataFactoryPipelineResource(pipelineResourceId);

            // Define pipeline parameters (if required)
            var parameters = new Dictionary<string, BinaryData>
            {
                // Example: If your pipeline requires parameters, add them here.
                // { "param1", BinaryData.FromString("value1") },
                // { "param2", BinaryData.FromString("value2") }
            };

            // Trigger the pipeline run
            var runResponse = await pipelineResource.CreateRunAsync(parameters);

            return Ok($"ADF Pipeline triggered successfully on {DateTime.UtcNow}");
        }
        catch (Exception ex)
        {
            return StatusCode(500, $"Error triggering ADF pipeline: {ex.Message}");
        }
    }
}

Step 5: Deploy the Code to Azure App Service

In Visual Studio, create an ASP.NET Core Web API project.
Copy the above C# code into your Controller or Service.
Deploy your code to Azure App Service using:
- Right-click on the project → Publish → Azure App Service.

Step 6: Test the Web App

Navigate to https://my-adf-web-app.azurewebsites.net.
Trigger the endpoint that executes the above code.
Your ADF pipeline should now run successfully!

This is the simplest way to invoke an ADF pipeline from Azure App Services using Managed Identity.

Possible Errors Expected,

Error triggering ADF pipeline: DefaultAzureCredential failed to retrieve a token from the included credentials. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/defaultazurecredential/troubleshoot

EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot
WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot
ManagedIdentityCredential authentication unavailable. No response received from the managed identity endpoint.
VisualStudioCredential authentication failed: Visual Studio Token provider can’t be accessed at D:DWASFilesSitesJBAPPLocalAppData.IdentityServiceAzureServiceAuthtokenprovider.json
AzureCliCredential authentication failed: Azure CLI not installed
AzurePowerShellCredential authentication failed: Az.Accounts module >= 2.2.0 is not installed.
AzureDeveloperCliCredential authentication failed: Azure Developer CLI could not be found.

Error triggering ADF pipeline: The client ’66a2225c-f1b5-439b-b375-78ae2b744f2f’ with object id ’66a2225c-f1b5-439b-b375-78ae2b744f2f’ does not have authorization to perform action ‘Microsoft.DataFactory/factories/pipelines/createRun/action’ over scope ‘/subscriptions/xxxx-xxxx-xxxx/resourceGroups/jbadf/providers/Microsoft.DataFactory/factories/jbadfapp/pipelines/jb_Copydata’ or the scope is invalid. If access was recently granted, please refresh your credentials.
Status: 403 (Forbidden)
ErrorCode: AuthorizationFailed

Content:
{“error”:{“code”:”AuthorizationFailed”,”message”:”The client ’66a2225c-f1b5-439b-b375-78ae2b744f2f’ with object id ’66a2225c-f1b5-439b-b375-78ae2b744f2f’ does not have authorization to perform action ‘Microsoft.DataFactory/factories/pipelines/createRun/action’ over scope ‘/subscriptions/xxxx-xxxx-xxxx/resourceGroups/jbadf/providers/Microsoft.DataFactory/factories/jbadfapp/pipelines/jb_Copydata’ or the scope is invalid. If access was recently granted, please refresh your credentials.”}}

Headers:
Cache-Control: no-cache
Pragma: no-cache
x-ms-failure-cause: REDACTED
x-ms-request-id: 896c9766-5ee3-4dbb-b3b6-71800e2ee564
x-ms-correlation-request-id: REDACTED
x-ms-routing-request-id: REDACTED
Strict-Transport-Security: REDACTED
X-Content-Type-Options: REDACTED
X-Cache: REDACTED
X-MSEdge-Ref: REDACTED
Date: Tue, 18 Feb 2025 08:23:20 GMT
Content-Length: 512
Content-Type: application/json; charset=utf-8
Expires: -1

Thank You,
Vivek Janakiraman

Disclaimer:
The views expressed on this blog are mine alone and do not reflect the views of my company or anyone else. All postings on this blog are provided “AS IS” with no warranties, and confers no rights.