When working with Azure Data Factory (ADF) pipelines, secure communication between various Azure services is crucial, especially when accessing sensitive data stored in resources like Azure Key Vault. Recently, while executing an ADF pipeline, I encountered the following error message:
Operation on target Web URLToken TenantId failed: {“error”:{“code”:”Forbidden”,”message”:”Client address is not authorized and caller is not a trusted service.\r\nClient address: xx.xx.xx.xxx\r\nCaller: appid=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx;oid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx;providers/Microsoft.DataFactory/factories/datafactoryjb;abc_er_sde=/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/jb_ADF_RG/providers/Microsoft.DataFactory/factories/datafactoryjb\r\nVault: jbkeyvault;location=northeurope”,”innererror”:{“code”:”ForbiddenByFirewall”}}}
This error indicates that the client address attempting to access the Azure Key Vault was not authorized, preventing the pipeline from retrieving necessary secrets stored in the vault. The root cause of this issue was that the Azure Key Vault had strict network security settings that only allowed authorized IP addresses to access it. The ADF instance running in the North Europe region was not on the list of authorized addresses, causing the connection to be blocked.
In this blog post, I will walk you through how I resolved the issue by whitelisting the required IP addresses for Data Factory and ensuring secure communication between Azure Data Factory and Azure Key Vault.
Understanding the Error: Client Address Not Authorized
The error message received during the pipeline execution highlights two key aspects:
- Client Address Not Authorized: The IP address attempting to connect to the Key Vault (ADF service in this case) was not whitelisted, which led to the request being blocked by the Key Vault’s firewall.
- Forbidden by Firewall: The error code
ForbiddenByFirewallsuggests that the Key Vault is configured with firewall rules that only allow specific IP addresses to access it.
By default, Azure Key Vault can restrict access to its secrets by using firewall settings to specify which IP ranges or services are allowed to connect. This provides an extra layer of security, but in this case, it became an obstacle because the IP addresses associated with the ADF instance were not permitted.
Steps to Resolve the Issue
To resolve this issue, I followed these steps to add the necessary IP addresses to the Azure Key Vault’s firewall settings:
1. Identify the Region of the Azure Data Factory
The first step was to identify the region in which my Azure Data Factory instance was running. In this case, the ADF instance was running in the North Europe region, as seen from the error message and confirmed in the Azure portal.
2. Retrieve the IP Address Range for ADF in North Europe
Azure services like Data Factory operate from a specific set of IP ranges depending on the region they are deployed in. Microsoft publishes the IP address ranges for each service and region, which are updated periodically. You can download the full list of IP ranges for different Azure services and regions from this Microsoft link.
In this case, the required IP addresses for Data Factory in the North Europe region were as follows:
{
"name": "DataFactory.NorthEurope",
"id": "DataFactory.NorthEurope",
"properties": {
"changeNumber": 7,
"region": "northeurope",
"regionId": 17,
"platform": "Azure",
"systemService": "DataFactory",
"addressPrefixes": [
"4.207.242.72/29",
"13.69.230.96/28",
"13.74.108.224/28",
"20.38.80.192/26",
"20.38.82.0/23",
"20.50.68.56/29",
"20.223.64.60/32",
"20.223.65.144/29",
"48.209.130.96/28",
"48.209.130.112/29",
"52.138.229.32/28"
]
}
}You can get the details from Download Azure IP Ranges and Service Tags – Public Cloud from Official Microsoft Download Center.
These IP addresses need to be whitelisted in the Azure Key Vault to allow ADF to access the secrets.
3. Whitelist the IP Addresses in Azure Key Vault
To whitelist the necessary IP ranges, follow these steps:
- Navigate to the Azure Key Vault in the Azure portal.
- Select Networking from the left-side menu.
- Under Firewalls and virtual networks, ensure that you have selected to allow access from selected networks.
- Add the necessary IP address ranges to the firewall rules:
- “4.207.242.72/29”
- “13.69.230.96/28”
- “13.74.108.224/28”
- “20.38.80.192/26”
- “20.38.82.0/23”
- “20.50.68.56/29”
- “20.223.64.60/32”
- “20.223.65.144/29”
- “48.209.130.96/28”
- “48.209.130.112/29”
- “52.138.229.32/28”
- Once the IP addresses are added, save the changes to apply the updated firewall rules.
4. Validate the Pipeline Run
After whitelisting the required IP addresses in the Key Vault, I re-ran the Azure Data Factory pipeline. This time, the pipeline successfully connected to the Key Vault and retrieved the required secrets, and the entire workflow executed without any issues.
Conclusion
The error "Client address is not authorized" occurred because the Azure Key Vault was configured with strict firewall rules, preventing Azure Data Factory from accessing it. By identifying the region of the ADF instance and retrieving the correct IP ranges for that region, I was able to whitelist the necessary addresses and resolve the issue.
This process highlights the importance of maintaining secure communication between Azure services, and how careful configuration of firewall rules can help both secure and enable workflows. Always ensure that the IP ranges for the services you are using are updated and whitelisted in sensitive resources like Azure Key Vault to prevent any disruptions in your pipelines.
Additional Resources:
Regards;
Vivek Janakiraman
Disclaimer:
The views expressed on this blog are mine alone and do not reflect the views of my company or anyone else. All postings on this blog are provided “AS IS” with no warranties, and confers no rights.