Azure Data Factory – HTTP/JSON dataset

Are you an Azure Data Engineer? Well it will not be a supprise that working with Azure Data Factory (ADF) is in most cases pretty straightforward.

Here’s how to create a Linked Service and a HTTP/JSON dataset within Azure Data Factory. Let’s start by creating a Linked Service to our API. Create a new Linked Service and select ‘HTTP‘ for the type. Click on ‘Continue‘, fill in the proper values in the next screen as seen below and click on ‘Create‘:

Now you need to create a HTTP/JSON dataset. Make sure to pass the ‘subscription-key‘ along the relative URL as seen below:

Voila! You’re now able to retrieve all your master data 🙂

Don’t forget to use one of the optional query parameters:

/consumers/?subscription-key=XXXX&modifiedinthepastminutes=60
OR
/consumers/?subscription-key=XXXX&modifiedsince=2020-05-01T01:33:22Z
OR
/consumers/?subscription-key=XXXX&id=b.wayne@wayneenterprises.com

More technical information about ADF’s Copy Activity can be found here.

If you have any questions, feel free to contact us. We’re happy to help!