Any Azure experts here?

[Spacerat]

There are so many Azure functions that I tend to get confused.

I have a WebAPI that inserts raw data into an Azure SQL table. This data needs to be enriched. So I need some kind of 'background worker' to take the inbound raw data and enrich it for reporting. So either I use one of the Azure functions to do this (dunno which one) or have to write it and run it in a VM. Ideally the receipt of raw data should trigger the background processing.

 

[lerouc]

And enriching it as part of the api that saves it into the db not an option?

 

[Traq]

Web jobs:

https://docs.microsoft.com/en-us/azure/app-service/web-sites-create-web-jobs

or Azure Functions

https://azure.microsoft.com/en-us/services/functions/

 

[Spacerat]

And enriching it as part of the api that saves it into the db not an option?

That was my first thought and will be my fallback. But the data come from various embedded devices across the world and I wanted to make the upload of the data as light-weight as possible. Just exploring some possible alternatives

 

[Spacerat]

Web jobs:

https://docs.microsoft.com/en-us/azure/app-service/web-sites-create-web-jobs

or Azure Functions

https://azure.microsoft.com/en-us/services/functions/

Tx will look int this. Have you used this before?

 

[Traq]

Tx will look int this. Have you used this before?

It depends on what you do to "enrich".

Yes, we use web jobs, and functions. For ETL we use Azure Data Factory to move from SQL Azure DB to the data warehousing services.

 

[Traq]

That was my first thought and will be my fallback. But the data come from various embedded devices across the world and I wanted to make the upload of the data as light-weight as possible. Just exploring some possible alternatives

For this specific requirement, if you wanted to make use of Azure features, will require Azure functions, and Azure IoT hub if it is streaming data sets.

 

[Spacerat]

It depends on what you do to "enrich".

Yes, we use web jobs, and functions. For ETL we use Azure Data Factory to move from SQL Azure DB to the data warehousing services.

The data from the device is raw event data. The WebApi stores each event as a row in an Azure SQL table. Each device has a unique id. Enriching is simply determining the facility and customer the event belongs to as well as some time based data and analytics. The enriched event is then stored in another table.

For this specific requirement, if you wanted to make use of Azure features, will require Azure functions, and Azure IoT hub if it is streaming data sets.

I am just thinking that by reusing Azure services, one does not have to develop and wire up everything. Which is how I normally do it. The volume of data is not huge at this stage. Not sure what value IoT hub adds as I already have a WebApi to consume and store raw events?

This is my 'issue' with Azure. I find it hard to figure out which service I should be using. But tx for the input

 

[marzbars]

Why not a SQL job?

 

[Traq]

<snip> Azure SQL table. </snip>

Why not a SQL job?

He would need to specifically make use of a VM to host the SQL just to get that functionality if he doesn't do so already. Alternatively he would need to use SQL Elastic Pools (which will provide some level of functionality similar to SQL Server Agent).

 

[Spacerat]

He would need to specifically make use of a VM to host the SQL just to get that functionality if he doesn't do so already. Alternatively he would need to use SQL Elastic Pools (which will provide some level of functionality similar to SQL Server Agent).

Yes.. dont want to run a VM just for this

 

[kripstoe]

Yes.. dont want to run a VM just for this

Yip, WebJobs should work for this.