Sign in

Microsoft certified Data and BI Professional. Enthusiastic about solving problems. Currently playing with Azure Functions and XMLA endpoints.

Developing a bare minimum REST API for Azure PostgreSQL Database using Azure Functions with Node.js in Azure Portal

Photo by on

Azure Functions is a serverless computing service running on Azure’s cloud-hosted execution environment. Azure Functions lets us run auto-scaled, event-triggered code on an abstracted underlying hosting environment.

An everyday use case scenario for Azure Functions is to perform work in response to an event and when that work can be completed quickly, typically in seconds. The on-demand execution feature makes Azure Functions a suitable candidate for developing REST APIs.

This article will create a stateless HTTP triggered RESTful Azure Functions app with Node.js to perform read operations on an Azure PostgreSQL database and return the results as a JSON object…

A step-by-step guide to turning COVID-19 data into stunning Power BI visuals using Microsoft Azure offerings.

Photo by on

Cloud, Big Data, and Business Intelligence are the three buzz words of the decade. Everyone is talking about them. Everyone wants to do it. But no one tells you how to do it. How do you use the cloud to process your big data and build intelligence around it to make business decisions?

There are multiple answers to that question, and in this guide, we tried to answer that question using Microsoft’s cloud solution (Azure) and Microsoft’s BI tool (Power BI) to get you started in the right direction.

Microsoft Azure is one of the leading cloud solutions providers, offering…

A handy guide for dummies to troubleshoot Azure Key Vault reference errors in Functions App

Photo by on

Azure Key Vault provides a great advantage of keeping your credentials, keys, and secret safe and centralized. However, the real power of the Key Vault sprawls in the seamless integration with various Azure components.

The primary benefit of referencing Key Vault secrets in Function Apps lies in enhanced security and reduced maintenance. The secrets can be updated in the Key Vault and are immediately available in the App.

This article will present few lessons learned while working with Key Vault references.

Caution: Microsoft Azure is a paid service, and following this article can cause financial liability to you or your…

A comprehensive guide on how to obtain user identity and authentication claims in a .NET Core Azure Functions App

Photo by on

Azure App Service provides built-in identity and authentication support, making user identification and authentication a hassle-free, low-effort task.

For ASP.NET 4.6 (and above) apps, ClaimsPrincipal.Currentis populated with authenticated user’s identities and claims, enabling developers to follow the standard .NET code patterns. However, in .NET Core-based Azure Functions, ClaimsPrincipal.Currentis not populated automatically and Claims must be obtained by different means.

This article will present four methods to access user identity and claims information in the .NET Core (C#) code.

Caution: Microsoft Azure is a paid service, and following this article can cause financial liability to you or your organization.

Please read…

Configure identity and authentication in Azure Functions App with Azure AD

Photo by on

Long gone are the days when a team of developers had to spent months setting up and debug the authentication for their apps and services. In the age of the cloud, almost everything is ‘plug-and-play.’

Azure comes with an in-built identity and authentication provider, Azure Active Directory (Azure AD), so verifying and signing-in users is a minimal to no code task (aka “EasyAuth”).

This article will present a guide to setup and configure authentication for Azure Functions App to sign in users with Azure AD as the identity and authentication provider.

Caution: Microsoft Azure is a paid service, and following…

Building an Azure Functions App to return JSON results from Azure Synapse Analytics with Node.js

Photo by on

Azure SQL Database and SQL Server 2016 (and later) supports an in-built feature for formatting query results as JSON. Then (currently Azure Synapse Analytics) was announced. Synapse Analytics supports querying and manipulating JSON data. However, there is no in Synapse to return SQL query results as JSON.

This article will create an HTTP triggered RESTful Azure Functions app with Node.js. This app will query Azure Synapse Analytics and convert and return the results as JSON.

Caution: Microsoft Azure is a paid service, and following this article can cause financial liability to…

Making Sense of Big Data

A step-by-step guide to importing CSV data from ADLS Gen2 to Azure Synapse Analytics by using PolyBase

Photo by on

Azure Synapse Analytics SQL pool supports various data loading methods. The fastest and most scalable way to load data is through PolyBase. PolyBase is a data virtualization technology that can access external data stored in Hadoop or Azure Data Lake Storage via the T-SQL language.

PolyBase shifts the data loading paradigm from ETL to ELT. The data is first loaded into a staging table followed by the transformation steps and finally loaded into the production tables.

In this article, we load a CSV file from an Azure Data Lake Storage Gen2 account to an Azure Synapse Analytics data warehouse by…

Referencing and Accessing Azure Key Vault Secrets in Azure Functions as Environment Variables in Node.js

Photo by on

Azure functions offer a great way to store and reference credentials, keys, and other secrets as application settings. Application settings are exposed as environment variables during execution. Application Settings are encrypted at rest and transmitted over an encrypted channel. However, you still run the risk of inadvertently exposing these secrets to unauthorized users.

A more favored approach to alleviate this issue is to store the credentials or keys in the Azure Key Vault as secrets and reference the secrets as environment variables in our Azure functions apps.

This article will set up an Azure functions app to access secrets stored…

A guide on adding and executing an Azure Databricks notebook in Azure Data Factory pipeline with Azure Key Vault safe Access Tokens.

Photo by on

Azure Data Factory is a great tool to create and orchestrate ETL and ELT pipelines. The Data Factory's power lies in seamlessly integrating vast sources of data and various compute and store components.

This article looks at how to add a Notebook activity to an Azure Data Factory pipeline to perform data transformations. We will execute a PySpark notebook with Azure Databricks cluster from a Data Factory pipeline while safeguarding Access Token in Azure Key Vault as a secret.

Caution: Microsoft Azure is a paid service, and following this article can cause financial liability to you or your organization.


Using PySpark to incrementally processing and loading schema drifted files to Azure Synapse Analytics data warehouse in Azure Databricks

Photo by on

Data is the blood of a business. Data comes in varying shapes and sizes, making it a constant challenging task to find the means of processing and consumption, without which it holds no value whatsoever.

This article looks at how to leverage Apache Spark’s parallel analytics capabilities to iteratively cleanse and transform schema drifted CSV files into queryable relational data to store in a data warehouse. We will work in a Spark environment and write code in PySpark to achieve our transformation goal.

Caution: Microsoft Azure is a paid service, and following this article can cause financial liability to you…

Dhyanendra Singh Rathore

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store