Jul 17, 2021 · 9 min read
Applications or databases running out of disk space are a common issue that the Operations team addresses regularly. This problem has existed since the days we used to host applications on bare metal servers and is still present in virtualized and container environments. However, since we have reached a stage where the compute and storage systems are decoupled from each other, granting additional storage to applications rarely requires updating the application or modifying the underlying application host infrastructure.
Bulk Copy Data Sharing Pattern for Applications in Azure with Data Explorer, Data Factory & Cosmos DB
Jul 12, 2021 · 12 min read
🎁Special Giveaway@ManningBooks just launched Data Engineering on #Azure by @vladris. I am giving away 5 eBooks to my Tweeps!! Rules 👇🏻 ♥ Like 👉🏻 Follow me 🔃 Retweet and tag your friends in comments who may be interested#contest #giveaways #ebooks pic.twitter.com/hc7jNqCj25 — Rahul Rai (@rahulrai_in) July 12, 2021 In the initial stages of a data platform development, data size is small, and you can easily share it via email or services such as Power BI.
Jun 01, 2021 · 3 min read
Do you want your LinkedIn audience to know what you are up to on Twitter? Here’s how I have set up Power Automate to crosspost specific tweets to LinkedIn. What is Power Automate Power Automate is one of the products of the Microsoft Power Platform family. It is a web-based service that helps you create automated workflows between your favorite apps and services to synchronize files, get notifications, collect data, and more.
May 08, 2021 · 10 min read
In this series: Distributed Tracing with Jaeger Simplifying the setup with Tye (this article) Tye is an experimental dotnet tool from Microsoft that aims to make developing, testing, and deploying microservices easier. Tye’s opinionated nature greatly simplifies the lifecycle of development and deployment of .NET Core microservices. To understand the benefits of Tye, let’s enumerate the steps involved in the development and deployment of the DCalculator application to Kubernetes:
Apr 26, 2021 · 7 min read
In this series: Distributed Tracing with Jaeger (this article) Simplifying the setup with Tye Modern microservices applications consist of many services deployed on various hosts such as Kubernetes, AWS ECS, and Azure App Services or serverless compute services such as AWS Lambda and Azure Functions. One of the key challenges of microservices is the reduced visibility of requests that span multiple services. In distributed systems that perform various operations such as database queries, publish and consume messages, and trigger jobs, how would you quickly find issues and monitor the behavior of services?
Apr 11, 2021 · 10 min read
In this series: Development environment and Event producer Event consumer Azure Event Hubs integration (this article) Azure Event Hubs is a horizontally scalable event ingestion service capable of receiving and processing millions of events per second. It supports Apache Kafka Producer and Consumer API that you can use as an alternative to running a self-managed Apache Kafka cluster. Now you can integrate the Kafka ecosystem applications such as Kafdrop and many others with Event Hubs.
Apr 04, 2021 · 8 min read
In this series: Development environment and Event producer Event consumer (this article) Azure Event Hubs integration Let’s carry our discussion forward and implement a consumer of the events published by the Employee service to the leave-applications Kafka topic. We will extend the application that we developed earlier to add two new services to demonstrate how Kafka consumers work: Manager service and Result reader service. Source Code The complete source code of the application and other artifacts is available in my GitHub repository.
Mar 29, 2021 · 12 min read
In this series: Development environment and Event producer (this article) Event consumer Azure Event Hubs integration An event-driven architecture utilizes events to trigger and communicate between microservices. An event is a change in the service’s state, such as an item being added to the shopping cart. When an event occurs, the service produces an event notification which is a packet of information about the event. The architecture consists of an event producer, an event router, and an event consumer.
Feb 05, 2021 · 12 min read
Many times, your application needs to run tasks or jobs on regular intervals. Scheduling might be necessary when polling an API every night, or dispatching emails with reports each week. And sometimes you may find that you need to trigger a host of tasks across multi- or hybrid-cloud and need a way to run these tasks in a reliable, cross-cloud manner. In this article, let’s look at one way to do that using a Heroku dyno for our task, and Microsoft Azure Logic Apps as our scheduler service.
Feb 03, 2021 · 11 min read
Imagine running a .NET Core application in Kubernetes, which suddenly starts being sluggish, and the telemetry fails to give you a complete picture of the issue. To remediate performance issues of applications, starting with .NET Core 3, Microsoft introduced several .NET Core runtime diagnostics tools to diagnose application issues. dotnet-counters to view Performance Counters. dotnet-dump to capture and analyze Dumps. dotnet-trace to capture runtime events and sample CPU stacks. dotnet-gcdump to collect Garbage Collector dumps of application.