{"id":190,"date":"2025-12-02T09:17:45","date_gmt":"2025-12-02T09:17:45","guid":{"rendered":"https:\/\/dataguide.tech\/?p=190"},"modified":"2025-12-02T09:17:45","modified_gmt":"2025-12-02T09:17:45","slug":"how-to-build-automated-monitoring-alerts-for-microsoft-fabric","status":"publish","type":"post","link":"https:\/\/dataguide.tech\/index.php\/2025\/12\/02\/how-to-build-automated-monitoring-alerts-for-microsoft-fabric\/","title":{"rendered":"How to Build Automated Monitoring Alerts for Microsoft Fabric"},"content":{"rendered":"\n

Introduction<\/h2>\n\n\n\n

If you’ve been working with Microsoft Fabric in production environments, you’ve probably faced this critical question: “How can we monitor our pipelines, notebooks, and processes with automated notifications when something goes wrong?”<\/p>\n\n\n\n

Here’s the thing\u2014as of now, there’s no built-in functionality in Microsoft Fabric that fully addresses this need. The recently announced Workspace Monitoring doesn’t cover dataflows, pipelines, or notebooks. For many teams, this is actually a “hard stop” for moving to production, since they can’t react quickly and efficiently when failures happen.<\/p>\n\n\n\n

Since this question keeps coming up with every Fabric implementation, I decided to dig deeper and explore what’s currently possible. While there’s no official documentation, I found a way to tap into the same data that powers Fabric’s Monitoring Hub. Let me show you how.<\/p>\n\n\n\n

Discovering the Monitoring API<\/h2>\n\n\n\n

The first step was understanding how the Monitoring Hub actually works under the hood. To do this, I used the browser’s developer tools to peek at what’s happening behind the scenes.<\/p>\n\n\n\n

Here’s what I did: I opened Developer Tools (hit F12 in most browsers), switched to the Network tab, and started recording all API calls. Then I opened the Monitoring Hub tab in Fabric:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

As you can see in the screenshot, several API calls are being made. The highlighted one is particularly interesting\u2014this is the actual endpoint that Monitoring Hub uses to fetch execution data.<\/p>\n\n\n\n

When you click into the details of that API call, you’ll see something like this:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

The highlighted sections show the key information you’ll need to construct your own REST API call to the same endpoint. This means you can access exactly the same data that Monitoring Hub displays\u2014programmatically!<\/p>\n\n\n\n

Understanding the API Payload<\/h2>\n\n\n\n

Now, let’s look at the “Payload” tab to see what parameters are being sent:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Here are the three main parameters you’ll work with:<\/p>\n\n\n\n

    \n
  • limit<\/strong> – Sets the maximum number of rows the API will return<\/li>\n\n\n\n
  • endTime<\/strong> – Defaults to the current time; filters executions by their Start Time<\/span><\/li>\n\n\n\n
  • startTime<\/strong> – Defaults to 1970-01-01; filters executions by their Start Time<\/span><\/li>\n<\/ul>\n\n\n\n

    Heads up:<\/strong> You can discover additional parameters by modifying filters in the Monitoring Hub and checking the subsequent requests. For example, when I filtered to show only failed processes:<\/p>\n\n\n

    \n
    \"\"<\/figure>\n<\/div>\n\n\n

    The payload revealed a new parameter called status<\/code> with values: 3, 5, 7, 8. These represent different failure statuses. I don’t know the exact difference between them since there’s no official Microsoft documentation for this API\u2014but the key point is that we can modify the API call to filter the output we need.<\/p>\n\n\n\n

    You can experiment with different filters in the Monitoring Hub and check the Network tab to see what parameters get added. This way, you can craft the perfect API call for your specific monitoring needs.<\/p>\n\n\n\n

    What Data Can You Access?<\/h2>\n\n\n\n

    The API response gives you everything displayed in the Monitoring Hub:<\/p>\n\n\n

    \n
    \"\"<\/figure>\n<\/div>\n\n\n

    This includes:<\/p>\n\n\n\n

      \n
    • Object IDs and workspace IDs<\/li>\n\n\n\n
    • Start and end times<\/li>\n\n\n\n
    • Job owner information<\/li>\n\n\n\n
    • Execution status<\/li>\n\n\n\n
    • The serviceExceptionJson<\/code> field\u2014which contains detailed error information<\/li>\n<\/ul>\n\n\n\n

      This last field is particularly valuable for understanding what went wrong in a failed execution.<\/p>\n\n\n\n

      Automating Monitoring with Python<\/h2>\n\n\n\n

      Ok, now that I knew what the Monitoring Hub uses behind the scenes, I wanted to automate the whole process using Fabric Notebooks. My goal was simple: load the execution logs into a Lakehouse table, then use that data to send email or Teams notifications when issues occur.<\/p>\n\n\n\n

      Below you can find the script I created.<\/strong><\/p>\n\n\n\n

      Import Libraries<\/h3>\n\n\n\n

      Start by importing the necessary libraries: requests<\/code> for making HTTP requests, pandas<\/code> for data manipulation, and json<\/code> for handling JSON data.<\/p>\n\n\n\n

      Python<\/span>