Azure status

Note: During this incident, as a result of a delay in determining exactly which customer subscriptions were impacted, azure status, we chose to communicate via the public Azure Status page. As described in our documentation, public PIR postings on this page are reserved for 'scenario 1' incidents - typically broadly impacting incidents across entire zones or regions, azure status, or even multiple zones or regions. Summary of Impact: Between and UTC on 07 Feb first azure statuscustomers attempting to view their resources through the Azure Portal may have experienced latency and delays.

Note: During this incident, as a result of a delay in determining exactly which customer subscriptions were impacted, we chose to communicate via the public Azure Status page. As described in our documentation, public PIR postings on this page are reserved for 'scenario 1' incidents - typically broadly impacting incidents across entire zones or regions, or even multiple zones or regions. Summary of Impact: Between and UTC on 07 Feb first occurrence , customers attempting to view their resources through the Azure Portal may have experienced latency and delays. Subsequently, impact was experienced between and UTC on 08 Feb second occurrence , the issue re-occurred with impact experienced in customer locations across Europe leveraging Azure services. Preliminary Root Cause: External reports alerted us to higher-than-expected latency and delays in the Azure Portal.

Azure status

.

Azure status of Impact: Between and UTC on 07 Feb first occurrencecustomers attempting to view their resources through the Azure Portal may have experienced latency and delays.

.

The Hybrid Connection Debug utility is provided to perform captures and troubleshooting of issues with the Hybrid Connection Manager. This utility acts as a mini-Hybrid Connection Manager and must be used instead of the existing Hybrid Connection Manager you have installed on your client. If you have production environments that use Hybrid Connections, you should create a new Hybrid Connection that only gets served by this utility and repro your issue with the new Hybrid Connection. The tool can be downloaded here: Hybrid Connection Debug Utility. Typically, for any troubleshooting of Hybrid Connections issues, Listener should be the only mode that is necessary.

Azure status

Impact Statement: Starting as early as UTC on 07 Feb , customers accessing their resources through the Azure Portal may experience latency and delays viewing their resources. Impact would be mostly seen in West Europe. Current Status: We identified a potential issue with Azure Resource Graph which has now been mitigated.

Calico cat kitten

Unbeknownst to us, this preview feature of the ARM CAE implementation contained a latent code defect that caused issues when authentication to Entra failed. By UTC we had correlated the preview feature to the ongoing impact. This page contains root cause analyses RCAs of previous service issues, each retained for 5 years. However, upon further investigation, we identified a potential network issue with the Azure Resource Manager service which caused impact to additional Azure services including the Azure Portal, Azure Data Factory, Azure Synapse Analytics and Databricks. In addition, several internal offerings depend on ARM to support on-demand capacity and configuration changes, leading to degradation and failure when ARM was unable to process their requests. Specific to Key Vault, we identified a latent bug which resulted in application crashes when latency to ARM from the Key Vault data plane was persistently high. How are we making incidents like this less likely or less impactful? Mitigation: During the first occurrence, we initially suspected an issue with Azure Resource Graph ARG and reverted a recent deployment as this was a potential root cause. Completed Our Key Vault team has fixed the code that resulted in applications crashing when they were unable to refresh their RBAC caches. Completed We have offboarded all tenants from the CAE private preview, as a precaution.

Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support. Azure offers a suite of experiences to keep you informed about the health of your cloud resources. This information includes current and upcoming issues such as service impacting events, planned maintenance, and other changes that may affect your availability.

How can we make our incident communications more useful? Specific to Key Vault, we identified a latent bug which resulted in application crashes when latency to ARM from the Key Vault data plane was persistently high. Due to these ongoing node restarts and failed startups, ARM began experiencing a gradual loss in capacity to serve requests. After further investigation, we determined that an issue impacting the Azure Resource Manager ARM service resulted in downstream impact for various Azure services. As described in our documentation, public PIR postings on this page are reserved for 'scenario 1' incidents - typically broadly impacting incidents across entire zones or regions, or even multiple zones or regions. How are we making incidents like this less likely or less impactful? This triggered the latent code defect and caused ARM nodes, which are designed to restart periodically, to fail repeatedly upon startup. While impact for the first occurrence was focused on West Europe, the second occurrence was reported across European regions including West Europe. How can we make our incident communications more useful? Automated communications to a subset of impacted customers began shortly thereafter and, as impact to additional regions became better understood, we decided to communicate publicly via the Azure Status page. ARM nodes restart periodically by design, to account for automated recovery from transient changes in the underlying platform, and to protect against accidental resource exhaustion such as memory leaks.