BreezeTech - Possible issues with NC and TX – Incident details

Possible issues with NC and TX

Resolved
Degraded performance
Started 27 days agoLasted about 6 hours

Affected

Data Centers

Operational from 4:45 PM to 10:53 PM

CLT01 - Charlotte, NC

Operational from 4:45 PM to 10:53 PM

DFW01 - Dallas TX

Operational from 4:45 PM to 10:53 PM

Updates
  • Resolved
    Resolved

    The network has been stable since this morning and we are closing this incident. Should you experience any issues, please notify support by creating a support ticket.

  • Update
    Update

    We have received the all clear from Cosmic our DDoS provider, and we are still monitoring the situation from our NOC. We have provided a timeline of the events to detail below and wanted to be transparent in our response.

    11:40AM EST  - We were notified of network issues across multiple locations

    11:41AM EST - An emergency bridge call was started with BreezeTech staff to triage the issue

    11:42AM EST - Charlotte network determined partial outage

    11:43AM EST - Chicago network determined unimpacted

    11:44AM EST - Dallas network determined partial outage

    11:46AM EST - Outage determined related to Cosmic. Cosmic was contacted to check on issues

    11:48AM EST - Cosmic confirmed outage

    11:49AM EST - Charlotte Cosmic routes shifted to primary/secondary ISP carriers. Confirmed solution

    11:52AM EST - Dallas Cosmic routes shifted to primary/secondary ISP carriers. Confirmed solution

    11:53AM EST - Los Angeles network determined partial outage

    11:54AM EST - Los Angeles Cosmic routes shifted to primary/secondary ISP carriers. Confirmed solution

    We are continuing to monitor the situation but all services should be online at this time. If you are experiencing any issues, please reach out to support directly.

  • Monitoring
    Monitoring

    The temporary fix has been implemented, and we are actively monitoring the situation.

  • Identified
    Identified

    We've identified the issue as originating from Cosmic Guard ranges. A temporary fix is currently being implemented.

  • Investigating
    Investigating
    We are currently investigating this incident.