Inbound calling performance issues: call drops/errors and call quality

Performance Issue

Inbound calling performance issues: call drops/errors and call quality

Status: closed
Start:
End:
Duration: 6 days 18 hours 28 minutes
Affected Components:
NA1 Work Calling NA37 NA38 NA39 NA30 NA29 NA28 NA27 NA20 NA19 NA18 NA17 NA10 NA9 NA8 NA7 SB1 NA15 NA14 NA13 NA12 NA11 NA6 NA5 NA4 NA3 NA2 Contact Center Americas Inbound Contact Center US1 Contact Center US2
Update

8x8 is working with an inbound telephony carrier on performance issues causing busy signals, error messages, call drops and call quality issues. This is intermittently affecting inbound calls only on a single USA based 8x8 telephony carrier. We will provide updates as they become available and we apologize for the inconvenience.

Update

8x8 NOC is continuing to monitor our underlying carriers status on the issue. Updates will be provided as they become available. 

Update

8x8’s underlying carrier has observed stability for the last hour and at this current time calls are completing as normal. We will be monitoring the issue over the next 24 hours to ensure customers stay updated on the progress. Unless there’s a significant update, 8x8’s next update will be posted at 7AM Pacific Time on Tuesday 9-28-2021.

Update

As a continuation of the performance experienced yesterday, 8x8 operations teams have received a series of alerts for potential inbound and outbound call quality issues for any calls routed through the affected telephony carrier.

Outbound calls originating from 8x8 has been routed by our Operations team over alternate carriers are not impacted at this time.

For Inbound traffic into 8x8, call flows traversing the affected carrier could be experiencing a set of performance issues like: call quality degradation, call termination, etc. Due to the static nature of phone provisioning to a single carrier, inbound carrier traffic can not be re-routed.

We will provide updates as they become available as we continue to work with the affected carrier.

Update

8x8's underlying carrier has reduced impact to inbound phone calls throughout the day, but continues to work on full resolution. 8x8 will continue to monitor and report any updates as they become available. Our next status update will be Wednesday 9-29-2021 at 7 AM Pacific Time. 

Update

8x8's underlying carrier has continued to see improved performance and are still working on full resolution. Some customers may still be having issue receiving calls from Verizon Wireless, but other call scenarios are performing normally. If you are still experiencing issues with inbound calls related to call completion or call quality please contact support so we can investigate the issue. Our next update will be tomorrow Thursday 9-30-2021 at 8 AM Pacific Time unless another update becomes available. 

Update

8x8's underlying carrier has seen service stabilized and previous symptoms have subsided. Currently calls are being processed normally, however all teams are still monitoring.  If you are still experiencing issues with inbound calls related to call completion or call quality please contact support so we can investigate the issue. Our next update will be later this evening at 5pm pacific time unless another update becomes available.

Update

8x8's underlying carrier has continued to see improved performance and are still working monitoring to ensure full resolution. Some customers may still be having intermittent issues with call completion and call quality. If you are still experiencing issues with inbound calls related to call completion or call quality please contact support so we can investigate the issue. Our next update will be tomorrow Friday 10-01-2021 morning pacific time unless another update becomes available

Update

8x8's underlying carrier has seen service stabilized and previous symptoms have subsided. Currently, calls are being processed normally, however all teams are still monitoring. Our next update will be tomorrow morning (10/2/21) pacific time unless another update becomes available.

Resolved

8x8's carrier has observed stability now over a few days and 8x8 considers the issue as resolved at this time. If you are still experiencing issues please contact 8x8 support to troubleshoot issues further. 8x8 is following up with the carrier to ensure roto cause is fully understood and we apologize for the inconvenience.