ClaimSearch - Users are unable to login and may get kicked out of the system
Incident Report for ISO ClaimSearch
Postmortem

TIMING:

 August 16, 2023, 9:54 AM to 10:25 AM MT

 DESCRIPTION: 

 ClaimSearch Customers were unable to log in to ClaimSearch services.

 IMPACT:

 Unable were unable to access ClaimSearch Services.

 ROOT CAUSE:

 The Domain Name System (DNS)[*] resolution for the ClaimSearch services is supported by the third-party DNS platform. On Wednesday, August 16 around 9:54 AM ET, the Verisk Infrastructure team made a configuration change to the DNS platform while addressing an issue for another business unit within Verisk. Unfortunately, due to human error, the configuration was erroneously applied to all domains within Verisk instead of only the domain intended to be impacted. This resulted in Verisk customers being unable to access all the ClaimSearch Services.

 CORRECTIVE ACTION:

 On Wednesday, August 16 around 10:25 AM ET, the Verisk Infrastructure team reverted the misconfiguration in the DNS platform to resolve the issue.

 PREVENTATIVE MEASURES: 

 ·         The Verisk leadership has identified and implemented training opportunities to prevent similar human errors during troubleshooting. This includes implementing peer reviews and approvals for the high-critical changes during troubleshooting.

·         The Verisk change management team has insisted that the Verisk Infrastructure team follow the change management process for disruptive infrastructure changes, including troubleshooting.

[*] Domain Name System (DNS) is a decentralized hierarchical naming system that translates human-readable domain names (such as www.example.com) into the numerical IP addresses (such as 192.0.2.1) used by computers to identify and communicate with each other on a network, such as the internet.

Posted Sep 06, 2023 - 11:33 EDT

Resolved
The incident is resolved. We apologize for any inconvenience
Posted Aug 16, 2023 - 15:42 EDT
Monitoring
A fix has been implemented and we are recovering. Services which require queue processing will take some time to catch up on the back-log
Posted Aug 16, 2023 - 14:43 EDT
Investigating
We are currently investigating this issue.
Posted Aug 16, 2023 - 14:19 EDT
This incident affected: ClaimSearch Website, Visual Platform, Claims Inquiry (Netmap), Claims Reporting, Decision Net and System to System Interfaces (XML).