Following the Path: Software Testing of Advanced Utility Website & IVR Expands Customer Experience

Chatbot Following IVR Path

Utility Companies are keeping up with the growing advances in technology on their websites: customers can easily stop, start, or even move their service from one premise to another – all online or on their mobile device.

Voice technology has enabled telephony features such as Chatbot where customers can “ask” their virtual assistant using voice recognition and artificial intelligence to answer questions pertaining to the utility service. Utility companies are offering online shopping by using Marketplace to buy energy-efficient items such as light bulbs and thermostats. Want to learn more about Solar Energy? Customers can click on the utility’s website to learn if their home is a candidate for solar panels, how many need to be installed, and – most helpfully – how much money a customer can save monthly. All these technology advances are making things easier for their customers.


Olenick keeps up with these advances by maintaining an inventory of the latest iOS and Android mobile devices and bringing our mobile testing expertise to support this new technology – not only making sure that the product functions properly, but also supporting Accessibility Testing to ensure each website is compliant with WCAG 2.1 standards.

Chatbot Testing


One of the biggest challenges with testing new website technology is test data. Incorrect data can lead to:

  • Incorrect test results
  • Increased number of defects where the Developer must take time to analyze, causing delays
  • Unreliable results
  • Project requires more time and expenses which impedes “speed to market” of new features
  • Sensitive customer data could be lost
  • Test data can get outdated
  • Increased production issues and cost


Having a solid test data strategy ensures a successful testing phase:

  • Analyze the data and making sure it supports end-to-end test cases/scenarios
  • Identify sensitive test data and “mask” it to keep secured and protected
  • Establish a test data refresh process, update existing data and add new data to support new functionality


In addition to a test data strategy, specific and clear test requirements are key to successful testing. Olenick software testers live in an Agile world and have fully embraced the values of light process, ‘good enough’ software, and continuous improvement….and then they hit a complicated IVR project!


As customers embrace new operational efficiencies though Integrated Voice Response (IVR) systems, Software Testers must also be prepared to amp up the efficiency on testing. How do you make sure you cover all the requirements and design of these complex systems? Follow the path!


One solution for limited test requirements is to look to the design document to be the oracle for testing. Typically, an IVR project will have a detailed flow showing all the paths that a user might take through the application. Path coverage is the concept of making sure each code path is covered at least once. For the IVR, make sure every path in the flow diagram is covered.


So how do you do it? One way is to start with a printout of the IVR flow documentation, sharpen pencils of many colors, and start manually tracing colored paths through the IVR to create test cases covering each unique path. Mark each path with a test case number to cross reference the test case to the design document.

Colored Pencils With Lines


While it might seem cumbersome at first to invest the time into marking up all the paths, it allows testers to review the marked documents and begin developing test cases right as the design is being coded. The Team Lead or a senior member of the testing team can continue to mark paths as the other testers begin to document, build, and execute test cases in a very short amount of time. The result is an efficient method to develop test cases which also provides proof of high-quality testing coverage!


Olenick has used this method on other projects, most recently on the testing of another operational efficiency – the introduction of a Chatbot to reduce live Customer Service interactions. For this Chatbot project there were highly developed, detailed flowcharts that became a guide. Testers were able to copy the flows into Word and mark them up online with paths of many colors to mark the future test cases. This process ensured the testing team had excellent testing coverage with the fewest amount of test cases.


When faced with the prospect of testing a customer-facing system with a lot of interactions, why not try going with the flow and using path coverage?


Virtual assistant technology, along with other advanced web and customer interaction features, are on the rise and continuously evolving. Contact Olenick to learn more about how we can ensure that your applications are running smoothly!


Contributing Editor Eric Byville

Patricia O'Donnell Headshot

Patricia O’Donnell    

Bryan Kulp Headshot

Bryan Kulp    

Kevin Flynn Headshot

Kevin Flynn          View Bio