Maintenance cycle system and acceptance testing

Share on facebook
Share on twitter
Share on linkedin

Client Profile

Our client delivers intelligent virtual assistants and enterprise chatbots for customer engagement and workforce support. They offer Fortune 5000 enterprises in the travel, investment, communications, and human capital management industries a conversational artificial intelligence (AI) platform that answers complex questions, executes solutions, and calls on various data sources to deliver the right outcomes and resources across an enterprise. This increases customer and workforce engagement on any digital and voice channel

The client’s response-manager module lets content administrators edit, review, approve and publish virtual assistant and chatbot responses, including publishing entirely new response sets without requiring technical resources.

 

Project Needs

The client selected ESN to test and ensure a robust product was developed that met the client’s specification, product release strategy, and schedule. Project tasks included:

  • Defining and implementing the proper sets of test cases to cover new and modified application features
  • Staying in sync with the development team’s sprint cycle to ensure test cases were in place once the development sprint ended and that test results were available for the launch of the quality sprint
  • Ensuring a dynamic, full regression-coverage schedule as the maintenance cycle reached the final release-to-production phase

 

Challenges

  • Creating and executing a test strategy that supported continuous-integration builds and provided quality metrics to support the product owner when making release decisions
  • Defining and executing consistent and thorough manual quality-assurance testing of a complex user interface (UI) that included:
    • Conditionally enabled UI elements
    • Complex workflows with data changes triggered by the AI content-management-system

 

Technology and Tools

  • C#
  • Microsoft Test Manager
  • Microsoft Hyper-V
  • Jira
  • Microsoft Team Foundation Server (TFS)

 

Results

ESN successfully ensured the product was released with the features and functions required by the client. While doing this, we:

  • Created and executed test scenarios and test cases to verify features behaved to specification
  • Integrated new and updated test cases into the standard test sets used to confirm that the application continued to conform to the regression standards after the client made changes to the application
  • Helped the client achieve a defect-leakage rate of less than 0.1% and a test-efficiency rate of 96.91%
  • Increased the client’s test coverage for the on-time release of application- maintenance updates.