Our client is the world leader in high-resolution scanned beam display and imaging systems built upon the companyâ€™s proprietary silicon micro-mirror technology. The company designs and develops high-precision scanning systems and related technologies that enable personal and projection displays and image capture products for a broad range of military, medical, industrial, professional and consumer products.
The client contacted ESN to test the Barcode SDK against various Windows platforms, Windows mobile devices and barcode device firmware versions. SDK can be operated in tethered and un-tethered modes, using serial as well as Bluetooth connections.
The challenges involved in this project are:
- Testing all the API calls in different modes and on a number of target platforms
- All possible situations custom application developers other SDK users might encounter shall be covered in the testing
- DVT results shall be analyzed, and ESN needs to advise Microvision on the release readiness and reliability of the SDK
- Test harnesses shall be developed to automate the testing of API calls with all possible combinations of arguments values
- Generation of reports from the test harness for automated tests to better understand failures
- The ESN team shall be able pinpoint defects in the SDK source code and provide inputs for fixing the defects
- To study and report the effects of battery condition of the Flic Device on the functioning of the SDK
- To manage and track the test results of each test iteration
Test Scope And Coverage
The SDK was tested on various platforms with different Firmware devices, Bluetooth dongle devices and Bluetooth Stacks for Windows. The SDK needs to test on all the latest available Windows Mobile devices.
After studying the SDK, the ESN Team has developed a test harness to test the SDK. The test harness will call each and every API with user inputs as arguments for that API and will display the result as returned by the API.
The team has collected all possible valid, as well as invalid arguments, for every API by using boundary value analysis, equivalence partitioning techniques. In addition, scenario-based test cases were prepared to make sure that every API was working as expected in all possible scenarios.
Test Scenarios are constructed to cover the following aspects
- To make sure that the order in which the calls to the DLL are made has no unwanted impact
- To make sure DLL is providing the expected functionality to the applications
Every API is tested to ensure that it is producing the same output on all listed firmware versions and scanner modes on all operating modes. Also all the APIs are tested with various operating systems.
As testing is carried out with various combinations, producing proper test result was major challenge as part of this project. A test result table was prepared to give detailed summary of each test iteration.
Test Metrics And Defect Analysis
The testing activity metrics and results are collected and analyzed to help client decision making and also to improve the efficiency of our team.
Defect Removal Efficiency:
The ESN team has achieved more than 95% of Defect Removal Efficiency.
Test Coverage: The test Coverage in this project is 100% in terms of API coverage and test configuration coverage. In addition to this, the ESN team has ensured 85% of the code coverage by proactively reviewing the source code of the SDK during the test cases design phase
- ESN tested the SDK thoroughly on all platforms and uncovered hidden defects
- Most of the APIâ€™s has been automated so that can save lot of time
- Comprehensive defect report helped our client to develop a SDK which is robust on all the platforms
- Releasing various metrics at the appropriate time improved the client decision-making process.
- An effective regression test suite developed by ESN helped the sanity testing of various engineering releases of the SDK and firmwareâ€™s of the barcode scanner