#Test Engineering
| Capability Level | Capability Name | Description | Observed Behaviour | Project Scope |
|---|---|---|---|---|
| 0 | Test Engineering: Manual Testing | All tests are manual except for unit tests. | Testers follow checklists but only test revenue generating services on environments whose configuration they don’t understand. Release cycles get longer as new features require more testing. Regression bugs are common. |
Formalise and publish a coherent test approach to be reviewed by all engineers. Choose an automated test mechanism and framework that supports a human readable domain specific language (DSL). Code the steps necessary to allow simple user interaction functionality (such as form submission and check status type actions) to be tested at build time. Provide training to testers enabling them to write automated using the DSL tests. |
| 1 | Test Engineering: Coherent Test Approach | Simple functions have rudimentary automated tests executed at build time. | All apps and services are now tested with a common approach. Basic service function is tested with automated tests at build time. Regression bugs are less common and the test cycle has been shortened but it keeps growing as new features are added. |
Have test engineers create user stories or use cases for the test steps needed to automate more of the standard tests. Involve test engineers in use case or user story creation to ensure new feature use cases or user stories have appropriate test steps specified or test created. |
| 2 | Test Engineering: Basic Automation | Basic steps coded for automated tests | Basic steps that can be combined to form complex test cases are coded and so all core service functionality can be tested. Manual testing is still required prior to release. |
Develop tests steps and tests alongside new features. Focus test engineers on UX and ensuring all major user activities are tested at build time with automated tests. |
| 3 | Test Engineering: Automated testing | Automated user story or use case testing | Automated tests exist for all user stories and use cases. Manual checks are no longer required to ensure core service functionality. Test engineers focus their time on exploratory testing and improving the sophistication of the test systems. |
Add a performance component to tests report changes in service performance at build time and in each environment. Have test engineers code tests for all bugs found during exploratory testing. |
| 4 | Test Engineering: Continuous Delivery | Tests developed alongside new features. | Regression bugs now impossible. New feature test cases are developed alongside the new features. |