Commit 1aaa1b4d authored by Xander Miller's avatar Xander Miller

(doc): adds regression testing section

No related merge requests found
......@@ -8,7 +8,9 @@ Good QA seeks to monitor and optimize the health of these pipelines while minimi
## Code Quality Assurance
Is achieved by multiple observation and evaluation steps:
### Current Process
Code QA is achieved by multiple observation and evaluation steps:
1. Developer Authored Automated Unit Tests
2. Merge Request Development Sandboxes
3. Peer Code Reviews of Merge Requests
......@@ -17,17 +19,17 @@ Is achieved by multiple observation and evaluation steps:
QA Director's primary responsibility is the health and consistent implementation of the E2E tests. Both the automated comprehensive E2E tests and the manual smoke E2E tests.
## Merge Requests Approvals
#### Merge Requests Approvals
Before merge requests are merged into master, they should complete a smoke test. Merge requests ready to be merged should be tagged with the ~"MR::Ready to Merge" label.
### QA Testers
#### QA Testers
| Username | List |
| ------ | ------ |
| @xander-miller | [Click here](https://gitlab.com/groups/minds/-/merge_requests?scope=all&utf8=%E2%9C%93&state=opened&approver_usernames[]=xander-miller&wip=no&label_name[]=MR%3A%3AReady%20to%20Merge) |
## Staging
#### Staging
**Master** will automatically be deployed to the staging environment. Following this, the automated e2e tests will run. Once the manual qa tests have been performed, the `manual:qa` job should be run/played.
......@@ -38,8 +40,52 @@ Before merge requests are merged into master, they should complete a smoke test.
| engine | https://gitlab.com/minds/engine/environments/777308 |
| front | https://gitlab.com/minds/front/environments/779466 |
## Regressions
#### Regressions
Regressions found during the QA process in **merge requests** should be left as comments on the respective MR.
Regressions found during the QA process on **staging** should be created as issues and labeled with ~"Type::Regression" and ~"Regression::Staging" and assigned to the relevant developer. Labels also exists for ~"Regression::Canary" and ~"Regression::Production".
#### Regression Testing
Regressions are changes that damage existing functionality. Testing for regresions is achieved by developers running the exisitng unit test suite. And QA running E2E tests both automated and manual.
#### New Feature Acceptance Testing
New Feature testing is preformed by developers writing unit tests on new features and interacting with new features in local development environments and development instances. It is also preformed by stakeholders behind feature flags. And a final layer of new feature acceptance testing is performed by canary users.
### Proposed Modifications
#### Regression Testing
Manual smoke testing and automated comprehensive testing is an effective combination. Still we should outline how our coverage will grow and what steps will be taken to enhance the process.
Increased cypress test coverage active maintainance of cypress test suite.
- We need a `master-sandbox` development instance running the current master code to use as a baseline to develop cypress tests and establish an all passing null hypothesis.
- We need to increase our cypress coverage every sprint
- We should created HTML ids on the front end to identify UX elements and make testing easier, future proof tests against design changes.
- We need our tests to be robust so the automated E2E tests failing is an anomaly requiring investigation not a frustration to be ignored
Our dev instances need to be improved for E2E testing
- We need ways to test complex fatures like wiring, etherium processes, off chain tokens rewards, phone number validation, messaging, jitsi gatherings, pro, plus, admin, boosting, videos.
- We need data in the dev instances for testing discovery, search and interactions.
Smoke tests coverage should be increased and refined but always managable
- manual smoke tests plans should be run on both mobile platforms as well as desktop browser
- smoke testing should be done routinely but optimally when there are changes, managed through good communication
Mobile automated E2E tests coverage should be maintained as cypress test are maintained
- release schedule of mobile app will make e2e testing different from cypress testing but a similar pipeline should be established
#### New Feature Acceptance Testing
The primary limiting factor on our New Feature acceptance testing has been a lack of detailed wireframes and specifications. Wireframes and specifications are needed to create the standard a new feature is being tested against.
## Communication Quality Assurance
### Current Process
### Proposed Modifications
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment