|
| 1 | +Testing for DevTools |
| 2 | +======================= |
| 3 | + |
| 4 | +DevTools is test covered by multiple types of tests, all of which are run on the CI for each DevTools PR / commit: |
| 5 | + |
| 6 | +1. Unit tests |
| 7 | + - tests for business logic |
| 8 | +2. Widget tests |
| 9 | + - tests for DevTools UI components using mock or fake data |
| 10 | + - some widget tests may contain golden image testing |
| 11 | +3. Partial integration tests |
| 12 | + - tests for DevTools UI and business logic with a real VM service connection to a test app |
| 13 | +4. Full integration tests |
| 14 | + - Flutter web integration tests that run DevTools as a Flutter web app and connect it to a real test app |
| 15 | + on multiple platforms (Flutter mobile, Flutter web, and Dart CLI) |
| 16 | + |
| 17 | +**The following instructions are for unit tests, widget tests, and partial integration tests** in DevTools. For instructions |
| 18 | +on running and writing full integration tests, please see [integration_test/README.md](integration_test/README.md). |
| 19 | +In general, we should first try to test cover new features and bug fixes with unit tests or widget tests |
| 20 | +before writing new integration tests, which are slower to run and are not as easy to debug or iterate upon. |
| 21 | + |
| 22 | +## Running DevTools tests |
| 23 | + |
| 24 | +Before running tests, make sure your Flutter SDK matches the version that will be used on |
| 25 | +the CI. To update your local flutter version, run: |
| 26 | + |
| 27 | +``` |
| 28 | +./tool/update_flutter_sdk.sh --local |
| 29 | +``` |
| 30 | +> Note: Running this command requires that you have followed the [set up instructions](CONTRIBUTING.md#set-up-your-devtools-environment) |
| 31 | +in the DevTools contributing guide regarding cloning the Flutter SDK from GitHub. |
| 32 | + |
| 33 | +Now you can proceed with running DevTools tests: |
| 34 | + |
| 35 | +``` |
| 36 | +cd packages/devtools_app |
| 37 | +flutter test test/ |
| 38 | +``` |
| 39 | + |
| 40 | +### Updating golden image files |
| 41 | + |
| 42 | +> Note: golden images should only be generated on MacOS. |
| 43 | +
|
| 44 | +Golden image tests will fail for one of three reasons: |
| 45 | + |
| 46 | +1. The UI has been _intentionally_ modified. |
| 47 | +2. Something changed in the Flutter framework that would cause downstream changes for our tests. |
| 48 | +3. The UI has been _unintentionally_ modified, in which case we should not accept the changes. |
| 49 | + |
| 50 | +For valid golden image updates (1 and 2 above), the failing golden images will need to be updated. This can |
| 51 | +be done in one of two ways: |
| 52 | + |
| 53 | +1. If the tests failed on the CI for a PR, we can download the generated golden images directly from GitHub. |
| 54 | + > If you are developing on a non-MacOS machine, this is the only way you'll be able to update the golden images. |
| 55 | + - Natvigate to the failed Actions run for your PR on GitHub. Example: |
| 56 | + |
| 57 | +  |
| 58 | + |
| 59 | + - Scroll to the bottom of the Summary view to see the errors from the `macos goldens` job, and the notice containing the golden update command: |
| 60 | + |
| 61 | +  |
| 62 | + |
| 63 | +2. Update the goldens locally by running the failing test(s) with the `--update-goldens` flag. |
| 64 | + |
| 65 | + - Before updating the goldens, ensure your version of Flutter matches the version of Flutter that is used |
| 66 | + on the CI. To update your local flutter version, run: |
| 67 | + |
| 68 | + ``` |
| 69 | + ./tool/update_flutter_sdk.sh --local |
| 70 | + ``` |
| 71 | + |
| 72 | + - Then proceed with updating the goldens: |
| 73 | + |
| 74 | + ``` |
| 75 | + flutter test <path/to/my/test> --update-goldens |
| 76 | + ``` |
| 77 | +
|
| 78 | + or to update goldens for all tests: |
| 79 | + ``` |
| 80 | + flutter test test/ --update-goldens |
| 81 | + ``` |
| 82 | +
|
| 83 | +## Writing DevTools tests |
| 84 | +
|
| 85 | +When you add a new feature or fix a bug, please add a corresponding test for your change. |
| 86 | +
|
| 87 | +- If there is an existing test file for the feature your code touches, you can add the test case |
| 88 | +there. |
| 89 | +- Otherwise, create a new test file with the `_test.dart` suffix, and place it in an appropriate |
| 90 | +location under the `test/` directory for the DevTools package you are working on. |
| 91 | +
|
0 commit comments