| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 1 | # ANGLE + dEQP |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 2 | |
| 3 | drawElements (dEQP) is a very robust and comprehensive set of open-source |
| 4 | tests for GLES2, GLES3+ and EGL. They provide a huge net of coverage for |
| 5 | almost every GL API feature. ANGLE by default builds dEQP testing targets for |
| 6 | testing against GLES 2, GLES 3, EGL, and GLES 3.1 (on supported platforms). |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 7 | |
| 8 | ## How to build dEQP |
| 9 | |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 10 | You should have dEQP as a target if you followed the [DevSetup](DevSetup.md) |
| Yuly Novikov | 1c9ac27 | 2023-12-13 17:34:19 | [diff] [blame] | 11 | instructions. Some of the current targets are: |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 12 | |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 13 | * `angle_deqp_egl_tests` for EGL 1.x tests |
| Yuly Novikov | 1c9ac27 | 2023-12-13 17:34:19 | [diff] [blame] | 14 | * `angle_deqp_gles2_tests` for GLES 2.0 tests |
| 15 | * `angle_deqp_gles3_tests` for GLES 3.0 tests |
| 16 | * `angle_deqp_gles31_tests` for GLES 3.1 tests |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 17 | |
| 18 | ## How to use dEQP |
| 19 | |
| Rahul Jain | d775684 | 2017-12-22 10:36:09 | [diff] [blame] | 20 | Note: |
| Yuly Novikov | 1c9ac27 | 2023-12-13 17:34:19 | [diff] [blame] | 21 | To run an individual test, use the `--gtest_filter` argument. |
| 22 | It supports simple wildcards. For example: `--gtest_filter=dEQP-GLES2.functional.shaders.linkage.*`. |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 23 | |
| angle-autoroll | 2937ee2 | 2024-01-08 20:01:48 | [diff] [blame] | 24 | The tests lists are sourced from the Android CTS mains in |
| 25 | `third_party/VK-GL-CTS/src/android/cts/main`. See `gles2-main.txt`, |
| 26 | `gles3-main.txt`, `gles31-main.txt` and `egl-main.txt`. |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 27 | |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 28 | If you're running a full test suite, it might take very long time. Running in |
| 29 | Debug is only useful to isolate and fix particular failures, Release will give |
| 30 | a better sense of total passing rate. |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 31 | |
| Shahbaz Youssefi | 2553690 | 2019-01-23 14:49:03 | [diff] [blame] | 32 | ### Choosing a Renderer |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 33 | |
| Shahbaz Youssefi | b7b46ee | 2024-08-05 14:22:48 | [diff] [blame] | 34 | By default ANGLE tests with Vulkan, except on Apple platforms where OpenGL is used. |
| Shahbaz Youssefi | 2553690 | 2019-01-23 14:49:03 | [diff] [blame] | 35 | To specify the exact platform for ANGLE + dEQP, use the arguments: |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 36 | |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 37 | * `--deqp-egl-display-type=angle-d3d11` for D3D11 (highest available feature level) |
| Geoff Lang | 23a22c8 | 2015-08-31 16:49:51 | [diff] [blame] | 38 | * `--deqp-egl-display-type=angle-d3d9` for D3D9 |
| 39 | * `--deqp-egl-display-type=angle-d3d11-fl93` for D3D11 Feature level 9_3 |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 40 | * `--deqp-egl-display-type=angle-gl` for OpenGL Desktop (OSX, Linux and Windows) |
| 41 | * `--deqp-egl-display-type=angle-gles` for OpenGL ES (Android/ChromeOS, some Windows platforms) |
| Jonah Ryan-Davis | 6e4004d | 2019-12-06 19:41:17 | [diff] [blame] | 42 | * `--deqp-egl-display-type=angle-metal` for Metal (Mac) |
| Jonah Ryan-Davis | 5afd5ec | 2019-10-25 16:53:40 | [diff] [blame] | 43 | * `--deqp-egl-display-type=angle-swiftshader` for Vulkan with SwiftShader as driver (Android, Linux, Mac, Windows) |
| Jonah Ryan-Davis | 6e4004d | 2019-12-06 19:41:17 | [diff] [blame] | 44 | * `--deqp-egl-display-type=angle-vulkan` for Vulkan (Android, Linux, Windows) |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 45 | |
| Shahbaz Youssefi | 2553690 | 2019-01-23 14:49:03 | [diff] [blame] | 46 | The flag `--use-angle=X` has the same effect as `--deqp-egl-display-type=angle-X`. |
| 47 | |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 48 | ### Check your results |
| 49 | |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 50 | If run from Visual Studio 2015, dEQP generates a test log to |
| Frank Henigman | 3c613c0 | 2017-11-17 04:45:34 | [diff] [blame] | 51 | `out/sln/obj/src/tests/TestResults.qpa`. To view the test log information, you'll need to |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 52 | use the open-source GUI |
| 53 | [Cherry](https://android.googlesource.com/platform/external/cherry). ANGLE |
| 54 | checks out a copy of Cherry to `angle/third_party/cherry` when you sync with |
| 55 | gclient. Note, if you are using ninja or another build system, the qpa file |
| 56 | will be located in your working directory. |
| Geoff Lang | 13e569d | 2015-06-05 18:33:37 | [diff] [blame] | 57 | |
| Frank Henigman | c677795 | 2017-11-27 05:15:54 | [diff] [blame] | 58 | See the [official Cherry README](https://android.googlesource.com/platform/external/cherry/+/master/README) |
| 59 | for instructions on how to run Cherry on Linux or Windows. |
| Jamie Madill | 5798c0f | 2015-08-27 21:18:57 | [diff] [blame] | 60 | |
| 61 | ### GoogleTest, ANGLE and dEQP |
| 62 | |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 63 | ANGLE also supports the same set of targets built with GoogleTest, for running |
| 64 | on the bots. We don't currently recommend using these for local debugging, but |
| Ian Elliott | 61dc769 | 2021-09-28 21:07:20 | [diff] [blame] | 65 | we do maintain lists of test expectations in `src/tests/deqp_support` (see |
| Ian Elliott | 7c1622c | 2021-10-01 22:42:29 | [diff] [blame] | 66 | [Handling Test Failures](TestingAndProcesses.md)). When |
| Jamie Madill | 495d0ac | 2016-12-21 18:22:14 | [diff] [blame] | 67 | you fix tests, please remove the suppression(s) from the relevant files! |
| Cody Northrop | 1cd52d2 | 2020-02-18 22:28:29 | [diff] [blame] | 68 | |
| 69 | ### Running dEQP on Android |
| 70 | |
| Roman Lavrov | cd4cce3 | 2022-05-17 22:35:36 | [diff] [blame] | 71 | When you only need to run a few tests with `--gtest_filter` you can use Android wrappers such as `angle_deqp_egl_tests` directly but beware that Android test runner wipes data by default (try `--skip-clear-data`). |
| 72 | |
| 73 | Running the tests not using the test runner is tricky, but is necessary in order to get a complete TestResults.qpa from the dEQP tests when running many tests (since the runner shards the tests, only the results of the last shard will be available when using the test runner). First, use the runner to install the APK, test data and test expectations on the device. After the tests start running, the test runner can be stopped with Ctrl+C. Then, run |
| Cody Northrop | 1cd52d2 | 2020-02-18 22:28:29 | [diff] [blame] | 74 | ``` |
| 75 | adb shell am start -a android.intent.action.MAIN -n org.chromium.native_test/.NativeUnitTestNativeActivity -e org.chromium.native_test.NativeTest.StdoutFile /sdcard/chromium_tests_root/out.txt |
| 76 | ``` |
| Roman Lavrov | cd4cce3 | 2022-05-17 22:35:36 | [diff] [blame] | 77 | After the tests finish, get the results with (requires `adb root`) |
| Cody Northrop | 1cd52d2 | 2020-02-18 22:28:29 | [diff] [blame] | 78 | ``` |
| Roman Lavrov | 41adabc | 2022-04-29 19:04:27 | [diff] [blame] | 79 | adb pull /data/data/com.android.angle.test/TestResults.qpa . |
| Cody Northrop | 1cd52d2 | 2020-02-18 22:28:29 | [diff] [blame] | 80 | ``` |
| 81 | Note: this location might change, one can double-check with `adb logcat -d | grep qpa`. |