[go: up one dir, main page]
Google Git
Sign in
chromium / infra / infra / main / . / doc / testing.md
blob: 7197a8149af2700030d1fb9f5ad341dc72172b87 [file] [log] [blame] [view]
Nodir Turakulovf1fa0c92015-06-22 15:20:37[diff] [blame]1# Testing in Infra.git
2
3[TOC]
4
5## The Bare Minimum
6
7All operations on tests are performed using the [test.py](../test.py) script.
8Here are some commands for the impatient:
9
10| Command | Meaning |
11| ---------------------------- | ------------------------------------------------------- |
12| `./test.py test` | Run all tests in the repository and report results. |
13| `./test.py list` | List all tests in the repository, without running them. |
14| `./test.py test infra` | Run only tests found in the infra package. |
15| `./test.py train` | Run all tests and write expectations. |
16| `./test.py test infra:*foo*` | Run tests from infra with 'foo' in their name. |
17
18By default, `test.py` collects coverage information, and not having 100%
19coverage is an error.
20
21## Writing Tests
22
23`test.py` enforces some constraints so as to maintain a clear structure
24in the repository:
25
Nodir Turakulovf8adcbf2015-07-14 19:19:11[diff] [blame]26* tests must be methods of subclasses of unittest.TestCase. test.py
27 will *not* look for standalone functions. In addition, the method
28 name must start with 'test'.
29* tests classes must be contained in files named like `*_test.py`.
30* the coverage information for file `foo.py` is only collected from
31 tests located in `test/foo_test.py` or `tests/foo_test.py`.
Nodir Turakulovf1fa0c92015-06-22 15:20:37[diff] [blame]32
33A test fails when an exception is raised, or if expectations don't match
34(read on). Test methods can return a value. When run in train mode,
35`test.py` stores these values on disk, in directories named like
36`*.expected/` next to the file containing tests. When run in test mode,
37the return values are compared to the ones previously stored, and the
38test fails if they don't match.
39
40Example
41
42```python
43import unittest
44import
45
46class FooTest(unittest.TestCase):
47 def test_sha1(self):
48 ret = hashlib.sha1("Unimportant text").hexdigest()
49 self.assertEqual(ret, '19c12dd68b216f1a7a26d5b0290355ceef8a35b2')
50
51 def test_sha1_expectations(self):
52 ret = hashlib.sha1("Unimportant text").hexdigest()
53 return ret
54```
55
56`test_sha1` and `test_sha1_expectations` performs the same task, in a
57different way. To have both tests pass, you have to run:
58
59 ./test.py train # record output of test_sha1_expectations
60 ./test.py test
61
62## Testing App Engine with Endpoints
63
64Writing unit tests for code that uses Google Cloud Endpoints can be
65difficult. More precisely, writing the unit tests is much like writing
66any unit test, but ensuring that one's unit tests will run can be
67painful.
68
69Almost ubiquitously, one finds that testing App Engine involves testbed
70and webtest. The former facilitates stubbing of various backend
71services; the latter creates a mock application on which one can make
72API calls and inspect the results. Some interactions between Endpoints
73and webtest may prove turbid even to those used to testing App Engine
74applications; what follows is a series of prescriptions concerning the
75least obvious of these interactions.
76
77For a more detailed description of the system, adapted to the novice and
78with pointers to enlightening reading, see
79[Testing novice](testing_novice.md). For high-level documentation
80intended for the seasoned App Engine/Cloud Endpoints developer, read on.
81
82### A Worked Example
83
84`something.py` contains the API:
85
86```python
87class GoodRequest(messages.Message):
88 data = messages.IntegerField(1)
89
90
Quinten Yearsley0bdaf3b2020-04-30 21:50:48[diff] [blame]91class GreatResponse(messages.Message):
Nodir Turakulovf1fa0c92015-06-22 15:20:37[diff] [blame]92 data = messages.IntegerField(1)
93
94
95@endpoints.api(name='someendpoint', version='v1')
96class SomeEndpoint(remote.Service):
97
98 @endpoints.method(GoodRequest, GreatResponse,
99 path='/exalt', http_method='POST',
100 name='exalt')
101 def glorify(self, request):
102 glorious_number = request.data
103 if glorious_number < 0:
104 raise endpoints.BadRequestException(
105 'Perhaps you wanted to make a PessimisticRequest?')
106 response = GreatResponse(data=request.data ** 2)
107```
108
109`test/something_test.py` contains our test suite:
110
111```python
112# other imports
113from something import SomeEndpoint
114from support import test_case
115
116
117class MyNiceTestSuite(test_case.EndpointsTestCase):
118
119 api_service_cls = SomeEndpoint
120
121 def setUp(self):
122 super(MyNiceTestSuite, self).setUp()
123 # testbed setup, stub initialization, etc. should go here
124
125 def testGlorifyPerformsWonderfulSquaring(self):
126 request = {'data': 4}
127 response = self.call_api('glorify', request).json_body
128 self.assertEquals(response, {'data': 16})
129
130 def testNegativeNumbersAreNotGloriousEnough(self):
131 request = {'data': -4}
132 with self.call_should_fail('400'):
133 self.call_api('glorify', request)
134```
135
136### test_case.EndpointsTestCase Is Balm to One Parched
137
138`test_case` module (DEPSed as `/luci/appengine/components/support/test_case.py`)
139hides some of the complexity of writing test cases for Endpoints code.
140To explicate, `EndpointsTestCase` provides the following facilities:
141
Nodir Turakulovf8adcbf2015-07-14 19:19:11[diff] [blame]142* explicit creation of `endpoints.api_server` and `webtest.testApp`
143 with `setUp`
144* correct routing to endpoints methods (the user no longer needs to write
145 `'/_ah/spi/IncredibleEndpointName.someLongMethodName'`) with
146 `call_api`
147* error management (which will become error handling pending a fix for
148 [bug in `call_should_fail`](https://code.google.com/p/googleappengine/issues/detail?id=10544))
Nodir Turakulovf1fa0c92015-06-22 15:20:37[diff] [blame]149
150Much of the obscurity in Endpoints testing now evaporates. By using
151`EndpointsTestCase`, we avoid the pitfalls that inhere in setting up and
152posting to such an API in a test environment. A few final points:
153
Nodir Turakulovf8adcbf2015-07-14 19:19:11[diff] [blame]154* `api_service_cls`, a class member of the test suite, must be set;
155 otherwise, the test suite will not be able to create a test
156 application and will not have any knowledge of the API's methods
Nodir Turakulovf1fa0c92015-06-22 15:20:37[diff] [blame]157
Nodir Turakulovf8adcbf2015-07-14 19:19:11[diff] [blame]158* `EndpointsTestCase.call_api` and `EndpointsTestCase.call_should_fail` are the
159 recommended ways to make an API call and to handle errors, respectively. Note
160 that the argument structure for `call_api` is
161 `(<method name>, <request body>)`; the method name is literally the name
162 to which a method is bound in the API code, not the name specified in the decorator
Nodir Turakulovf1fa0c92015-06-22 15:20:37[diff] [blame]163
164Happy testing!