fdb7c0f747
The aim of this commit is to clarify the command line options available. While they are available as well as in the CLI with --help, it's useful to document them and provide a few examples. Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
144 lines
4.3 KiB
ReStructuredText
144 lines
4.3 KiB
ReStructuredText
.. _writingtests:
|
|
|
|
Writing tests
|
|
=============
|
|
|
|
Tests in MicroPython are located at the path ``tests/``. The following is a listing of
|
|
key directories and the run-tests.py runner script:
|
|
|
|
.. code-block:: bash
|
|
|
|
.
|
|
├── basics
|
|
├── extmod
|
|
├── float
|
|
├── micropython
|
|
├── run-tests.py
|
|
...
|
|
|
|
There are subfolders maintained to categorize the tests. Add a test by creating a new file in one of the
|
|
existing folders or in a new folder. It's also possible to make custom tests outside this tests folder,
|
|
which would be recommended for a custom port.
|
|
|
|
For example, add the following code in a file ``print.py`` in the ``tests/unix/`` subdirectory:
|
|
|
|
.. code-block:: python
|
|
|
|
def print_one():
|
|
print(1)
|
|
|
|
print_one()
|
|
|
|
If you run your tests, this test should appear in the test output:
|
|
|
|
.. code-block:: bash
|
|
|
|
$ cd ports/unix
|
|
$ make tests
|
|
skip unix/extra_coverage.py
|
|
pass unix/ffi_callback.py
|
|
pass unix/ffi_float.py
|
|
pass unix/ffi_float2.py
|
|
pass unix/print.py
|
|
pass unix/time.py
|
|
pass unix/time2.py
|
|
|
|
Tests are run by comparing the output from the test target against the output from CPython.
|
|
So any test should use print statements to indicate test results.
|
|
|
|
For tests that can't be compared to CPython (i.e. micropython-specific functionality),
|
|
you can provide a ``.py.exp`` file which will be used as the truth for comparison.
|
|
|
|
The other way to run tests, which is useful when running on targets other than the Unix port, is:
|
|
|
|
.. code-block:: bash
|
|
|
|
$ cd tests
|
|
$ ./run-tests.py
|
|
|
|
Then to run on a board:
|
|
|
|
.. code-block:: bash
|
|
|
|
$ ./run-tests.py -t /dev/ttyACM0
|
|
|
|
And to run only a certain set of tests (eg a directory):
|
|
|
|
.. code-block:: bash
|
|
|
|
$ ./run-tests.py -d basics
|
|
$ ./run-tests.py float/builtin*.py
|
|
|
|
Using run-tests.py
|
|
------------------
|
|
|
|
The ``run-tests.py`` script supports several parameters to customize test execution:
|
|
|
|
**Target and Device Selection:**
|
|
|
|
* ``-t, --test-instance``
|
|
|
|
The -t option accepts the following for the test instance:
|
|
|
|
- **unix** - use the unix port of MicroPython, specified by the MICROPY_MICROPYTHON
|
|
environment variable (which defaults to the standard variant of either the unix
|
|
or windows ports, depending on the host platform)
|
|
- **webassembly** - use the webassembly port of MicroPython, specified by the
|
|
MICROPY_MICROPYTHON_MJS environment variable (which defaults to the standard
|
|
variant of the webassembly port)
|
|
- **port:<device>** - connect to and use the given serial port device
|
|
- **a<n>** - connect to and use /dev/ttyACM<n>
|
|
- **u<n>** - connect to and use /dev/ttyUSB<n>
|
|
- **c<n>** - connect to and use COM<n>
|
|
- **exec:<command>** - execute a command and attach to it's stdin/stdout
|
|
- **execpty:<command>** - execute a command and attach to the printed /dev/pts/<n> device
|
|
- **<a>.<b>.<c>.<d>** - connect to the given IPv4 address
|
|
- anything else specifies a serial port
|
|
|
|
**Test Selection:**
|
|
|
|
* ``-d, --test-dirs`` - Specify one or more test directories to run
|
|
* ``-i, --include REGEX`` - Include tests matching regex pattern
|
|
* ``-e, --exclude REGEX`` - Exclude tests matching regex pattern
|
|
* ``files`` - Specific test files to run
|
|
|
|
**Execution Options:**
|
|
|
|
* ``--emit <EMITTER>`` - MicroPython emitter, EMITTER can be bytecode or native. Default: bytecode
|
|
* ``--via-mpy`` - Compile .py files to .mpy first
|
|
* ``--heapsize`` - Set heap size for tests
|
|
* ``-j, --jobs N`` - Number of tests to run simultaneously
|
|
|
|
Set the MICROPY_MPYCROSS environment variable to use a specific version of ``mpy-cross`` when using ``--via-mpy``.
|
|
|
|
**Result Management:**
|
|
|
|
* ``-r, --result-dir`` - Directory for test results. Default: results/
|
|
* ``--print-failures`` - Show diff of failed tests and exit
|
|
* ``--clean-failures`` - Delete .exp and .out files from prior failed tests
|
|
* ``--run-failures`` - Re-run only previously failed tests
|
|
|
|
**Examples:**
|
|
|
|
.. code-block:: bash
|
|
|
|
# Run only basic tests with native emitter
|
|
$ ./run-tests.py --emit native -d basics extmod
|
|
|
|
# Run tests excluding async functionality
|
|
$ ./run-tests.py -e async
|
|
|
|
# Run tests matching *_pep_*
|
|
$ ./run-tests.py -i *_pep_*
|
|
|
|
# Run specific test files in parallel
|
|
$ ./run-tests.py -j 4 basics/list*.py
|
|
|
|
# Test on connected ESP32 board
|
|
$ ./run-tests.py -t /dev/ttyUSB0
|
|
# or
|
|
$ ./run-tests.py -t u0
|
|
|
|
# Re-run only failed tests from previous run
|
|
$ ./run-tests.py --run-failures
|