This commit adds a new test to make sure ROMFS files are mounted and
read correctly, to be run as part of the CI process.
The changes also include the source binary files that have been used to
create the pre-baked ROMFS partition image used in the test, along with
a Makefile to allow recreating said file.
The CI test ROMFS image is mounted only if no other ROMFS partition is
mounted in slot 0. The specific test is executed only if there actually
is a ROMFS partition mounted and if the partition is identified as the
one used to run tests on. This allows for user images to be mounted and
for a successful test run if that is the case.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit renames the "tests/frozen" directory into "tests/assets" to
make it more explicit that it does contain files that are needed for
other tests to function.
Right now there's only a single pre-compiled module being used for
miscellaneous tests, but it will soon hold ROMFS test data as well.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Otherwise, a very deeply nested regular expression like
re.compile("(" * 65536)
can exhaust the host stack during the compile phase. This turns
that into a `RuntimeError: maximum recursion depth exceeded`
instead.
This crash was found via fuzzing.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
Properly cast tuple subclasses to the native base class before performing
`__add__` or `__iadd_`.
Closes: #7304
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
It is a mistake in Python code if the result of readinto()/write()
is more bytes than requested. This can lead to misbehavior
e.g., in mp_stream_rw if the invariant is not respected.
CPython appears to largely ignore the values returned from
readinto()/write(), at least from `print()` & `json.load()`.
Consequently, an expected output file is needed for the
new test.
Closes: #18845
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
Document this in cpydiff and add a test with expected output
for coverage testing.
As discussed in #11441, the code growth from handling this
case seems to outweigh the benefit of implementing it
properly.
Closes: #11439
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
This commit optimises the lookup of opcodes' condition codes, in order
to take up less code than before.
The original data was held in a table containing the condition code
value (an incrementing 0-based integer) and the two condition ASCII
characters. Given that the condition code value also matches the
entry's index in the table, that can be safely omitted.
This saves 52 bytes when compiled for Thumb.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds an implementation for "socket.sendall" to the Unix
port.
Right now the implementation is a wrapper over a single "socket.send"
call, since it wasn't possible to let "socket.send" perform a partial
transfer. With no partial transfers it is not possible to have a
testable implementation following the expected behaviour, so a single
send operation is done and OSErr(EINTR) is raised if a partial transfer
occurs.
The discussion for the issue linked to this PR contains more information
about the efforts made to let partial transfers happen.
This fixes#16803.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit extends the parser's behaviour to also recognise constant
integer assignments even when the assignment itself has a type
annotation.
Now declarations like "var: int = const(val)" can be treated as
constants by the compiler and thus be folded in the rest of the program
as it sees fit. Before this change, that line would generate a variable
creation and its value assignment, without any folding being performed.
This fixes#15608.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Document the behavior on expression annotations rejected by CPython but
accepted by MicroPython.
Closes issue #19031.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
If a port enables t-strings then it is required to have the
`string.templatelib` package (at least to run the tests). That's
automatically the case if `MICROPY_PY_TSTRINGS` is enabled.
If a port freezes in the micropython-lib `string` extension package then
the latest version of this package will include the built-in
`string.templatelib` classes. So the feature check for t-strings no longer
needs to check if they are available.
Signed-off-by: Damien George <damien@micropython.org>
The webassembly port needs some additional weakref tests due to the fact
that garbage collection only happens when Python execution finishes and
JavaScript resumes.
The `tests/ports/webassembly/heap_expand.py` expected output also needs to
be updated because the amount of GC heap got smaller (weakref WTB takes
some of the available RAM).
Signed-off-by: Damien George <damien@micropython.org>
This adds support for the standard `weakref` module, to make weak
references to Python objects and have callbacks for when an object is
reclaimed by the GC.
This feature was requested by PyScript, to allow control over the lifetime
of external proxy objects (distinct from JS<->Python proxies).
Addresses issue #646 (that's nearly a 12 year old issue!).
Functionality added here:
- `weakref.ref(object [, callback])` create a simple weak reference with
optional callback to be called when the object is reclaimed by the GC
- `weakref.finalize(object, callback, /, *args, **kwargs)` create a
finalize object that holds a weak reference to an object and allows more
convenient callback usage and state change
The new module is enabled at the "everything" level.
The implementation aims to be as efficient as possible, by adding another
bit-per-block to the garbage collector, the WTB (weak table). Similar to
the finalizer bit (FTB), if a GC block has its corresponding WTB bit set
then a weak reference to that block is held. The details of that weak
reference are stored in a global map, `mp_weakref_map`, which maps weak
reference to ref/finalize objects, allowing the callbacks to be efficiently
found when the object is reclaimed.
With this feature enabled the overhead is:
- 1/128th of the available memory is used for the new WTB table (eg a 128k
heap now needs an extra 1k for the WTB).
- Code size is increased.
- At garbage collection time, there is a small overhead to check if the
collected objects had weak references. This check is the same as the
existing FTB finaliser scan, so shouldn't add much overhead. If there
are weak reference objects alive (ref/finalize objects) then additional
time is taken to call the callbacks and do some accounting to clean up
the used weak reference.
Signed-off-by: Damien George <damien@micropython.org>
Because socket objects have a finaliser they must be created carefully, in
case an exception is raised during the population of their members, eg
invalid input argument or out-of-memory when allocating additional arrays.
Prior to the fix in this commit, the finaliser would crash due to
`incoming.udp_raw.array` being an invalid pointer in the following cases:
- if a SOCK_RAW was created with a proto argument that was not an integer
- if a SOCK_DGRAM or SOCK_RAW was created where the allocation of
`lwip_incoming_packet_t` failed
- if an integer was passed in for the socket type but it was not one of
SOCK_STREAM, SOCK_DGRAM or SOCK_RAW
Furthermore, if the allocation of `lwip_incoming_packet_t` failed then it
may have led to corruption within lwIP when freeing `socket->pcb.raw`
because that PCB was not fully set up with its callbacks.
This commit fixes all of these issues by ensuring:
- `pcb.tcp` and `incoming.udp_raw.array` are initialised to NULL early on
- the proto argument is parsed before allocating the PCB
- the allocation of `lwip_incoming_packet_t` occurs befor allocating the
PCB
- `incoming.udp_raw.array` is checked for NULL in the finaliser code
The corresponding test (which already checked most of these causes of
failure) has been updated to include a previously-uncovered scenario.
Signed-off-by: Damien George <damien@micropython.org>
Implemented according to API docs in a parent comment.
Adds new multi_extmod/machine_can_* tests which pass when testing between
NUCLEO_G474RE, NUCLEO_H723ZG and PYBDV11.
This work was mostly funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Simplifies the pattern of an optional arg which can be a list of at
least a certain length, otherwise one is lazily initialised.
Modify pyb.CAN and ESP-NOW APIs to use the helper. Note this changes
the return type of pyb.CAN.recv() from tuple to list.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Reclassify failures of tests listed in flaky_tests_to_ignore as "ignored"
instead of retrying them. Ignored tests still run and their output is
reported, but they don't affect the exit code. The ci.sh --exclude lists
for these tests are removed so they run normally.
Signed-off-by: Andrew Leech <andrew.leech@planet-innovation.com>
Includes corresponding .exp files because this feature is only available in
Python 3.14+.
Tests for `!a` conversion specifier and space after `!` are not included
because they are not supported by MicroPython.
Signed-off-by: Koudai Aono <koxudaxi@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support for t-strings by leveraging the existing f-string
parser in the lexer. It includes:
- t-string parsing in `py/lexer.c`
- new built-in `__template__()` function to construct t-string objects
- new built-in `Template` and `Interpolation` classes which implement all
the functionality from PEP 750
- new built-in `string` module with `templatelib` sub-module, which
contains the classes `Template` and `Interpolation`
The way the t-string parser works is that an input t-string like:
t"hello {name:5}"
is converted character-by-character by the lexer/tokenizer to:
__template__(("hello ", "",), name, "name", None, "5")
For reference, if it were an f-string it would be converted to:
"hello {:5}".format(name)
Some properties of this implementation:
- it's enabled by default at the full feature level,
MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_FULL_FEATURES
- when enabled on a Cortex-M bare-metal port it costs about +3000 bytes
- there are no limits on the size or complexity of t-strings, and it allows
arbitrary levels of nesting of f-strings and t-strings (up to the memory
available to the compiler)
- the 'a' (ascii) conversion specifier is not supported (MicroPython does
not have the built-in `ascii` function)
- space after conversion specifier, eg t"{x!r :10}", is not supported
- arguments to `__template__` and `Interpolation` are not fully validated
(it's not necessary, it won't crash if the wrong arguments are passed in)
Otherwise the implementation here matches CPython.
Signed-off-by: Damien George <damien@micropython.org>
Because it requires a different configuration of the pins (in `setUp`).
Eg on mimxrt pins used for a `machine.Counter` cannot be read.
Signed-off-by: Damien George <damien@micropython.org>
Because it requires a different configuration of the pins (in `setUp`).
Eg on mimxrt pins used for an Encoder cannot be read.
Signed-off-by: Damien George <damien@micropython.org>
As per CPython behaviour, `b"".hex()` should return an empty str object
(not an empty bytes object).
Fixes issue #18807.
Signed-off-by: Damien George <damien@micropython.org>
As of CPython 3.7 `bytes.fromhex()` skips all ASCII whitespace. And as of
CPython 3.8 `bytes.hex()` supports the optional separator argument. So
this test no longer needs a .exp file.
Signed-off-by: Damien George <damien@micropython.org>
Fixes rp2 issue where socket.getaddrinfo() could block indefinitely if an
interface goes down and still has a DNS server configured, as the LWIP
timer stops running and can't time out the DNS query.
Adds a regression test under multi_wlan that times out on rp2 without this
fix.
Fixes issue #18797.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Occasionally, it's useful to be able to compare MicroPython's performance
figures to CPython's. This change adds the ability to run the internalbench
test runner with `--test-instance=cpython` in order to execute the same
test routines against CPython and produce a benchmark performance report in
the same format as MicroPython.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Previously, mpycert.der was the Intermediate certificate which is regularly
re-issued by Letsencrypt.
Also changes ssl_cert.py to load the cert data from the same file as
test_sslcontext_client.py, so the DER string doesn't have to be pasted
into the source.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Fixes fatal crash if serial port access returns an error (for example: port
is native USB-CDC and the host hard faults during the test run). Instead of
crashing, have the runner mark this as a test run error and continue.
It's not certain the next test will run successfully, but this provides the
context of output showing what was happening when the communication error
occurred. Without this change, that output is lost when the fatal exception
terminates the runner process.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Test file paths which get passed to the run_tests function can be
absolute or relative and with or without leading slash in the latter
case, depending on the arguments to run-tests.py, but since that path
is used to:
- display which tests run
- record which tests ran in the results.json
- craft the filename for the .exp/.out file for failed tests
it is desirable to always use the same file path irregardless of
how the user passed the path.
In practice this means that all forms of running our own tests like:
>python ./run-tests.py -i extmod
>python ./run-tests.py -d extmod
>python ./run-tests.py -d ./extmod
>python ./run-tests.py -d ../tests/extmod
>python ./run-tests.py -d /full/path/to/tests/extmod
will now consistently all display the tests like
pass extmod/time_time_ns.py
FAIL extmod/some_failing_test.py
and produce output files like
results/extmod_some_failing_test.py.exp
results/extmod_some_failing_test.py.out
instead of displaying/using the exact path as passed.
For external tests, meaning not in the tests/ directory, we also want
to be consistent so there the choice was made to always use absolute
paths.
Signed-off-by: stijn <stijn@ignitron.net>
Scan the --test-dirs argument for the main tests directory being
passed and if so do the same thing as if running from within that
main test directory.
In practice this makes the following (which used to counterintuitively
try and fail to run the .py files in the tests/ directory itself)
>python micropython/tests/run-tests.py -d micropython/tests
do the same thing as
>cd micropython/tests
>python ./run-tests.py
which is logical and convenient.
Signed-off-by: stijn <stijn@ignitron.net>
Test file paths which get passed to the run_tests function can be
absolute or relative and with or without leading slash in the latter
case, depending on the arguments to run-tests.py, but the skip_tests
list with tests to skip only contains relative paths so using simple
string equality comparison easily leads to false negatives.
Compare the full absolute path instead such that it doesn't matter
anymore in which form the tests are passed. Note:
- use realpath to resolve symlinks plus make the comparison case
insensitive on windows
- the test_file passed to run_one_test is not altered by this commit,
such that when the user inputs relative paths the tests are also
still displayed with relative paths
- likewise the test_file_abspath is not modified because functions
like run_micropython rely on it having forward slashes
In practice this means that it used to be so that the only forms
of running tests for which the skip_tests lists actually worked were:
>python ./run-tests.py
>python ./run-tests.py -d extmod
whereas it now works consistently so also for these invocations,
which in the end all point to the exact same path:
>python ./run-tests.py -d ./extmod
>python ./run-tests.py -d ../tests/extmod
>python ./run-tests.py -d /full/path/to/tests/extmod
These examples used to not skip any of the tests in the extmod/
directory thereby leading to test failures.
Signed-off-by: stijn <stijn@ignitron.net>
This is convenient when trying to figure out the correct values
for --include/--exclude/--test-dirs/... arguments.
Signed-off-by: stijn <stijn@ignitron.net>
This commit introduces a test that should check whether viper load or
store operations won't clobber either the buffer address or the index
value being used.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces support for writing inline assembler code
snippets when targeting Xtensa cores that use register windows (eg.
the whole ESP32 family).
Opcodes support is still limited to what the ESP8266 supports (as in,
LX3 cores opcodes), however each LX core version is guaranteed to
support all previous versions' opcodes as well. The ESP32 does not have
the inline assembler enabled by default, following the existing
expectations when it comes to firmware footprint.
Since now emitted functions may have one of two possible exit sequences,
the L32I test had to be fixed. It would return the word containing the
L32I opcode itself, but the upper 8 bits of the word came from the
following opcode - which can change depending on the exit code sequence.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
