Properly cast tuple subclasses to the native base class before performing
`__add__` or `__iadd_`.
Closes: #7304
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
Document this in cpydiff and add a test with expected output
for coverage testing.
As discussed in #11441, the code growth from handling this
case seems to outweigh the benefit of implementing it
properly.
Closes: #11439
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
The webassembly port needs some additional weakref tests due to the fact
that garbage collection only happens when Python execution finishes and
JavaScript resumes.
The `tests/ports/webassembly/heap_expand.py` expected output also needs to
be updated because the amount of GC heap got smaller (weakref WTB takes
some of the available RAM).
Signed-off-by: Damien George <damien@micropython.org>
Includes corresponding .exp files because this feature is only available in
Python 3.14+.
Tests for `!a` conversion specifier and space after `!` are not included
because they are not supported by MicroPython.
Signed-off-by: Koudai Aono <koxudaxi@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
As per CPython behaviour, `b"".hex()` should return an empty str object
(not an empty bytes object).
Fixes issue #18807.
Signed-off-by: Damien George <damien@micropython.org>
As of CPython 3.7 `bytes.fromhex()` skips all ASCII whitespace. And as of
CPython 3.8 `bytes.hex()` supports the optional separator argument. So
this test no longer needs a .exp file.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this fix, `mp_import_all()` assumed that its argument was exactly
a native module instance. That would lead to a crash if something else was
passed in, eg a user class via a custom `__import__` implementation or by
writing to `sys.modules`.
MicroPython already supports injecting non-module objects into the import
machinery, so it makes sense to round out that implementation by supporting
`from x import *` where `x` is a non-module object.
Fixes issue #18639.
Signed-off-by: Damien George <damien@micropython.org>
It turns out that it's relatively simple to support nested f-strings, which
is what this commit implements.
The way the MicroPython f-string parser works at the moment is:
1. it extracts the f-string arguments (things in curly braces) into a
temporary buffer (a vstr)
2. once the f-string ends (reaches its closing quote) the lexer switches to
tokenizing the temporary buffer
3. once the buffer is empty it switches back to the stream.
The temporary buffer can easily hold f-strings itself (ie nested f-strings)
and they can be re-parsed by the lexer using the same algorithm. The only
thing stopping that from working is that the temporary buffer can't be
reused for the nested f-string because it's currently being parsed.
This commit fixes that by adding a second temporary buffer, which is the
"injection" buffer. That allows arbitrary number of nestings with a simple
modification to the original algorithm:
1. when an f-string is encountered the string is parsed and its arguments
are extracted into `fstring_args`
2. when the f-string finishes, `fstring_args` is inserted into the current
position in `inject_chrs` (which is the start of that buffer if no
injection is ongoing)
3. `fstring_args` is now cleared and ready for any further f-strings
(nested or not)
4. the lexer switches to `inject_chrs` if it's not already reading from it
5. if an f-string appeared inside the f-string then it is in `inject_chrs`
and can be processed as before, extracting its arguments into
`fstring_args`, which can then be inserted again into `inject_chrs`
6. once `inject_chrs` is exhausted (meaning that all levels of f-strings
have been fully processed) the lexer switched back to tokenizing the
stream.
Amazingly, this scheme supports arbitrary numbers of nestings of f-strings
using the same quote style.
This adds some code size and a bit more memory usage for the lexer. In
particular for a single (non-nested) f-string it now makes an extra copy of
the `fstring_args` data, when copying it across to `inject_chrs`.
Otherwise, memory use only goes up with the complexity of nested f-strings.
Signed-off-by: Damien George <damien@micropython.org>
The null byte cannot exist in source code (per CPython), so use it to
indicate the end of the input stream (instead of `(mp_uint_t)-1`). This
allows the cache chars (chr0/1/2 and their saved versions) to be 8-bit
bytes, making it clear that they are not `unichar` values. It also saves a
bit of memory in the `mp_lexer_t` data structure. (And in a future commit
allows the saved cache chars to be eliminated entirely by storing them in
a vstr instead.)
In order to keep code size down, the frequently used `chr0` is still of
type `uint32_t`. Having it 32-bit means that machine instructions to load
it are smaller (it adds about +80 bytes to Thumb code if `chr0` is changed
to `uint8_t`).
Also add tests for invalid bytes in the input stream to make sure there are
no regressions in this regard.
Signed-off-by: Damien George <damien@micropython.org>
If a function has children then the code object returned from __code__ must
contain an `mp_raw_code_t` (actually `mp_raw_code_truncated_t` is enough)
that points to the child table.
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes tests that will always fail when using CPython 3.14
to get a known-good output to compare MicroPython's behaviour against.
Starting from CPython 3.14, statements inside a `finally` block that
will prevent the execution flow to reach the block's last statement will
raise a SyntaxWarning. That text would end up in the comparison data
and thus make known-good tests fail.
Given that those tests explicitly exercise flow control interruptions
in finally blocks, there is no real workaround that can be applied to
the tests themselves. Therefore those tests will now check
MicroPython's behaviour against expected output files recorded from
the tests' output with CPython 3.11.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This does not actually cover any additional lines, but it does cover new
functionality not previously covered.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
This test reproduces the bug that gave rise to the esp32 segfaults in
issues #18061 and #18481 on all platforms.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Currently, dict views (eg `dict.keys()`, `dict.values()`) do not implement
the `bool` or `len` unary operations. That may seem like a reasonable
omission for MicroPython to keep code size down, but it actually leads to
silently incorrect bool operations, because by default things are true.
Eg we currently have:
>>> bool(dict().keys())
True
which is wrong, it should be `False` because the dict is empty.
This commit implements `bool` and `len` unary operations on dict views by
simply delegating to the existing dict unary op function.
Fixes issue #12385.
Signed-off-by: Damien George <damien@micropython.org>
Correctly format integers when there are leading zeros with a grouping
character, such as "{:04,d}".format(0x100) -> "0,256".
The new padding patterns for commas-and-zeroes and underscores-and-zeroes
are smooshed together into the existing pad_zeroes to save space.
Only the two combinations of (decimal + commas) and (other bases +
underscores) are properly supported.
Also add a test for it.
Fixes issue #18082.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
With the aim of getting consistency, and removing the need to learn an
additional term, replace uses of uPy/uPython with MPy/MicroPython.
Rule of thumb was to use "MPy" abbreviation where "CPy" is used nearby, but
the full word MicroPython otherwise.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
These two tests can't run on the unix minimal build because it doesn't have
the relevant build options enabled. So skip them.
Signed-off-by: Damien George <damien@micropython.org>
Running the tests now requires CPython 3.8.2 or newer, which was
released February 2020 and should be widely available.
A few examples of features that were previously not supported by CPython,
but which are now:
- %-formatting for bytes and bytearray (PEP 461), CPython 3.5
- annotated variables (PEP 526), CPython 3.6
- assignment expressions (PEP 572), CPython 3.8
Note that `basics/fun_code_full.py.exp` is added here because that requires
CPython 3.10 or newer.
Signed-off-by: Damien George <damien@micropython.org>
This change follows CPython behaviour, allowing use of:
from instance import method
to import a bound method from a class instance, eg registered via
setting `sys.modules["instance"] = instance`.
Admittedly this is probably a very rarely used pattern in Python, but it
resolves a long standing comment about whether or not this is actually
possible (it turns out it is possible!). A test is added to show how it
works.
The main reason for this change is to fix a problem with imports in the
webassembly port: prior to this fix, it was not possible to do `from
js_module import function`, where `js_module` is a JavaScript object
registered to be visible to Python through the webassembly API function
`registerJsModule(js_module)`. But now with this fix that is possible.
Signed-off-by: Damien George <damien@micropython.org>
Re-organize `mp_parse_num_integer()` (for longlong) slightly so that the
most negative 64-bit integer can be parsed.
Fixes issue #17932.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Instead of using a feature check. This is more consistent with how other
optional modules are skipped.
Signed-off-by: Damien George <damien@micropython.org>
Including the stochastic tests needed to guarantee sensitivity to the
potential iterate-while-modifying hazard a naive implementation might have.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
This is to fix an outstanding TODO. The test cases is using a range as
this will exist in all builds, but `mp_obj_get_int` is used in many
different parts of code where an overflow is more likely to occur.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
The recently merged 5e9189d6d1 now allows
temporary slices to be allocated on the C stack, which is much better than
allocating them on the GC heap.
Unfortunately there are cases where the C-allocated slice can escape and be
retained as an object, which leads to crashes (because that object points
to the C stack which now has other values on it).
The fix here is to add a new `MP_TYPE_FLAG_SUBSCR_ALLOWS_STACK_SLICE`.
Native types should set this flag if their subscr method is guaranteed not
to hold on to a reference of the slice object.
Fixes issue #17733 (see also #17723).
Signed-off-by: Damien George <damien@micropython.org>
Long long big integer support now raises an exception on overflow rather
than returning an undefined result.
Also adds an error when shifting by a negative value.
The new arithmetic checks are added in the misc.h header.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
These will run on all ports which support them, but importantly
they'll also run on ports that don't support arbitrary precision
but do support 64-bit long ints.
Includes some test workarounds to account for things which will overflow
once "long long" big integers overflow (added in follow-up commit):
- uctypes_array_load_store test was failing already, now won't parse.
- all the ffi_int tests contain 64-bit unsigned values, that won't parse
as long long.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit adds a fast-path optimisation for when a BUILD_SLICE is
immediately followed by a LOAD/STORE_SUBSCR for a native type, to avoid
needing to allocate the slice on the heap.
In some cases (e.g. `a[1:3] = x`) this can result in no allocations at all.
We can't do this for instance types because the get/set/delattr
implementation may keep a reference to the slice.
Adds more tests to the basic slice tests to ensure that a stack-allocated
slice never makes it to Python, and also a heapalloc test that verifies
(when using bytecode) that assigning to a slice is no-alloc.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Previously, there was no test coverage of the "write failed" path. In
fact, the assertion would fire instead of gracefully raising a Python
exception.
Slightly re-organize the code to place the assertion later. Add a test
case which exercises all paths, and update the expected output.
Signed-off-by: Jeff Epler <jepler@gmail.com>
As suggested by @dpgeorge, factor out part of array_construct to allow it
to be used for construction & extension.
Note that extending with a known-length list (or tuple) goes through the
slow path of calling array_extend once per element.
Fixes issue #7408.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This groups non-decimal values by fours, such as bbb_bbbb_bbbb. It also
supports `{:_d}` to use underscore for decimal numbers (grouped in threes).
Use of incorrect ":,b" is not diagnosed.
Thanks to @dpgeorge for the suggestion to reduce code size.
Signed-off-by: Jeff Epler <jepler@gmail.com>
This fix handles attrtuple as well, eg. os.uname(). A test case has been
added in basics/attrtuple2.py.
Fixes issue #16969.
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
For a given MicroPython firmware/executable it can be sometimes important
to know how it was built, which variant/board configuration it came from.
This commit adds a new field `sys.implementation._build` that can help
identify the configuration that MicroPython was built with.
For now it's either:
* <VARIANT> for unix, webassembly and windows ports
* <BOARD>-<VARIANT> for microcontroller ports (the variant is optional)
In the future additional elements may be added to this string, separated by
a hyphen.
Resolves issue #16498.
Signed-off-by: Damien George <damien@micropython.org>
This change allows tuples to be passed as the prefix/suffix argument to the
`str.startswith()` and `str.endswith()` methods. The methods will return
`True` if the string starts/ends with any of the prefixes/suffixes in the
tuple.
Also adds full support for the `start` and `end` arguments to both methods
for compatibility with CPython.
Tests have been updated for the new behaviour.
Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
This allows retrieving the code object of a function using
`function.__code__`, and then reconstructing a function from a code object
using `FunctionType(code_object)`.
This feature is controlled by `MICROPY_PY_FUNCTION_ATTRS_CODE` and is
enabled at the full-features level.
Signed-off-by: Damien George <damien@micropython.org>
This includes making int("01") parse in base 10 like standard Python.
When a base of 0 is specified it means auto-detect based on the prefix, and
literals begining with 0 (except when the literal is all 0's) like "01" are
then invalid and now throw an exception.
The new error message is different from CPython. It says e.g.,
`SyntaxError: invalid syntax for integer with base 0: '09'`
Additional test cases were added to cover the changed & added code.
Co-authored-by: Damien George <damien@micropython.org>
Signed-off-by: Jeff Epler <jepler@gmail.com>
