This is a fix for an algorithmic error in mpremote mip, that throws an
error due to a '\n' used in the concatenation and split when there is more
than one lib path in `sys.path`.
Signed-off-by: Jos Verlinde <Jos.Verlinde@microsoft.com>
Allow inclusion of large integer constants in frozen files using long-long
representation (mpy-cross option -mlongint-impl=longlong).
Signed-off-by: Yoctopuce <dev@yoctopuce.com>
Prior to this fix the current working path in the remote VFS would always
be prepended to the requested path to get the full path, even if the
requested path was already absolute, ie starting with "/".
So `os.chdir("/remote/dir1")` would set the working path to "/dir1/", and
a subsequent call with an absolute path like `os.listdir("/remote/dir2")`
would try to list the directory "/dir1/dir2/".
Fixes issue #15308.
Signed-off-by: Damien George <damien@micropython.org>
MPY files can now hold generated RV32IMC native code. This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
When no usable manifest directives are found (as opposed to no manifest
being set in the makefile), non-compiling code was generated for the
empty frozen constants pool block.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This tool is used to compute size differences in the firmware (eg as part
of CI), but it doesn't currently check any firmware that has bare-metal
lwIP/networking, making it hard to see how firmware size changes when
networking related changes are made.
So, change the board selection for the rp2 port to RPI_PICO_W. Changes in
size to standard RPI_PICO firmware will be very similar to other bare-metal
boards like PYBV10.
Signed-off-by: Damien George <damien@micropython.org>
The `run-natmodtests.py` script now properly excludes tests that don't use
the corresponding native module.
Signed-off-by: Damien George <damien@micropython.org>
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e911b088836f4e7c37fac2bc0f67917
- 5e3006f1172d0eabbbefeb3268dfb942ec7cf9cd
- e2ff00e81113d7a3f32f860652017644b5d68bf1
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41c2fa66a224725fdb2411846c91cf1a.
Signed-off-by: Damien George <damien@micropython.org>
These are needed to read/write array.array objects, which is useful in
native code to provide fast extensions that work with big arrays of data.
Signed-off-by: Damien George <damien@micropython.org>
It destroys a few manual alignments, but these seem minor compared to
the benefit of automated code style consistency.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
The documentation for `freeze()` says that:
- If `script` is `None`, all files in `path` will be frozen.
- If `script` is an iterable then `freeze()` is called on all items of the
iterable.
This commit makes sure this behaviour is followed when an empty tuple/list
is passed in for `script` (previously an empty tuple/list froze all files).
Fixes issue #14125.
Signed-off-by: Damien George <damien@micropython.org>
The argument to MP_BC_MAKE_FUNCTION (raw code index) was being encoded as a
byte instead of a variable unsigned int. That meant that if there were
more than 128 merged mpy files the encoding would be invalid.
Fix that by using `mp_encode_uint(idx)` to encode the raw code index. And
also use `Opcode` constants for the opcode values to make it easier to
understand the code.
Signed-off-by: Damien George <damien@micropython.org>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a433d6253a61cb0f987835fbc06b2de. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Follow up to 35dd959133fb233d75e9b3cddbf98b2ed01c6594, allows explicitly
adding the unix-ffi library path from the command line.
This option is needed when building unix-ffi manifests in micropython-lib
CI.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Allows bytecode itself to be used instead of an mp_raw_code_t in the simple
and common cases of a bytecode function without any children.
This can be used to further reduce frozen code size, and has the potential
to optimise other areas like importing.
Signed-off-by: Damien George <damien@micropython.org>
To simplify their access and reduce code size.
The `scope_flags` member is only ever used to determine if a function is a
generator or not, so make it reflect that fact as a bool type.
Signed-off-by: Damien George <damien@micropython.org>
The `asm_n_pos_args` and `asm_type_sig` members of `mp_raw_code_t` are only
used for raw codes of type MP_CODE_NATIVE_ASM, which are rare, for example
in frozen code. So using a truncated `mp_raw_code_t` in these cases helps
to reduce frozen code size on targets that have MICROPY_EMIT_INLINE_ASM
enabled.
With this, change in firmware size of RPI_PICO builds is -648.
Signed-off-by: Damien George <damien@micropython.org>
The mp_raw_code_t struct has been reordered and some members resized. The
`n_pos_args` member is renamed to `asm_n_pos_args`, and `type_sig` renamed
to `asm_type_sig` to indicate that these are used only for the inline-asm
emitters. These two members are also grouped together in the struct.
The justifications for resizing the members are:
- `fun_data_len` can be 32-bits without issue
- `n_children` is already limited to 16-bits by
`mp_emit_common_t::ct_cur_child`
- `scope_flags` is already limited to 16-bits by `scope_t::scope_flags`
- `prelude_offset` is already limited to 16-bits by the argument to
`mp_emit_glue_assign_native()`
- it's reasonable to limit the maximim number of inline-asm arguments to 12
(24 bits for `asm_type_sig` divided by 2)
This change helps to reduce frozen code size (and in some cases RAM usage)
in the following cases:
- 64-bit targets
- builds with MICROPY_PY_SYS_SETTRACE enabled
- builds with MICROPY_EMIT_MACHINE_CODE enabled but MICROPY_EMIT_INLINE_ASM
disabled
With this change, unix 64-bit builds are -4080 bytes in size. Bare-metal
ports like rp2 are unchanged (because mp_raw_code_t is still 32 bytes on
those 32-bit targets).
Signed-off-by: Damien George <damien@micropython.org>
It's rare to freeze .mpy files without specifying a qstr header from a
firmware build, but it can be useful for testing, eg
`mpy-tool.py -f test.mpy`. Fix this case so static qstrs are properly
excluded from the frozen qstr list.
Signed-off-by: Damien George <damien@micropython.org>
This commit changes how library packages are searched for when a manifest
file is loaded: there is now simply a list of library paths that is
searched in order for the given package. This list defaults to the
main directories in micropython-lib, but can be added to -- either appended
or prepended -- by using `add_library()`.
In particular the way unix-ffi library packages are searched has changed,
because the `unix_ffi` argument to `require()` is now removed. Instead, if
a build wants to include packages from micropython-lib/unix-ffi, then it
must explicitly add this to the list of paths to search using:
add_library("unix-ffi", "$(MPY_LIB_DIR)/unix-ffi")
Work done in collaboration with Jim Mussared.
Signed-off-by: Damien George <damien@micropython.org>
Similar to ruff.yaml, it's simpler to run the codespell command directly
from a workflow file. And developers can run codespell directly from the
command line without the need for options, or just use pre-commit.
This commit also applies a specific version to codespell, same as
pre-commit (introduced in a166d805f43).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This disables using qstr hashes altogether, which saves RAM and flash
(two bytes per interned string on a typical build) as well as code size.
On PYBV11 this is worth over 3k flash.
qstr comparison will now be done just by length then data. This affects
qstr_find_strn although this has a negligible performance impact as, for a
given comparison, the length and first character will ~usually be
different anyway.
String hashing (e.g. builtin `hash()` and map.c) now need to compute the
hash dynamically, and for the map case this does come at a performance
cost.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Required modifying the gen-cpydiff.py code to allow a "preamble" section to
be inserted at the top of any of the generated files.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
No longer require importlib_metadata on new Python versions as it is
included in the standard distribution.
Signed-off-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>
This adds a `add_library(name, path)` method for use in manifest.py that
allows registering an external path (e.g. to another repo) by name.
This name can then be passed to `require("package", library="name")` to
reference packages in that repo/library rather than micropython-lib.
Within the external library, `require()` continues to work as normal
(referencing micropython-lib) by default, but they can also specify the
library name to require another package from that repo/library.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Also, IDF v5.1.2 is now supported, just not used by default.
IDF v5.0.2 still builds but we cannot guarantee continued support for this
version moving forward.
Signed-off-by: IhorNehrutsa <IhorNehrutsa@gmail.com>
CCaches are scoped per-job.
Uses https://github.com/hendrikmuhs/ccache-action to get desired behaviour
(updating the cache on each run).
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Allows splitting the esp32 job into multiple parts without too much
boilerplate. The matrix is parameterised using the name of the function to
call in tools/ci.sh, to minimise the dependency on GitHub Actions.
This can get esp32 build times down around 3m if IDF is cached already.
If the cache is cold, the cache preparation step on each job can double up
against each other. However, restructuring the workflow to not do this
seems either complex or requires copy-pasting the entire cache step.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Cache is keyed on the ESP-IDF version used in CI, so there shouldn't be any
cache invalidation issues when ESP-IDF version changes.
Restoring from cache takes approx 15s, compared to 2-3m to perform these
steps (ESP-IDF tools install, ESP-IDF clone, ESP-IDF submodule clone) the
first time.
Cache size is approx 1.6GB, the git clone is tweaked as much as possible to
keep the size down.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Speeds up importing files from mounted filesystem.
Also fix the return code for invalid / unsupported ioctl requests.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Remove port-specific test directories and excluded tests from
tinytest-codegen, and let it read said information from an external file.
This way tinytest-codegen is not limited to always generate tests for the
`qemu-arm` target.
This allows having port-specific test directory and excluded tests for more
than one QEMU bare-metal target.
The `qemu-arm` port Makefile was modified to work with the generator
changes and a tests profile file was added to said port.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Requires additions to tools/boardgen.py for stm32 pin generation.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
For now, this implements the functionality required for esp32 and rp2,
including support for numeric pins, rp2 alternate functions, and rp2
extended pins.
This also updates the rp2 port to use the same structure for pins.h and
pins.csv as for esp32, and moves the pin definitions directly into the
table (rather than having a table of pointers), which is a small code size
improvement.
Support for "hidden" pins in pins.csv is added (matching the stm32
implementation).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This updates a small number of files that change with ruff-format's (vs
black's) rules.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
- Add config for [tool.ruff.format] to pyproject.toml.
- Update pre-commit to run both ruff and ruff-format (and only check C
files when running codeformat.py)
- Update CI.
- Simplify codeformat.py to remove all the Python-specific logic (just run
"ruff format" directly).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This port is largely unmaintained, has limited features (the only hardware
support is for GPIO and timer, and no machine module), only supports a
small number of Teensy boards, and can be confused with the mimxrt support
for Teensy 4.x.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
