This fixes a regression in db59e55fe7a0b67d3af868990468e7b8056afe42: prior
to that commit `mpremote` supported trailing slashes on the destination of
a normal (non-recursive) copy.
Add back support for that, with the semantics that a trailing slash
requires the destination to be an existing directory.
Also add a test for this.
Signed-off-by: Damien George <damien@micropython.org>
mpremote error messages now go to stderr, so make sure stdout is flushed
before printing them.
Also update the test runner to capture error messages.
Signed-off-by: Damien George <damien@micropython.org>
The variable `written` was being used before it was defined in the
`fs_writefile()` method of the Transport class. This was causing an
`UnboundLocalError` to be raised when the `progress_callback` was not
provided.
Fixes issue #16084.
Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
A long time ago when there was only the `stm` port, Ctrl-C would trigger a
preemptive NLR jump to break out of running code. Then in commit
124df6f8d07b53542b6960dbeea9b63bff469a67 a more general approach to
asynchronous `KeyboardInterrupt` exceptions was implemented, and `stmhal`
supported both approaches, with the general (soft) interrupt taking
priority.
Then in commit bc1488a05f509cd5be8bfab9574babfcb993806f `pyboard.py` was
updated with a corresponding change to make it issue a double Ctrl-C to
break out of any existing code when entering the raw REPL (two Ctrl-C
characters were sent in order to more reliably trigger the preemptive NLR
jump).
No other port has preemptive NLR jumps and so a double Ctrl-C doesn't
really behave any differently to a single Ctrl-C: with USB CDC the double
Ctrl-C would most likely be in the same USB packet and so processed in the
same low-level USB callback, so it's just setting the keyboard interrupt
flag twice in a row. The VM/runtime then just sees one keyboard interrupt
and acts as though only one Ctrl-C was sent.
This commit changes the double Ctrl-C to a single Ctrl-C in `pyboard.py`
and `mpremote`. That keeps things as simple as they need to be.
Signed-off-by: Damien George <damien@micropython.org>
This adds a -f/--force option to the "cp" command, which forces
unconditional copies, in particular does not check the hash.
Signed-off-by: Damien George <damien@micropython.org>
These tests are specifically for the command-line interface and cover:
- resume/soft-reset/connect/disconnect
- mount
- fs cp,touch,mkdir,cat,sha256sum,rm,rmdir
- eval/exec/run
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Makes the filesystem command give standard error messages rather than
just printing the exception from the device.
Makes the distinction between CommandError and TransportError clearer.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Changes in this commit:
- Adds transport API `fs_hashfile` to compute the hash of a file with given
algorithm.
- Adds commands `mpremote <...>sum file` to compute and print hashes of
various algorithms.
- Adds shortcut `mpremote sha256sum file`.
- Uses the hash computation to improve speed of recursive file copy to
avoid copying a file where the target is identical.
For recursive copy, if possible it will use the board's support (e.g.
built-in hashlib or hashlib from micropython-lib), but will fall back to
downloading the file and using the local implementation.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This introduces a Python filesystem API on `Transport` that is implemented
entirely with eval/exec provided by the underlying transport subclass.
Updates existing mpremote filesystem commands (and `edit) to use this API.
Also re-implements recursive `cp` to allow arbitrary source / destination.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This is a step towards making the transport expose a Python API rather than
functions that mostly print to stdout.
Most use cases of `transport.eval()` are to get some state back from the
device, so have it return as a value directly by default.
Updates uses of `transport.eval()` to remove the parse argument where it
now isn't needed, make the `rtc` command use eval/exec, and update the
`mip` command to use eval's parsing.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.
Tested on frdm_k64f.
Signed-off-by: Maureen Helm <maureen.helm@analog.com>
This commit adds the `VIRT_RV32` board to the list of targets for
calculating code size changes as part of the CI pipeline.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit adds the Qemu-based RISC-V 32 bits `VIRT_RV32` board to the
list of ports/boards to be built for measuring code size changes.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets the RV64 port use the version of libffi that is bundled
as a submodule in the MicroPython source tree, as the packaged libffi
library coming from Ubuntu's RISC-V repository trashes foreign function
call results on exit.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Native .mpy files targetting armv6m (eg RP2040) cannot currently have more
than about 2kiB of native code (between the start of the file and the init
function).
This commit fixes that by using bigger jumps to jump to the init function.
Signed-off-by: Damien George <damien@micropython.org>
The Unix port's Arm target CI steps have been updated to be more in
line with the other targets (the MicroPython binary doesn't need an
environment variable to be set in order to run now).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The FFI helper definition was accidentally omitted when committing the
necessary shell code for building RV64 Unix builds in the CI
environment.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
The Unix port's MIPS target CI steps have been updated to be more in
line with the other targets (the MicroPython binary now runs as a
dynamic executable), and the test exceptions for ffi have been lifted.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
With the recent qemu (d9a0fdda9a7b0db55c1115b55bb1b83cd5ce739c and
0426934969d06aa649ba903f5408cb331b5b9c2d) and zephyr
(05cad7b56f5d460db26a468a05bfdeabe4a656db) changes to how their tests are
run, two things became unused:
- The tinytest framework, which embedded a set of tests and their expected
output within firmware, so these tests could be run stand-alone.
- The `--write-exp` and `--list-tests` options to `tests/run-tests.py`,
which were needed primarily to generated the expected test output for
tinytest (also the associated `tests/run-tests-exp.py/.sh` scripts are
now unused).
This commit removes the tinytest component and all its helper code. This
eliminates a maintenance burden.
Signed-off-by: Damien George <damien@micropython.org>
As reported in #14430 the Xtensa compiler can add R_XTENSA_ASM_EXPAND
relocation relaxation entries in object files, and they were not
supported by mpy_ld.
This commit adds handling for that entry, doing nothing with it, as it
is only of real use for an optimising linker.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Currently both the qemu-arm and qemu-riscv ports share a lot of code and
functionality. This commit merges the qemu-riscv port into the qemu-arm
port. The only real differences between the two are the toolchains used to
build the code, and the initialisation/startup framework. Everything else
is pretty much the same, so this brings the following benefits:
- less code duplication
- less burden on maintenance
- generalised qemu port, could in the future support other architectures
A new board `VIRT_RV32` has been added to the qemu-arm port which is the
existing RISC-V board from the qemu-riscv port. To build it:
$ make BOARD=VIRT_RV32 repl
To cleanly separate the code for the different architectures, startup code
has been moved to ports/qemu-arm/mcu/<arch>/.
Signed-off-by: Damien George <damien@micropython.org>
Currently, the qemu-arm (and qemu-riscv) port has two build modes:
- a simple test that executes a Python string; and
- a full test that uses tinytest to embed all tests within the firmware,
then executes that and captures the output.
This is very different to all the other ports. A difficulty with using
tinytest is that with the large number of tests the firmware overflows its
virtual flash size. It's also hard to run tests via .mpy files and with
the native emitter. Being different to the other ports also means an extra
burden on maintenance.
This commit reworks the qemu-arm port so that it has a single build target
that creates a standard firmware which has a REPL. When run under
qemu-system-arm, the REPL acts like any other bare-metal port, complete
with soft reset (use machine.reset() to turn it off and exit
qemu-system-arm).
This approach gives many benefits:
- allows playing with a REPL without hardware;
- allows running the test suite as it would on a bare-metal board, by
making qemu-system-arm redirect the UART serial of the virtual device to
a /dev/pts/xx file, and then running run-tests.py against that serial
device;
- skipping tests is now done via the logic in `run-tests.py` and no longer
needs multiple places to define which tests to skip
(`tools/tinytest-codegen.py`, `ports/qemu-arm/tests_profile.txt` and also
`tests/run-tests.py`);
- allows testing/using mpremote with the qemu-arm port.
Eventually the qemu-riscv port would have a similar change.
Prior to this commit the test results were:
743 tests ok. (121 skipped)
With this commit the test results are:
753 tests performed (22673 individual testcases)
753 tests passed
138 tests skipped
More tests are skipped because more are included in the run. But overall
more tests pass.
Signed-off-by: Damien George <damien@micropython.org>
With this change, the zephyr port is tested against the standard test suite
via the following scheme:
- the zephyr port is built with the `qemu_cortex_m3` board and the
`prj_minimal.conf` configuration
- `qemu-system-arm` runs `zephyr.elf`
- the zephyr console is redirected to a pts/pty
- `tests/run-tests.py` is run in bare-metal mode against the pts/pty device
This allows testing the zephyr port as though it were a physical board
attached over a serial port.
Signed-off-by: Damien George <damien@micropython.org>
The pts line printed by qemu-system-arm goes to stdout, not stderr.
Redirect stderr to stdout in case other tools do print to stderr.
Signed-off-by: Damien George <damien@micropython.org>
IDF 3 builds are very old now (it seems like the last successful builds are
from 2021), and the current IDF 5 is stable. So remove IDF 3 variants.
Signed-off-by: Matt Trentini <matt.trentini@gmail.com>
macos-11.0 is no longer available.
With this change in the macos version, some tests which previously failed
now pass, and some different tests now fail. Exclude those that fail from
the CI until they can be fixed properly.
Signed-off-by: Damien George <damien@micropython.org>
This is a fix for an algorithmic error in mpremote mip, that throws an
error due to a '\n' used in the concatenation and split when there is more
than one lib path in `sys.path`.
Signed-off-by: Jos Verlinde <Jos.Verlinde@microsoft.com>
Allow inclusion of large integer constants in frozen files using long-long
representation (mpy-cross option -mlongint-impl=longlong).
Signed-off-by: Yoctopuce <dev@yoctopuce.com>
Prior to this fix the current working path in the remote VFS would always
be prepended to the requested path to get the full path, even if the
requested path was already absolute, ie starting with "/".
So `os.chdir("/remote/dir1")` would set the working path to "/dir1/", and
a subsequent call with an absolute path like `os.listdir("/remote/dir2")`
would try to list the directory "/dir1/dir2/".
Fixes issue #15308.
Signed-off-by: Damien George <damien@micropython.org>
MPY files can now hold generated RV32IMC native code. This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
When no usable manifest directives are found (as opposed to no manifest
being set in the makefile), non-compiling code was generated for the
empty frozen constants pool block.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This tool is used to compute size differences in the firmware (eg as part
of CI), but it doesn't currently check any firmware that has bare-metal
lwIP/networking, making it hard to see how firmware size changes when
networking related changes are made.
So, change the board selection for the rp2 port to RPI_PICO_W. Changes in
size to standard RPI_PICO firmware will be very similar to other bare-metal
boards like PYBV10.
Signed-off-by: Damien George <damien@micropython.org>
The `run-natmodtests.py` script now properly excludes tests that don't use
the corresponding native module.
Signed-off-by: Damien George <damien@micropython.org>
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e911b088836f4e7c37fac2bc0f67917
- 5e3006f1172d0eabbbefeb3268dfb942ec7cf9cd
- e2ff00e81113d7a3f32f860652017644b5d68bf1
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41c2fa66a224725fdb2411846c91cf1a.
Signed-off-by: Damien George <damien@micropython.org>
These are needed to read/write array.array objects, which is useful in
native code to provide fast extensions that work with big arrays of data.
Signed-off-by: Damien George <damien@micropython.org>
It destroys a few manual alignments, but these seem minor compared to
the benefit of automated code style consistency.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
The documentation for `freeze()` says that:
- If `script` is `None`, all files in `path` will be frozen.
- If `script` is an iterable then `freeze()` is called on all items of the
iterable.
This commit makes sure this behaviour is followed when an empty tuple/list
is passed in for `script` (previously an empty tuple/list froze all files).
Fixes issue #14125.
Signed-off-by: Damien George <damien@micropython.org>
