WIP: Align to micropython v1.20 #65

amirgon · 2023-06-05T21:46:07Z

TODO:

Fix and test Micropython ports
Push LVGL and lv_bindings_micropython submodules to master and point lv_micropython to them

Signed-off-by: Damien George <[email protected]>

Total code size change of this and previous commit: bare-arm: +0 +0.000% minimal x86: +0 +0.000% unix x64: +32 +0.004% standard stm32: +24 +0.006% PYBV10 cc3200: +16 +0.009% esp8266: +20 +0.003% GENERIC esp32: +44 +0.003% GENERIC[incl +8(data)] mimxrt: +32 +0.009% TEENSY40 renesas-ra: +24 +0.004% RA6M2_EK nrf: +0 +0.000% pca10040 rp2: +24 +0.005% PICO samd: +32 +0.012% ADAFRUIT_ITSYBITSY_M4_EXPRESS Addresses issue micropython#7920. Signed-off-by: Damien George <[email protected]>

Signed-off-by: Damien George <[email protected]>

LAN8710 uses the same drivers as LAN8720, so this commit just adds the names. Alternatively, both could be summarised under LAN87xx, like the esp-idf does.

Both the direction and the Pin used for ref_clk can now be configured. It Requires at least idf v4.4. The new keyword arguments to the constructor are: - ref_clk_mode=mode: with mode being Pin.IN or Pin.OUT. If it is not set, then the default configuration is used, which may be configured by kconfig settings. - ref_clk=pin_obj: which defines the Pin used for ref_clk. This is either Pin(0), Pin(16) or Pin(17). No check is done for the pin number. If it is the wrong one, it simply will not work. Besides that, no harm.

Incorporating PR micropython#7356.

Add support for various SPI-based ethernet chips (W5500, KSZ8851SNL, DM9051) to the ESP32 port. This leverages the existing support in ESP-IDF for these chips -- which configures these chips in "MAC raw" mode -- and the existing support for network.LAN in the ESP32 port. In particular, this doesn't leverage the wiznet5k support that is used on the rp2 and stm32 ports (because that's for native use of lwIP). Tested on the POE Featherwing (with the SJIRQ solder jumper bridged) and a ESP32-S3 feather. A note about the interrupt pin: The W5500 implementation within ESP-IDF relies on hardware interrupt, and requires the interrupt pin from the W5500 to be wired to a GPIO. This is not the case by default on the Adafruit Ethernet FeatherWing, which makes it not directly compatible with this implementation.

`esp_eth_ioctl(ETH_CMD_S_MAC_ADDR)` sets the MAC address of the hardware device, but we also need to notify the upper layers of the change so that e.g. DHCP work properly.

Signed-off-by: Damien George <[email protected]>

Specifying DMA channel 2 causes an "invalid dma channel" error. In ESP-IDF, the only available channel for ESP32S3 is SPI_DMA_CH_AUTO=3.

It has been around for a long time and the API is mature enough. Signed-off-by: Damien George <[email protected]>

MicroPython doesn't need the shared library and disabling it in the configure could ease the build of a static version of MicroPython.

Based on and tested on the rp2 port. Signed-off-by: Liao Jingyi <liaojingyi2@gmail> Signed-off-by: Damien George <[email protected]>

Signed-off-by: enriquezgarc <[email protected]>

pre-commit manages its own dependencies otherwise (including Black), but this one is a C/C++ binary so needs to be installed independently. Signed-off-by: Angus Gratton <[email protected]>

Not all DFU bootloaders exit after download, this modifier is required for some boards to reset, otherwise they sit idle in DFU mode.

The major setting is about the PHY interface configuration. The configuration matches the Olimex ESP32 Gateway as well. Tested with esp-idf v4.2.4 and Olimex ESP32 POE boards.

Tested to work on an ESP32-MINI-1, which is a single core ESP32-U4DWH. Signed-off-by: Clayton Cronk <[email protected]>

This is available since ESP-IDF v4.4. Signed-off-by: Damien George <[email protected]>

The delay is 1 ms. It avoids the crashes reported by the issues micropython#8289, micropython#8792 and micropython#9236 with esp-idf versions >= 4.2, but does not solve an underlying problem in the esp-idf.

32-bit platforms only support a slice offset start of 24 bit max due to the limited size of the mp_obj_array_t.free member. Similarly on 64-bit platforms the limit is 56 bits. This commit adds an OverflowError if the user attempts to slice a memoryview beyond this limit. Signed-off-by: Damien George <[email protected]>

This is important for literal tuples, e.g. f"{a,b,}, {c}" --> "{}".format((a,b), (c),) which would otherwise result in either a syntax error or the wrong result. Fixes issue micropython#9635. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <[email protected]>

Unless MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_D, these macros only work with values and "->"/"." expressions as their sole argument. In other words, the macros are broken with expressions which contain operations of lower precedence than the cast operator. Depending on situation, the old code either results in compiler error: MP_OBJ_TO_PTR(flag ? o1 : o2) expands into "(void *)flag ? o1 : o2", which some compiler configurations will reject (e.g. GCC -Wint-conversion -Wint-to-pointer-cast -Werror) Or in an incorrect address calculation: For ptr declared as "uint8_t *" the MP_OBJ_FROM_PTR(ptr + off) expands into ((mp_obj_t)ptr) + off, resulting in an obviously wrong address. Signed-off-by: Alex Riesen <[email protected]>

Signed-off-by: Damien George <[email protected]>

When accessing the argument of the irq function.

This will also catch / and /= when float support is disabled. Fixes issue micropython#10544. Signed-off-by: Damien George <[email protected]>

ghost · 2023-06-14T02:36:53Z

I'm hoping you have the time to update the RP2 port for the new pico_w support in MPY 1.20

amirgon · 2023-06-14T06:25:55Z

I'm hoping you have the time to update the RP2 port for the new pico_w support in MPY 1.20

I would like to, but I don't have an RP2 device to test it.
After it compiles I'll need some help testing the different ports.

ghost · 2023-06-14T20:44:24Z

I am using directions to compile the Raspberry Pi Pico port, and it's working for me:
MicroPython v1.19.1-685-gb9fae5f2f on 2023-06-13; Raspberry Pi Pico with RP2040
I also see lgvl on the help("modules") list, and I can "import lgvl as lv". Connecting to REPL with a USB cable, I can run_tests.py, with 16 fails related to assembler, math, and nanoseconds. Uhmmm... what's the best way to write this up, and/or can I buy you a pico-w? I'm not having much luck trying to hack this together, would a zip file of the whole text of a clone and compile help?

amirgon · 2023-06-14T21:49:48Z

Connecting to REPL with a USB cable, I can run_tests.py, with 16 fails related to assembler, math, and nanoseconds.

Did you check this?

lv_micropython/ports/rp2/README.md

Lines 110 to 116 in 808d72d

    
           ### Troubleshooting 
        
           If you experience unstable behaviour, it is worth checking the value of 
        
           *MICROPY_HW_FLASH_STORAGE_BASE* against the value of *__flash_binary_end* 
        
           from the firmware.elf.map file. If the storage base is lower than the 
        
           binary end, parts of the firmware will be overwritten when the micro- 
        
           python filesystem is initialised.

Another thing I would check is Garbage Collection. LVGL relies on MP gc so if anything goes wrong there it could cause nasty crashes.
The easiest thing to do is to import gc and call gc.collect() while running some LVGL code and see if it crashes due to gc.

How do these failures look like? Do you have a stack trace for them? Can you connect a debugger? Or are these just Python errors?

Regardless of the failures, were you able to run some hello world with a display? The tests are good for logic but don't test the display driver and the event loop. If you are able to run some examples such as the advanced_demo.py, I would be optimistic about the rest.

what's the best way to write this up, and/or can I buy you a pico-w? I'm not having much luck trying to hack this together, would a zip file of the whole text of a clone and compile help?

Could you open a GitHub issue for that on lv_binding_micropython?
Try to describe the problem, how to reproduce it, the errors you get, stack trace etc.

Currently I have very little time to put into it. For now I won't be able to debug this myself, but if you are willing to try, I'll be happy to discuss, provide feedback etc. Hopefully others would step in and help too.

amirgon · 2023-06-16T23:17:21Z

@embeddedt - I think the stm32 port and js port (renamed to webassembly) need to be fixed for Micropython v1.20
Can you help?

sam0910 · 2023-06-17T20:56:26Z

Thank you so much for MP1.20
it works fine with ESP32-SPIRAM, ESP-IDF v4.4.5
New micropython 1.20 features(mip, mpy v6.1) are available :)

I found ./py/mpconfig.h shows,

#define MICROPY_VERSION_MAJOR 1
#define MICROPY_VERSION_MINOR 20
#define MICROPY_VERSION_MICRO 0

I don't know why REPL shows,
MicroPython v1.19.1-1735-gbcaf84455-dirty on 2023-06-18;

embeddedt · 2023-06-18T00:09:11Z

I've fixed STM32 compilation, however attempting to run advanced_demo results in a MemManage fault. It happens when gc_free runs so I wonder if this is a missing GC root. Do we have a way of tracking those on the simulator instead of having to debug on the actual platform?

#0  0x0809a332 in boardctrl_fatal_error (msg=msg@entry=0x80ede33 "MemManage") at boardctrl.c:49
#1  0x0809ab64 in MemManage_Handler () at stm32_it.c:248
#2  <signal handler called>
#3  0x08020362 in gc_free (ptr=<optimized out>) at ../../py/gc.c:779
#4  0x080209c4 in gc_free (ptr=<optimized out>) at ../../py/gc.c:759
#5  0x08020294 in m_free (ptr=<optimized out>) at ../../py/malloc.c:199
#6  0x08069260 in lv_free (data=<optimized out>) at ../../lib/lv_bindings/lvgl/src/misc/lv_mem.c:106
#7  0x08063460 in lv_img_cache_set_size_builtin (new_entry_cnt=new_entry_cnt@entry=1) at ../../lib/lv_bindings/lvgl/src/draw/lv_img_cache_builtin.c:191
#8  0x08063604 in _lv_img_cache_builtin_init () at ../../lib/lv_bindings/lvgl/src/draw/lv_img_cache_builtin.c:71
#9  0x08055530 in lv_init () at ../../lib/lv_bindings/lvgl/src/core/lv_obj.c:169
#10 0x08041c86 in mp_lv_anim_del_all (mp_n_args=<optimized out>, mp_args=<optimized out>, lv_func_ptr=<optimized out>) at build-STM32F7DISC/lvgl/lv_mpy.c:33359
#11 0x0803962c in mp_execute_bytecode (code_state=code_state@entry=0x2004fc28, inject_exc=inject_exc@entry=0x0) at ../../py/vm.c:1042
#12 0x0803051a in fun_bc_call (self_in=0xc002f640, n_args=<optimized out>, n_kw=0, args=0x0) at ../../py/objfun.c:278
#13 0x0802c0ba in mp_parse_compile_execute (lex=0xc002f630, parse_input_kind=parse_input_kind@entry=MP_PARSE_FILE_INPUT, globals=0xc002f540, locals=<optimized out>) at ../../py/runtime.c:1600
#14 0x08037210 in do_load_from_lexer (context=context@entry=0xc002f520, lex=<optimized out>) at ../../py/builtinimport.c:166
#15 0x0803742e in do_load (module_obj=0xc002f520, file=file@entry=0x2004fd38) at ../../py/builtinimport.c:260
#16 0x08037654 in process_import_at_level (override_main=false, path=0x2004fd38, outer_module_obj=0x0, level_mod_name=4443, full_mod_name=4443) at ../../py/builtinimport.c:460
#17 mp_builtin___import___default (n_args=n_args@entry=5, args=args@entry=0x2004fdf4) at ../../py/builtinimport.c:554
#18 0x0802c1ba in mp_import_name (name=<optimized out>, fromlist=<optimized out>, level=<optimized out>) at ../../py/runtime.c:1508
#19 0x08039542 in mp_execute_bytecode (code_state=code_state@entry=0x2004fea0, inject_exc=inject_exc@entry=0x0) at ../../py/vm.c:1233
--Type <RET> for more, q to quit, c to continue without paging--c
#20 0x0803051a in fun_bc_call (self_in=0xc002f510, n_args=<optimized out>, n_kw=0, args=0x0) at ../../py/objfun.c:278
#21 0x0808f80a in parse_compile_execute (source=source@entry=0x2004ff80, input_kind=<optimized out>, exec_flags=exec_flags@entry=22) at ../../shared/runtime/pyexec.c:123
#22 0x0808fbd6 in pyexec_friendly_repl () at ../../shared/runtime/pyexec.c:675
#23 0x0809a960 in stm32_main (reset_mode=<optimized out>) at main.c:617

amirgon · 2023-06-18T06:57:14Z

Thank you so much for MP1.20
it works fine with ESP32-SPIRAM, ESP-IDF v4.4.5
New micropython 1.20 features(mip, mpy v6.1) are available :)

@sam0910 - Thank you very much for testing this branch on ESP32!

I don't know why REPL shows,
MicroPython v1.19.1-1735-gbcaf84455-dirty on 2023-06-18;

Probably I need to push the v1.20.0 tag to GitHub.

@embeddedt - Thanks for the help!

I believe the GC problem is specific to stm32 because I tested the unix port, and GC works fine there.
My guess is that it's related to MP_REGISTER_ROOT_POINTER (see this commit) and to the stm32 Makefile.

In Micropython v1.20, some Python script scans the sources and looks for MP_REGISTER_ROOT_POINTER in the sources. This macro is defined to nothing in C, but used to hint the build system to add a root pointer.

Maybe the source (driver/stm32/STM32F7DISC/modrk043fn48h.c) is not added to the correct Makefile variable. Perhaps we can add it to SRC_EXTMOD_C instead, like done for lv_mpy.c.
lv_mpy.c also defines MP_REGISTER_ROOT_POINTER and it works well there. On the other hand, modrk043fn48h is not exactly an external module (it's specific to stm32) so I'm not sure SRC_EXTMOD_C is the right parameter here.

embeddedt · 2023-06-18T18:12:46Z

False alarm. My LVGL submodule was somehow out of date and therefore didn't have lvgl/lvgl#4269, which meant lvgl_root_pointers was null. STM32 appears to be working fine now.

embeddedt · 2023-06-18T20:14:12Z

JS will definitely have a merge conflict but it is on a separate branch so I can update it later, shouldn't block any other development.

amirgon · 2023-06-18T20:44:46Z

JS will definitely have a merge conflict but it is on a separate branch so I can update it later, shouldn't block any other development.

Right. https://github.com/lvgl/lv_micropython/actions/runs/5305529951/jobs/9602554249

I suspect it's more than a simple merge problem. The js port was renamed "webassembly" and I'm not sure what other changes were done.
Do you use root pointers anywhere in the js port?

embeddedt · 2023-06-23T00:18:11Z

JS port should be updated now. (I guess it should now be called the WASM port?)

The real URL is now https://sim.lvgl.io/v9.0/micropython/ports/webassembly/index.html but I implemented a symlink so https://sim.lvgl.io/v9.0/micropython/ports/javascript/index.html still works too.

Let me know if there are any issues.

amirgon · 2023-06-23T06:15:06Z

Thank you @embeddedt !

dpgeorge and others added 30 commits January 18, 2023 09:15

lib/re1.5: Reduce code size when checking for named class char.

bd86ce5

Signed-off-by: Damien George <[email protected]>

tests/extmod/ure_namedclass: Add tests for named classes in class sets.

fc745d8

Signed-off-by: Damien George <[email protected]>

esp32/network_lan: Add support for LAN8710 PHY.

efb4bd3

LAN8710 uses the same drivers as LAN8720, so this commit just adds the names. Alternatively, both could be summarised under LAN87xx, like the esp-idf does.

docs/esp32/quickref: Add docs for the LAN interface constructor.

30bac47

Incorporating PR micropython#7356.

esp32/network_lan: Fix setting MAC address of ethernet devices.

c7301b8

`esp_eth_ioctl(ETH_CMD_S_MAC_ADDR)` sets the MAC address of the hardware device, but we also need to notify the upper layers of the change so that e.g. DHCP work properly.

esp32/network_lan: Fix compilation on IDF lower than v4.4.

67097d8

Signed-off-by: Damien George <[email protected]>

esp32/machine_sdcard: Change dma_channel to SPI_DMA_CH_AUTO for ESP32S3.

fb0dd86

Specifying DMA channel 2 causes an "invalid dma channel" error. In ESP-IDF, the only available channel for ESP32S3 is SPI_DMA_CH_AUTO=3.

esp8266/modnetwork: Add ability to set tx power for WLAN interface.

e30e543

esp8266/README: Remove notice about port being "experimental".

4eb7063

It has been around for a long time and the API is mature enough. Signed-off-by: Damien George <[email protected]>

unix/Makefile: Disable building the shared library of libffi.

720f2cf

MicroPython doesn't need the shared library and disabling it in the configure could ease the build of a static version of MicroPython.

docs/library/machine.Timer: Add freq argument to machine.Timer.

7f6345a

Based on and tested on the rp2 port. Signed-off-by: Liao Jingyi <liaojingyi2@gmail> Signed-off-by: Damien George <[email protected]>

docs/reference/constrained: Add missing heap-dump symbols to mem_info().

23ef297

docs/zephyr/quickref: Fix zsensor module usage examples.

5c10727

Signed-off-by: enriquezgarc <[email protected]>

CODECONVENTIONS: Explain uncrustify dependency for pre-commit setup.

3634781

pre-commit manages its own dependencies otherwise (including Black), but this one is a C/C++ binary so needs to be installed independently. Signed-off-by: Angus Gratton <[email protected]>

stm32/Makefile: Leave DFU mode after download when using dfu-util.

3b5cafc

Not all DFU bootloaders exit after download, this modifier is required for some boards to reset, otherwise they sit idle in DFU mode.

esp32/boards: Add board definition for Olimex ESP32 PoE boards.

32a858e

The major setting is about the PHY interface configuration. The configuration matches the Olimex ESP32 Gateway as well. Tested with esp-idf v4.2.4 and Olimex ESP32 POE boards.

esp32/boards/GENERIC_UNICORE: Add board definition for unicore chips.

54e85fe

Tested to work on an ESP32-MINI-1, which is a single core ESP32-U4DWH. Signed-off-by: Clayton Cronk <[email protected]>

esp32/network_lan: Add support for Ethernet PHY KSZ8081.

30db33d

This is available since ESP-IDF v4.4. Signed-off-by: Damien George <[email protected]>

esp32: Add a small delay before leaving wlan.active().

d6bc34a

The delay is 1 ms. It avoids the crashes reported by the issues micropython#8289, micropython#8792 and micropython#9236 with esp-idf versions >= 4.2, but does not solve an underlying problem in the esp-idf.

unix/main: Free pathbuf when there's an error opening a file.

94ee1b6

embed: Add new "embed" port which builds a simple C package.

a8a1ad1

Signed-off-by: Damien George <[email protected]>

examples/embedding: Rework example to use ports/embed.

4f3780a

Signed-off-by: Damien George <[email protected]>

rp2: Fix crash in a pin.irq handler.

d4a4cde

When accessing the argument of the irq function.

py/objint_mpz: Catch and reject @ and @= operating on big integers.

d387ae3

This will also catch / and /= when float support is disabled. Fixes issue micropython#10544. Signed-off-by: Damien George <[email protected]>

Remove VARIANT=dev on unix port

d3f379d

amirgon changed the title ~~WIP: Update micropython v1.20~~ WIP: Align to micropython v1.20 Jun 5, 2023

amirgon mentioned this pull request Jun 5, 2023

Align to Micropython v1.20.0 lvgl/lv_binding_micropython#272

Closed

Update LVGL

808d72d

amirgon added 3 commits June 16, 2023 23:48

Move unix specific build rules

c8299de

Fix symlink path

6c20437

ESP32 fixes in lv_bindings submodule

753e1ff

amirgon added 2 commits June 17, 2023 09:15

stm32: fix include path

7e9fa61

stm32: fix manifest

52615da

amirgon force-pushed the update_micropython_v1.20 branch from bb2fd00 to 52615da Compare June 17, 2023 07:36

amirgon mentioned this pull request Jun 17, 2023

Support ESP32-S2, ESP32-C3, ESP32-S3 lvgl/lv_binding_micropython#227

Closed

7 tasks

stm32 update root pointer

bcaf844

amirgon force-pushed the update_micropython_v1.20 branch from c4c3b9c to bcaf844 Compare June 17, 2023 12:58

Update CFLAGS variable name for stm32

4766e94

Rebase lv_bindings and update LVGL

fb095f2

amirgon merged commit 3f25a76 into master Jun 18, 2023

amirgon mentioned this pull request Jun 18, 2023

Integration as external c module with unmodified micropython. lvgl/lv_binding_micropython#242

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: Align to micropython v1.20 #65

WIP: Align to micropython v1.20 #65

Uh oh!

amirgon commented Jun 5, 2023

Uh oh!

ghost commented Jun 14, 2023

Uh oh!

amirgon commented Jun 14, 2023

Uh oh!

ghost commented Jun 14, 2023

Uh oh!

amirgon commented Jun 14, 2023

Uh oh!

amirgon commented Jun 16, 2023

Uh oh!

sam0910 commented Jun 17, 2023

Uh oh!

embeddedt commented Jun 18, 2023

Uh oh!

amirgon commented Jun 18, 2023

Uh oh!

embeddedt commented Jun 18, 2023

Uh oh!

embeddedt commented Jun 18, 2023

Uh oh!

amirgon commented Jun 18, 2023

Uh oh!

embeddedt commented Jun 23, 2023

Uh oh!

amirgon commented Jun 23, 2023

Uh oh!

Uh oh!

WIP: Align to micropython v1.20 #65

WIP: Align to micropython v1.20 #65

Uh oh!

Conversation

amirgon commented Jun 5, 2023

Uh oh!

ghost commented Jun 14, 2023

Uh oh!

amirgon commented Jun 14, 2023

Uh oh!

ghost commented Jun 14, 2023

Uh oh!

amirgon commented Jun 14, 2023

Uh oh!

amirgon commented Jun 16, 2023

Uh oh!

sam0910 commented Jun 17, 2023

Uh oh!

embeddedt commented Jun 18, 2023

Uh oh!

amirgon commented Jun 18, 2023

Uh oh!

embeddedt commented Jun 18, 2023

Uh oh!

embeddedt commented Jun 18, 2023

Uh oh!

amirgon commented Jun 18, 2023

Uh oh!

embeddedt commented Jun 23, 2023

Uh oh!

amirgon commented Jun 23, 2023

Uh oh!

Uh oh!