Add Disassembler

.github/workflows/run_tests.yaml

@@ -91,3 +91,10 @@ jobs:
         export PATH=$PATH:${{ steps.fetch_binutils.outputs.bin_dir }}
         cd tests
         ./02_compat_rtc_tests.sh
+
+    - name: Run disassembler tests
+      id: disassembler_tests
+      run: |
+        export PATH=$PATH:${{ steps.build_micropython.outputs.bin_dir }}
+        cd tests
+        ./03_disassembler_tests.sh

README.rst

@@ -35,6 +35,7 @@ The following features are supported:
 * expressions in assembly code and constant definitions
 * RTC convenience macros (e.g. ``WRITE_RTC_REG``)
 * many ESP32 ULP code examples found on the web will work unmodified
+* a simple disassembler is also provided
 
 
 Quick start

docs/disassembler.rst

@@ -0,0 +1,146 @@
+=====================
+Disassembler
+=====================
+
+micropython-esp32-ulp contains a disassembler for disassembling code for the
+ESP32 ULP (Ultra Low-Power) Co-Processor.
+
+The main purpose of this tool is to inspect what instructions our assembler
+created, what value each field is set to, and to compare this with the output
+created by the assembler from Espressif (part of their `binutils-gdb fork <https://github.com/espressif/binutils-gdb/tree/esp32ulp-elf-2.35>`_),
+which we use as our reference implementation.
+
+
+Usage
+------------------------
+
+To disassemble a ULP binary, simply run:
+
+.. code-block:: bash
+
+   micropython -m tools.disassemble path/to/binary.ulp
+
+You can also specify additional options to ``disassemble.py`` as follows:
+
++--------------------------+----------------------------------------------------------------+
+| Option                   | Description                                                    |
++==========================+================================================================+
+| ``-h``                   | Show help text                                                 |
++--------------------------+----------------------------------------------------------------+
+|| ``-m <bytes sequence>`` || Disassemble a provided sequence of hex bytes                  |
+||                         || (in this case any filename specified is ignored)              |
++--------------------------+----------------------------------------------------------------+
+| ``-v``                   | Verbose mode (shows ULP header and fields of each instruction) |
++--------------------------+----------------------------------------------------------------+
+
+
+Disassembling a file
+------------------------
+
+The simplest and default mode of the disassembler is to disassemble the
+specified file.
+
+Note that the ULP header is validates and files with unknown magic bytes will be
+rejected. The correct 4 magic bytes at the start of a ULP binary are ``ulp\x00``.
+
+Example:
+
+.. code-block:: shell
+
+   $ micropython -m tools.disassemble path/to/binary.ulp
+   .text
+   0000  040000d0  LD r0, r1, 0
+   0004  0e0400d0  LD r2, r3, 1
+   0008  84010068  ST r0, r1, 0
+   000c  8b090068  ST r3, r2, 2
+   .data
+   0000  00000000  <empty>
+
+
+Disassembling a byte sequence
+-----------------------------
+
+The ``-m`` option allows disassembling a sequences hex letters representing
+ULP instructions.
+
+This option expects the actual instructions directly, without any ULP header.
+
+The sequence must contain a number of hex letters exactly divisible by 8, i.e.
+8, 16, 24, etc, because each 32-bit word is made up of 8 hex letters. Spaces
+can be included in the sequence and they are ignored.
+
+The typical use case for this feature is to copy/paste some instructions from
+a hexdump (e.g. xxd output) for analysis.
+
+Example:
+
+.. code-block:: shell
+
+   # hexdump binary.ulp
+   $ xxd path/to/binary.ulp
+   00000000: 756c 7000 0c00 2400 0400 0000 9300 8074  ulp...$........t
+   00000010: 2a80 0488 2004 8074 1c00 0084 0000 0040  *... ..t.......@
+   (...)
+
+   # analyse the last 2 instructions
+   $ micropython -m tools.disassemble -m "1c00 0084 0000 0040"
+   0000  1c000084  JUMPS 0, 28, LT
+   0004  00000040  NOP
+
+
+Verbose mode
+------------------------
+
+In verbose mode the following extra outputs are enabled:
+
+* ULP header (except when using ``-m``)
+* The fields of each instruction and their values
+
+For example:
+
+.. code-block::
+
+   header
+   ULP magic    : b'ulp\x00' (0x00706c75)
+   .text offset : 12 (0x0c)
+   .text size   : 36 (0x24)
+   .data offset : 48 (0x30)
+   .data size   : 4 (0x04)
+   .bss size    : 0 (0x00)
+   ----------------------------------------
+   .text
+   0000  93008072  MOVE r3, 9
+                    dreg       =   3
+                    imm        =   9
+                    opcode     =   7
+                    sel        =   4 (MOV)
+                    sreg       =   0
+                    sub_opcode =   1
+                    unused     =   0
+   (...detail truncated...)
+   0020  000000b0  HALT
+                    opcode     =  11 (0x0b)
+                    unused     =   0
+   ----------------------------------------
+   .data
+   0000  00000000  <empty>
+
+
+Disassembling on device
+-----------------------------
+
+The disassembler also works when used on an ESP32.
+
+To use the disassembler on a real device:
+
+* ensure ``micropython-esp32-ulp`` is installed on the device (see `docs/index.rst </docs/index.rst>`_).
+* upload ``tools/disassemble.py`` to the device (any directory will do)
+* run the following:
+
+  .. code-block:: python
+
+     from disassemble import disassemble_file
+     # then either:
+     disassemble_file('path/to/file.ulp')  # normal mode
+     # or:
+     disassemble_file('path/to/file.ulp', True)  # verbose mode

docs/index.rst

@@ -136,6 +136,15 @@ found as part of Arduino/ESP-IDF projects.
 The preprocessor and how to use it is documented here: `Preprocessor support </docs/preprocess.rst>`_.
 
 
+Disassembler
+------------
+There is a disassembler for disassembling ULP binary code. This is mainly used to
+inspect what instructions our assembler created, however it can be used to analyse
+any ULP binaries.
+
+The disassembler and how to use it is documented here: `Disassembler </docs/disassembler.rst>`_.
+
+
 Limitations
 -----------
 

tests/00_unit_tests.sh

@@ -4,7 +4,9 @@
 
 set -e
 
-for file in opcodes assemble link util preprocess definesdb; do
+LIST=${1:-opcodes assemble link util preprocess definesdb disassemble}
+
+for file in $LIST; do
     echo testing $file...
     micropython $file.py
 done

tests/03_disassembler_tests.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+
+set -e
+
+test_disassembling_a_file() {
+    local verbose
+    if [ "$1" == verbose ]; then
+        verbose=-v
+        echo -e "Testing disassembling a file in VERBOSE mode"
+    else
+        echo -e "Testing disassembling a file in NORMAL mode"
+    fi
+
+    testname=all_opcodes
+    fixture=fixtures/${testname}.S
+    echo -e "\tBuilding $fixture using micropython-esp32-ulp"
+
+    log_file="${testname}.log"
+    ulp_file="fixtures/${testname}.ulp"
+    micropython -m esp32_ulp $fixture 1>$log_file   # generates $ulp_file
+
+    lst_file="${testname}.lst"
+    lst_file_fixture=fixtures/${testname}${verbose}.lst
+    echo -e "\tDisassembling $ulp_file using micropython-esp32-ulp disassembler"
+    micropython tools/disassemble.py $verbose $ulp_file > $lst_file
+
+    if ! diff $lst_file_fixture $lst_file 1>/dev/null; then
+        echo -e "\tDisassembled output differs from expected output!"
+        echo ""
+        echo "Disassembly test failed for $fixture"
+        echo "micropython-esp32-ulp log:"
+        cat $log_file
+        echo "Diff of disassembly: expected vs actual"
+        diff -u $lst_file_fixture $lst_file
+    fi
+}
+
+test_disassembling_a_manual_sequence() {
+    local verbose
+    if [ "$1" == verbose ]; then
+        verbose=-v
+        echo -e "Testing disassembling a manual byte sequence in VERBOSE mode"
+    else
+        echo -e "Testing disassembling a manual byte sequence in NORMAL mode"
+    fi
+
+    sequence="e1af 8c72 0100 0068 2705 cc19 0005 681d 0000 00a0 0000 0074"
+
+    lst_file="manual_bytes.lst"
+    lst_file_fixture=fixtures/manual_bytes${verbose}.lst
+    echo -e "\tDisassembling manual byte sequence using micropython-esp32-ulp disassembler"
+    micropython tools/disassemble.py $verbose -m $sequence > $lst_file
+
+    if ! diff $lst_file_fixture $lst_file 1>/dev/null; then
+        echo -e "\tDisassembled output differs from expected output!"
+        echo ""
+        echo "Disassembly test failed for manual byte sequence"
+        echo "Diff of disassembly: expected vs actual"
+        diff -u $lst_file_fixture $lst_file
+    fi
+}
+
+test_disassembling_a_file
+test_disassembling_a_file verbose
+
+test_disassembling_a_manual_sequence
+test_disassembling_a_manual_sequence verbose