You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This post goes over the RustPython parser. You can see the source code at [RustPython/parser/](https://github.com/RustPython/RustPython/tree/master/parser).
8
+
This post goes over the RustPython parser. You can see the source code in the [rustpython-parser](https://github.com/RustPython/RustPython/tree/main/compiler/parser) crate.
9
9
10
10
When you write code in Python and run it, an interpreter, such as the RustPython interpreter, acts as the translator between you and your machine.
11
11
12
-
The interpreter has the job of turning your human code into bytecode that a Python virtual machine can run. Bytecode is an intermediate code between source code and machine code. This makes it portable across multiple hardware and operating systems. Bytecode "works" as long as you implement a virtual machine (vm) that can run it. There is a performance penalty for this flexibility. RustPython has a vm under [RustPython/vm/](https://github.com/RustPython/RustPython/tree/master/vm). Other posts will go into the details of that vm but now let's figure out how to turn code into bytecode.
12
+
The interpreter has the job of turning your human code into bytecode that a Python virtual machine can run. Bytecode is an intermediate code between source code and machine code. This makes it portable across multiple hardware and operating systems. Bytecode "works" as long as you implement a virtual machine (vm) that can run it. There is a performance penalty for this flexibility. RustPython also [has a vm](https://github.com/RustPython/RustPython/tree/main/vm) that interprets the generated bytecode, other posts will go into the details of that vm but now let's figure out how to turn code into bytecode.
13
13
14
14
15
15
## What bytecode looks like
@@ -54,10 +54,7 @@ If you want to sound fancy:
54
54
- The parsing process is called "lexical analysis"
The code for the lexing stage lives in [lex.rs](https://github.com/RustPython/RustPython/blob/main/compiler/parser/src/lexer.rs) of the parser crate.
61
58
62
59
63
60
If you want to dive into the details of lexical analysis, check out [Python in a nutshell / Lexical structure](https://learning.oreilly.com/library/view/python-in-a/9781491913833/ch03.html#python_language-id00003)
@@ -76,7 +73,7 @@ As the presenter puts it, this is the spirit of the beast (Python) and it is onl
76
73
77
74
So, we have the rules or grammar of a programming language in a machine encoded format... now we need to write something that verifies that those rules were followed... This sounds like something that other people could use and like something that should exist as an open source project! 🤔
78
75
79
-
Sure enough, there is a whole Rust framework called `LALRPOP`. It takes the tokens generated by the lexer, verifies the syntax and turns the tokens into an AST (Abstract Syntax Tree). More information and a tutorial can be found in the [LALRPOP book](https://lalrpop.github.io/lalrpop/README.html).
76
+
Sure enough, there is a whole Rust framework called `LALRPOP`. It takes the tokens generated by the lexer, verifies the syntax and turns the tokens into an AST (Abstract Syntax Tree). More information and a tutorial can be found in the [LALRPOP book](https://lalrpop.github.io/lalrpop/index.html).
80
77
81
78
RustPython does one nice extra thing on top of `LALRPOP`. It masks the errors and provides you with safer, nicer errors. You can see the code for this in `RustPython/parser/src/error.rs`
82
79
@@ -101,4 +98,4 @@ As a recap, when you write a line of Python code and "run it", here is what the
101
98
⬇️ compile the AST into bytecode
102
99
**OUTPUT: bytecode** (in `__pycache__/file.pyc` or in memory)
103
100
104
-
The compiler is under **`RustPython/compiler`**. Keep an eye on the blog for a future post about the details or the compiler. In the meantime, check out the parser source code in [RustPython/parser/](https://github.com/RustPython/RustPython/tree/master/parser).
101
+
The compiler is located in the [rustpython-compiler](https://github.com/RustPython/RustPython/tree/main/compiler) crate. Keep an eye on the blog for a future post about the details or the compiler. In the meantime, check out the parser source code in [rustpython-parser](https://github.com/RustPython/RustPython/tree/main/compiler/parser).
0 commit comments