| 
1 | 1 | # The Parser  | 
2 | 2 | 
 
  | 
3 | 3 | The parser is responsible for converting raw Rust source code into a structured  | 
4 |  | -form which is easier for the compiler to work with, usually called an *Abstract  | 
5 |  | -Syntax Tree*. The bulk of the parser lives in the [libsyntax] crate.  | 
 | 4 | +form which is easier for the compiler to work with, usually called an [*Abstract  | 
 | 5 | +Syntax Tree*][ast]. An AST mirrors the structure of a Rust program in memory,   | 
 | 6 | +using a `Span` to link a particular AST node back to its source text.  | 
6 | 7 | 
 
  | 
7 |  | -The parsing process is made up of roughly 3 stages,  | 
 | 8 | +The bulk of the parser lives in the [libsyntax] crate.  | 
 | 9 | + | 
 | 10 | +Like most parsers, the parsing process is composed of two main steps,  | 
8 | 11 | 
 
  | 
9 | 12 | - lexical analysis - turn a stream of characters into a stream of token trees  | 
10 |  | -- macro expansion - run `proc-macros` and expand `macro_rules` macros  | 
11 | 13 | - parsing - turn the token trees into an AST  | 
12 | 14 | 
 
  | 
 | 15 | +The `syntax` crate contains several main players,  | 
 | 16 | + | 
 | 17 | +- a [`CodeMap`] for mapping AST nodes to their source code  | 
 | 18 | +- the [ast module] contains types corresponding to each AST node  | 
 | 19 | +- a [`StringReader`] for lexing source code into tokens  | 
 | 20 | +- the [parser module] and [`Parser`] struct are in charge of actually parsing  | 
 | 21 | +  tokens into AST nodes,  | 
 | 22 | +- and a [visit module] for walking the AST and inspecting or mutating the AST  | 
 | 23 | +  nodes.  | 
 | 24 | + | 
 | 25 | +The main entrypoint to the parser is via the various `parse_*` functions  | 
 | 26 | +in the [parser module]. They let you do things like turn a filemap into a  | 
 | 27 | +token stream, create a parser from the token stream, and then execute the  | 
 | 28 | +parser to get a `Crate` (the root AST node).  | 
 | 29 | + | 
 | 30 | +To minimise the amount of copying that is done, both the `StringReader` and   | 
 | 31 | +`Parser` have lifetimes which bind them to the parent `ParseSess`. This contains  | 
 | 32 | +all the information needed while parsing, as well as the `CodeMap` itself.  | 
13 | 33 | 
 
  | 
14 | 34 | [libsyntax]: https://github.com/rust-lang/rust/tree/master/src/libsyntax  | 
 | 35 | +[rustc_errors]: https://github.com/rust-lang/rust/tree/master/src/librustc_errors  | 
 | 36 | +[ast]: https://en.wikipedia.org/wiki/Abstract_syntax_tree  | 
 | 37 | +[`CodeMap`]: https://github.com/rust-lang/rust/blob/master/src/libsyntax/codemap.rs  | 
 | 38 | +[ast module]: https://github.com/rust-lang/rust/blob/master/src/libsyntax/ast.rs  | 
 | 39 | +[parser module]: https://github.com/rust-lang/rust/tree/master/src/libsyntax/parse  | 
 | 40 | +[`Parser`]: https://github.com/rust-lang/rust/blob/master/src/libsyntax/parse/parser.rs  | 
 | 41 | +[`StringReader`]: https://github.com/rust-lang/rust/blob/master/src/libsyntax/parse/lexer/mod.rs  | 
 | 42 | +[visit module]: https://github.com/rust-lang/rust/blob/master/src/libsyntax/visit.rs  | 
0 commit comments