Skip to content

Commit 4c06c85

Browse files
wooormwooorm
committed
.
0 parents  commit 4c06c85

34 files changed

+8997
-0
lines changed

Diff for: ‎.editorconfig

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
root = true
2+
3+
[*]
4+
indent_style = space
5+
indent_size = 2
6+
end_of_line = lf
7+
charset = utf-8
8+
trim_trailing_whitespace = true
9+
insert_final_newline = true
10+
11+
[*.rs]
12+
indent_size = 4

Diff for: ‎.github/workflows/main.yml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: main
2+
on:
3+
- pull_request
4+
- push
5+
jobs:
6+
main:
7+
name: ${{matrix.rust}}
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: actions/checkout@v2
11+
- uses: actions-rs/toolchain@v1
12+
with:
13+
toolchain: ${{matrix.rust}}
14+
components: rustfmt, clippy
15+
- run: cargo clippy -- -W clippy::pedantic
16+
- run: cargo fmt --all -- --check
17+
- run: cargo test
18+
- run: cargo install cargo-tarpaulin && cargo tarpaulin --out Xml
19+
- uses: codecov/codecov-action@v1
20+
strategy:
21+
matrix:
22+
rust:
23+
- stable
24+
- beta

Diff for: ‎.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.DS_Store
2+
*.log
3+
*.lock
4+
coverage/
5+
target

Diff for: ‎Cargo.toml

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
[package]
2+
name = "micromark"
3+
version = "0.0.0"
4+
authors = ["Titus Wormer <[email protected]>"]
5+
edition = "2015"
6+
rust-version = "1.56"
7+
description = "small commonmark compliant markdown parser with positional info and concrete tokens"
8+
homepage = "https://github.com/micromark/micromark-rs"
9+
repository = "https://github.com/micromark/micromark-rs"
10+
license = "MIT"
11+
keywords = ["commonmark", "markdown", "parse", "render", "tokenize"]
12+
categories = ["compilers", "encoding", "parser-implementations", "parsing", "text-processing"]
13+
include = ["src/", "license"]
14+
publish = false
15+
16+
[dependencies]
17+
log = "0.4"
18+
env_logger = "0.9"

Diff for: ‎Untitled.txt

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
micromark.js: unquoted: is `completeAttributeValueUnquoted`s case for `completeAttributeNameAfter` missing a `/`?. I’ve added it here.

Diff for: ‎examples/lib.rs

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
extern crate micromark;
2+
use micromark::{micromark, micromark_with_options, CompileOptions};
3+
4+
fn main() {
5+
// Turn on debugging.
6+
// You can show it with `RUST_LOG=debug cargo run --example lib`
7+
env_logger::init();
8+
9+
// Safely turn (untrusted?) markdown into HTML.
10+
println!("{:?}", micromark("# Hello, world!"));
11+
12+
// Turn trusted markdown into HTML.
13+
println!(
14+
"{:?}",
15+
micromark_with_options(
16+
"<div style=\"color: tomato\">\n\n# Hello, tomato!\n\n</div>",
17+
&CompileOptions {
18+
allow_dangerous_html: true
19+
}
20+
)
21+
);
22+
}

Diff for: ‎funding.yml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
github: wooorm

Diff for: ‎license

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
(The MIT License)
2+
3+
Copyright (c) 2022 Titus Wormer <[email protected]>
4+
5+
Permission is hereby granted, free of charge, to any person obtaining
6+
a copy of this software and associated documentation files (the
7+
'Software'), to deal in the Software without restriction, including
8+
without limitation the rights to use, copy, modify, merge, publish,
9+
distribute, sublicense, and/or sell copies of the Software, and to
10+
permit persons to whom the Software is furnished to do so, subject to
11+
the following conditions:
12+
13+
The above copyright notice and this permission notice shall be
14+
included in all copies or substantial portions of the Software.
15+
16+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
17+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
19+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
20+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
21+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
22+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Diff for: ‎readme.md

+183
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# micromark-rs
2+
3+
Here be dragons!
4+
🐉
5+
There’s a lot to do.
6+
Some major to dos are described here, more smaller ones are in the code.
7+
8+
## Some useful scripts for now
9+
10+
Run examples:
11+
12+
```sh
13+
RUST_BACKTRACE=1 RUST_LOG=debug cargo run --example lib
14+
```
15+
16+
Format:
17+
18+
```sh
19+
cargo fmt --all
20+
```
21+
22+
Lint:
23+
24+
```sh
25+
cargo fmt --all -- --check && cargo clippy -- -W clippy::pedantic
26+
```
27+
28+
Tests:
29+
30+
```sh
31+
RUST_BACKTRACE=1 cargo test
32+
```
33+
34+
Docs:
35+
36+
```sh
37+
cargo doc --document-private-items
38+
```
39+
40+
(add `--open` to open them in a browser)
41+
42+
## To do
43+
44+
### Some major obstacles
45+
46+
- [ ] (8) Subtokenization: figure out a good, fast way to deal with constructs in
47+
one content type that also are another content type
48+
- [ ] (1) Setext headings: can they be solved in content, or do they have to be
49+
solved in flow somehow
50+
- [ ] (8) Can content (and to a lesser extent string and text) operate more
51+
performantly than checking whether other flow constructs start a line,
52+
before exiting and actually attempting flow constructs?
53+
- [ ] (5) Figure out definitions and sharing those identifiers, and references
54+
before definitions
55+
- [ ] (3) Interrupting: sometimes flow can or cannot start depending on the
56+
previous construct (typically paragraph)
57+
- [ ] (5) Containers: this will be rather messy, and depends a lot on how
58+
subtokenization is solved
59+
- [ ] (3) Concrete constructs: HTML or code (fenced) cannot be “pierced” into by
60+
containers
61+
- [ ] (3) Lazy lines, in containers, in flow and content in a paragraph, a line
62+
does not need to be indented
63+
- [ ] (5) There’s a lot of rust-related choosing whether to pass (mutable)
64+
references or whatever around that should be refactored
65+
- [ ] (5) Figure out extensions
66+
- [ ] (1) Support turning off constructs
67+
68+
### Small things
69+
70+
- [ ] (3) Clean compiler
71+
- [ ] (1) Optionally remove dangerous protocols when compiling
72+
- [ ] (1) Use preferred line ending style in markdown
73+
- [ ] (1) Handle BOM at start
74+
- [ ] (1) Make sure tabs are handled properly and that positional info is perfect
75+
- [ ] (1) Make sure crlf/cr/lf are working perfectly
76+
- [ ] (3) Figure out lifetimes of things (see `life time` in source)
77+
- [ ] (3) Use `commonmark` tests
78+
- [ ] (3) Share a bunch of tests with `micromark-js`
79+
- [ ] (5) Do some research on rust best practices for APIs, e.g., what to accept,
80+
how to integrate with streams or so?
81+
- [ ] (1) Go through clippy rules, and such, to add strict code styles
82+
- [ ] (1) Make sure that rust character groups match CM character groups (e.g., is
83+
`unicode_whitespace` or so the same?)
84+
- [ ] (1) Any special handling of surrogates?
85+
- [ ] (1) Make sure debugging is useful for other folks
86+
- [ ] (3) Add some benchmarks, do some perf testing
87+
- [ ] (3) Write comparison to other parsers
88+
- [ ] (3) Add node/etc bindings?
89+
- [ ] (8) After all extensions, including MDX, are done, see if we can integrate
90+
this with SWC to compile MDX
91+
- [ ] (3) Bunch of docs
92+
- [ ] (5) Site
93+
94+
### Constructs
95+
96+
- [ ] (5) attention (strong, emphasis) (text)
97+
- [ ] (1) autolink
98+
- [x] blank line
99+
- [ ] (5) block quote
100+
- [x] character escape
101+
- [x] character reference
102+
- [x] code (fenced)
103+
- [x] code (indented)
104+
- [ ] (1) code (text)
105+
- [ ] (3) content
106+
- [ ] (3) definition
107+
- [ ] (1) hard break escape
108+
- [x] heading (atx)
109+
- [ ] (1) heading (setext)
110+
- [x] html (flow)
111+
- [ ] html (text)
112+
- [ ] (3) label end
113+
- [ ] (3) label start (image)
114+
- [ ] (3) label start (link)
115+
- [ ] (8) list
116+
- [ ] (1) paragraph
117+
- [x] thematic break
118+
119+
### Content types
120+
121+
- [ ] (8) container
122+
- [ ] block quote
123+
- [ ] list
124+
- [ ] (1) flow
125+
- [x] blank line
126+
- [x] code (fenced)
127+
- [x] code (indented)
128+
- [ ] content
129+
- [x] heading (atx)
130+
- [x] html (flow)
131+
- [x] thematic break
132+
- [ ] (3) content
133+
- [ ] definition
134+
- [ ] heading (setext)
135+
- [ ] paragraph
136+
- [ ] (5) text
137+
- [ ] attention (strong, emphasis) (text)
138+
- [ ] autolink
139+
- [x] character escape
140+
- [x] character reference
141+
- [ ] code (text)
142+
- [ ] hard break escape
143+
- [ ] html (text)
144+
- [ ] label end
145+
- [ ] label start (image)
146+
- [ ] label start (link)
147+
- [x] string
148+
- [x] character escape
149+
- [x] character reference
150+
151+
### Extensions
152+
153+
The main thing here is is to figure out if folks could extend from the outside
154+
with their own code, or if we need to maintain it all here.
155+
Regardless, it is essential for the launch of `micromark-rs` that extensions
156+
are theoretically or practically possible.
157+
The extensions below are listed from top to bottom from more important to less
158+
important.
159+
160+
- [ ] (1) frontmatter (yaml, toml) (flow)
161+
— [`micromark-extension-frontmatter`](https://github.com/micromark/micromark-extension-frontmatter)
162+
- [ ] (3) autolink literal (GFM) (text)
163+
— [`micromark-extension-gfm-autolink-literal`](https://github.com/micromark/micromark-extension-gfm-autolink-literal)
164+
- [ ] (3) footnote (GFM) (content, text)
165+
— [`micromark-extension-gfm-footnote`](https://github.com/micromark/micromark-extension-gfm-footnote)
166+
- [ ] (3) strikethrough (GFM) (text)
167+
— [`micromark-extension-gfm-strikethrough`](https://github.com/micromark/micromark-extension-gfm-strikethrough)
168+
- [ ] (5) table (GFM) (flow)
169+
— [`micromark-extension-gfm-table`](https://github.com/micromark/micromark-extension-gfm-table)
170+
- [ ] (1) task list item (GFM) (text)
171+
— [`micromark-extension-gfm-task-list-item`](https://github.com/micromark/micromark-extension-gfm-task-list-item)
172+
- [ ] (3) math (flow, text)
173+
— [`micromark-extension-math`](https://github.com/micromark/micromark-extension-math)
174+
- [ ] (8) directive (flow, text)
175+
— [`micromark-extension-directive`](https://github.com/micromark/micromark-extension-directive)
176+
- [ ] (8) expression (MDX) (flow, text)
177+
— [`micromark-extension-mdx-expression`](https://github.com/micromark/micromark-extension-mdx-expression)
178+
- [ ] (5) JSX (MDX) (flow, text)
179+
— [`micromark-extension-mdx-jsx`](https://github.com/micromark/micromark-extension-mdx-jsx)
180+
- [ ] (3) ESM (MDX) (flow)
181+
— [`micromark-extension-mdxjs-esm`](https://github.com/micromark/micromark-extension-mdxjs-esm)
182+
- [ ] (1) tagfilter (GFM) (n/a, renderer)
183+
— [`micromark-extension-gfm-tagfilter`](https://github.com/micromark/micromark-extension-gfm-tagfilter)

0 commit comments

Comments
 (0)