Releases: mincrmatt12/nmfu
0.5.0a1
This is a preview of some of the new features in the upcoming 0.5.0 release.
New Features
- Greedy case statements: explicit ambiguity resolution can be turned on as an option in case statements now (52e5418)
- Finish codes: explicit finish statements can now return custom status codes, instead of just always giving
_DONE
(1ffe4d4) - Yield: parsers can now interrupt execution with a return code and be restarted; for example to build a lexer (1ffe4d4, 41b2c23)
Bugs Fixed
- Loops that have (possibly conditional) breaks before any matching statements no longer think they're infinite (a687b3b)
Infrastructure Changes
- Tests now use finish codes to better check correctness (9f1c20e)
- Various examples/docs have been updated to use finish codes instead of result enumerations (976fc60, a777a5e)
- New lexer example demonstrating yield functionality (a9a96a8)
- Various documentation improvements:
- Updated to lark 1.0 (f3dd61e)
- Appimages now build against Ubuntu 20.04 as their base (9dc256c)
- Note snaps may be removed in a future release due to issues in reliably building them in CI
0.4.1
(0.4.0 was missing a commit and so is not listed here as a release)
This is a major feature release, vastly improving NMFU's capability for parsing binary protocols, in addition to fixing a number of bugs. There's also now a proper documentation site for NMFU, available at nmfu.rtfd.io.
New Features
- Expanded type system: (ea5e297)
- Integers can now have a custom size and signedness:
out int{size 4, unsigned} wow;
- Strings can now be unterminated:
out unterminated str[56] look_no_nulls;
- Integers can now have a custom size and signedness:
- Conditional blocks:
if
/elif
/else
statements now exist (be41210, f3f71de). - Expanded expressions:
- Binary matches (8b2f09b, e202041):
- Strings and regexes can now be made "binary", where instead of them having characters they instead only contain hex.
- For example,
"f0 0ffa"b
orb/ff+ (3456|56 77{2-3})/
. - This also adds support for
\xHH
escapes within normal strings.
- Character constants now support escape sequences (45196d6, 1cdd324)
- Improved debugging tools for working on NMFU itself:
- New optimizer that tries to shortcircuit fallthrough transitions if possible (3043adc, db126e2)
Bugs Fixed
- Better ambiguity detection and resolution for chaining matches, especially those with inverted matches (1026f83).
- Properly handle actions after certain blocks when there is nothing following them (c5f1806).
- Correct short-circuit behaviour for action side-effects (such as
outofspace
handling) (bbda784). - More accurate line/column information for errors concerning regexes (92cd8f2).
- Ensure chaining DFAs doesn't drop error handling-transitions in certain circumstances (64e89f2)
- Fix
DFA.dfs()
not properly consideringMAY_GOTO_TARGET
-type actions, causing the optimizer to mistakenly drop states (ca72b3b) - Fixed loop body start actions being ignored (ccb54a3)
- Macros now bind arguments in a sane way, fixing various problems with nesting them. (8f87354, 6191b42)
- Fallthrough loop detection now considers multi-state cycles. (222d971)
- Action-only conditional nodes now properly propagate subsequent actions (ad3a917)
- Foreach nodes only apply their actions to non-fallthrough transitions to ensure they only run once per character (709d314)
- Rewrote the majority of the loop implementation to make it more compatible with action side-effects (5b520bb)
DTAG
system now does more checks for garbage-collection and uses weakrefs to be slightly more memory-efficient (e5d58f3)
Infrastructure Changes
- Proper documentation, powered by mkdocs (ce5429c) as well as a basic tutorial (a1ea939, 3af8af2, 3897641)
- New example showcasing binary features which parses
TripUpdate
messages from a GTFS Realtime feed (f4a2edb, e7c8b1d) - Demonstration of more advanced string manipulation in the RDF example (92ff838, 5fa891e)
- Full integration tests now will run the generated code through an actual compiler to check for correctness. While they're skipped if one isn't found, please make sure any potential patches are tested with one installed. (acbff31)
- Certain integration tests now check for both compilation success and correct matching (a2fb61a)
- Code coverage is now reported in CI (2b9152f)
- More tests, both for new and old features. (lots)
0.4.0a2
This is another early preview of some more new features in the upcoming 0.4.0 release, as well as some fixes for regressions in 0.4.0a1.
New Features
- More math operators: unary not and negation (5f24078), string length and indexing (353f391) and bitwise logic/bitshifts (6936042).
- Character constants now support escape sequences (45196d6)
- Expanded type system: (ea5e297)
- Integers can now have a custom size and signedness:
out int{size 4, unsigned} wow;
- Strings can now be unterminated:
out unterminated str[56] look_no_nulls;
- Integers can now have a custom size and signedness:
- Better integer literals; now supporting hexadecimal and binary:
0b0110
,-0x54
(1e61988) - Improved NMFU-debugging:
- New optimizer that tries to shortcircuit fallthrough transitions if possible (3043adc, db126e2)
Bugs Fixed
- Ensure chaining DFAs doesn't drop error handling-transitions in certain circumstances (64e89f2)
- Fix
DFA.dfs()
not properly consideringMAY_GOTO_TARGET
-type actions, causing the optimizer to mistakenly drop states (ca72b3b) - Fixed loop body start actions being ignored (ccb54a3)
- Macros now bind arguments in a sane way, fixing various problems with nesting them. (8f87354, 6191b42)
- Fallthrough loop detection now considers multi-state cycles. (222d971)
- Action-only conditional nodes now properly propagate subsequent actions (ad3a917)
- Foreach nodes only apply their actions to non-fallthrough transitions to ensure they only run once per character (709d314)
- Rewrote the majority of the loop implementation to make it more compatible with action side-effects (5b520bb)
DTAG
system now does more checks for garbage-collection and uses weakrefs to be slightly more memory-efficient (e5d58f3)
Infrastructure Changes
- New test showcasing binary features which parses
TripUpdate
messages from a GTFS Realtime feed (f4a2edb, e7c8b1d) - Demonstration of more advanced string manipulation in the RDF example (92ff838, 5fa891e)
- Full integration tests now will run the generated code through an actual compiler to check for correctness. While they're skipped properly if one isn't found, please make sure any potential patches are tested with one installed. (acbff31)
0.4.0a1
This is an early preview of some of the new features in the upcoming 0.4.0 release.
New Features
- Conditional blocks:
if
/elif
/else
statements now exist (be41210, f3f71de). - Boolean expressions:
||
and&&
, as well as comparison operators (baa2ba1, 3369969) - Binary matches (8b2f09b, e202041):
- Strings and regexes can now be made "binary", where instead of them having characters they instead only contain hex.
- For example,
"f0 0ffa"b
orb/ff+ (3456|56 77{2-3})/
. - This also adds support for
\xHH
escapes within normal strings.
- Option for dumping the parse tree for debugging (1b7242a).
Bugs Fixed
- Better ambiguity detection and resolution for chaining matches, especially those with inverted matches (1026f83).
- Properly handle actions after certain blocks when there is nothing following them (c5f1806).
- Correct short-circuit behaviour for action side-effects (such as
outofspace
handling) (bbda784). - More accurate line/column information for errors concerning regexes (92cd8f2).
Infrastructure Changes
0.3.1
0.3.0
This is a feature release which adds a bunch of miscellaneous features, in addition to changing the public-facing API.
New Features
- New API: (cbd45c4)
- Instead of taking in single characters, the
_feed
function now takes in blocks of characters at once. The function can also
optionally report how many characters it read in case of early finishes, to aid in chaining parsers. - Instead of having a parameter for
is_end
, there is now an optional separate_end
function for handling EOF detection, enabled
by the-feof-support
flag. - These changes are designed to improve generated parser efficiency.
- Instead of taking in single characters, the
- Fallthrough semantics: (cce57c4, af6728f)
- Instead of
else
matches in case nodes and catch blocks consuming characters, they now forward the error/else to the first match
in their block. - This makes error handling more robust, as well as removing a lot of the uses for
some_var += [$last]
, simplifying parsers.
- Instead of
- Revamped regexes: (50f219c, 5603d44)
- Operator precedence is now more consistent with typical regexes, so
(option1|option2)
now works correctly. - Repetition syntax (
\d{4-6}
) is now supported.
- Operator precedence is now more consistent with typical regexes, so
- Foreach statement: (9cda4ac)
- Parsers can now execute statements on each character matched by some region.
- This greatly simplifies things like reading numbers:
foreach { \d+\; } do { num = [num * 10 + ($last - '0')]; }
- Macros can now take arguments. (07e4a04)
- Character constants can now be used in math expressions. (fc93c91)
Bugs Fixed
- Fixed problem where having a wait match after a try except could break due to incomplete error-handling redirection. (87de753)
- Fixed internal error with
allow_replace
inDFState.transition
which could hide ambiguous transitions. (0e1ad3b) - Fixed problems with incomplete
else
handling on case nodes with regex-matches with inverted character sets/wildcards. (417fb41, 3fa4d93)
Infrastructure Changes
0.3.0a2
This is another early preview of some more changes in the upcoming 0.3.0 release.
New Features
- Added a mode for making the
start
pointer indirect, allowing_feed
to report how much of its input it consumed. (e589c6f) - Changed regex grammar to more closely match other regex syntax (by fixing the precedence of the
|
operator.) (50f219c) - Added support for regex repetition matches (5603d44)
- Added a new block syntax,
foreach
, which can execute statements on each character consumed by a block. (9cda4ac) - Added character constants in math expressions. (fc93c91)
Bugs Fixed
- Various problems with the new fallthrough transitions from 0.3.0a1. (4203c2e, 68e95a3)
- Fixed problem where having a wait match after a try except could break due to incomplete error-handling redirection. (87de753)
- Fixed internal error with
allow_replace
inDFState.transition
which could hide ambiguous transitions. (0e1ad3b) - Fixed problems with incomplete
else
handling on case nodes with regex-matches with inverted character sets/wildcards. (417fb41, 3fa4d93) - Fix end matches, which were broken in 0.3.0a1. (4e3b6db, 4f4dba9)
Infrastructure Changes
- The test suite has been thoroughly expanded, and now covers a lot more.
- There is now an automated version update script.
0.3.0a1
This is an early preview of some of the breaking changes in the upcoming 0.3.0 release.
New Features
- New API: instead of taking in single characters, the
_feed
function now takes in blocks of characters at once, to improve efficiency end
support is now optional. This is part of the API changes, because now end handling is through a separate_end
function.- Fallthrough semantics: the
else
path of acase
node, as well as jumps intocatch
blocks no longer consume a character, and instead
properly forward it to the first match in the target block.
Infrastructure Changes
- NMFU now also builds AppImages
0.2.1
This is a minor bugfix release with a few new features
New Features
- New option
-fuse-cplusplus-guard
which makes the generated headers C++ compatible
Bugs Fixed
- Previously, a try-except block with an empty handler would cause an error; now it correctly does nothing and exit on an error.
0.2.0
This is a feature release which adds hooks and a few miscellaneous features.
New Features
- Hooks:
- Parsers can now call arbitrary callbacks at arbitrary points like macros
- These hooks get a pointer to the current state and input character
- In the future, these may be used for things like buffering large outputs
- Append-character expressions
- Append expressions can now add individual character codes from a math expression
- e.g:
out_var += [$last - 3];
Bugs Fixed
- Case labels with inverted character classes (or non-erroring Else transitions) would accept too few inputs
- Macros with more than one statement would be truncated
- Custom Else transitions were not properly dealt with by the ambiguous detection logic and so a variety of things involving wildcards would fail in strange ways.
Infrastructure Changes
- NMFU now has a test suite, install the necessary dependencies with the
tests
extra and run withpytest
- NMFU now has CI, see the badge in the README
- NMFU can now build as a snap