Skip to content

Release/7.0#74

Merged
tnunnink merged 14 commits into
mainfrom
release/7.0
May 24, 2026
Merged

Release/7.0#74
tnunnink merged 14 commits into
mainfrom
release/7.0

Conversation

@tnunnink

Copy link
Copy Markdown
Owner

This release improves TagName and Instruction text parsing to correctly capture and extract nested indirect tag references found in arrays. This is accomplished by moving away from Regex pattern matching to new NeutralText and NeutralStream lexer/tokenization pattern.

New classes that can be used to stream tokens from a string of neutral text include.

  • NeutralText: This is a light wrapper around a string that provides the ability to tokenize the string.
  • NeutralToken: This is a single token in the stream of natural text. It contains the token type, value, starting index, and length.
  • TokenType: This is an enumeration on the token class that indicates the type of the token (Identifier, Literal, Bracket, Dot, Parenthesis, Comma, etc.).
  • NeutralStream: This is a stream interface for enumerating a sequence of tokens for processing.

I have also updated the API for TagName to make the properties a little more explicit.

  • Path > FullPath
  • LocalPath (same)
  • Operand > RelativePath
  • Member > MemberPath
  • Base > BaseName
  • Element > MemberName

Argument has also been simplified. It now mostly centers around identification of the argument value (reference, literal, expression, etc). It is now up to user to determine if an argument needs to be parsed further (expression case). This can be done by using TagName.Parse static method to extract tag names from the argument expression if needed.

Instruction was also updated to just provide the main Arguments collection since you can filter arguments by type. I think this simplifies the API a bit as it was already getting a little confusing and circular.

tnunnink and others added 14 commits May 18, 2026 10:46
…pdated related classes, methods, and tests to align with the new API. Added new `ClearData` method in `StringData`.
…crape` to `ExtractAll` for improved clarity and consistency. Updated all references, methods, and tests to align with the new naming.
…hanced parsing of diverse atomic formats. Updated tests to ensure coverage for multiple argument scenarios.
…types for enhanced format checking. Updated corresponding tests to ensure complete coverage for validity checks. Refined exception handling in `Parse` methods for consistent error responses.
…erator-based splitting for improved performance and maintainability. Introduced new properties (`IsInvalid`, `IsLiteral`, `IsTag`, `IsAtomic`, `IsExpression`) for enhanced argument type checks. Updated `ArgumentType` and related APIs to align with the changes. Revised and expanded test coverage to reflect the updates.
…gument` class for improved expression parsing, and added corresponding tests for enhanced functionality and coverage.
…clarity and context. Updated associated methods, properties, enums, and exceptions. Added `NeutralText` and `NeutralTextExtensions` for tokenization support. Expanded and updated tests to cover new functionality and changes.
…ciency and added new methods for advanced token stream navigation. Enhanced `NeutralText` and `NeutralTextExtensions` with updated operator handling, additional tokenization features, and streamlined helper methods. Replaced `EOF` with `None` in `NeutralToken` for improved clarity.
…factored `NeutralStream` for iterator-based token processing, simplifying functionality and enhancing memory efficiency. Updated `TagName` with the new `Append` method and switched from concatenation to method chaining for creating hierarchical tag names throughout the codebase. Replaced unused logic in `Argument` with `NotImplementedException`. Expanded `NeutralText` test cases for broader tokenization coverage.
…gic and unused methods, simplifying the implementation. Improved `Instruction` tokenization and processing with enhanced `NeutralStream` integration. Added comprehensive tests for `NeutralStream` and updated test cases around `Instruction` and `TagName`.
…pdated associated tests to improve format checking and error handling.
@tnunnink tnunnink merged commit bb9df6b into main May 24, 2026
1 check passed
@tnunnink tnunnink deleted the release/7.0 branch May 24, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant