CL Syntax Checker

[bobcozzi]

This is my pr to integrate the syntax checker into CLLE

[worksofliam]

@bobcozzi Is this PR ready for review?

[worksofliam]

@bobcozzi We're going to need to use the components API in this PR like we discussed.

[bobcozzi]

Hey, welcome back. The PR currently is using cmd/ctrl+F5 to trigger syntax checking on the current line. I did not try to change the existing code. Perhaps it needs to also be triggered on an Enter key?
But YES aside from that it is ready and if you want to integrate it into the original code, I will push the PR this morning.

[bobcozzi]

@bobcozzi We're going to need to use the components API in this PR like we discussed.

I take it you got pulled into a million things to do before you "turn the page". I don't know what those "merging is blocked" message means so I am going to leave this with you and go back to the other extensions. Let me know if you need/want me to tweak anything regarding the CL syntax checker.

[SanjulaGanepola]

Hello @bobcozzi. This is an awesome PR! There are some changes which will need to be made - one in particular is to move to use the component API provided by Code for IBM i. This will standardize the versioning and installation process to fit in with how we install other pieces on the IBM i. Would you be interested in doing this or would you like me to take a look at this if you don't mind me pushing directly to this branch?

[bobcozzi]

I haven't had success implementing the component API in my extensions, so yes, I'd like to see one in action. Feel free to push directly into this branch. Thanks!

[SanjulaGanepola]

@bobcozzi Sorry about the delay with getting to this. I was able to finish a first pass at making the changes to use the component API. In addition to that, I made some significant changes to how and when the syntax checker is triggered (so it is more inline with how the SQL syntax checker behaves in the Db2 for IBM i extension). I am open to discuss if you think it makes more sense for it to be user invoked as it was before rather than automatically (details below). I personally find having it automatically check to be very convenient.

Changes

• Registered a new CLSyntaxChecker component in the Code for IBM i component registry. This will be automatically installed like you had before and can be found in the components tab of the connection settings
• Restructured code / files to be similar to the SQL syntax checker code in Db2i for IBM i (more details in the bullet below on the actual changes to functionality). For reference that implementation is here
• Removed the vscode-clle.CLSyntaxCheck command and keybinding in favour of having the same behavior as the SQL syntax checker:
  • By default, it will automatically run the syntax checker on the command you are typing on. There is a wait period (1.5 seconds) which is also configurable
  • To validate all commands in the entire document, you can also now use a button in the top right of the editor. This will run the syntax checker on each individual command
    
  • Added new extension settings:
    
• Refactored collectCLCmds logic to use an existing utility function (getCommandString)
• Added a progress status bar at the bottom to indicate when the syntax checker is running

Currently, these changes are in my own branch (feature/cl-syntax-checker-component). I can go ahead and merge those changes into this branch if these changes sound good.

I do have a couple questions for you:

1. How can we have the syntax checker handle variables?
   
2. Do lines with expressions have to be ignored?
   

[bobcozzi]

I sorry I was out of town the last few weeks.
So the host program has a 2nd parameter as does the UDTF. That 2nd parameter controls any type of CL command environment to test for syntax errors. All support environments are available, Command Entry (i.e., no CL variables) CL Program (labels, variables expressions), Command Line (limited user menu command line) etc. Here is the C/C++ comments on that:
//////////////////////////////////////////////
// Input Indicator Fields
//////////////////////////////////////////////
inIndy(CMD);
inIndy(CHECKOPT);
// Options may be any 1 of the following:
// Note upper/lower case and leading asterisk are ignored.
// NULL or empty = Command Entry (non-program) CL (default)
// *CL - Command Entry (non-program) CL
// *CLLE - ILE CL Program syntax
// *CLP - OPM CL Program syntax
// *LIMIT - Limited User Commands
// *CMD - Command Definition Commands
// *BND - Binder Language Commands

The UDTF's CHECKOPT parameter, accepts any of those values. Right nowt he default is *CL (command entry) which I selected because Liam was an advocate of developers running CL commands (which I don't think most people do during development). But that can be modified before we ship it.
So to get CL Program for ILE CL (probably the best choices for a default) is:

  SELECT * FROM TABLE(CL_SYNTAX_CHECK(cmd=>'CHECKFILE:  PGM PARM(&FILE &FILEATTR &OPTION)' ,
 CHECKOPT=>'*CLLE'))

Note my UDTF parameter values such as *CLLE, *PDM, etc. are case insensitive.

[bobcozzi]

Your 2nd question about when to check---yes I agree that it should be when a statement is entered.
However I do hear developer saying they dislike (as I do as well) syntax checking when I'm still writing the statement. So maybe if we do a check on Enter or when we use the mouse or cursor keys to leave the current line? Checking it every 1.5 seconds seems overkill for CL commands. I suppose we could also do it on a user idle-wait then go ahead and check it. But we don't want to annoy the very folks who requested it by telling them the CL command they are typing in has syntax errors while they are still typing it--IBM CL developers don't work that way.

[SanjulaGanepola]

So to get CL Program for ILE CL

@bobcozzi Even with CHECKOPT set to *CLLE, I see that it still fails in the cases I mentioned earlier. I have included the examples again below:

From my testing, while this worked:

SELECT * FROM TABLE(
    CL_SYNTAX_CHECK(cmd=>'CHECKFILE:  PGM PARM(&FILE &FILEATTR &OPTION)' , CHECKOPT=>'*CLLE'));

this kinda thing fails:

SELECT * FROM TABLE(
    CL_SYNTAX_CHECK(cmd=>'CRTORD     CUID(&CUID)' , CHECKOPT=>'*CLLE'));

same for stuff like:

SELECT * FROM TABLE(
    CL_SYNTAX_CHECK(cmd=>'IF (&YYYY2 *LT 40) THEN(DO)' , CHECKOPT=>'*CLLE'));

I am I using the UDTF incorrectly?

Checking it every 1.5 seconds seems overkill for CL commands. I suppose we could also do it on a user idle-wait then go ahead and check it.

I probably didn't explain this clearly, but it doesn't actually check every 1.5 seconds constantly. Rather there needs to be a file change which starts a timer. Basically whenever a change is made, it starts a 1.5 second timer. If it hits the end the timer, it will run the syntax checker. If a change is made before that, it will restart the timer. This is to ensure it's not constantly being run as the user is typing. We could increase this to like 3 seconds if you think that is better. Users can also disable this automatic checking all together if they prefer (using the settings below). What do you think?

[bobcozzi]

I’m out of the country right now be back next week. Don’t have any computer stuff with me to check this out, so I’ll look at it next week thanks.Sent from my iPhoneOn Oct 7, 2025, at 7:06 PM, Sanjula Ganepola ***@***.***> wrote:SanjulaGanepola left a comment (IBM/vscode-clle#35) So to get CL Program for ILE CL @bobcozzi Even with CHECKOPT set to *CLLE, I see that it still fails in the cases I mentioned earlier. I have included the examples again below: From my testing, while this worked: SELECT * FROM TABLE( CL_SYNTAX_CHECK(cmd=>'CHECKFILE: PGM PARM(&FILE &FILEATTR &OPTION)' , CHECKOPT=>'*CLLE')); this kinda thing fails: SELECT * FROM TABLE( CL_SYNTAX_CHECK(cmd=>'CRTORD CUID(&CUID)' , CHECKOPT=>'*CLLE')); same for stuff like: SELECT * FROM TABLE( CL_SYNTAX_CHECK(cmd=>'IF (&YYYY2 *LT 40) THEN(DO)' , CHECKOPT=>'*CLLE')); I am I using the UDTF incorrectly? Checking it every 1.5 seconds seems overkill for CL commands. I suppose we could also do it on a user idle-wait then go ahead and check it. I probably didn't explain this clearly, but it doesn't actually check every 1.5 seconds constantly. Rather there needs to be a file change which starts a timer. Basically whenever a change is made, it starts a 1.5 second timer. If it hits the end the timer, it will run the the syntax checker. If a change is made before that, it will restart the timer. This is to ensure its not constantly being run as the user is typing. We could increase this to like 3 seconds if you think that is better. Users can also disable this automatic checking all together if they prefer: image.png (view on web) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[bobcozzi]

I was able to reproduce the issue on my system. I will look into it.

[bobcozzi]

The issue was that the documentation was unclear on the actual purpose of option 0 and 1. The new update will default to Option 3 which is Command Entry syntax checking. The following options will be available for the CHECKOPT parameter (upper/lower case is ignored):

• 'RUN' = The default Check a Cl command to be run on Command Entry or similar environment. ('CL' is an alias)
• 'CLLE' = ILE CL Programs (source type CLLE)
• 'CLP' = OPM CL Programs (source type CLP)
• 'CMD' = CL Command Definition source (i.e., commands CMD, PARM, QUAL, ELEM, DEP, PMTCTL)
• 'BND' = Binder Language commands
• 'PDM' = Commands with PDM &N style options embedded in them.

I'll post this on Monday or Tuesday/

[bobcozzi]

I have updated the code to use the appropriate syntax check environment option (see above msg) and have just now pushed the PR update. Hopefully I did the git push correctly.

[SanjulaGanepola]

@bobcozzi Thanks for making those changes. I gave it a try and it is work as expected now which is great! We are very close to getting these changes in.

Did you by chance take a look at my clarification regarding the automatic syntax checking. If not, I have included the details again below:

I probably didn't explain this clearly, but it doesn't actually check every 1.5 seconds constantly. Rather there needs to be a file change which starts a timer. Basically whenever a change is made, it starts a 1.5 second timer. If it hits the end the timer, it will run the syntax checker. If a change is made before that, it will restart the timer. This is to ensure it's not constantly being run as the user is typing. Users can also disable this automatic checking all together if they prefer (using the settings below) or adjust this time interval. What do you think?
image

A couple additional changes I made today:

• I increased the default check interval to 3 seconds
• The progress bar at the bottom will also now indicate the number of commands it is checking:
  

This will probably all make more sense if you give this a try. I went ahead and pushed all my changes (as described throughout this PR) directly to this same branch. Can you do a git pull on your machine and then debug the extension to give it a try? Let me know if you find any issues. If all looks good in your testing, I can go ahead and merge + create a new release.

[bobcozzi]

I can do this tomorrow morning. Not sure why we need a progress bar but I'll see what it looks like--unless it is only appearing on the "Syntax Check On Open" option, then I get it.

[SanjulaGanepola]

Not sure why we need a progress bar but I'll see what it looks like--unless it is only appearing on the "Syntax Check On Open" option, then I get it.

It would appear on both Check on Edit and Check on Open (basically anytime the syntax checker is being run (we do the same in the database extension). It is useful for Check on Edit if you are making edits across multiple lines and they span different commands in which case the checker is run on each command individually. It is also useful if you use the button at the top of the editor to check the entire document.

[bobcozzi]

I had to update some parameters that were changed on the update() and one other function.
Also when I used it, I typed in a CL command, pressed Enter. Then sat there until the timer evoked the syntax checker. Were you going to add it to the Enter key action as well? Just wondering. Those few seconds seem like centuries. :)

[SanjulaGanepola]

I just pushed a commit to immediately run the syntax checker if the enter key is pressed. Can you give this a try?

As for when the user is typing in general, what do you think about lowering the default check interval back to 1500 (or lower?)? It feels a bit more responsive, but I'm open to keeping it at 3000 if you think that is better.

[bobcozzi]

I've got a family obligation today, suddenly, so I'll test your addition Enter key this evening when I get home. If that's too late, I'm sure it is fine. (I assume you applied my changes)

[bobcozzi]

I pulled down everything I believe I should have and do not see the Enter key processing.
Also the few small changes I made were not there for parameter/syntax issues--but I was able to just add a hand-ful of ? and it worked like before (after xx ms it does the syntax checker.)
I assume your Enter key was COMMITed? But it didn't come down with my PR pull.
Let me know if I should try that again.

[SanjulaGanepola]

@bobcozzi This branch should have the enter key changes as well as the changes you made as well.

The specific changes for the enter key can be seen here: b4c4883

Can you double check that you are on the cl-syntax-checker branch and not the cl-syntax-checker-ibm-latest branch which I see you also created. That one is 4 commits behind:

[bobcozzi]

When I press Enter, syntax checking doesn't seems to react to the Enter key. The timer still does however.
If I press Enter on another CL command above the one with the error being displayed, the message seems to bounce up to the prior CL command. e.g., if the error is online 12, and I move up to line 10 and go to the end of line, and press Enter, the error now moves up to line 11 and seems to stick there until I "Enter" a few more times above it and then it might move up another line. So those errors aren't sticking with the CL command they apply to.

[UPDATE] I updated the logic and now the sequence is that the message follows the original statement, however once the 1500 ms timer fires, all messages are cleared (as designed), but it only checks the current line.
I will work on a more user-friendly method. If I can get it to work I will push those changes otherwise I will post another command indicating my failure to do so. :)

[bobcozzi]

Okay, so while I managed to get the syntax errors to be more efficient in their use of the CL Syntax Checker (not calling it for all statements when one is changed or a new line is inserted) VS Code's desire to keep a Diagnostic message assigned to a specific line number ("hard coded") means that when a line is inserted or deleted above any existing Diagnostic messages (if they are not removed) are not shifted with the code itself. So they appear to be on the "wrong" line.
It seems VS Code is designed for syntax checking the entire source file on virtually every keystroke--very cool for in-memory synta checkers written in TS. But for remote calls to the IBM i host -- yikes!

Lastly, I'm wondering why you're not using the VS CODE "eol" identifier instead of '\r\n' and similar?

return document.eol === EndOfLine.CRLF ? '\r\n' : '\n';

[bobcozzi]

Perhaps we need to only syntax check the current statement and leave it up until the next one. That's kind of how you had it working. What do you think???

[SanjulaGanepola]

Looking into this now

[SanjulaGanepola]

When I press Enter, syntax checking doesn't seems to react to the Enter key. The timer still does however.

This is still odd as this logic here should handle that. Can you try debugging this logic on your end and see if it properly detects if there is an enter and then validates right away?

VS Code's desire to keep a Diagnostic message assigned to a specific line number ("hard coded") means that when a line is inserted or deleted above any existing Diagnostic messages (if they are not removed) are not shifted with the code itself. So they appear to be on the "wrong" line.

Hmm I hadn't noticed this earlier as I always resolved the error before adding or removing a line above. This is an interesting problem since diagnostics are just tied to ranges in the document like you said. I gave this a try with the SQL syntax checker and looks like it also has this issue.

@worksofliam Any suggestions here?

Lastly, I'm wondering why you're not using the VS CODE "eol" identifier instead of '\r\n' and similar?

Good point. I updated this.

[bobcozzi]

That looks correct (the EOL patcth)

[bobcozzi]

I think the only thing we can do with CL short of porting the CL syntax checker to TypeScript (LOL) is to only check the current statement on either a timer or a keydown that is a character key (i.e., X, Y Z) and not an arrow key. Then when the next line is syntax checked, it currently clears that old line's diagnostics anyway. So if the error isn't cleared it will vanish until they get to it again. Then we can add in that "Check All" option, which unfortunately unless I update the UDTF will be a lengthy process. What do you think?

[bobcozzi]

I've experimented with the CL Syntax Checker UDTF and found that it's not practical to modify it to accept multiple source lines. Since CL statements can span multiple lines, passing an array of "lines" would require the UDTF to re-assemble each CL command from potentially fragmented lines. While associating errors with the primary line of a command is straightforward, reliably reconstructing commands from lines is complex and error-prone.

Current Limitations

• The UDTF is best suited to checking one CL command at a time.
• Returns errors for the specific CL command disregarding which is already re-assembled by the extension and passed to the UDTF.

Recommended Approach

• Syntax check only the current CL statement (the one being edited)
• Use a keyboard pause-timer to trigger syntax checking:
  • Listen for onDidChangeTextDocument events to detect typing.
  • Reset/clear this idle timer on each onDidChangeTextDocument event.
  • If the timer elapses (e.g., after 1500ms of inactivity), call the syntax checker for the current statement.
  • if Enter is pressed, or an Arrow key or mouse click moves the cursor off "this" line, then call the syntax checker for the statement, then process the key's actions.

This approach checks one statement at a time, triggering syntax checking on a keyboard pause or the Enter key, and balances performance and functionality for CL development.

• User types → timer resets
• User stops typing → 1.5 seconds elapse → syntax checker runs for current command
• User presses Enter → syntax checker runs immediately, then new line is inserted

---

Summary

• Natural editing experience: Diagnostics appear after a short pause or command completion, not while parameters are still being entered.
• Reduced false positives: Avoids showing incomplete or misleading errors while typing.
• Optimized performance: Fewer host calls per statement.
• Minimized diagnostic drift: Since syntax checks apply only to the active command, line insertions above it have minimal impact.

---

Optional Enhancements

• Configurable Timer Delay
• Allow developers to adjust the syntax check interval (e.g., clle.syntaxCheckDelayMs) in settings.

---

Summary

This approach delivers a practical, efficient, and developer-friendly syntax-checking experience for CL development in VS Code.
By validating one CL statement at a time — triggered by user activity rather than every keystroke — it:

• Avoids complex multi-line parsing on the host,
• Works within VS Code’s diagnostic model, and
• Provides a balanced mix of performance and usability familiar to IBM i developers.

[worksofliam]

VS Code's desire to keep a Diagnostic message assigned to a specific line number ("hard coded") means that when a line is inserted or deleted above any existing Diagnostic messages (if they are not removed) are not shifted with the code itself. So they appear to be on the "wrong" line.

Hmm I hadn't noticed this earlier as I always resolved the error before adding or removing a line above. This is an interesting problem since diagnostics are just tied to ranges in the document like you said. I gave this a try with the SQL syntax checker and looks like it also has this issue.

@worksofliam Any suggestions here?

@SanjulaGanepola In the RPGLE extension, we have a feature where the diagnostic will clear when the range is edited: https://github.com/codefori/vscode-ibmi/blob/473c34ab435e26c70f4392231f2df84eb4a5edc5/src/ui/diagnostics.ts#L29-L42

Perhaps the CL and SQL extensions could follow this same configuration setting?

[SanjulaGanepola]

@worksofliam Yes I do remember that, but the issue here is actually when changes are made outside the range (in particular above it). If a line above is removed or added, the diagnostics no longer apply to the correct line.

[worksofliam]

@SanjulaGanepola I realise that now. But, it might be good if we wrote standard logic to be able to move diagnostics in accordance to the user edit that we could use across all the extensions.

[bobcozzi]

Yes, the Diagnostics object has a range associated with it.
If an insert or delete "line" occurs then any existing Diagnostics object would need to be scanned for its starting line at or exceeding where the insert/delete line occurred and if so, update its range. The Diag message should then move to the correct location. I see this kind of happening in typescript editing if you get a syntax error online 14 and move the cursor up to line 12, and press Enter, you'll momentarily see the error drift up to relative line 14 and then jump back to the original line as the syntax-checker catches up--which places it on what is now line 15.

The code below may doe this for the CL syntax checker, if you'd like to integrate it into it.

import * as vscode from 'vscode';

// Assume this is our existing DiagnosticCollection
const diagnosticCollection = vscode.languages.createDiagnosticCollection('clle');

// Example function to handle document changes
function handleDocumentChange(event: vscode.TextDocumentChangeEvent) {
    const doc = event.document;
    const changes = event.contentChanges;

    // Iterate over all changes in this event
    changes.forEach(change => {
        const changeStartLine = change.range.start.line;
        const changeEndLine = change.range.end.line;
        const insertedLineCount = change.text.split('\n').length - 1;
        const deletedLineCount = changeEndLine - changeStartLine;

        // Calculate net line shift
        const netLineShift = insertedLineCount - deletedLineCount;

        if (netLineShift === 0) return; // Nothing to adjust

        // Get current diagnostics for this document
        const diagnostics = diagnosticCollection.get(doc.uri) || [];

        // Update diagnostics ranges
        const updatedDiagnostics = diagnostics.map(diag => {
            const startLine = diag.range.start.line;
            const endLine = diag.range.end.line;

            if (startLine > changeEndLine) {
                // Shift diagnostic lines downward or upward
                const newStart = startLine + netLineShift;
                const newEnd = endLine + netLineShift;

                return new vscode.Diagnostic(
                    new vscode.Range(newStart, diag.range.start.character, newEnd, diag.range.end.character),
                    diag.message,
                    diag.severity
                );
            }
            // Diagnostics above the change stay the same
            return diag;
        });

        // Update the diagnostic collection
        diagnosticCollection.set(doc.uri, updatedDiagnostics);
    });
}

// Example: Listen for document changes
vscode.workspace.onDidChangeTextDocument(handleDocumentChange);