Grit TypeScript API

[morgante]

I've heard many people are interested in using Grit from other applications, but without needing to learn the GritQL syntax.

With that in mind, I'm working on a TypeScript SDK which would allow you to interact with GritQL directly from TypeScript. I'd love early feedback on the API.

Top-level API

The main find method is used to execute a GritQL query. It returns a promise which can be awaited to get the results.

To find matches inside a directory, you should the find($pattern, $paths) method. $pattern is any valid Grit query, while $paths is an array of strings representing the paths to search in.

Query Builder

Queries are built by composing patterns together. All patterns are imported from a language-specific import that determines the target language.

The most basic pattern is a simple snippet pattern which can be used to match equivalent snippets of code in the target language. Snippets are created using a js.code template string, like this:

import { find, js } from "@getgrit/ql";

const results = await find(js.code`console.log($msg)`, ["src"]);

AST Nodes

Use js.ast.$node to match a specific (named) AST node. An object can be passed to filter named fields of the node.

For example, this pattern will find any variable assigned the value of 42:

const pattern = js.ast.augmented_assignment_expression({
  right: js.code`42`,
});

Variables

Use js.var.$name to create a metavariable named $name. This is particularly useful for binding variables to be used in other patterns.
For example, this can be combined with the AST node pattern to capture the name of the variable being assigned:

const pattern = js.ast.augmented_assignment_expression({
  left: js.var.$foo,
  right: js.code`42`,
});

Constraints

Constraints can be added to patterns by calling additional methods on the pattern.

Contains

You can use contains to require that a pattern contains another pattern. For example, to find a function that contains a variable assignment:

const pattern = js.code`function $_() { $body }`.contains(
  js.ast.augmented_assignment_expression()
);

Within

You can use within to require that a pattern is within another pattern. For example, to find a function that is within a class:

const pattern = js.code`function $_() { $body }`.within(
  js.ast.class_declaration()
);

Filter

The filter method can be used to provide a callback which will be called with each match. If the callback returns false, the match will be discarded.

For example, this function will only match functions that mention the word "myFunction":

const pattern = js.code`function $_() { $body }`.filter((match) => {
  return match.text().includes("myFunction");
});

The match argument to the callback is an object that represents the match. You can call additional arguments on the match to get more information about the match and search for more patterns.

For example, this pattern will find all functions that contain a variable assignment:

const pattern = js.code`function $_() { $body }`.filter((match) => {
  return match.contains(js.ast.augmented_assignment_expression());
});

The second argument to the callback is a context argument that represents the context in which the match was found. This is primarily useful for retrieving variable values.

For example, this can be match all functions that have a name starting with foo:

const pattern = js.code`function $name() { $body }`.filter((match, context) => {
  return context.var.$name.startsWith("foo");
});

Each

The each method can be used to provide a callback which will be called with each match. The callback function has the same signature as the filter method, but the return value is ignored.

You can use this to gather information about a program. For example, this will build a list of all classes in a program:

const classes = [];

const results = await find(
  js.ast.class_declaration({ name: js.var.$name }).each((match, context) => {
    classes.push(context.var.$name);
  }),
  ["src"]
);

Rewrites

The rewrite method can be used to rewrite any matching pattern. You should provide a replacement template string to the rewrite method.

For example, this will replace all instances of console.log with console.error:

const pattern = js.code`console.log($msg)`.rewrite(
  js.code`console.error($msg)`
);
await find(pattern, ["src"]);

You can also replace matches inside each or filter methods. For example, this will replace all instances of console.log with console.error only if the message contains the word "error":

const pattern = js.code`console.log($msg)`.each((match) => {
  if (match.text().includes("error")) {
    match.rewrite(js.code`console.error($msg)`);
  }
});

[arendjr]

I think that syntax for composing patterns is really intuitive, well done!

For Biome specifically, I think we wouldn’t use the find() API since we would like to drive the filesystem traversal instead of letting the plugins do that. But the other stuff looks very much reusable.

I have a slight concern regarding the callbacks with filter() and each(). It could end up with a lot of roundtrips between Rust and JS. That said, it seems very much a “you pay for what you use” kind of situation, so the overhead might be totally acceptable.

Are you planning on allowing an escape hatch that allows people to parse a GritQL pattern and use it as a (sub)pattern within this API?

[morgante]

I have a slight concern regarding the callbacks with filter() and each(). It could end up with a lot of roundtrips between Rust and JS. That said, it seems very much a “you pay for what you use” kind of situation, so the overhead might be totally acceptable.

Yeah, I had similar concerns (and it's definitely less performant than pure patterns) but if you do filtering up front I found performance to be acceptable even on large repos.

If it becomes an issue, I think I could also add a variant of where to allow additional Rust-only filtering:

const pattern = js.code`function $_() { $body }`.where(js.var.$body.contains(js.ast.augmented_assignment_expression())

For Biome specifically, I think we wouldn’t use the find() API since we would like to drive the filesystem traversal instead of letting the plugins do that. But the other stuff looks very much reusable.

Yeah, the find function is mainly for use in migrations/workflows. I expect plugins could just export a pattern and Biome handles searching for it.

[psalv]

I've used a number of AST tools (like eslint and ts-morph) but this syntax feels the most intuitive of any I've seen:

const pattern = js.code`function $_() { $body }`.filter((match) => {
  return match.contains(js.ast.augmented_assignment_expression());
});

Things I really like about this:

• The potential to create reusable fragments.
• There are certain classes of problem that are pretty difficult to write as eslint rules that I think could become fairly easy with this syntax which opens the door to a ton of new guardrails.
• This feels very testable.

each is also a really interesting, doing complex enumerations can be a pain and this seems to solve that problem.

Something I would love to use this for is a combination of deterministic + non-deterministic code generation, being able to identify the exact code I want to change across a large surface area, but then using something like GPT to adapt only that specific code to a new pattern feels like it would work really well with the rewrite syntax (albeit a little slow, but that's not a huge concern for me):

const pattern = js.code`console.log($msg)`.each((match) => {
  if (match.text().includes("error")) {
    match.rewrite(js.code`console.error($msg)`);
  }
});

Overall I think this is great and solves several problems that today do not have great solutions!

[arendjr]

Nit: Maybe it makes sense to make match.text() a read-only property instead of a function? Might make the API feel a bit more idiomatic.