Skip to content

Commit 5e15044

Browse files
authored
Add documentation for how plugin CLI commands can be made lazy loaded (#4027)
* Add docs for making plugins lazy loaded Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Suggestions from reviews: Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Small changes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
1 parent e2b20a4 commit 5e15044

1 file changed

Lines changed: 48 additions & 3 deletions

File tree

‎docs/source/extend_kedro/plugins.md‎

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ def commands():
2222
pass
2323

2424

25-
@commands.command()
25+
@commands.command(name="to_json")
2626
@click.pass_obj
2727
def to_json(metadata):
2828
"""Display the pipeline in JSON format"""
@@ -48,7 +48,6 @@ Commands must be provided as [`click` `Groups`](https://click.palletsprojects.co
4848

4949
The `click Group` will be merged into the main CLI Group. In the process, the options on the group are lost, as is any processing that was done as part of its callback function.
5050

51-
5251
## Project context
5352

5453
When they run, plugins may request information about the current project by creating a session and loading its context:
@@ -78,7 +77,53 @@ Plugins may also add commands to the Kedro CLI, which supports two types of comm
7877

7978
## Suggested command convention
8079

81-
We use the following command convention: `kedro <plugin-name> <command>`, with `kedro <plugin-name>` acting as a top-level command group. This is our suggested way of structuring your plugin bit it is not necessary for your plugin to work.
80+
We use the following command convention: `kedro <plugin-name> <command>`, with `kedro <plugin-name>` acting as a top-level command group. This is our suggested way of structuring your plugin but it is not necessary for your plugin to work.
81+
82+
## Advanced: Lazy loading of plugin commands
83+
84+
If you are developing a plugin with a large set of CLI commands or with certain large libraries that are slow to import but are used by a small subset of commands, consider using lazy loading of these commands. This can significantly improve the performance of the plugin as well as the overall performance of Kedro CLI. To do this, you can follow [the instructions on the
85+
`click` documentation on lazy loading of commands](https://click.palletsprojects.com/en/8.1.x/complex/#lazily-loading-subcommands). From Kedro 0.19.7, the [Kedro commands are declared as lazy loaded command groups](https://github.com/kedro-org/kedro/blob/main/kedro/framework/cli/cli.py) that you can use as a reference for the implementation.
86+
87+
Consider the previous example of the `kedrojson` plugin. Suppose the plugin has two commands, `kedro to_json pipelines` and `kedro to_json nodes`. The `to_json pipelines` command is used more frequently than the `to_json nodes` command and the `to_json nodes` command requires a large library to be imported. In this case, you can define your commands to be lazily loaded with delayed imports as follows:
88+
89+
In `kedrojson/plugin.py`:
90+
```python
91+
import click
92+
from kedro.framework.project import pipelines
93+
from kedro.framework.cli.utils import LazyGroup
94+
95+
@click.group()
96+
def commands():
97+
pass
98+
99+
@commands.group(
100+
name="to_json",
101+
cls=LazyGroup,
102+
lazy_subcommands={
103+
"nodes": "kedrojson.plugin.nodes",
104+
"pipelines": "kedrojson.plugin.pipelines"
105+
}
106+
)
107+
def to_json():
108+
"""Convert Kedro nodes and pipelines to JSON"""
109+
pass
110+
111+
@click.command(name="nodes")
112+
def nodes():
113+
"""Convert Kedro nodes to JSON"""
114+
import some_large_library
115+
print("Converting nodes to JSON")
116+
...
117+
118+
@click.command("pipelines")
119+
def pipelines():
120+
"""Convert Kedro pipelines to JSON"""
121+
print("Converting pipelines to JSON")
122+
...
123+
```
124+
125+
The loading of the individual `nodes` and `pipelines` commands and subsequently the imports of the large libraries will be delayed until the respective commands are called.
126+
82127

83128
## Hooks
84129

0 commit comments

Comments
 (0)