Skip to content

Commit

Permalink
Doc: Document usage of dynamic log links
Browse files Browse the repository at this point in the history
Signed-off-by: Fabio Grätz <[email protected]>
  • Loading branch information
Fabio Grätz committed Feb 27, 2025
1 parent 5fe7a73 commit df11707
Showing 1 changed file with 89 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,95 @@ task_logs:
- "https://..."
```

### Configure dynamic log links

Dynamic log links are links which are 1. not shown by default for all tasks and 2. which can use template variables provided during task registration.


Configure dynamic log links in the flytepropeller configuration the following way:

```yaml
configmap:
task_logs:
plugins:
logs:
dynamic-log-links:
- log_link_a: # Name of the dynamic log link
displayName: Custom dynamic log link A
templateUris: 'https://some-service.com/{{ .taskConfig.custom_param }}'
```

In `flytekit`, dynamic log links are activated and configured using a so-called `ClassDecorator`.
You can define such a custom decorator for controlling dynamic log links for instance as follows:

```py
from flytekit.core.utils import ClassDecorator
class configure_log_links(ClassDecorator):
"""
Task function decorator to configure dynamic log links.
"""
def __init__(
self,
task_function: Optional[Callable] = None,
enable_log_link_a: Optional[bool] = False,
custom_param: Optional[str] = None,
**kwargs,
):
"""
Configure dynamic log links for a task.
Args:
task_function (function, optional): The user function to be decorated. If the decorator is called
with arguments, task_function will be None. If the decorator is called without arguments,
task_function will be function to be decorated.
enable_log_link_a (bool, optional): Activate dynamic log link `log_link_a` configured in the backend.
custom_param (str, optional): Custom parameter for log link templates configured in the backend.
"""
self.enable_log_link_a = enable_log_link_a
self.custom_param = custom_param
super().__init__(
task_function,
enable_log_link_a=enable_log_link_a,
custom_param=custom_param,
**kwargs,
)
def execute(self, *args, **kwargs):
output = self.task_function(*args, **kwargs)
return output
def get_extra_config(self) -> dict[str, str]:
"""Return extra config for dynamic log links."""
extra_config = {}

log_link_types = []
if self.enable_log_link_a:
log_link_types.append("log_link_a")

if self.custom_param:
extra_config["custom_param"] = self.custom_param
# Activate other dynamic log links as needed

extra_config[self.LINK_TYPE_KEY] = ",".join(log_link_types)
return extra_config


@task
@configure_log_links(
enable_log_link_a=True,
custom_param="test-value",
)
def my_task():
...

```
For inspiration, consider how the flytekit [wandb](https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-wandb/flytekitplugins/wandb/tracking.py), [neptune](https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-neptune/flytekitplugins/neptune/tracking.py) or [vscode](https://github.com/flyteorg/flytekit/blob/master/flytekit/interactive/vscode_lib/decorator.py) plugins make use of dynamic log links.
## Datadog integration
To send your Flyte workflow logs to Datadog, you can follow these steps:
Expand Down

0 comments on commit df11707

Please sign in to comment.