Skip to content

IBX-9621: Document AI / Connect integration #2690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 9, 2025
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 25 additions & 6 deletions docs/ai_actions/ai_actions_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@
AI Actions is an extensible solution for integrating features provided by AI services into your workflows, all managed through a user-friendly interface.

Out-of-the-box, AI Actions solution includes two essential components: a framework package and an OpenAI connector package.
It comes pre-configured with the following action types:

As of version v4.6.19, AI Actions can integrate with [[[= product_name_connect =]]](https://doc.ibexa.co/projects/connect/en/latest/general/ibexa_connect/), to give you an opportunity to build complex data transformation workflows without having to rely on custom code.

Check failure on line 16 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L16

[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'
Raw output
{"message": "[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 16, "column": 15}}}, "severity": "ERROR"}
From the developer's perspective, the integration removes the burden of maintaining third-party AI handlers, and accelerates the deployment of AI-based solutions.

AI Actions solution comes pre-configured with the following action types:

- [Refine text](#refining-text): Rewrite existing text according to instructions set in a prompt
- [Generate alternative text](#generating-alternative-text): Generate alt text for images for accessibility purposes
Expand All @@ -26,20 +30,28 @@
## Availability

AI Actions are an opt-in capability available as an [LTS update](editions.md#lts-updates) starting with the v4.6.12 version of [[= product_name =]], regardless of its edition.
[[= product_name_connect =]] integration is available starting with v4.6.19.

Check failure on line 33 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L33

[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'
Raw output
{"message": "[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 33, "column": 69}}}, "severity": "ERROR"}
To begin using AI Actions, you must first [install the required packages and perform initial configuration](install_ai_actions.md).

!!! note "API Key"
### Prerequisites

The OpenAI connector requires that you first [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).

Integration with [[= product_name_connect =]] requires that you first [get the credentials](https://doc.ibexa.co/projects/connect/en/latest/general/ibexa_connect/#access-ibexa-connect) to your account, and the [API token](install_ai_actions.md#token).

The OpenAI connector requires that you first [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).
!!! note "[[= product_name_connect =]] Availability"

[[= product_name_connect =]] comes with all contracts signed from 2023.
If you signed your contract earlier, contact your customer success manager to use [[= product_name_connect =]].

## How it works

AI Actions LTS update relies on an extensible AI framework, which is responsible for gathering information from various sources, such as AI action types, AI action configurations, and contextual details like SiteAccess, user details, locale settings, and more.
This data can then be combined with user input.
It's then passed to a service connector, such as the default OpenAI connector, for final processing on [[= product_name =]] side.
The service connector wraps all data into a prompt or another suitable format and sends it to an external AI service.
It's then passed to a service connector, such as the default OpenAI connector or the [[= product_name_connect =]] connector, for final processing on [[= product_name =]] side.
The service connector wraps all data into a prompt or another suitable format and sends it to an external service.

When the AI Service returns a response, the response goes back through the service connector and passes to the framework.
When the external service returns a response, the response goes back through the service connector and passes to the framework.
It can then be presented to the user in any way necessary.

### Core concepts
Expand Down Expand Up @@ -147,3 +159,10 @@
![Alt text generation](img/alt_text_use_ai.png)

With some customization, administrators could use the API to run a batch process against a larger collection of illustrations.

### Performing advanced image to text analysis

With some additional customization, store managers could benefit from automating part of product management by integrating their [[= product_name =]] with Google Cloud Vision and [PIM](pim_guide.md) by using [[= product_name_connect =]].
Instead of manually selecting and linking images stored in a [DAM](add_image_asset_from_dam.md) solution to their products, they could use of a no-code workflow where an AI service, for example, Google Cloud Vision, extracts text and attributes from product images, which are then matched with existing items in a product catalog.

Check warning on line 166 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L166

[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.
Raw output
{"message": "[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 166, "column": 96}}}, "severity": "WARNING"}

This would enable automatic product identification, tagging, and catalog updates, resulting in less manual work and more efficient product management.

Check warning on line 168 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L168

[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.
Raw output
{"message": "[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 168, "column": 1}}}, "severity": "WARNING"}
Binary file added docs/ai_actions/img/connect_api_token.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
64 changes: 63 additions & 1 deletion docs/ai_actions/install_ai_actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,72 @@

```bash
###> ibexa/connector-openai ###
OPENAI_API_KEY=sk-svcacct-AFCrCt1h2s3i4s5i6s7t8h9e0a1p2i3c4o5d6e
OPENAI_API_KEY=<your_api_key>
###< ibexa/connector-openai ###
```

### Configure access to [[= product_name_connect =]]

First, get the credentials by contacting [Ibexa Support](https://support.ibexa.co).

#### Create team

In [[= product_name_connect =]], set up the account, and [create a team](https://doc.ibexa.co/projects/connect/en/latest/access_management/teams/#creating-teams).
Navigate to the team details page and note down the numerical value of the **Team id** variable.

Creating a team matters, because [scenarios](https://doc.ibexa.co/projects/connect/en/latest/scenarios/creating_a_scenario/) that process data coming from your AI action are associated with a team.
This way, if your organization has more than one application, each application can be linked to a different team and so can be scenarios coming from those applications.

Check warning on line 49 in docs/ai_actions/install_ai_actions.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/install_ai_actions.md#L49

[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.
Raw output
{"message": "[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.", "location": {"path": "docs/ai_actions/install_ai_actions.md", "range": {"start": {"line": 49, "column": 1}}}, "severity": "WARNING"}

If specific users from the team are supposed to modify scenario settings, you must [assign the right roles](https://doc.ibexa.co/projects/connect/en/latest/access_management/teams/#managing-teams) to them.

#### Create token

Navigate to your [[= product_name_connect =]] user's profile, and on the **API ACCESS** tab, create a new token.
Copy the token code that appears next to the label.

![Creating an API token](img/connect_api_token.png)

#### Set up credentials

In the root folder of your project, modify the `.env` file.
Replace a placeholder value of the `IBEXA_CONNECT_TOKEN` variable with the token that you got from [[= product_name_connect =]] and provide a value of the `IBEXA_CONNECT_TEAM_ID` variable.

```bash
###> ibexa/connect ###
IBEXA_CONNECT_HOST=https://connect.ibexa.co
IBEXA_CONNECT_API_PATH=/api/v2/
# Token can be created in the user's profile in Ibexa Connect, under the 'API ACCESS' section.
IBEXA_CONNECT_TOKEN=<your_api_token>
# Use the URL below to read more on Ibexa Connect teams.
# https://doc.ibexa.co/projects/connect/en/latest/access_management/teams/
IBEXA_CONNECT_TEAM_ID=2
###< ibexa/connect ###
```

#### Initiate integration

Initiate the models provided by the handler by issuing the following command:

```bash
php bin/console ibexa:connect:init-connect-ai <team_id> <language> <model list>
```

For example:

```bash
php bin/console ibexa:connect:init-connect-ai 2 en connect-image-to-text connect-text-to-text
```

!!! note "Support for multiple [[= product_name_connect =]] languages"

The [`language` attribute](https://developers.make.com/api-documentation/api-reference/templates#templates-1) determines the language in which template details such as module names will be displayed in [[= product_name_connect =]]'s UI.

#### Customize templates

Check warning on line 95 in docs/ai_actions/install_ai_actions.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/install_ai_actions.md#L95

[Ibexa.EOLWhitespace] Remove whitespace characters from the end of the line.
Raw output
{"message": "[Ibexa.EOLWhitespace] Remove whitespace characters from the end of the line.", "location": {"path": "docs/ai_actions/install_ai_actions.md", "range": {"start": {"line": 95, "column": 25}}}, "severity": "WARNING"}

Return to the [[= product_name_connect =]] dashboard and modify the **Template for connect...handler** [templates](https://doc.ibexa.co/projects/connect/en/latest/scenarios/scenario_templates/) by defining the logic needed to process the data.

Once the templates are ready, you can build scenarios from them, either directly in [[= product_name_connect =]] or in [[[= product_name =]]'s user interface]([[= user_doc =]]/ai_actions/work_with_ai_actions/#create-new-ai-actions).

### Modify the database schema

Create the `add_ai_actions.sql` file that contains the following code.
Expand Down
2 changes: 1 addition & 1 deletion docs/content_management/images/add_image_asset_from_dam.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The configuration for each connector depends on the requirements of the specific

You can create your own connectors, or use the provided example DAM connector for [Unsplash](https://unsplash.com/).

To add the Unsplash connector to your system add the `ibexa/connector-unsplash` bundle to your installation.
To add the Unsplash connector to your system, add the `ibexa/connector-unsplash` bundle to your installation.

## Add Image Asset in Page Builder [[% include 'snippets/experience_badge.md' %]] [[% include 'snippets/commerce_badge.md' %]]

Expand Down