Skip to content

IBX-9621: Document AI / Connect integration #2690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/administration/project_organization/bundles.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ To remove a bundle (either one you created yourself, or an out-of-the-box one th
|---------|-----------|
|ibexa/oss|Core packages|
|ibexa/content-tree|content tree functionality|
|ibexa/connect|[[[= product_name_connect =]]](https://doc.ibexa.co/projects/connect/en/latest/)|
|ibexa/connect|[[[= product_name_connect =]]]([[= connect_doc =]]/)|
|ibexa/calendar|Calendar tab with a calendar widget|
|ibexa/connector-dam|Connector for DAM (Digital Asset Management) systems|
|ibexa/elasticsearch|Integration with Elasticsearch search engine|
Expand Down
31 changes: 25 additions & 6 deletions docs/ai_actions/ai_actions_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@
AI Actions is an extensible solution for integrating features provided by AI services into your workflows, all managed through a user-friendly interface.

Out-of-the-box, AI Actions solution includes two essential components: a framework package and an OpenAI connector package.
It comes pre-configured with the following action types:

As of version v4.6.19, AI Actions can integrate with [[[= product_name_connect =]]]([[= connect_doc =]]/general/ibexa_connect/), to give you an opportunity to build complex data transformation workflows without having to rely on custom code.

Check failure on line 16 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L16

[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'
Raw output
{"message": "[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 16, "column": 15}}}, "severity": "ERROR"}
From the developer's perspective, the integration removes the burden of maintaining third-party AI handlers, and accelerates the deployment of AI-based solutions.

AI Actions solution comes pre-configured with the following action types:

- [Refine text](#refining-text): Rewrite existing text according to instructions set in a prompt
- [Generate alternative text](#generating-alternative-text): Generate alt text for images for accessibility purposes
Expand All @@ -26,20 +30,28 @@
## Availability

AI Actions are an opt-in capability available as an [LTS update](editions.md#lts-updates) starting with the v4.6.12 version of [[= product_name =]], regardless of its edition.
[[= product_name_connect =]] integration is available starting with v4.6.19.

Check failure on line 33 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L33

[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'
Raw output
{"message": "[Ibexa.VariablesVersion] Use global variable '[[= latest_tag_4_6 =]]' instead of 'v4.6.19'", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 33, "column": 69}}}, "severity": "ERROR"}
To begin using AI Actions, you must first [install the required packages and perform initial configuration](install_ai_actions.md).

!!! note "API Key"
### Prerequisites

The OpenAI connector requires that you first [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).

Integration with [[= product_name_connect =]] requires that you first [get the credentials]([[= connect_doc =]]/general/ibexa_connect/#access-ibexa-connect) to your account, and the [API token](install_ai_actions.md#token).

The OpenAI connector requires that you first [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).
!!! note "[[= product_name_connect =]] Availability"

[[= product_name_connect =]] comes with all contracts signed from 2023.
If you signed your contract earlier, contact your customer success manager to use [[= product_name_connect =]].

## How it works

AI Actions LTS update relies on an extensible AI framework, which is responsible for gathering information from various sources, such as AI action types, AI action configurations, and contextual details like SiteAccess, user details, locale settings, and more.
This data can then be combined with user input.
It's then passed to a service connector, such as the default OpenAI connector, for final processing on [[= product_name =]] side.
The service connector wraps all data into a prompt or another suitable format and sends it to an external AI service.
It's then passed to a service connector, such as the default OpenAI connector or the [[= product_name_connect =]] connector, for final processing on [[= product_name =]] side.
The service connector wraps all data into a prompt or another suitable format and sends it to an external service.

When the AI Service returns a response, the response goes back through the service connector and passes to the framework.
When the external service returns a response, the response goes back through the service connector and passes to the framework.
It can then be presented to the user in any way necessary.

### Core concepts
Expand Down Expand Up @@ -147,3 +159,10 @@
![Alt text generation](img/alt_text_use_ai.png)

With some customization, administrators could use the API to run a batch process against a larger collection of illustrations.

### Performing advanced image to text analysis

With some additional customization, store managers could benefit from automating part of product management by integrating their [[= product_name =]] with Google Cloud Vision and [PIM](pim_guide.md) by using [[= product_name_connect =]].
Instead of manually selecting and linking images stored in a [DAM](add_image_asset_from_dam.md) solution to their products, they could use of a no-code workflow where an AI service, for example, Google Cloud Vision, extracts text and attributes from product images, which are then matched with existing items in a product catalog.

Check warning on line 166 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L166

[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.
Raw output
{"message": "[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 166, "column": 96}}}, "severity": "WARNING"}

This would enable automatic product identification, tagging, and catalog updates, resulting in less manual work and more efficient product management.

Check warning on line 168 in docs/ai_actions/ai_actions_guide.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/ai_actions_guide.md#L168

[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.
Raw output
{"message": "[Ibexa.OxfordComma] Use a comma before the last 'and' or 'or' in a list of four or more items.", "location": {"path": "docs/ai_actions/ai_actions_guide.md", "range": {"start": {"line": 168, "column": 1}}}, "severity": "WARNING"}
Binary file added docs/ai_actions/img/connect_api_token.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
116 changes: 92 additions & 24 deletions docs/ai_actions/install_ai_actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,37 +6,21 @@
# Install AI Actions

AI Actions are available as an LTS update to [[= product_name =]] starting with version v4.6.12 or higher, regardless of its edition.
To use this feature you must first install the packages and configure them.
To use this feature you must first install the package.
You can then install and configure the service connectors or build your own ones.

## Install packages
## Install the framework

Run the following commands to install the packages:
Run the following command to install the package:

``` bash
composer require ibexa/connector-ai
composer require ibexa/connector-openai
```

This command adds the framework code, a service connector with the OpenAI service, service handlers, Twig templates, and configurations required for using AI Actions.
This command adds the framework code, service handlers, Twig templates, and configurations required for using AI Actions.
It also modifies the permission system to account for the new functionality.

## Configure AI Actions

Once the packages are installed, before you can start using AI Actions, you must enable them by following these instructions.

### Configure access to OpenAI

Create an OpenAI account, [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key), and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).

Then, in the root folder of your project, modify the `.env` file: find the `OPENAI_API_KEY` variable and replace a placeholder value with the API key that you got from the AI service.

```bash
###> ibexa/connector-openai ###
OPENAI_API_KEY=sk-svcacct-AFCrCt1h2s3i4s5i6s7t8h9e0a1p2i3c4o5d6e
###< ibexa/connector-openai ###
```

### Modify the database schema
## Modify the database schema

Create the `add_ai_actions.sql` file that contains the following code.

Expand Down Expand Up @@ -68,9 +52,31 @@

This command modifies the existing database schema by adding database configuration required for using AI Actions.

You can now restart you application and start [working with the AI Actions feature]([[= user_doc =]]/ai_actions/work_with_ai_actions/).
!!! note "Next steps"

Once the framework is installed and configured, before you can start using AI Actions, you can configure access to [[ = product_name_base = ]]-made service connectors by following the instructions below, or [create your own](extend_ai_actions.md#create-custom-action-handler).

Only then you can restart you application and start [working with the AI Actions feature]([[= user_doc =]]/ai_actions/work_with_ai_actions/).

## Configure access to OpenAI

Run the following command to install the service connector with the OpenAI service:

``` bash
composer require ibexa/connector-openai
```

Create an OpenAI account, [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key), and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).

Then, in the root folder of your project, modify the `.env` file: find the `OPENAI_API_KEY` variable and replace a placeholder value with the API key that you got from the AI service.

```bash
###> ibexa/connector-openai ###
OPENAI_API_KEY=<your_api_key>
###< ibexa/connector-openai ###
```

### Install sample AI action configurations (optional)
### Install sample OpenAI action configurations (optional)

By installing a collection of sample AI action configurations you can quickly start using the feature.
You do it by following a standard [data migration](importing_data.md) procedure:
Expand All @@ -81,3 +87,65 @@
```

Based on these examples, which reflect the most common use cases, you can learn to configure your own AI actions with greater ease.

## Configure access to [[= product_name_connect =]]

First, get the credentials by contacting [Ibexa Support](https://support.ibexa.co).

### Create team

In [[= product_name_connect =]], set up the account, and [create a team]([[= connect_doc =]]/access_management/teams/#creating-teams).
Navigate to the team details page and note down the numerical value of the **Team id** variable.

Creating a team matters, because [scenarios]([[= connect_doc =]]/scenarios/creating_a_scenario/) that process data coming from your AI action are associated with a team.
This way, if your organization has more than one [[= product_name =]] project, each project can be linked to a different team and so can be scenarios used in those projects.

If specific users from the team are supposed to modify scenario settings, you must [assign the right roles]([[= connect_doc =]]/access_management/teams/#managing-teams) to them.

### Create token

Navigate to your [[= product_name_connect =]] user's profile, and on the **API ACCESS** tab, create a new token.
Copy the token code that appears next to the label.

![Creating an API token](img/connect_api_token.png)

### Set up credentials

In the root folder of your project, modify the `.env` file.
Replace a placeholder value of the `IBEXA_CONNECT_TOKEN` variable with the token that you got from [[= product_name_connect =]] and provide a value of the `IBEXA_CONNECT_TEAM_ID` variable.

```bash
###> ibexa/connect ###
IBEXA_CONNECT_HOST=https://connect.ibexa.co
IBEXA_CONNECT_API_PATH=/api/v2/
# Token can be created in the user's profile in Ibexa Connect, under the 'API ACCESS' section.
IBEXA_CONNECT_TOKEN=<your_api_token>
# Use the URL below to read more on Ibexa Connect teams.
# https://doc.ibexa.co/projects/connect/en/latest/access_management/teams/
IBEXA_CONNECT_TEAM_ID=2
###< ibexa/connect ###
```

### Initiate integration

Initiate the models provided by the handler by issuing the following command:

```bash
php bin/console ibexa:connect:init-connect-ai <team_id> <language> <action handler identifiers>
```

For example:

```bash
php bin/console ibexa:connect:init-connect-ai 2 en connect-image-to-text connect-text-to-text
```

!!! note "Support for multiple [[= product_name_connect =]] languages"

The [`language` attribute](https://developers.make.com/api-documentation/api-reference/templates#templates-1) determines the language in which template details such as module names will be displayed in [[= product_name_connect =]]'s UI.

### Customize templates

Check warning on line 147 in docs/ai_actions/install_ai_actions.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/ai_actions/install_ai_actions.md#L147

[Ibexa.EOLWhitespace] Remove whitespace characters from the end of the line.
Raw output
{"message": "[Ibexa.EOLWhitespace] Remove whitespace characters from the end of the line.", "location": {"path": "docs/ai_actions/install_ai_actions.md", "range": {"start": {"line": 147, "column": 24}}}, "severity": "WARNING"}

Return to the [[= product_name_connect =]] dashboard and modify the **Template for connect...handler** [templates]([[= connect_doc =]]/scenarios/scenario_templates/) by defining the logic needed to process the data.

Once the templates are ready, you can build scenarios from them, either directly in [[= product_name_connect =]] or in [[[= product_name =]]'s user interface]([[= user_doc =]]/ai_actions/work_with_ai_actions/#create-new-ai-actions).
2 changes: 1 addition & 1 deletion docs/content_management/images/add_image_asset_from_dam.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The configuration for each connector depends on the requirements of the specific

You can create your own connectors, or use the provided example DAM connector for [Unsplash](https://unsplash.com/).

To add the Unsplash connector to your system add the `ibexa/connector-unsplash` bundle to your installation.
To add the Unsplash connector to your system, add the `ibexa/connector-unsplash` bundle to your installation.

## Add Image Asset in Page Builder [[% include 'snippets/experience_badge.md' %]] [[% include 'snippets/commerce_badge.md' %]]

Expand Down
6 changes: 3 additions & 3 deletions docs/ibexa_engage/integrate_ibexa_connect.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ edition: experience

# Use Ibexa Connect

You can use [[[= product_name_connect =]]](https://doc.ibexa.co/projects/connect/en/latest/general/ibexa_connect/) to create workflows.
You can use [[[= product_name_connect =]]]([[= connect_doc =]]/general/ibexa_connect/) to create workflows.
[[= product_name_engage =]] collects user data and passes it directly to [[= product_name_connect =]].
With this data, you can create scenarios, for example, to add a user to newsletter, or to specific user segment group.

For more information, see [[[= product_name_connect =]] documentation](https://doc.ibexa.co/projects/connect/en/latest/).
For more information, see [[[= product_name_connect =]] documentation]([[= connect_doc =]]/).

## Integrate [[= product_name_engage =]] with [[= product_name_connect =]]

Expand Down Expand Up @@ -55,4 +55,4 @@ To do it, perform the following actions::
![Configure Ibexa Engage](configure_ibexa_engage.png "Configure Ibexa Engage")

4\. Then, go to **Engage** -> **Integrations** -> **Push rules** to define the default or specific rules for new campaign or website.
Select the created webhook.
Select the created webhook.
2 changes: 1 addition & 1 deletion docs/personalization/integrate_recommendation_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ To let your visitors receive emails with recommendations:
1\. With the [User API](api_reference/user_api.md#post-requests), add the `e-mail` attribute (or another identifier) to the user record.

2\. Prepare an endpoint to intercept push messages and pass them on, for example, to your mailing system.
Out of many options, it could be an [[= product_name_connect =]] [webhook](https://doc.ibexa.co/projects/connect/en/latest/tools/webhooks/).
Out of many options, it could be an [[= product_name_connect =]] [webhook]([[= connect_doc =]]/tools/webhooks/).
The endpoint must meet the following requirements:

- must support POST requests
Expand Down
2 changes: 1 addition & 1 deletion docs/release_notes/ibexa_dxp_v4.4.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ You can now take advantage of [[[= product_name_connect =]]](https://www.ibexa.c
an iPaaS (integration platform-as-a-service) which allows you to connect [[= product_name =]] with third-party applications.
[[= product_name_connect =]] features a low-code drag-and-drop interface and hundreds of connectors to different services that help you automate business processes.

See [[[= product_name_connect =]] documentation](https://doc.ibexa.co/projects/connect/en/latest/).
See [[[= product_name_connect =]] documentation]([[= connect_doc =]]/).

![Example of an Ibexa Connect scenario](4.4_connect_scenario_example.png)

Expand Down
2 changes: 1 addition & 1 deletion docs/release_notes/ibexa_dxp_v4.5.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ For more information, see [Breached passwords](https://doc.ibexa.co/en/4.5/users

### [[= product_name_connect =]]

For list of changes in [[= product_name_connect =]], see [Ibexa app release notes](https://doc.ibexa.co/projects/connect/en/latest/general/ibexa_app_release_notes/).
For list of changes in [[= product_name_connect =]], see [Ibexa app release notes]([[= connect_doc =]]/general/ibexa_app_release_notes/).

### Deprecations

Expand Down
2 changes: 1 addition & 1 deletion docs/release_notes/ibexa_dxp_v4.6.md
Original file line number Diff line number Diff line change
Expand Up @@ -873,7 +873,7 @@ A new helper method `ibexa.helpers.contentType.getContentTypeDataByHref` has bee

#### [[= product_name_connect =]]

For a list of changes in [[= product_name_connect =]], see [Ibexa app release notes](https://doc.ibexa.co/projects/connect/en/latest/general/ibexa_app_release_notes/).
For a list of changes in [[= product_name_connect =]], see [Ibexa app release notes]([[= connect_doc =]]/general/ibexa_app_release_notes/).

##### Scenario block

Expand Down