Incorrect path to the Wazuh indexer binary `performance-analyzer-agent-cli` #2321

rauldpm · 2023-07-27T17:04:04Z

Description

An error has been detected in the wazuh-indexer-performance-analyzer service as a result of what was reported by a community user and @jftuduri

root@ubuntu-jammy:~# systemctl status wazuh-indexer-performance-analyzer.service
× wazuh-indexer-performance-analyzer.service - Wazuh-indexer Performance Analyzer
      Loaded: loaded (/lib/systemd/system/wazuh-indexer-performance-analyzer.service; disabled; vendor preset: enabled)
      Active: failed (Result: exit-code) since Wed 2023-07-26 16:24:46 UTC; 681ms ago
     Process: 51055 ExecStart=/usr/share/wazuh-indexer/bin/performance-analyzer-agent-cli (code=exited, status=203/EXEC)
    Main PID: 51055 (code=exited, status=203/EXEC)
         CPU: 7ms

Jul 26 16:24:46 ubuntu-jammy systemd[1]: wazuh-indexer-performance-analyzer.service: Scheduled restart job, restart counter is at 5.
Jul 26 16:24:46 ubuntu-jammy systemd[1]: Stopped Wazuh-indexer Performance Analyzer.
Jul 26 16:24:46 ubuntu-jammy systemd[1]: wazuh-indexer-performance-analyzer.service: Start request repeated too quickly.
Jul 26 16:24:46 ubuntu-jammy systemd[1]: wazuh-indexer-performance-analyzer.service: Failed with result 'exit-code'.
Jul 26 16:24:46 ubuntu-jammy systemd[1]: Failed to start Wazuh-indexer Performance Analyzer.

As commented by @Desvelao, the error seems to come from the service file, since ExecStart is not pointing to the binary correctly

wazuh-packages/stack/indexer/base/files/usr/lib/systemd/system/wazuh-indexer-performance-analyzer.service

Line 7 in 014bd30

ExecStart=/usr/share/wazuh-indexer/bin/performance-analyzer-agent-cli

In the OpenSearch GitHub repository of the plugin, it is possible to see that it does not have the correct path

https://github.com/opensearch-project/performance-analyzer/blob/7c32918b5434c04af40c072d0a972ddf7818b556/packaging/opensearch-performance-analyzer.service#L6

ExecStart=/usr/share/opensearch/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli

Possibly this is a lack of updating procedure with respect to OpenSearch since the initial versions of said file do share the path with respect to our service file

https://github.com/opensearch-project/performance-analyzer/blob/1.0.1.0/packaging/opensearch-performance-analyzer.service

ExecStart=/usr/share/opensearch/bin/performance-analyzer-agent-cli

It is necessary to adapt the correct path for the service to work normally

    
           %dir %attr(750, %{USER}, %{GROUP}) %{INSTALL_DIR}/bin/opensearch-performance-analyzer 
        
           %attr(750, %{USER}, %{GROUP}) %{INSTALL_DIR}/bin/opensearch-performance-analyzer/performance-analyzer-agent 
        
           %attr(750, %{USER}, %{GROUP}) %{INSTALL_DIR}/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli

Tasks

Fix the performance-analyzer-agent-cli path in the service file

Validation

Wazuh indexer works without error
The plugin service starts and works normally

The text was updated successfully, but these errors were encountered:

rauldpm · 2023-09-04T12:29:46Z

Update report

Still happening in Wazuh indexer 4.6.0 - 2.8.0

root@debian11:/home/vagrant# cat /usr/share/wazuh-indexer/VERSION 
4.6.0
root@debian11:/home/vagrant# systemctl start wazuh-indexer-performance-analyzer.service
root@debian11:/home/vagrant# systemctl status wazuh-indexer-performance-analyzer.service
● wazuh-indexer-performance-analyzer.service - Wazuh-indexer Performance Analyzer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer-performance-analyzer.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-09-04 12:28:16 UTC; 897ms ago
    Process: 9535 ExecStart=/usr/share/wazuh-indexer/bin/performance-analyzer-agent-cli (code=exited, status=203/EXEC)
   Main PID: 9535 (code=exited, status=203/EXEC)
        CPU: 3ms

Sep 04 12:28:16 debian11 systemd[1]: wazuh-indexer-performance-analyzer.service: Scheduled restart job, restart counter is at 5.
Sep 04 12:28:16 debian11 systemd[1]: Stopped Wazuh-indexer Performance Analyzer.
Sep 04 12:28:16 debian11 systemd[1]: wazuh-indexer-performance-analyzer.service: Start request repeated too quickly.
Sep 04 12:28:16 debian11 systemd[1]: wazuh-indexer-performance-analyzer.service: Failed with result 'exit-code'.
Sep 04 12:28:16 debian11 systemd[1]: Failed to start Wazuh-indexer Performance Analyzer.

rauldpm · 2023-10-16T16:21:40Z

Update report

Still happening in Wazuh indexer 4.8.0 - 2.10.0

root@ubuntu20stack:/home/vagrant# cat /usr/share/wazuh-indexer/VERSION 
4.8.0
root@ubuntu20stack:/home/vagrant# cat /usr/share/wazuh-indexer/VERSION 
4.8.0
root@ubuntu20stack:/home/vagrant# systemctl start wazuh-indexer-performance-analyzer.service
root@ubuntu20stack:/home/vagrant# systemctl status wazuh-indexer-performance-analyzer.service
● wazuh-indexer-performance-analyzer.service - Wazuh-indexer Performance Analyzer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer-performance-analyzer.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-10-16 16:21:58 UTC; 957ms ago
    Process: 6049 ExecStart=/usr/share/wazuh-indexer/bin/performance-analyzer-agent-cli (code=exited, status=203/EXEC)
   Main PID: 6049 (code=exited, status=203/EXEC)

Oct 16 16:21:58 ubuntu20stack systemd[1]: wazuh-indexer-performance-analyzer.service: Scheduled restart job, restart counter is at 5.
Oct 16 16:21:58 ubuntu20stack systemd[1]: Stopped Wazuh-indexer Performance Analyzer.
Oct 16 16:21:58 ubuntu20stack systemd[1]: wazuh-indexer-performance-analyzer.service: Start request repeated too quickly.
Oct 16 16:21:58 ubuntu20stack systemd[1]: wazuh-indexer-performance-analyzer.service: Failed with result 'exit-code'.
Oct 16 16:21:58 ubuntu20stack systemd[1]: Failed to start Wazuh-indexer Performance Analyzer.

Deblintrake09 · 2023-12-19T21:56:23Z

Update

After fixing the path and building new package, the system still does not start. A new error appears:

The job identifier is 13115 and the job result is done.
dic 19 22:38:28 pop-os systemd[1]: Started Wazuh-indexer Performance Analyzer.
░░ Subject: A start job for unit wazuh-indexer-performance-analyzer.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit wazuh-indexer-performance-analyzer.service has finished successfully.
░░ 
░░ The job identifier is 13115.
dic 19 22:38:28 pop-os performance-analyzer-agent-cli[386208]: OPENSEARCH_PATH_CONF variable needs to be set or passed in as a parameter.
dic 19 22:38:28 pop-os performance-analyzer-agent-cli[386208]: Note: It should be set to /usr/share/wazuh-indexer/config (for docker or tar) or /etc/opensearch (for rpm or deb)
dic 19 22:38:28 pop-os systemd[1]: wazuh-indexer-performance-analyzer.service: Main process exited, code=exited, status=1/FAILURE

The deb package was installed in an ubuntu endpoint but neither of both proposed paths were present in the system.

The file uses the OPENSEARCH_PATH_CONF variable as the path to locate a configuration file /opensearch-performance-analyzer/log4j2.xml

# Make sure the OPENSEARCH_PATH_CONF environment variable is set
if [ -z "$OPENSEARCH_PATH_CONF" ]; then
    echo "OPENSEARCH_PATH_CONF variable needs to be set or passed in as a parameter."
    echo "Note: It should be set to $OPENSEARCH_HOME/config (for docker or tar) or /etc/opensearch (for rpm or deb)"
    exit 1
fi

PA_AGENT_JAVA_OPTS="-Dlog4j.configurationFile=$OPENSEARCH_PATH_CONF/opensearch-performance-analyzer/log4j2.xml \
              -Xms64M -Xmx64M -XX:+UseSerialGC -XX:CICompilerCount=1 -XX:-TieredCompilation -XX:InitialCodeCacheSize=4096 \
              -XX:MaxRAM=400m"

Deblintrake09 · 2023-12-20T08:59:42Z

Research 20/12

Looking at the perferormance-analyzer repository we can see that there is only one instance of the log4j2.xml file, located in the expected path of the script.

performance-analyzer

We cannot find that file inside our build:

# find /usr/share/wazuh-indexer/ -name 'log4j2*' -type f
/usr/share/wazuh-indexer/performance-analyzer-rca/config/log4j2.xml

The only file we find in our build inside the same folder the one coming from performance-analyzer-rca repository

performance-analyzer-rca

When looking through all the file system of the test endpoint, we can finth that the only log4j2.xml is located at /etc/wazuh-indexer/opensearch-performance-analyzer

# find / -name 'log4j2*' -type f
/etc/wazuh-indexer/opensearch-performance-analyzer/log4j2.xml
/etc/wazuh-indexer/log4j2.properties
find: ‘/run/user/1000/gvfs’: Permiso denegado
find: ‘/run/user/1000/doc’: Permiso denegado
/home/qa/Descargas/opensearch-build/docker/release/config/opensearch/log4j2.properties
find: ‘/proc/12610/task/12610/net’: Argumento inválido
find: ‘/proc/12610/net’: Argumento inválido
/usr/share/wazuh-indexer/performance-analyzer-rca/config/log4j2.xml

When adding Environment=OPENSEARCH_PATH_CONF=/etc/wazuh-indexer/ enviroment variable to the /lib/systemd/system/wazuh-indexer-performance-analyzer.service file the analyzer starts correctly.

# systemctl status wazuh-indexer-performance-analyzer
● wazuh-indexer-performance-analyzer.service - Wazuh-indexer Performance Analyzer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer-performance-analyzer.service; disabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-12-20 11:06:20 CET; 8min ago
   Main PID: 20951 (java)
      Tasks: 24 (limit: 21396)
     Memory: 130.8M
        CPU: 11.042s
     CGroup: /system.slice/wazuh-indexer-performance-analyzer.service
             └─20951 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Xms4m -Xmx64m -XX:+UseSerialGC -Dlog4j.configurationFile=/etc/wazuh-indexer//opensearch-performance-an>

dic 20 11:14:40 pop-os performance-analyzer-agent-cli[20951]: INFORMACIÓN: Single batch             : No bind variables have been provided with a single statement batch execut>

pro-akim · 2023-12-20T15:25:22Z

Review Notes

LGTM

Tested 🟢

root@mediumubuntu:/home/vagrant/wazuh-packages/stack/indexer/deb# systemctl status wazuh-indexer-performance-analyzer
● wazuh-indexer-performance-analyzer.service - Wazuh-indexer Performance Analyzer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer-performance-analyzer.service; disabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-12-20 15:19:33 UTC; 5s ago
   Main PID: 35732 (java)
      Tasks: 24 (limit: 14266)
     Memory: 116.2M
     CGroup: /system.slice/wazuh-indexer-performance-analyzer.service
             └─35732 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Xms4m -Xmx64m -XX:+UseSerialGC -Dlog4j.configurationFile=/etc/wazuh-indexe>

davidjiglesias · 2023-12-22T11:08:46Z

Please @Deblintrake09, ensure all commits for the PR are signed.

davidjiglesias · 2024-01-03T11:00:34Z

Still 1 commit unsigned.

davidjiglesias · 2024-01-04T09:21:19Z

LGTM

rauldpm added level/task Subtask issue type/bug Bug issue labels Jul 27, 2023

rauldpm mentioned this issue Oct 16, 2023

Bump Wazuh indexer to OpenSearch 2.10.0 #2520

Closed

13 tasks

Deblintrake09 self-assigned this Dec 19, 2023

Deblintrake09 mentioned this issue Dec 19, 2023

Fix wazuh-indexer-performance-analyzer-cli binary startup #2709

Merged

Deblintrake09 linked a pull request Dec 19, 2023 that will close this issue

Fix wazuh-indexer-performance-analyzer-cli binary startup #2709

Merged

davidjiglesias closed this as completed Jan 4, 2024

wazuhci added this to Release 4.9.0 Apr 4, 2024

wazuhci moved this to Done in Release 4.9.0 Apr 4, 2024

Enaraque mentioned this issue Sep 9, 2024

[BUG] wazuh-indexer-performance-analyzer does not work as expected wazuh/wazuh-indexer#387

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incorrect path to the Wazuh indexer binary `performance-analyzer-agent-cli` #2321

Incorrect path to the Wazuh indexer binary `performance-analyzer-agent-cli` #2321

rauldpm commented Jul 27, 2023

rauldpm commented Sep 4, 2023

rauldpm commented Oct 16, 2023 •

edited

Loading

Deblintrake09 commented Dec 19, 2023 •

edited

Loading

Deblintrake09 commented Dec 20, 2023 •

edited

Loading

pro-akim commented Dec 20, 2023

davidjiglesias commented Dec 22, 2023

davidjiglesias commented Jan 3, 2024

davidjiglesias commented Jan 4, 2024

Incorrect path to the Wazuh indexer binary performance-analyzer-agent-cli #2321

Incorrect path to the Wazuh indexer binary performance-analyzer-agent-cli #2321

Comments

rauldpm commented Jul 27, 2023

Description

Related:

Tasks

Validation

rauldpm commented Sep 4, 2023

Update report

rauldpm commented Oct 16, 2023 • edited Loading

Update report

Deblintrake09 commented Dec 19, 2023 • edited Loading

Update

Deblintrake09 commented Dec 20, 2023 • edited Loading

Research 20/12

pro-akim commented Dec 20, 2023

Review Notes

davidjiglesias commented Dec 22, 2023

davidjiglesias commented Jan 3, 2024

davidjiglesias commented Jan 4, 2024

Incorrect path to the Wazuh indexer binary `performance-analyzer-agent-cli` #2321

Incorrect path to the Wazuh indexer binary `performance-analyzer-agent-cli` #2321

rauldpm commented Oct 16, 2023 •

edited

Loading

Deblintrake09 commented Dec 19, 2023 •

edited

Loading

Deblintrake09 commented Dec 20, 2023 •

edited

Loading