[Feature][Engine UI] partial pages of the engine (#7602)

Author: QiaoJ-Chen

.github/workflows/backend.yml

@@ -306,6 +306,37 @@ jobs:
           npm set strict-ssl false
           npm install
           npm run build
+
+  seatunnel-ui:
+    if: needs.changes.outputs.api == 'true'
+    needs: [ changes, sanity-check ]
+    name: Build SeaTunnel UI
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    steps:
+      - name: Checkout PR
+        uses: actions/checkout@v3
+
+      - uses: actions/setup-node@v2
+        with:
+          node-version: 20.x
+
+      - name: Install Dependencies and Check Code Style
+        run: |
+          cd seatunnel-engine/seatunnel-engine-ui/
+          npm install
+          npm run lint
+
+      - name: Run unit tests
+        run: |
+          cd seatunnel-engine/seatunnel-engine-ui/
+          npm run test:unit
+
+      - name: Build SeaTunnel UI
+        run: |
+          cd seatunnel-engine/seatunnel-engine-ui/
+          npm run build
+
   unit-test:
     needs: [ changes, sanity-check ]
     if: needs.changes.outputs.api == 'true' || (needs.changes.outputs.api == 'false' && needs.changes.outputs.ut-modules != '')

.gitignore

@@ -55,3 +55,8 @@ seatunnel-examples
 
 /lib/*
 version.properties
+node/
+
+dist/
+
+seatunnel-engine/seatunnel-engine-server/**/ui/*

docs/en/seatunnel-engine/web-ui.md

@@ -0,0 +1,48 @@
+# Apache SeaTunnel Web UI Documentation
+
+## Access
+
+Before accessing the web ui we need to enable the http rest api. first you need to configure it in the `seatunnel.yaml` configuration file
+
+```
+seatunnel:
+  engine:
+    http:
+      enable-http: true
+      port: 8080
+
+```
+
+Then visit `http://ip:8080/#/overview`
+
+## Overview
+
+The Web UI of Apache SeaTunnel offers a user-friendly interface for monitoring and managing SeaTunnel jobs. Through the Web UI, users can view real-time information on currently running jobs, finished jobs, and the status of worker and master nodes within the cluster. The main functional modules include Jobs, Workers, and Master, each providing detailed status information and operational options to help users efficiently manage and optimize their data processing workflows.
+![overview.png](../../images/ui/overview.png)
+
+## Jobs
+
+### Running Jobs
+
+The "Running Jobs" section lists all SeaTunnel jobs that are currently in execution. Users can view basic information for each job, including Job ID, submission time, status, execution time, and more. By clicking on a specific job, users can access detailed information such as task distribution, resource utilization, and log outputs, allowing for real-time monitoring of job progress and timely handling of potential issues.
+![running.png](../../images/ui/running.png)
+![detail.png](../../images/ui/detail.png)
+
+### Finished Jobs
+
+The "Finished Jobs" section displays all SeaTunnel jobs that have either successfully completed or failed. This section provides execution results, completion times, durations, and failure reasons (if any) for each job. Users can review past job records through this module to analyze job performance, troubleshoot issues, or rerun specific jobs as needed.
+![finished.png](../../images/ui/finished.png)
+
+## Workers
+
+### Workers Information
+
+The "Workers" section displays detailed information about all worker nodes in the cluster, including each worker's address, running status, CPU and memory usage, number of tasks being executed, and more. Through this module, users can monitor the health of each worker node, promptly identify and address resource bottlenecks or node failures, ensuring the stable operation of the SeaTunnel cluster.
+![workers.png](../../images/ui/workers.png)
+
+## Master
+
+### Master Information
+
+The "Master" section provides the status and configuration information of the master node in the SeaTunnel cluster. Users can view the master's address, running status, job scheduling responsibilities, and overall resource allocation within the cluster. This module helps users gain a comprehensive understanding of the cluster's core management components, facilitating cluster configuration optimization and troubleshooting.
+![master.png](../../images/ui/master.png)

docs/images/ui/detail.png

@@ -206,7 +206,8 @@ const sidebars = {
                 "seatunnel-engine/rest-api-v2",
                 "seatunnel-engine/user-command",
                 "seatunnel-engine/logging",
-                "seatunnel-engine/telemetry"
+                "seatunnel-engine/telemetry",
+                "seatunnel-engine/web-ui"
             ]
         },
         {

docs/images/ui/finished.png

@@ -0,0 +1,47 @@
+# Apache SeaTunnel Web UI 文档
+
+## 访问
+
+在访问 web ui 之前我们需要开启 http rest api。首先需要在`seatunnel.yaml`配置文件中配置
+
+```
+seatunnel:
+  engine:
+    http:
+      enable-http: true
+      port: 8080
+```
+
+然后访问 `http://ip:8080/#/overview`
+
+## 概述
+
+Apache SeaTunnel 的 Web UI 提供了一个友好的用户界面，用于监控和管理 SeaTunnel 作业。通过 Web UI，用户可以实时查看当前运行的作业、已完成的作业，以及集群中工作节点和管理节点的状态。主要功能模块包括 Jobs、Workers 和 Master，每个模块都提供了详细的状态信息和操作选项，帮助用户高效地管理和优化其数据处理流程。
+![overview.png](../../images/ui/overview.png)
+
+## 作业
+
+### 运行中的作业
+
+“运行中的作业”模块列出了当前正在执行的所有 SeaTunnel 作业。用户可以查看每个作业的基本信息，包括作业 ID、提交时间、状态、执行时间等。点击具体作业可以查看更多详细信息，如任务分布、资源使用情况和日志输出，便于用户实时监控作业进度并及时处理潜在问题。
+![running.png](../../images/ui/running.png)
+![detail.png](../../images/ui/detail.png)
+
+### 已完成的作业
+
+“已完成的作业”模块展示了所有已成功完成或失败的 SeaTunnel 作业。此部分提供了每个作业的执行结果、完成时间、耗时以及失败原因（如果有）。用户可以通过此模块回顾过去的作业记录，分析作业性能，进行故障排查或重复执行某些特定作业。
+![finished.png](../../images/ui/finished.png)
+
+## 工作节点
+
+### 工作节点信息
+
+“工作节点”模块展示了集群中所有工作节点的详细信息，包括每个工作节点的地址、运行状态、CPU 和内存使用情况、正在执行的任务数量等。通过该模块，用户可以监控各个工作节点的健康状况，及时发现和处理资源瓶颈或节点故障，确保 SeaTunnel 集群的稳定运行。
+![workers.png](../../images/ui/workers.png)
+
+## 管理节点
+
+### 管理节点信息
+
+“管理节点”模块提供了 SeaTunnel 集群中主节点的状态和配置信息。用户可以查看 Master 节点的地址、运行状态、负责的作业调度情况以及整体集群的资源分配情况。该模块帮助用户全面了解集群的核心管理部分，便于进行集群配置优化和故障排查。
+![master.png](../../images/ui/master.png)

docs/images/ui/master.png

@@ -32,6 +32,7 @@
         <module>seatunnel-engine-core</module>
         <module>seatunnel-engine-storage</module>
         <module>seatunnel-engine-serializer</module>
+        <module>seatunnel-engine-ui</module>
     </modules>
 
     <properties>

docs/images/ui/overview.png

@@ -33,6 +33,11 @@
             <artifactId>seatunnel-engine-core</artifactId>
             <version>${project.version}</version>
         </dependency>
+        <dependency>
+            <groupId>org.apache.seatunnel</groupId>
+            <artifactId>seatunnel-engine-ui</artifactId>
+            <version>${project.version}</version>
+        </dependency>
         <dependency>
             <groupId>org.apache.seatunnel</groupId>
             <artifactId>checkpoint-storage-hdfs</artifactId>

docs/images/ui/running.png

@@ -47,6 +47,7 @@
 
 import javax.servlet.DispatcherType;
 
+import java.net.URL;
 import java.util.EnumSet;
 
 import static org.apache.seatunnel.engine.server.rest.RestConstant.ENCRYPT_CONFIG;
@@ -88,7 +89,15 @@ public void createJettyServer() {
         FilterHolder filterHolder = new FilterHolder(new ExceptionHandlingFilter());
         context.addFilter(filterHolder, "/*", EnumSet.of(DispatcherType.REQUEST));
 
-        context.addServlet(new ServletHolder("default", new DefaultServlet()), "/");
+        ServletHolder defaultServlet = new ServletHolder("default", DefaultServlet.class);
+        URL uiResource = JettyService.class.getClassLoader().getResource("ui");
+        if (uiResource != null) {
+            defaultServlet.setInitParameter("resourceBase", uiResource.toExternalForm());
+        } else {
+            log.warn("UI resources not found in classpath");
+        }
+
+        context.addServlet(defaultServlet, "/");
 
         ServletHolder overviewHolder = new ServletHolder(new OverviewServlet(nodeEngine));
         ServletHolder runningJobsHolder = new ServletHolder(new RunningJobsServlet(nodeEngine));

docs/images/ui/workers.png

@@ -0,0 +1,22 @@
+ # Licensed to the Apache Software Foundation (ASF) under one or more
+ # contributor license agreements.  See the NOTICE file distributed with
+ # this work for additional information regarding copyright ownership.
+ # The ASF licenses this file to You under the Apache License, Version 2.0
+ # (the "License"); you may not use this file except in compliance with
+ # the License.  You may obtain a copy of the License at
+ #
+ #     http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+NODE_ENV=development
+
+# if you want to use the online server, you need to set this to `http://124.221.211.72:8081`
+# VITE_APP_API_SERVICE='http://localhost:5801'
+VITE_APP_API_SERVICE='http://124.221.211.72:8081'
+# the context path of api service
+VITE_APP_API_BASE='/api'

docs/sidebars.js

@@ -0,0 +1,19 @@
+ # Licensed to the Apache Software Foundation (ASF) under one or more
+ # contributor license agreements.  See the NOTICE file distributed with
+ # this work for additional information regarding copyright ownership.
+ # The ASF licenses this file to You under the Apache License, Version 2.0
+ # (the "License"); you may not use this file except in compliance with
+ # the License.  You may obtain a copy of the License at
+ #
+ #     http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+NODE_ENV=production
+
+# the context path of api service
+VITE_APP_API_BASE=''

docs/zh/seatunnel-engine/web-ui.md

@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* eslint-env node */
+require('@rushstack/eslint-patch/modern-module-resolution')
+
+module.exports = {
+  root: true,
+  'extends': [
+    'plugin:vue/vue3-essential',
+    'eslint:recommended',
+    '@vue/eslint-config-typescript',
+    '@vue/eslint-config-prettier/skip-formatting'
+  ],
+  overrides: [
+    {
+      files: [
+        'cypress/e2e/**/*.{cy,spec}.{js,ts,jsx,tsx}',
+        'cypress/support/**/*.{js,ts,jsx,tsx}'
+      ],
+      'extends': [
+        'plugin:cypress/recommended'
+      ]
+    }
+  ],
+  parserOptions: {
+    ecmaVersion: 'latest'
+  },
+  rules: {
+    "vue/multi-word-component-names": "off"
+  }
+}

seatunnel-engine/pom.xml

@@ -0,0 +1,30 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
+.DS_Store
+dist
+dist-ssr
+coverage
+*.local
+
+/cypress/videos/
+/cypress/screenshots/
+
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?
+
+*.tsbuildinfo

seatunnel-engine/seatunnel-engine-server/pom.xml

@@ -0,0 +1,8 @@
+{
+  "$schema": "https://json.schemastore.org/prettierrc",
+  "semi": false,
+  "tabWidth": 2,
+  "singleQuote": true,
+  "printWidth": 100,
+  "trailingComma": "none"
+}

seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/JettyService.java

@@ -0,0 +1,60 @@
+# seatunnel-engine-ui
+
+## Development Environment Dependencies
+
+- Node 18+/20+ required
+- npm 7+
+
+- modify `VITE_APP_API_SERVICE` and `VITE_APP_API_BASE` in `.env.development`
+- quick start
+
+```sh
+npm install
+npm run dev
+```
+
+## Project Setup
+
+```sh
+npm install
+```
+
+### Compile and Hot-Reload for Development
+
+```sh
+npm run dev
+```
+
+### Type-Check, Compile and Minify for Production
+
+```sh
+npm run build
+```
+
+### Run Unit Tests with [Vitest]
+
+```sh
+npm run test:unit
+```
+
+### Run End-to-End Tests with [Cypress]
+
+```sh
+npm run test:e2e:dev
+```
+
+This runs the end-to-end tests against the Vite development server.
+It is much faster than the production build.
+
+But it's still recommended to test the production build with `test:e2e` before deploying (e.g. in CI environments):
+
+```sh
+npm run build
+npm run test:e2e
+```
+
+### Lint with [ESLint]
+
+```sh
+npm run lint
+```