Skip to content

Commit 9a8b8ab

Browse files
committed
Add unit tests, Docker integration, and update documentation
1 parent 4445b99 commit 9a8b8ab

24 files changed

+1090
-134
lines changed

.gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -165,4 +165,5 @@ cython_debug/
165165
daily_progress/*
166166
src/jupyter/*.md
167167
src/jupyter/daily_progress/*
168-
run/*
168+
run/*
169+
test_results.txt

Dockerfile

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Dockerfile
2+
3+
# 使用官方的 Python 基础镜像
4+
FROM python:3.10-slim
5+
6+
# 设置工作目录
7+
WORKDIR /app
8+
9+
# 复制 requirements.txt 并安装依赖
10+
COPY requirements.txt .
11+
12+
# 安装 Python 依赖
13+
RUN pip install --no-cache-dir -r requirements.txt
14+
15+
# 复制项目的所有文件到容器
16+
COPY . .
17+
18+
# 复制并执行 validate_tests.sh 脚本
19+
COPY validate_tests.sh .
20+
RUN chmod +x validate_tests.sh
21+
RUN ./validate_tests.sh
22+
23+
# 设置容器入口
24+
CMD ["python", "src/daemon_process.py"]

README-EN.md

+157-34
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,33 @@
11
# GitHub Sentinel
22

3+
## Table of Contents
4+
5+
- [GitHub Sentinel](#github-sentinel)
6+
- [Features](#features)
7+
- [Quick Start](#quick-start)
8+
- [1. Install Dependencies](#1-install-dependencies)
9+
- [2. Configure the Application](#2-configure-the-application)
10+
- [3. How to Run](#3-how-to-run)
11+
- [A. Run as a Command Line Tool](#a-run-as-a-command-line-tool)
12+
- [B. Run as a Background Service](#b-run-as-a-background-service)
13+
- [C. Run as a Gradio Server](#c-run-as-a-gradio-server)
14+
- [Ollama Installation and Service Deployment](#ollama-installation-and-service-deployment)
15+
- [Unit Testing](#unit-testing)
16+
- [Unit Testing and Validation Script `validate_tests.sh`](#unit-testing-and-validation-script-validate_testssh)
17+
- [Purpose](#purpose)
18+
- [Functionality](#functionality)
19+
- [Building and Validating with Docker](#building-and-validating-with-docker)
20+
- [1. `Dockerfile`](#1-dockerfile)
21+
- [Purpose](#purpose)
22+
- [Key Steps](#key-steps)
23+
- [2. `build_image.sh`](#2-build_imagesh)
24+
- [Purpose](#purpose)
25+
- [Functionality](#functionality)
26+
- [Contributing](#contributing)
27+
- [License](#license)
28+
- [Contact](#contact)
29+
30+
331
![GitHub stars](https://img.shields.io/github/stars/DjangoPeng/GitHubSentinel?style=social)
432
![GitHub forks](https://img.shields.io/github/forks/DjangoPeng/GitHubSentinel?style=social)
533
![GitHub watchers](https://img.shields.io/github/watchers/DjangoPeng/GitHubSentinel?style=social)
@@ -9,18 +37,20 @@
937
![GitHub last commit](https://img.shields.io/github/last-commit/DjangoPeng/GitHubSentinel?color=red)
1038

1139
<p align="center">
12-
<br> English | <a href="README.md">中文</a>
40+
<br> <a href="README.md">中文</a> | English
1341
</p>
1442

15-
GitHub Sentinel is an open-source tool AI Agent designed for developers and project managers. It automatically retrieves and aggregates updates from subscribed GitHub repositories on a regular basis (daily/weekly). Key features include subscription management, update retrieval, notification system, and report generation.
43+
GitHub Sentinel is an open-source tool AI agent designed for developers and project managers. It automatically retrieves and aggregates updates from subscribed GitHub repositories periodically (daily/weekly). Key features include subscription management, update retrieval, notification system, and report generation.
1644

1745
## Features
18-
- Subscription management
19-
- Update retrieval
20-
- Notification system
21-
- Report generation
2246

23-
## Getting Started
47+
- **Subscription Management**: Manage your subscription list of GitHub repositories.
48+
- **Update Retrieval**: Automatically retrieve and aggregate the latest updates from subscribed repositories, including commits, issues, and pull requests.
49+
- **Notification System**: Notify subscribers about the latest project progress via email.
50+
- **Report Generation**: Generate detailed project progress reports based on retrieved updates, supporting multiple formats and templates.
51+
- **Multi-Model Support**: Support natural language report generation through OpenAI and Ollama models.
52+
53+
## Quick Start
2454

2555
### 1. Install Dependencies
2656

@@ -32,64 +62,64 @@ pip install -r requirements.txt
3262

3363
### 2. Configure the Application
3464

35-
Edit the `config.json` file to set up your GitHub token, Email settings(e.g.Tencent Exmail), subscription file, update settings and LLM settings(both support OpenAI GPT API and Ollama REST API so far):
36-
65+
Edit the `config.json` file to set up your GitHub Token, Email settings (using Tencent WeCom Email as an example), subscription file, update settings, and large model service configurations (supporting OpenAI GPT API and Ollama private large model service):
3766

3867
```json
3968
{
40-
"github_token": "your_github_token",
69+
"github": {
70+
"token": "your_github_token",
71+
"subscriptions_file": "subscriptions.json",
72+
"progress_frequency_days": 1,
73+
"progress_execution_time": "08:00"
74+
},
4175
"email": {
4276
"smtp_server": "smtp.exmail.qq.com",
4377
"smtp_port": 465,
4478
"from": "[email protected]",
4579
"password": "your_email_password",
4680
4781
},
48-
"slack_webhook_url": "your_slack_webhook_url",
49-
"subscriptions_file": "subscriptions.json",
50-
"github_progress_frequency_days": 1,
51-
"github_progress_execution_time":"08:00",
5282
"llm": {
53-
"model_type": "openai",
83+
"model_type": "ollama",
5484
"openai_model_name": "gpt-4o-mini",
5585
"ollama_model_name": "llama3",
5686
"ollama_api_url": "http://localhost:11434/api/chat"
87+
},
88+
"report_types": ["github"],
89+
"slack": {
90+
"webhook_url": "your_slack_webhook_url"
5791
}
5892
}
59-
6093
```
61-
**For security reasons:** It is recommended to configure the GitHub Token and Email Password using environment variables to avoid storing sensitive information in plain text, as shown below:
94+
95+
**For security reasons:** The GitHub Token and Email Password settings support using environment variables to avoid configuring sensitive information in plain text, as shown below:
6296

6397
```shell
64-
# GitHub
98+
# Github
6599
export GITHUB_TOKEN="github_pat_xxx"
66100
# Email
67101
export EMAIL_PASSWORD="password"
68102
```
69103

70-
#### Ollama: Installation and Deployment
71-
72-
[Ollama Installation and Deployment](docs/ollama.md)
73-
74104
### 3. How to Run
75105

76-
GitHub Sentinel supports the following three modes of operation:
106+
GitHub Sentinel supports the following three running modes:
77107

78-
#### A. Run as a Command-Line Tool
108+
#### A. Run as a Command Line Tool
79109

80-
You can interactively run the application from the command line:
110+
You can run the application interactively from the command line:
81111

82112
```sh
83113
python src/command_tool.py
84114
```
85115

86-
In this mode, you can manually enter commands to manage subscriptions, retrieve updates, and generate reports.
116+
In this mode, you can manually input commands to manage subscriptions, retrieve updates, and generate reports.
87117

88118
#### B. Run as a Background Service
89119

90-
To run the application as a background service (daemon), it will automatically update according to the configured schedule.
120+
To run the application as a background service (daemon process), it will automatically update periodically according to the relevant configuration.
91121

92-
You can use the daemon management script [daemon_control.sh](daemon_control.sh) to start, check the status, stop, and restart:
122+
You can directly use the daemon management script [daemon_control.sh](daemon_control.sh) to start, check the status, stop, and restart:
93123

94124
1. Start the service:
95125

@@ -99,8 +129,8 @@ You can use the daemon management script [daemon_control.sh](daemon_control.sh)
99129
DaemonProcess started.
100130
```
101131

102-
- This will launch [./src/daemon_process.py], generating reports periodically as set in `config.json`, and sending emails.
103-
- Service logs will be saved to `logs/DaemonProcess.log`, with historical logs also appended to `logs/app.log`.
132+
- This will start [./src/daemon_process.py], which will periodically generate reports and send emails according to the update frequency and time point set in `config.json`.
133+
- The service log will be saved to the `logs/DaemonProcess.log` file. At the same time, historical cumulative logs will also be appended to the `logs/app.log` log file.
104134

105135
2. Check the service status:
106136

@@ -138,20 +168,113 @@ python src/gradio_server.py
138168
![gradio_demo](images/gradio_demo.png)
139169

140170
- This will start a web server on your machine, allowing you to manage subscriptions and generate reports through a user-friendly interface.
141-
- By default, the Gradio server will be accessible at `http://localhost:7860`, but you can share it publicly if needed.
171+
- By default, the Gradio server will be accessible at `http://localhost:7860`, but it can be shared publicly if needed.
172+
173+
## Ollama Installation and Service Deployment
174+
175+
Ollama is a private large model management tool that supports local and containerized deployment, command-line interaction, and REST API calls.
176+
177+
For detailed instructions on Ollama installation and private large model service deployment, please refer to [Ollama Installation and Service Deployment](docs/ollama.md).
178+
179+
### Ollama Brief Official Installation
180+
181+
To use Ollama for calling private large model services in GitHub Sentinel, follow these steps for installation and configuration:
182+
183+
1. **Install Ollama**:
184+
Download and install the Ollama service according to the official Ollama documentation. Ollama supports multiple operating systems, including Linux, Windows, and macOS.
185+
186+
2. **Start the Ollama Service**:
187+
After installation, start the Ollama service with the following command:
188+
189+
```bash
190+
ollama serve
191+
```
192+
193+
By default, the Ollama API will run on `http://localhost:11434`.
194+
195+
3. **Configure Ollama for Use in GitHub Sentinel**:
196+
In the `config.json` file, configure the relevant information for the Ollama API:
197+
198+
```json
199+
{
200+
"llm": {
201+
"model_type": "ollama",
202+
"ollama_model_name": "llama3",
203+
"ollama_api_url": "http://localhost:11434/api/chat"
204+
}
205+
}
206+
```
207+
208+
4. **Validate the Configuration**:
209+
Start GitHub Sentinel and generate a report with the following command to verify that the Ollama configuration is correct:
210+
211+
```bash
212+
python src/command_tool.py
213+
```
214+
215+
If the configuration is correct, you will be able to generate reports using the Ollama model.
216+
217+
## Unit Testing
218+
219+
To ensure the quality and reliability of the code, GitHub Sentinel uses the `unittest` module for unit testing. For detailed explanations of `unittest` and related tools (such as `@patch` and `MagicMock`), please refer to [Detailed Unit Test Explanation](docs/unit_test.md).
220+
221+
### Unit Testing and Validation Script `validate_tests.sh`
222+
223+
#### Purpose
224+
`validate_tests.sh` is a shell script used to run unit tests and validate the results. It is executed during the Docker image build process to ensure the correctness and stability of the code.
225+
226+
#### Functionality
227+
- The script runs all unit tests and outputs the results to the `test_results.txt` file.
228+
- If the tests fail, the script outputs the test results and causes the Docker build to fail.
229+
- If all tests pass, the script continues the build process.
230+
231+
## Building and Validating with Docker
232+
233+
To facilitate building and deploying the GitHub Sentinel project in various environments, we provide Docker support. This support
234+
235+
includes the following files and functionalities:
236+
237+
### 1. `Dockerfile`
238+
239+
#### Purpose
240+
The `Dockerfile` is a configuration file used to define how to build a Docker image. It describes the steps to build the image, including installing dependencies, copying project files, running unit tests, etc.
241+
242+
#### Key Steps
243+
- Use `python:3.10-slim` as the base image and set the working directory to `/app`.
244+
- Copy the project's `requirements.txt` file and install Python dependencies.
245+
- Copy all project files to the container and grant execution permission to the `validate_tests.sh` script.
246+
- During the build process, execute the `validate_tests.sh` script to ensure that all unit tests pass. If the tests fail, the build process will be aborted.
247+
- After a successful build, the container will default to running `src/main.py` as the entry point.
248+
249+
### 2. `build_image.sh`
250+
251+
#### Purpose
252+
`build_image.sh` is a shell script used to automatically build a Docker image. It retrieves the branch name from the current Git branch and uses it as the tag for the Docker image, facilitating the generation of different Docker images on different branches.
253+
254+
#### Functionality
255+
- Retrieve the current Git branch name and use it as the tag for the Docker image.
256+
- Use the `docker build` command to build the Docker image and tag it with the current Git branch name.
257+
258+
#### Usage Example
259+
```bash
260+
chmod +x build_image.sh
261+
./build_image.sh
262+
```
263+
264+
With these scripts and configuration files, you can ensure that Docker images built in different development branches are based on code that has passed unit tests, thereby improving code quality and deployment reliability.
142265
143266
## Contributing
144267
145-
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated. If you have any suggestions or feature requests, please open an issue first to discuss what you would like to change.
268+
Contributions make the open-source community a wonderful place to learn, inspire, and create. Any contributions you make are **greatly appreciated**. If you have any suggestions or feature requests, please start an issue to discuss what you would like to change.
146269
147270
<a href='https://github.com/repo-reviews/repo-reviews.github.io/blob/main/create.md' target="_blank"><img alt='Github' src='https://img.shields.io/badge/review_me-100000?style=flat&logo=Github&logoColor=white&labelColor=888888&color=555555'/></a>
148271
149272
## License
150273
151-
This project is licensed under the terms of the Apache-2.0 License . See the [LICENSE](LICENSE) file for details.
274+
This project is licensed under the terms of the Apache-2.0 License. See the [LICENSE](LICENSE) file for details.
152275
153276
## Contact
154277
155278
Django Peng - [email protected]
156279
157-
Project Link: https://github.com/DjangoPeng/GitHubSentinel
280+
Project Link: https://github.com/DjangoPeng/GitHubSentinel

0 commit comments

Comments
 (0)