Skip to content

Commit a2b5e46

Browse files
martinibachmartinibach
authored
Merge pull request #46 from restackio/defense_quickstart
2 parents bd5ac1c + 1098563 commit a2b5e46

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+8105
-0
lines changed

‎examples/defense_quickstart_news_scraper_summarizer/.env.Example

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# (Required) Example-specific environment variables
2+
3+
OPENBABYLON_API_URL=<OPENBABYLON_API_URL>
4+
5+
# (Optional) Restack Cloud - You only need to set these if you are using Restack Cloud
6+
7+
RESTACK_ENGINE_ID=<RESTACK_ENGINE_ID>
8+
RESTACK_ENGINE_ADDRESS=<RESTACK_ENGINE_ADDRESS:443>
9+
RESTACK_ENGINE_ADDRESS_WITHOUT_PORT=<RESTACK_ENGINE_ADDRESS_WITHOUT_PORT>
10+
RESTACK_ENGINE_API_KEY=<RESTACK_ENGINE_API_KEY>
11+
12+
RESTACK_CLOUD_TOKEN=<RESTACK_CLOUD_TOKEN>

‎examples/defense_quickstart_news_scraper_summarizer/.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.env

‎examples/defense_quickstart_news_scraper_summarizer/README.md

+200
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
# Quickstart: War news scraper & summarizer - USS Hornet Defense Tech Hackathon
2+
3+
This project demonstrates how to build a Next.js application that scrapes and summarizes defense news articles using OpenBabylon AI. Built for the USS Hornet Defense Tech Hackathon, it features a Next.js frontend for displaying summarized news and a backend service that handles RSS feed scraping, content translation, and AI-powered summarization of defense-related articles.
4+
5+
## OpenBabylon Credentials
6+
7+
During the hackathon, OpenBabylon provides a public url. No API key is needeed.
8+
9+
```
10+
OPENBABYLON_API_URL=64.139.222.109:80/v1
11+
```
12+
13+
## Functions
14+
15+
A core feature of Restack is the ability to build [TypeScript functions](https://docs.restack.io/libraries/typescript/reference/functions) that can be executed as workflow steps. Each function supports rate limiting, retries, error handling and replay capabilities.
16+
17+
1. **rssPull**: Fetches and parses RSS feeds with encoding detection, supporting multiple RSS formats and character encodings
18+
2. **crawlWebsite**: Extracts clean article content from web pages using Mozilla's Readability
19+
3. **splitContent**: Splits large texts into manageable chunks (4096 token limit)
20+
4. **llmChat**: Integrates with OpenBabylon's Mistral model for translation and summarization
21+
22+
## Example Workflow
23+
24+
Functions can be executed as steps in a [workflow](https://docs.restack.io/features/workflows). The example includes one main workflow:
25+
26+
**rssDigest**: Multi-stage workflow that:
27+
28+
- Fetches RSS feeds from specified sources (default: pravda.com.ua)
29+
- Crawls full article content
30+
- Splits content into manageable chunks
31+
- Translates content to English
32+
- Generates per-chunk summaries
33+
- Creates a final consolidated digest
34+
35+
## Project Structure
36+
37+
- `frontend/`: Next.js frontend application with:
38+
39+
- Server Actions for workflow triggering
40+
- Tailwind CSS styling
41+
- Type-safe API integration to Restack Engine
42+
43+
- `backend/`: Restack backend with Node.js featuring:
44+
- `rssDigest` workflow
45+
- `llmChat` function to OpenBabylon Endpoint
46+
47+
## Prerequisites
48+
49+
- Node.js
50+
- Docker
51+
- OpenBabylon AI API URL (No API KEY NEEDED)
52+
- Restack Engine local
53+
54+
55+
## Getting Started
56+
57+
### 1. Install Restack Web UI & Engine on local
58+
59+
First, install the Restack Web UI and Engine using Docker:
60+
61+
```bash
62+
docker run -d --pull always --name studio -p 5233:5233 -p 6233:6233 -p 7233:7233 ghcr.io/restackio/restack:main
63+
```
64+
65+
- Required for any function and workflow execution as the orchestration layer
66+
- Access the Web UI at http://localhost:5233 to monitor workflow runs, view logs, and manage functions
67+
68+
![Restack Engine UI](./restack-engine-ui.png)
69+
70+
71+
### 2. Set Up Backend with Restack AI
72+
73+
1. Navigate to the backend directory:
74+
75+
```bash
76+
cd backend
77+
```
78+
79+
2. Install dependencies:
80+
81+
```bash
82+
pnpm install
83+
```
84+
85+
3. Create a `.env` file with your credentials in the backend folder:
86+
87+
```
88+
# (Required) Example-specific environment variables
89+
90+
OPENBABYLON_API_URL=<OPENBABYLON_API_URL>
91+
92+
# (Optional) Restack Cloud - You only need to set these if you are using Restack Cloud
93+
94+
RESTACK_ENGINE_ID=<RESTACK_ENGINE_ID>
95+
RESTACK_ENGINE_ADDRESS=<RESTACK_ENGINE_ADDRESS:443>
96+
RESTACK_ENGINE_API_KEY=<RESTACK_ENGINE_API_KEY>
97+
```
98+
99+
4. Start the backend service:
100+
101+
```bash
102+
pnpm run dev
103+
```
104+
105+
### 3. Set Up Frontend
106+
107+
1. Navigate to the frontend directory:
108+
109+
```bash
110+
cd frontend
111+
```
112+
113+
2. Install dependencies:
114+
115+
```bash
116+
pnpm install
117+
```
118+
119+
3. (Optional) Create a `.env` file:
120+
121+
```
122+
# Optional:
123+
RESTACK_ENGINE_ID=your_engine_id
124+
RESTACK_ENGINE_ADDRESS=your_engine_address
125+
RESTACK_ENGINE_API_KEY=your_engine_api_key
126+
```
127+
128+
4. Start the development server:
129+
130+
```bash
131+
pnpm run dev
132+
```
133+
134+
Visit [http://localhost:3000](http://localhost:3000) to see the application.
135+
136+
![Restack Engine UI](./frontend.png)
137+
138+
## Deploy on Restack Cloud
139+
140+
To deploy the example on Restack Cloud, follow these steps:
141+
142+
1. [Sign-up to restack cloud](https:console.restack.io)
143+
2. Go to `Workspace > Settings > Token` and generate an API token
144+
3. Create a new cloud engine
145+
146+
--- Back in your IDE --- 4. Copy the env variables from `env.example` into your `.env` file at the root of the example folder.
147+
Ensure to set the values correctly.
148+
149+
```
150+
151+
OPENBABYLON_API_URL=<OPENBABYLON_API_URL>
152+
153+
RESTACK_ENGINE_ID=<RESTACK_ENGINE_ID>
154+
RESTACK_ENGINE_ADDRESS=<RESTACK_ENGINE_ADDRESS:443>
155+
RESTACK_ENGINE_ADDRESS_WITHOUT_PORT=<RESTACK_ENGINE_ADDRESS_WITHOUT_PORT>
156+
RESTACK_ENGINE_API_KEY=<RESTACK_ENGINE_API_KEY>
157+
158+
RESTACK_CLOUD_TOKEN=<RESTACK_CLOUD_TOKEN>
159+
160+
```
161+
162+
5. At root of the folder `defense_quickstart_news_scraper_summarizer`, run:
163+
164+
```bash
165+
pnpm install
166+
```
167+
168+
It will install `restack/cloud` and other dependencies to deploy easily.
169+
170+
6. Run the restack:up command
171+
172+
```bash
173+
pnpm restack:up
174+
```
175+
176+
Alternatively you can run:
177+
178+
```bash
179+
EXPORT RESTACK_CLOUD_TOKEN=<TOKEN> && node restack_up.mjs
180+
```
181+
182+
- Confirm the deployment plan
183+
184+
For detailed deployment information, see the [Restack Cloud documentation](https://docs.restack.io/restack-cloud/deployrepo).
185+
186+
--- In Webbrowser ---
187+
188+
6. Go back to [Restack Cloud](https://console.restack.io)
189+
190+
Your deployment will include:
191+
192+
- A frontend application
193+
- A backend service
194+
- A Restack Engine instance connected to your cloud engine
195+
196+
![Restack Engine UI](./restack-cloud-overview.png)
197+
198+
## Contributing
199+
200+
Feel free to submit issues and enhancement requests.

‎examples/defense_quickstart_news_scraper_summarizer/backend/.env.Example

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# (Required) Example-specific environment variables
2+
3+
OPENBABYLON_API_URL=<OPENBABYLON_API_URL>
4+
5+
# (Optional) Restack Cloud - You only need to set these if you are using Restack Cloud
6+
7+
RESTACK_ENGINE_ID=<RESTACK_ENGINE_ID>
8+
RESTACK_ENGINE_ADDRESS=<RESTACK_ENGINE_ADDRESS:443>
9+
RESTACK_ENGINE_API_KEY=<RESTACK_ENGINE_API_KEY>
10+
11+
RESTACK_CLOUD_TOKEN=<RESTACK_CLOUD_TOKEN>

‎examples/defense_quickstart_news_scraper_summarizer/backend/.prettierrc

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{}

‎examples/defense_quickstart_news_scraper_summarizer/backend/Dockerfile

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Build stage
2+
FROM node:20-bullseye AS builder
3+
4+
WORKDIR /app
5+
6+
# Install pnpm
7+
RUN npm install -g pnpm
8+
9+
# Copy package files and env file if it exists
10+
COPY package*.json .env* ./
11+
12+
# Copy package files
13+
COPY package*.json ./
14+
15+
# Install dependencies including TypeScript
16+
RUN pnpm install
17+
RUN pnpm add -D typescript
18+
19+
# Copy source code
20+
COPY . .
21+
22+
# Build TypeScript code
23+
RUN pnpm run build
24+
25+
# Production stage
26+
FROM node:20-bullseye
27+
28+
WORKDIR /app
29+
30+
# Install pnpm
31+
RUN npm install -g pnpm
32+
33+
# Copy package files and built code
34+
COPY --from=builder /app/package*.json ./
35+
COPY --from=builder /app/dist ./dist
36+
37+
# Install production dependencies only
38+
RUN pnpm install --prod
39+
40+
# Define environment variables
41+
ARG RESTACK_ENGINE_ID
42+
ENV RESTACK_ENGINE_ID=${RESTACK_ENGINE_ID}
43+
44+
ARG RESTACK_ENGINE_ADDRESS
45+
ENV RESTACK_ENGINE_ADDRESS=${RESTACK_ENGINE_ADDRESS}
46+
47+
ARG RESTACK_ENGINE_API_KEY
48+
ENV RESTACK_ENGINE_API_KEY=${RESTACK_ENGINE_API_KEY}
49+
50+
EXPOSE 3000
51+
52+
CMD ["pnpm", "tsx", "dist/services.js"]

‎examples/defense_quickstart_news_scraper_summarizer/backend/package.json

+36
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
{
2+
"name": "restack-examples-ts-defense-quickstart-backend",
3+
"version": "0.0.1",
4+
"description": "Basic Node.js backend example for Next.js frontend using Restack AI, OpenBabylon",
5+
"scripts": {
6+
"dev": "dotenv -e .env tsx watch --include src src/services.ts",
7+
"clean": "rm -rf node_modules",
8+
"build": "tsc --build",
9+
"docker:build": "docker build -t restack-def-qs-news-scraper-summarizer-backend .",
10+
"docker:run": "docker run -p 8000:8000 --network restack-net --env-file .env --env RESTACK_ENGINE_ADDRESS=http://localhost:6233 restack-def-qs-news-scraper-summarizer-backend",
11+
"docker:dev": "pnpm docker:build && pnpm docker:run"
12+
},
13+
"dependencies": {
14+
"@mozilla/readability": "^0.5.0",
15+
"@restackio/ai": "^0.0.85",
16+
"@temporalio/workflow": "^1.11.2",
17+
"axios": "^1.7.7",
18+
"dotenv": "^16.4.5",
19+
"dotenv-cli": "^7.4.1",
20+
"iconv-lite": "^0.6.3",
21+
"jsdom": "^25.0.1",
22+
"openai": "^4.73.0",
23+
"tsx": "^4.19.2",
24+
"xml2js": "^0.6.2",
25+
"zod": "^3.23.8",
26+
"zod-to-json-schema": "^3.23.5"
27+
},
28+
"devDependencies": {
29+
"@types/express": "^5.0.0",
30+
"@types/jsdom": "^21.1.7",
31+
"@types/node": "^20.16.9",
32+
"@types/xml2js": "^0.4.14",
33+
"ts-node": "^10.9.2",
34+
"typescript": "^5.6.3"
35+
}
36+
}

0 commit comments

Comments
 (0)