lmnt-all-voices

Author: martinibach

community/lmnt/.env.Example

@@ -0,0 +1,8 @@
+LMNT_API_KEY=<your-lmnt-api-key>
+
+# Restack Cloud (Optional)
+
+# RESTACK_ENGINE_ID=<your-engine-id>
+# RESTACK_ENGINE_API_KEY=<your-engine-api-key>
+# RESTACK_ENGINE_ADDRESS=<your-engine-address>
+# RESTACK_ENGINE_API_ADDRESS=<your-engine-api-address>

community/lmnt/.gitignore

@@ -0,0 +1,10 @@
+__pycache__
+.pytest_cache
+venv
+.env
+.vscode
+poetry.lock
+
+src/media/*
+!src/media/.gitkeep
+

community/lmnt/Dockerfile

@@ -0,0 +1,22 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y
+
+RUN pip install poetry
+
+COPY pyproject.toml ./
+
+COPY . .
+
+# Configure poetry to not create virtual environment
+RUN poetry config virtualenvs.create false
+
+# Install dependencies
+RUN poetry install --no-interaction --no-ansi
+
+# Expose port 80
+EXPOSE 80
+
+CMD poetry run python -m src.services

community/lmnt/README.md

@@ -0,0 +1,164 @@
+# Restack AI - LMNT Example
+
+This repository contains a simple example project to help you scale with Restack AI.
+It demonstrates how to scale reliably to millions of workflows on a local machine with a local LLM provider.
+
+## Walkthrough video
+
+https://www.youtube.com/watch?v=WsUtQYC74og
+
+## Motivation
+
+When scaling AI workflows, you want to make sure that you can handle failures and retries gracefully.
+This example demonstrates how to do this with Restack AI.
+
+### Workflow Steps
+
+The table below shows the execution of 50 workflows in parallel, each with three steps.
+Steps 2 and 3 are LLM functions that must adhere to a rate limit of 1 concurrent call per second.
+
+| Step | Workflow 1 | Workflow 2 | ... | Workflow 50 |
+| ---- | ---------- | ---------- | --- | ----------- |
+| 1    | Basic      | Basic      | ... | Basic       |
+| 2    | LLM        | LLM        | ... | LLM         |
+| 3    | LLM        | LLM        | ... | LLM         |
+
+### Traditional Rate Limit Management
+
+When running multiple workflows in parallel, managing the rate limit for LLM functions is crucial. Here are common strategies:
+
+1. **Task Queue**: Use a task queue (e.g., Celery, RabbitMQ) to schedule LLM calls, ensuring only one is processed at a time.
+2. **Rate Limiting Middleware**: Implement middleware to queue requests and process them at the allowed rate.
+3. **Semaphore or Locking**: Use a semaphore or lock to control access, ensuring only one LLM function runs per second.
+
+### With Restack
+
+Restack automates rate limit management, eliminating the need for manual strategies. Define the rate limit in the service options, and Restack handles queuing and execution:
+
+```python
+client.start_service(
+    task_queue="llm",
+    functions=[llm_generate, llm_evaluate],
+    options=ServiceOptions(
+        rate_limit=1,
+        max_concurrent_function_runs=1
+    )
+)
+```
+
+Focus on building your logics while Restack ensures efficient and resilient workflow execution.
+
+### On Restack UI
+
+You can see from the parent workflow how long each child workflow stayed in queue and how long was the execution time.
+
+![Parent Workflow](./ui-parent.png)
+
+And for each child workflow, for each step you can see how long the function stayed in queue, how long the function took to execute and how many retries happened.
+
+![Child Workflow](./ui-child.png)
+
+## Prerequisites
+
+- Python 3.10 or higher
+- Poetry (for dependency management)
+- Docker (for running the Restack services)
+- Local LLM provider (we use LMStudio and a Meta Llama 3.1 8B Instruct 4bit model in this example)
+
+## Start LM stduio for local LLM provider
+
+Start local server with Meta Llama 3.1 8B Instruct 4bit model
+
+https://lmstudio.ai
+
+## Prerequisites
+
+- Docker (for running Restack)
+- Python 3.10 or higher
+
+## Start Restack
+
+To start the Restack, use the following Docker command:
+
+```bash
+docker run -d --pull always --name restack -p 5233:5233 -p 6233:6233 -p 7233:7233 ghcr.io/restackio/restack:main
+```
+
+## Start python shell
+
+```bash
+poetry env use 3.10 && poetry shell
+```
+
+## Install dependencies
+
+```bash
+poetry install
+```
+
+```bash
+poetry env info # Optional: copy the interpreter path to use in your IDE (e.g. Cursor, VSCode, etc.)
+```
+
+```bash
+poetry run dev
+```
+
+## Run workflows
+
+### from UI
+
+You can run workflows from the UI by clicking the "Run" button.
+
+![Run workflows from UI](./ui-endpoints.png)
+
+### from API
+
+You can run one workflow from the API by using the generated endpoint:
+
+`POST http://localhost:6233/api/workflows/ChildWorkflow`
+
+or multiple workflows by using the generated endpoint:
+
+`POST http://localhost:6233/api/workflows/ExampleWorkflow`
+
+### from any client
+
+You can run workflows with any client connected to Restack, for example:
+
+```bash
+poetry run schedule
+```
+
+executes `schedule_workflow.py` which will connect to Restack and execute the `ChildWorkflow` workflow.
+
+```bash
+poetry run scale
+```
+
+executes `schedule_scale.py` which will connect to Restack and execute the `ExampleWorkflow` workflow.
+
+```bash
+poetry run interval
+```
+
+executes `schedule_interval.py` which will connect to Restack and execute the `ChildWorkflow` workflow every second.
+
+## Deploy on Restack Cloud
+
+To deploy the application on Restack, you can create an account at [https://console.restack.io](https://console.restack.io)
+
+## Project Structure
+
+- `src/`: Main source code directory
+  - `client.py`: Initializes the Restack client
+  - `functions/`: Contains function definitions
+  - `workflows/`: Contains workflow definitions
+  - `services.py`: Sets up and runs the Restack services
+- `schedule_workflow.py`: Example script to schedule and run a workflow
+- `schedule_interval.py`: Example script to schedule and a workflow every second
+- `schedule_scale.py`: Example script to schedule and run 50 workflows at once
+
+# Deployment
+
+Create an account on [Restack Cloud](https://console.restack.io) and follow instructions on site to create a stack and deploy your application on Restack Cloud.

community/lmnt/pyproject.toml

@@ -0,0 +1,33 @@
+# Project metadata
+[tool.poetry]
+name = "community_lmnt"
+version = "0.0.1"
+description = "A simple example to get started with the restack-ai SDK"
+authors = [
+    "Restack Team <[email protected]>",
+]
+readme = "README.md"
+packages = [{include = "src"}]
+
+[tool.poetry.dependencies]
+python = ">=3.10,<4.0"
+restack-ai = "^0.0.52"
+watchfiles = "^1.0.0"
+pydantic = "^2.10.5"
+lmnt = "1.1.4"
+
+[tool.poetry.dev-dependencies]
+pytest = "6.2"  # Optional: Add if you want to include tests in your example
+
+# Build system configuration
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"
+
+# CLI command configuration
+[tool.poetry.scripts]
+dev = "src.services:watch_services"
+services = "src.services:run_services"
+workflow = "schedule_workflow:run_schedule_workflow"
+interval = "schedule_interval:run_schedule_interval"
+scale = "schedule_scale:run_schedule_scale"

community/lmnt/schedule_interval.py

@@ -0,0 +1,28 @@
+import asyncio
+import time
+from restack_ai import Restack
+from restack_ai.restack import ScheduleSpec, ScheduleIntervalSpec
+from datetime import timedelta
+
+async def main():
+
+    client = Restack()
+
+    workflow_id = f"{int(time.time() * 1000)}-ChildWorkflow"
+    await client.schedule_workflow(
+        workflow_name="ChildWorkflow",
+        workflow_id=workflow_id,
+        schedule=ScheduleSpec(
+            intervals=[ScheduleIntervalSpec(
+                every=timedelta(seconds=1)
+            )]
+        )
+    )
+
+    exit(0)
+
+def run_schedule_scale():
+    asyncio.run(main())
+
+if __name__ == "__main__":
+    run_schedule_scale()

community/lmnt/schedule_scale.py

@@ -0,0 +1,24 @@
+import asyncio
+import time
+from restack_ai import Restack
+
+from src.workflows.workflow import ExampleWorkflowInput
+
+async def main():
+
+    client = Restack()
+
+    workflow_id = f"{int(time.time() * 1000)}-ExampleWorkflow"
+    await client.schedule_workflow(
+        workflow_name="ExampleWorkflow",
+        workflow_id=workflow_id,
+        input=ExampleWorkflowInput(amount=50)
+    )
+
+    exit(0)
+
+def run_schedule_scale():
+    asyncio.run(main())
+
+if __name__ == "__main__":
+    run_schedule_scale()

community/lmnt/schedule_workflow.py

@@ -0,0 +1,26 @@
+import asyncio
+import time
+from restack_ai import Restack
+
+async def main():
+
+    client = Restack()
+
+    workflow_id = f"{int(time.time() * 1000)}-ExampleWorkflow"
+    run_id = await client.schedule_workflow(
+        workflow_name="ChildWorkflow",
+        workflow_id=workflow_id
+    )
+
+    await client.get_workflow_result(
+        workflow_id=workflow_id,
+        run_id=run_id
+    )
+
+    exit(0)
+
+def run_schedule_workflow():
+    asyncio.run(main())
+
+if __name__ == "__main__":
+    run_schedule_workflow()

community/lmnt/src/__init__.py

@@ -0,0 +1,21 @@
+import os
+from restack_ai import Restack
+from restack_ai.restack import CloudConnectionOptions
+from dotenv import load_dotenv  
+
+# Load environment variables from a .env file
+load_dotenv()
+
+
+engine_id = os.getenv("RESTACK_ENGINE_ID")
+address = os.getenv("RESTACK_ENGINE_ADDRESS")
+api_key = os.getenv("RESTACK_ENGINE_API_KEY")
+api_address = os.getenv("RESTACK_ENGINE_API_ADDRESS")
+
+connection_options = CloudConnectionOptions(
+    engine_id=engine_id,
+    address=address,
+    api_key=api_key,
+    api_address=api_address
+)
+client = Restack(connection_options)