Skip to content

Commit 71ac8db

Browse files
authored
Add competition docs (#67)
1 parent 604ccd8 commit 71ac8db

File tree

3 files changed

+98
-1
lines changed

3 files changed

+98
-1
lines changed

docs/competitions/QUICKSTART.md

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Competition Quick Start Guide
2+
3+
## Prerequisites
4+
5+
- Python 3.8+
6+
- Storage provider credentials (R2/S3)
7+
- Registration on subnet 2 (use `btcli register` if not already registered)
8+
9+
> [!NOTE]
10+
> While miners can run on any platform, testing on macOS arm64 is recommended since validators use this architecture for evaluation. This helps ensure your circuit will perform consistently when being evaluated.
11+
12+
## Getting Started
13+
14+
1. **Configure Storage**
15+
16+
The following variables configure remote storage for your circuit files.
17+
18+
```bash
19+
export STORAGE_PROVIDER="r2" # or "s3"
20+
export STORAGE_BUCKET="your-bucket"
21+
export STORAGE_ACCOUNT_ID="your-account-id"
22+
export STORAGE_ACCESS_KEY="your-access-key"
23+
export STORAGE_SECRET_KEY="your-secret-key"
24+
export STORAGE_REGION="auto" # or AWS region for S3
25+
```
26+
27+
2. **Start Miner**
28+
29+
Start the miner process normally using `pm2` or your preferred process manager.
30+
31+
## Circuit Submission Flow
32+
33+
1. **Prepare Circuit Files**
34+
35+
- Compile your circuit
36+
- Generate proving/verification keys
37+
- Test locally with sample inputs
38+
39+
2. **Deploy Circuit**
40+
41+
- Move your files into the `competition_circuit/` directory.
42+
- The miner process will automatically detect your circuit
43+
- New circuit files are uploaded to remote storage
44+
- Verification key is committed to the chain
45+
46+
3. **Monitor Evaluation**
47+
- Watch logs for validator requests for your circuit
48+
- Track SOTA competition metrics on https://accelerator.omron.ai

docs/competitions/README.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Competition Technical Guide
2+
3+
## Overview
4+
5+
This guide provides a deep technical dive into participating in competitions as a miner. Competitions are a mechanism for miners to submit optimized zero-knowledge circuits that prove neural network execution, with rewards based on circuit performance across multiple metrics.
6+
7+
## Circuit Evaluation
8+
9+
The scoring system evaluates circuits based on accuracy (40% weight), proof size (30% weight), and response time (30% weight). Accuracy measures how closely circuit outputs match the baseline model using MSE loss and exponential transformation. Proof size evaluates the compactness of generated zero-knowledge proofs relative to current SOTA. Response time measures proof generation speed normalized against SOTA performance.
10+
11+
The final score calculation uses an exponential decay formula that creates a score between 0 and 1, where higher scores indicate better performance relative to the current SOTA. The formula penalizes poor performance exponentially, encouraging continuous improvement and optimization:
12+
13+
```
14+
score = exp(-(
15+
0.4 * max(0, sota_accuracy - accuracy) +
16+
0.3 * max(0, (proof_size - sota_proof_size)/sota_proof_size) +
17+
0.3 * max(0, (response_time - sota_response_time)/sota_response_time)
18+
))
19+
```
20+
21+
## Technical Requirements
22+
23+
Your circuit must process inputs matching the competition config shape and produce a matching output shape.
24+
25+
The submission package must include several key files: a compiled circuit (model.compiled), proving and verification keys (pk.key and vk.key), and a settings.json configuration file. These files work together to enable proof generation and verification.
26+
27+
## Evaluation Process
28+
29+
The evaluation process runs through multiple rounds of testing to ensure consistent performance. Each round generates random test inputs that are fed through both your circuit and a baseline model. The baseline comparison uses either PyTorch or ONNX models, supporting flexible implementation approaches.
30+
31+
Your circuit must generate valid proofs that verify successfully. The system measures proof generation time and size across 10 evaluation rounds, averaging the metrics to determine final scores. All verifications must pass for a valid submission - a single failure results in disqualification.
32+
33+
## Deployment Architecture
34+
35+
The competition system uses cloud storage (R2/S3) for circuit file management. When validators request your circuit, they receive signed URLs for secure file access.
36+
37+
The commitment process anchors your verification key hash on-chain. This creates an immutable record of your submission and prevents tampering. The system verifies that local and chain commitments match before proceeding with evaluation.
38+
39+
## Optimization Guidelines
40+
41+
Circuit optimization requires balancing multiple competing factors. Reducing circuit complexity generally improves proof generation speed and size but may impact accuracy. The scoring formula's weights guide this tradeoff - accuracy carries the highest weight at 40%.
42+
43+
Resource management plays a crucial role in performance. Proof generation demands significant GPU power and memory. Monitor system resources during testing to ensure your circuit operates within validator timeout limits. Profile your operations to identify and eliminate bottlenecks.
44+
45+
## Platform Requirements
46+
47+
Currently, validators run using macOS arm64 architecture. This requirement ensures consistent evaluation environments across all participants. While you can develop and test on other platforms, final submissions must be validated on the required architecture to maintain consensus and provide the most optimal benchmark for the use case.

neurons/_validator/core/validator_loop.py

+3-1
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,8 @@ def log_health(self):
175175
log_queue_metrics(queue_size, est_latency)
176176

177177
def update_processed_uids(self):
178-
if len(self.processed_uids) >= len(self.queryable_uids):
178+
total_processed = len(self.processed_uids) + len(self.active_tasks)
179+
if total_processed >= len(self.queryable_uids):
179180
self.processed_uids.clear()
180181

181182
@with_rate_limit(period=ONE_MINUTE)
@@ -274,6 +275,7 @@ async def maintain_request_pool(self):
274275
pass
275276

276277
slots_available = self.current_concurrency - len(self.active_tasks)
278+
277279
if slots_available > 0:
278280
self.update_processed_uids()
279281
available_uids = [

0 commit comments

Comments
 (0)