Merge pull request #39 from ASSERT-KTH/38-improve-readme-documentation

Enhance README Documentation

README.md

@@ -1,31 +1,103 @@
 # HACK-LABS
 
-Contains actual smart contract attacker contracts for major benchmarks, starting with smartbugs-curated.
-
-It includes an evaluation [tool](https://github.com/ASSERT-KTH/solidity-hack-labs/tree/main/evaluator) for automated patch assessment.
-The tool allows you to test patched contracts from the dataset against these exploits to evaluate the effectiveness of the applied patches.
-
-### Smartbugs-Curated Dataset
-Total contracts: 143
-
-Details per category (following the [DASP taxonomy](https://dasp.co/)): 
-
-| Vulnerability            | Total Contracts | Exploits  |
-|--------------------------|-----------------|-----------|
-| Reentrancy               | 31              | 26        |
-| Access Control           | 18              | 16        |
-| Arithmetic               | 15              | 13        |
-| Unchecked Low Level Calls| 52              | 20        |
-| Denial Of Service        | 6               | 4         |
-| Bad Randomness           | 8               | 4         |
-| Front Running            | 4               | 3         |
-| Time Manipulation        | 5               | 3         |
-| Short Addresses          | 1               | 0         |
-| Others                   | 3               | 2         |
-| Total                    | 143             | 91        |
-
-Details:
-
-- 1 duplicate contract:
-  - 0x627fa62ccbb1c1b04ffaecd72a53e37fc0e17839.sol on reentrancy and unchecked_low_level
-  - Not exploitable for reentrancy
+[![Test Suite](https://github.com/ASSERT-KTH/solidity-hack-labs/actions/workflows/ci.yml/badge.svg)](https://github.com/ASSERT-KTH/solidity-hack-labs/actions/workflows/ci.yml)
+
+A comprehensive framework for testing smart contract security patches against real-world exploits. This project provides:
+1. A reproducible dataset of smart contract exploits for SmartBugs-Curated dataset
+2. An automated tool for evaluating security patches
+
+## 🎯 Project Structure
+
+```
+.
+├── smartbugs-curated/     # Exploit dataset and test framework
+│   └── 0.4.x/            # Solidity 0.4.x contracts and exploits
+│       ├── contracts/     # Vulnerable contracts and exploits
+│       └── test/         # Automated exploit tests
+│
+└── evaluator/            # Patch evaluation tool
+    └── src/             # Source code for patch testing
+```
+
+## 📊 Exploit Dataset Coverage
+
+Our dataset is based on the [DASP taxonomy](https://dasp.co/) and covers the following vulnerabilities:
+
+| Vulnerability            | Total Contracts | Working Exploits | Coverage % |
+|--------------------------|-----------------|------------------|------------|
+| Reentrancy               | 31              | 26              | 83.9%      |
+| Access Control           | 18              | 16              | 88.9%      |
+| Arithmetic               | 15              | 13              | 86.7%      |
+| Unchecked Low Level Calls| 52              | 20              | 38.5%      |
+| Denial Of Service        | 6               | 4               | 66.7%      |
+| Bad Randomness           | 8               | 4               | 50.0%      |
+| Front Running            | 4               | 3               | 75.0%      |
+| Time Manipulation        | 5               | 3               | 60.0%      |
+| Short Addresses          | 1               | 0               | 0.0%       |
+| Others                   | 3               | 2               | 66.7%      |
+| **Total**                | **143**         | **91**          | **63.6%**  |
+
+### Coverage Limitations and Technical Constraints
+
+The following table summarizes why certain contracts in our dataset could not be exploited:
+
+| Challenge Type              | Number of Contracts |
+|----------------------------|-------------------|
+| Not Exploitable            | 31                |
+| Missing Methods            | 9                 |
+| Mislabeled Solidity Version| 7                 |
+| Exceeded Time Limit        | 3                 |
+| Incompatible Solidity Version| 1               |
+| Honeypot Contracts         | 1                 |
+| Hash Cracking Required     | 1                 |
+
+For a detailed breakdown of non-exploitable contracts and their specific reasons, see [not-exploitable.md](./not-exploitable.md).
+
+## 🚀 Quick Start
+
+### Testing Exploits
+Navigate to the exploit dataset:
+```bash
+cd smartbugs-curated/0.4.x
+npm ci
+npx hardhat test
+```
+
+### Evaluating Patches
+Navigate to the evaluator tool:
+```bash
+cd evaluator
+python -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+Test a patch:
+```bash
+python src/main.py \
+    --format solidity \
+    --patch ./your-patch.sol \
+    --contract-file ./target-contract.sol \
+    --main-contract ContractName
+```
+
+## 📚 Documentation
+
+- [Exploit Dataset Documentation](./smartbugs-curated/0.4.x/README.md)
+- [Patch Evaluator Documentation](./evaluator/README.md)
+
+## 🤝 Contributing
+
+Contributions are welcome! Please check our contribution guidelines before submitting:
+1. Follow the existing project structure
+2. Include tests for new exploits
+3. Document any new features or changes
+
+## 📝 Notes
+
+- Dataset contains one duplicate contract (`0x627fa62ccbb1c1b04ffaecd72a53e37fc0e17839.sol`) listed under both reentrancy and unchecked_low_level categories
+- All exploits are provided for educational and testing purposes only
+
+## 🙏 Acknowledgements
+
+- [SmartBugs](https://github.com/smartbugs) - For the smart contract vulnerability dataset

smartbugs-curated/0.4.x/README.md

@@ -1,33 +1,97 @@
-# Automatic exploits for smartbugs-curated
+# Automatic Exploit Testing for SmartBugs Contracts
 
-This directory contains attacks which expoit the vulnerabilities in contracts from smartbugs-curated.
+This repository contains automated exploit tests for vulnerabilities found in the SmartBugs-curated smart contract dataset. The tests are implemented using the Hardhat testing framework.
 
-These attacks are automated using Hardhat testing framework.
+## Overview
 
-## Directory structure
-- `contracts/`: Contains Solidity smart contracts.
-    + `dataset/`: Dataset directory corresponding to smartbugs-curated contracts.
-    + `<category>/`: intermediate directory where `<category>` corresponds to the category of vulnerabilities.
-        - `<file_name>_attack.sol`: attacker contract w.r.t. `dataset/<category>/<file_name>.sol`
+The project demonstrates various security vulnerabilities in Ethereum smart contracts by providing:
+- Vulnerable contracts from the SmartBugs-Curated dataset
+- Exploit contracts that demonstrate the vulnerabilities
+- Automated tests that verify the exploits
 
-- `test/`: Contains test files. Hardhat uses Mocha testing framework by default.
-    + `<category>`: intermediate directory where `<category>` corresponds to the category of vulnerabilities.
-        - `<file_name>_test.js`: test simulating the attack `<file_name>_attack.sol` on `<file_name>.sol`.
+## Project Structure
 
-- `artifacts/`: Stores compiled contract artifacts generated by Hardhat.
+```
+.
+├── contracts/
+│   ├── dataset/           # Original vulnerable contracts from SmartBugs
+│   └── <category>/        # Exploit contracts organized by vulnerability type
+│       └── *_attack.sol   # Exploit contract for each vulnerability
+│
+├── test/
+│   └── <category>/        # Test files organized by vulnerability type
+│       └── *_test.js      # Test scripts that demonstrate exploits
+│
+├── artifacts/             # Compiled contract files (generated)
+└── hardhat.config.js      # Hardhat configuration
+```
 
-- `hardhat.config.js`: Configuration file for Hardhat.
-
-## How to get started
+## Getting Started
 
 ### Prerequisites
-- Nodejs with npm7+ (tested using node:v20.14.0 and npm:10.7.0)
+
+- Node.js (v20.14.0 or later)
+- npm (v10.7.0 or later)
 
 ### Installation
-1. Make sure that Nodejs is installed.
-2. Clone repo and go into `smartbugs-curated/0.4.x` directory.
-3. Run `npm ci`.
 
-### Running tests
-- To test all the exploits run `npx hardhat test`.
-- To test a specific exploit run `npx hardhat test <path_to_test>`.
+1. Clone the repository
+
+```bash
+git clone <repository-url>
+cd smartbugs-curated/0.4.x
+```
+
+2. Install dependencies
+
+```bash
+npm ci
+```
+
+### Running Tests
+
+Run all exploit tests:
+```bash
+npx hardhat test
+```
+
+Run a specific exploit test:
+```bash
+npx hardhat test test/<category>/<test_name>_test.js
+```
+
+## Example
+
+To test a specific reentrancy vulnerability:
+```bash
+npx hardhat test test/reentrancy/simple_dao_test.js
+```
+
+## Test Results
+
+Each test file contains two types of tests:
+
+### 1. Sanity Tests
+Verify normal contract behavior without exploiting vulnerabilities. These ensure the contract works as intended under normal circumstances.
+
+### 2. Exploit Tests
+Demonstrate the vulnerability by executing attack sequences that exploit the security weakness.
+
+Example output:
+```bash
+    Reentrancy Attack for simpleDAO.sol
+    ✔ sanity check: reentrancy/simpleDAO.sol (632ms)
+    ✔ should successfully drain funds through reentrancy attack
+
+  2 passing (651ms)
+
+```
+
+## Contributing
+
+Feel free to contribute by:
+1. Adding new exploit contracts
+2. Improving existing tests
+3. Enhancing documentation
+
+Please ensure all new exploits follow the existing directory structure and naming conventions.