Create Tokenomics README.md (#32)

MichaelFrazzy · web-flow · commit 5a678c01a93f · 2023-09-18T11:08:44.000-05:00
* Create Tokenomics README.md

* Create placeholder.txt

* Add files via upload

* Update ETH_Gas_Data_Infura.py

Added async/parallel processing updates and error handling to all 3 scripts, making them 50-100x more efficient. Chainstack is currently the main version of the script for block data scraping, with the other 2 (Infura and Etherscan) are additional tools for use with chains besides Ethereum or as backups.

* Update ETH_Gas_Data_Etherscan.py

Same as previous update

* Update ETH_Gas_Data_Chainstack.py

Same updates as the last two scripts, this is the current main version between the three. Aync/concurrency and error handling updates, as well as additional columns to assist in GNOT fee simulation.

* Update README.md

Added a description of the revised/improved script functionality
diff --git a/tokenomics/README.md b/tokenomics/README.md
@@ -0,0 +1,16 @@
+# Tokenomics 
+A central location to store all tokenomic scripts, data, and documentation that doesn't fall under other repos or directories. 
+
+## GNOT Fee Equation & Simulation
+
+[Fee Equation Spreadsheet](https://docs.google.com/spreadsheets/d/16LdahdcIRMHNQXULCSeO90PGbHTbZbuefwDVbDeKpWk/edit?usp=sharing)
+
+In order to test variants of the GNOT/Gno.land fee equation, a number of scripts, simulations, and files have been created to aid in the process. Async/concurrency has been added to reduce request time from a week to roughly 4 hours per chain. 
+
+- A series of scripts and queries utilizing Etherscan, Infura, and a Chainstack node that pull block data from ETH and similar smart contract platforms/L1s before exporting to a CSV.
+  - Fee data will be used to weight the GNOT fee simulations, resulting in a more realistic distribution. Wei Ethereum data is then converted to Gwei to better align with GNOT notation.
+  - Gas limit, gas used, the percentage used out of the max, median wei price, and median gwei price are all scrapped and added to a CSV. The percentage of gas used each block will be used to weight the GNOT fee equation's network congestion metric during simulations. 
+  
+- Once the base Gno.land fee equation is finalized, a Monte Carlo simulation will be created to test a number of Gno.land network conditions. This includes testing against exploits such as spam attacks, various levels of network congestion, etc. Individual parameters within the fee equation such as CPU cycles required, bytes stored, and the threshold at which fee cost begins increasing expotentially (to combat exploits) will also be tested. 
+
+If for any reason the Monte Carlo simulation does not provide adoquete insight, a seperate Cartesian Product simulation may be created to brute force additional results. By virtue of testing every possible parameter input against every other parameter input, a Cartesian Product sim can further substantiate any findings as necessary (trading efficiency for thoroughness). 
diff --git a/tokenomics/gnot-fee/ETH_Gas_Data_Chainstack.py b/tokenomics/gnot-fee/ETH_Gas_Data_Chainstack.py
@@ -0,0 +1,79 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# In[1]:
+
+
+import nest_asyncio
+nest_asyncio.apply()
+import asyncio
+import aiohttp
+import pandas as pd
+
+CONCURRENCY_LIMIT = 50  # Adjust this to set the desired concurrency level
+semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)
+
+async def fetch_gas_data_for_block(session, rpc_url, block_num):
+    async with semaphore:  # Use the semaphore to limit concurrency
+        headers = {"Content-Type": "application/json"}
+        payload = {
+            "jsonrpc": "2.0",
+            "id": 1,
+            "method": "eth_getBlockByNumber",
+            "params": [hex(block_num), True]
+        }
+
+        async with session.post(rpc_url, headers=headers, json=payload) as response:
+            if response.headers.get("Content-Type") != "application/json":
+                print(f"Unexpected content type for block {block_num}: {response.headers.get('Content-Type')}")
+                return {
+                    "timestamp": None,
+                    "block_gas_limit": None,
+                    "block_gas_used": None,
+                    "median_gas_price": None,
+                    "error": f"Unexpected content type: {response.headers.get('Content-Type')}"
+                }
+            
+            data = await response.json()
+            if data["result"]:
+                block_gas_limit = int(data["result"]["gasLimit"], 16)
+                block_gas_used = int(data["result"]["gasUsed"], 16)
+                timestamp = int(data["result"]["timestamp"], 16)
+                
+                tx_gas_prices = [int(tx["gasPrice"], 16) for tx in data["result"]["transactions"]]
+                median_gas_price = sorted(tx_gas_prices)[len(tx_gas_prices) // 2] if tx_gas_prices else 0
+
+                return {
+                    "timestamp": timestamp, 
+                    "block_gas_limit": block_gas_limit,
+                    "block_gas_used": block_gas_used,
+                    "median_gas_price": median_gas_price,
+                    "error": None
+                }
+            else:
+                return {
+                    "timestamp": None,
+                    "block_gas_limit": None,
+                    "block_gas_used": None,
+                    "median_gas_price": None,
+                    "error": "No result in response"
+                }
+
+async def fetch_gas_data_chainstack(rpc_url, start_block, end_block):
+    tasks = []
+    async with aiohttp.ClientSession() as session:
+        for block_num in range(start_block, end_block + 1):
+            task = fetch_gas_data_for_block(session, rpc_url, block_num)
+            tasks.append(task)
+        return await asyncio.gather(*tasks)
+
+RPC_URL = "ADD_RPC_URL"
+START_BLOCK = 15537393  
+END_BLOCK = 17971893  
+
+gas_data = asyncio.run(fetch_gas_data_chainstack(RPC_URL, START_BLOCK, END_BLOCK))
+df = pd.DataFrame(gas_data)
+df['median_gas_price_gwei'] = df['median_gas_price'] / 1e9
+df['percentage_used'] = df['block_gas_used'] / df['block_gas_limit']
+df = df[['timestamp', 'block_gas_limit', 'block_gas_used', 'percentage_used', 'median_gas_price', 'median_gas_price_gwei', 'error']]
+df.to_csv('ETH_historic_gas_data_chainstack.csv', index=False)
diff --git a/tokenomics/gnot-fee/ETH_Gas_Data_Etherscan.py b/tokenomics/gnot-fee/ETH_Gas_Data_Etherscan.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# In[ ]:
+
+
+import nest_asyncio
+nest_asyncio.apply()
+import asyncio
+import aiohttp
+import pandas as pd
+
+CONCURRENCY_LIMIT = 50
+semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)
+
+async def fetch_gas_data_for_block(session, api_key, block_num):
+    base_url = "https://api.etherscan.io/api"
+    params = {
+        "module": "proxy",
+        "action": "eth_getBlockByNumber",
+        "tag": hex(block_num),
+        "boolean": "true",
+        "apikey": api_key
+    }
+
+    async with session.get(base_url, params=params) as response:
+        data = await response.json()
+        if data["result"]:
+            block_gas_limit = int(data["result"]["gasLimit"], 16)
+            block_gas_used = int(data["result"]["gasUsed"], 16)
+            timestamp = int(data["result"]["timestamp"], 16)
+            
+            tx_gas_prices = [int(tx["gasPrice"], 16) for tx in data["result"]["transactions"]]
+            median_gas_price = sorted(tx_gas_prices)[len(tx_gas_prices) // 2] if tx_gas_prices else 0
+
+            return {
+                "timestamp": timestamp, 
+                "block_gas_limit": block_gas_limit,
+                "block_gas_used": block_gas_used,
+                "median_gas_price": median_gas_price,
+                "error": None
+            }
+        else:
+            return {
+                "timestamp": None,
+                "block_gas_limit": None,
+                "block_gas_used": None,
+                "median_gas_price": None,
+                "error": "No result in response"
+            }
+
+async def fetch_gas_data_etherscan(api_key, start_block, end_block):
+    tasks = []
+    async with aiohttp.ClientSession() as session:
+        for block_num in range(start_block, end_block + 1):
+            task = fetch_gas_data_for_block(session, api_key, block_num)
+            tasks.append(task)
+        return await asyncio.gather(*tasks)
+
+API_KEY = "ETHERSCAN_API_KEY"
+START_BLOCK = 15537393  
+END_BLOCK = 17971893  
+
+gas_data = asyncio.run(fetch_gas_data_etherscan(API_KEY, START_BLOCK, END_BLOCK))
+df = pd.DataFrame(gas_data)
+df['median_gas_price_gwei'] = df['median_gas_price'] / 1e9
+df['percentage_used'] = df['block_gas_used'] / df['block_gas_limit']
+df = df[['timestamp', 'block_gas_limit', 'block_gas_used', 'percentage_used', 'median_gas_price', 'median_gas_price_gwei', 'error']]
+df.to_csv('ETH_historic_gas_data_etherscan.csv', index=False)
diff --git a/tokenomics/gnot-fee/ETH_Gas_Data_Infura.py b/tokenomics/gnot-fee/ETH_Gas_Data_Infura.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# In[ ]:
+
+
+import nest_asyncio
+nest_asyncio.apply()
+import asyncio
+import aiohttp
+import pandas as pd
+
+CONCURRENCY_LIMIT = 50
+semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)
+
+async def fetch_gas_data_for_block(session, api_key, block_num):
+    base_url = "https://mainnet.infura.io/v3/" + api_key
+    headers = {"Content-Type": "application/json"}
+    payload = {
+        "jsonrpc": "2.0",
+        "id": 1,
+        "method": "eth_getBlockByNumber",
+        "params": [hex(block_num), True]
+    }
+
+    async with session.post(base_url, headers=headers, json=payload) as response:
+        data = await response.json()
+        if data["result"]:
+            block_gas_limit = int(data["result"]["gasLimit"], 16)
+            block_gas_used = int(data["result"]["gasUsed"], 16)
+            timestamp = int(data["result"]["timestamp"], 16)
+            
+            tx_gas_prices = [int(tx["gasPrice"], 16) for tx in data["result"]["transactions"]]
+            median_gas_price = sorted(tx_gas_prices)[len(tx_gas_prices) // 2] if tx_gas_prices else 0
+
+            return {
+                "timestamp": timestamp, 
+                "block_gas_limit": block_gas_limit,
+                "block_gas_used": block_gas_used,
+                "median_gas_price": median_gas_price,
+                "error": None
+            }
+        else:
+            return {
+                "timestamp": None,
+                "block_gas_limit": None,
+                "block_gas_used": None,
+                "median_gas_price": None,
+                "error": "No result in response"
+            }
+
+async def fetch_gas_data_infura(api_key, start_block, end_block):
+    tasks = []
+    async with aiohttp.ClientSession() as session:
+        for block_num in range(start_block, end_block + 1):
+            task = fetch_gas_data_for_block(session, api_key, block_num)
+            tasks.append(task)
+        return await asyncio.gather(*tasks)
+
+API_KEY = "INFURA_API_KEY"
+START_BLOCK = 15537393  
+END_BLOCK = 17971893  
+
+gas_data = asyncio.run(fetch_gas_data_infura(API_KEY, START_BLOCK, END_BLOCK))
+df = pd.DataFrame(gas_data)
+df['median_gas_price_gwei'] = df['median_gas_price'] / 1e9
+df['percentage_used'] = df['block_gas_used'] / df['block_gas_limit']
+df = df[['timestamp', 'block_gas_limit', 'block_gas_used', 'percentage_used', 'median_gas_price', 'median_gas_price_gwei', 'error']]
+df.to_csv('ETH_historic_gas_data_infura.csv', index=False)
diff --git a/tokenomics/gnot-fee/placeholder.txt b/tokenomics/gnot-fee/placeholder.txt
@@ -0,0 +1 @@
+GNOT Fee scripts & simulations
