Skip to content

Metrics api use curl #410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Mar 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
4926f78
Create automate-metrics.yaml
jukent Mar 11, 2024
7b0acfa
outline for automate-metrics.yaml
jukent Mar 11, 2024
b8511a2
Create get-metrics.py
jukent Mar 11, 2024
a9ed412
Update automate-metrics.yaml
jukent Mar 11, 2024
37cd664
Update automate-metrics.yaml
jukent Mar 11, 2024
aa0463b
Update footer-menu.html with metrics
jukent Mar 11, 2024
8b91ca4
Create metrics.md
jukent Mar 11, 2024
9f664fa
add json_extract to extensions
jukent Mar 11, 2024
2dafc21
Update get-metrics.py to also write file
jukent Mar 11, 2024
47fad36
Update get-metrics.py with returned condition
jukent Mar 11, 2024
9a4a657
Update automate-metrics.yaml
jukent Mar 11, 2024
f6dbf6d
pre-commit
jukent Mar 11, 2024
238edf2
use os to access passed in keys
jukent Mar 11, 2024
314b93d
fix double quoted strings
jukent Mar 11, 2024
ce7f60e
add jsonextract to environment.yml
jukent Mar 11, 2024
49ba11b
write metrics.md in a python file
jukent Mar 11, 2024
53c472c
rm from config
jukent Mar 11, 2024
99143ec
use env files
jukent Mar 11, 2024
7da6027
update comment
jukent Mar 11, 2024
e15d349
state with a table
jukent Mar 11, 2024
1d7c268
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 11, 2024
6e55f19
rm table because it won't format and add to toc tree
jukent Mar 11, 2024
73ad6f1
Merge branch 'metrics-api' of https://github.com/jukent/projectpythia…
jukent Mar 11, 2024
99f8acf
add dispatch
jukent Mar 11, 2024
ac5d69e
starting metrics file, and - vs _
jukent Mar 11, 2024
2345bb0
pre-commit
jukent Mar 11, 2024
fc92d94
fix write-metrics
jukent Mar 11, 2024
4a18be6
make a metrics folder for holding future graphs
jukent Mar 12, 2024
5c449f0
missed a path herr
jukent Mar 12, 2024
ed3a388
add metrics automation as a step before site building
jukent Mar 12, 2024
820b976
rm automate-metrics schedule
jukent Mar 12, 2024
546fec2
Add last updated time to write-metrics
jukent Mar 12, 2024
411e54d
pre-commit
jukent Mar 13, 2024
a907dff
use curl
jukent Mar 15, 2024
7a3c583
Merge branch 'metrics-gh-action' into metrics-api
jukent Mar 15, 2024
f36f4b2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 3 additions & 14 deletions .github/workflows/automate-metrics.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@ name: Update User Metrics

on:
workflow_dispatch:
schedule:
# Weekly Every Monday at 1PM UTC
- cron: '0 13 * * 1'

env:
portal_id: ${{ secrets.GA4_PORTAL_ID }}
Expand Down Expand Up @@ -32,17 +29,9 @@ jobs:
pip install google-analytics-data

- name: Get and Write Metrics to JSON
run: python get-metrics.py
run: |
curl -O https://raw.githubusercontent.com/jukent/projectpythia.github.io/metrics-api/.github/workflows/get-metrics.py
python get_metrics.py

- name: Write Markdown File
run: python write-metrics-md.py

- name: Push Changes
if: steps.Get-and-Write-Metrics.outputs == 1 # Conditional step execution
run: |
git config --global user.name "${{ github.actor }}"
git config --global user.email "${{ github.actor }}@users.noreply.github.com"
git add user_metrics.json
git add ../portal/metrics.md
git commit -m "Update user metrics"
git push
48 changes: 3 additions & 45 deletions .github/workflows/get-metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,6 @@


def _run_total_users_report(property_id):
"""Fetches total users for a given property ID

Args:
property_id: The Google Analytics 4 property ID

Returns:
int: The total number of active users
"""

client = BetaAnalyticsDataClient()

Expand All @@ -38,48 +30,14 @@ def _run_total_users_report(property_id):


def get_metrics(portal_id, foundations_id, cookbooks_id):
"""Retrieves total users for specified GA4 properties and writes to file if changes are significant."""

metrics_dict = {}
metrics_dict['Portal'] = _run_total_users_report(str(portal_id))
metrics_dict['Foundations'] = _run_total_users_report(str(foundations_id))
metrics_dict['Cookbooks'] = _run_total_users_report(str(cookbooks_id))

return metrics_dict # Return the metrics dictionary


def write_metrics(metrics_dict):
"""Reads existing metrics, compares for significant change, and writes to file if necessary."""

# Read existing user counts (handle potential file absence)
try:
with open('user_metrics.json') as f:
user_data = json.load(f)
except FileNotFoundError:
user_data = {}

# Define a threshold for significant change (adjust as needed)
threshold = 100
has_significant_change = False
for property, user_count in metrics_dict.items():
existing_count = user_data.get(property, 0)
if abs(existing_count - user_count) > threshold:
user_data[property] = user_count
has_significant_change = True

# Write to file if significant change detected
if has_significant_change:
with open('user_metrics.json', 'w') as outfile:
json.dump(metrics_dict, outfile)
return 1 # Signals significant change
else:
return 0 # Signals no significant change
with open('user_metrics.json', 'w') as outfile:
json.dump(metrics_dict, outfile)


if __name__ == '__main__':
metrics = get_metrics(PORTAL_ID, FOUNDATIONS_ID, COOKBOOKS_ID)
exit_code = write_metrics(metrics)
if exit_code == 1:
print('Significant change detected in user metrics.')
else:
print('No significant change in user metrics.')
get_metrics(PORTAL_ID, FOUNDATIONS_ID, COOKBOOKS_ID)
10 changes: 10 additions & 0 deletions .github/workflows/nightly-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ on:
- cron: '0 0 * * *' # Daily “At 00:00”

jobs:
update-metrics:
uses: automate-metrics.yaml@main

build:
if: ${{ github.repository_owner == 'ProjectPythia' }}
uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main
Expand All @@ -20,3 +23,10 @@ jobs:
uses: ./.github/workflows/sphinx-link-checker.yaml
with:
path_to_source: 'portal'

deploy:
needs: build
uses: ProjectPythia/cookbook-actions/.github/workflows/deploy-book.yaml@main
with:
cname: projectpythia.org
publish_dir: 'portal/_build/html'
3 changes: 3 additions & 0 deletions .github/workflows/publish-site.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ on:
workflow_dispatch:

jobs:
update-metrics:
uses: automate-metrics.yaml@main

build:
uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main
with:
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/trigger-site-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ on:
pull_request:

jobs:
update-metrics:
uses: automate-metrics.yaml@main

build:
uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main
with:
Expand Down
9 changes: 4 additions & 5 deletions .github/workflows/write-metrics-md.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import datetime
import json


Expand All @@ -9,25 +10,23 @@ def process_user_data(user_data_file, markdown_file):
user_data_file: Path to the JSON file containing user data.
markdown_file: Path to the output markdown file.
"""
now = datetime.datetime.now()

with open(user_data_file, 'r') as f:
user_data = json.load(f)

# table_header = '| Portal | Foundations | Cookbooks |\n'
# table_row = f"| {' | '.join([str(user_data[key]) for key in user_data])} |\n"
# table = table_header + table_row

# Write processed data to markdown file
with open(markdown_file, 'w') as f:
f.write('# Metrics \n\n')
f.write('Total Users:\n\n')
for key in user_data:
f.write(f'{key}: {user_data[key]}\n')
f.write('\n')
f.write(f'Last Updated: {now}\n')
f.close()


if __name__ == '__main__':
user_data_file = 'user_metrics.json'
markdown_file = '../../portal/metrics.md'
process_user_data(user_data_file, markdown_file)
print('User data report generated: ', markdown_file)
6 changes: 0 additions & 6 deletions portal/metrics.md
Original file line number Diff line number Diff line change
@@ -1,7 +1 @@
# Metrics

Total Users:

Portal: 0
Foundations: 0
Cookbooks: 0
Loading