Feature: Enhance Analaytics module to fetch Ontology visits, User visits, and the Most visited pages in the month #17

syphax-bouazzouni · 2023-10-09T09:37:24Z

Changes

Upgrade our code to the latest release of NCBO: https://github.com/ncbo/ncbo_cron/releases/tag/v5.26.0
- Add an option to not generate the annotator dictionary on each ontology parsing
- update ncbo_ontology_archive_old_submissions script
- update ncbo_ontology_import script, to create contacts if inexistent
Update the analytics module to use the new version of the Google Analytics client (starting 2023), this needs a new configuration file

 # Google Analytics GA4 config
  config.analytics_path_to_key_file          = "config/your_analytics_key.json"
  config.analytics_property_id                  = "123456789"
  config.analytics_path_to_ga_data_file = "data/your_ga_data.json"  # will be created if not-existent 
  config.analytics_start_date                    = '2015-01-01' # since when we want to fetch Google Analytics

Update the Analytics module, to fetch and save, in addition to Ontology visits, now also User visits, and the Most visited pages in the previous month.
Add a script to import and save Google Analytics UA (2013-2023)

- create contact instance if it doesn't exist - changed --from-api to --from-apikey - minor linting

…puller-script Add a script to pull a new version of an ontology on demand

automatically create contact in ontology import script

…cessing (and vise versa)

Resolves warning messages when we exclude .git directory from docker image

+ AG v8

…-v5.26.0

…m-v5.26.0

codecov · 2023-12-25T19:08:17Z

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (development@987fba3). Click here to learn what that means.

Additional details and impacted files

@@              Coverage Diff               @@
##             development      #17   +/-   ##
==============================================
  Coverage               ?   55.05%           
==============================================
  Files                  ?       20           
  Lines                  ?     1475           
  Branches               ?        0           
==============================================
  Hits                   ?      812           
  Misses                 ?      663           
  Partials               ?        0

Flag	Coverage Δ
unittests	`55.05% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…migration

…ode clean of it

…its, and the Most visited pages in the month (#17) * remove forgot variables * fix for #61 - create contact instance if it doesn't exist - changed --from-api to --from-apikey - minor linting * Restore branch specifier to develop * Optimization - remove repeated query * Gemfile.lock update * Gemfile.lock update * Gemfile.lock update * Gemfile had references to develop branch * implemented #64 - ability to generate labels independently of RDF processing (and vise versa) * Gemfile.lock update * fixed a bug in #64 * Relocate docker-compose file and update default configs * Add GH workflow for publishing docker images * use ruby native method for listing files instead of a git function Resolves warning messages when we exclude .git directory from docker image * remove comment * capitalize argument in order to be consistent with other scripts * add arm/64 platform * additional error handling for SPAM deletion script, #60 * additional error handling for SPAM deletion script, #60 * implemented #67 - improved corrupt data and error handling * Gemfile.lock update * exclude test/data/dictionary.txt from git commits * update version of solr-ut * Gemfile.lock update * Restore branch specifier to master * fixed configuration for the analytics module * Gemfile.lock update * implemented #69 - scheduled annotator dictionary file generation should be a configurable option instead of the default * Gemfile.lock update * gem update * create new rake taks for updating purls for all ontologies moved from ontologies_api/fix_purls.rb * initial implementation of #70 - Google Analytics v4 Update Compatibility Issue * added the /data folder to ignore * update gems * Gemfile.lock update * Gemfile.lock update * Gemfile.lock update * use patched version of agraph v7.3.1 * unpin faraday gem * A chnage to reference Analytics Redis from LinkedData block * Gemfile.lock update * Gemfile.lock update * Gemfile.lock update * Gemfile.lock update * use assert_operator instead of assert minitest style guide adherence. encountered an intermittent unit test failure so assert_operator will provide better failure feedback than assert * use local solr to pass the tests * fixed ncbo_ontology_archive_old_submissions error output * Gemfile.lock update * Gemfile.lock update * Gemfile update * Gemfile update * fixes to the analytics script and a new script to generate UA analytics for documentation * Gemfile.lock update * Gemfile.lock update * implemented the first pass at bmir-radx/radx-project#37 * implemented the first pass at bmir-radx/radx-project#37 * set bundler version to be comptatible with ruby 2.7 + AG v8 * refactor ontologies analytics job to handle the new google analytics migration * add user analytics fetching the monthly user visits count * add page visits analytics fetching last month most visited pages * extract google analytics UA import code to a script to make current code clean of it * add option to force submission archiving even if already archived --------- Co-authored-by: Alex Skrenchuk <[email protected]> Co-authored-by: mdorf <[email protected]> Co-authored-by: Jennifer Vendetti <[email protected]>

syphax-bouazzouni and others added 30 commits December 5, 2022 10:03

remove forgot variables

8535b30

fix for #61

f7aa1be

- create contact instance if it doesn't exist - changed --from-api to --from-apikey - minor linting

Merge pull request #59 from ontoportal-lirmm/pr/feature/add-ontology-…

d1f8aa7

…puller-script Add a script to pull a new version of an ontology on demand

Restore branch specifier to develop

f03b2aa

Optimization - remove repeated query

e383b54

Merge pull request #62 from ncbo/fix_contact_in_ontology_import_script

c496e7e

automatically create contact in ontology import script

Gemfile.lock update

a0297f2

Gemfile.lock update

99f389c

Gemfile.lock update

2dfd9da

Gemfile.lock update

7f0589e

Merge staging for release

6d0b1c8

Gemfile had references to develop branch

6db9916

implemented #64 - ability to generate labels independently of RDF pro…

f47a07b

…cessing (and vise versa)

Gemfile.lock update

dbd4060

fixed a bug in #64

5c6cf91

Relocate docker-compose file and update default configs

c09d76e

Merge branch 'master' into develop

9317cbd

Add GH workflow for publishing docker images

6c6a5c8

use ruby native method for listing files instead of a git function

1988608

Resolves warning messages when we exclude .git directory from docker image

remove comment

ed26123

capitalize argument in order to be consistent with other scripts

a366805

add arm/64 platform

6ec1174

additional error handling for SPAM deletion script, #60

a461322

additional error handling for SPAM deletion script, #60

3b9fdb0

implemented #67 - improved corrupt data and error handling

030930c

Gemfile.lock update

3308da0

Gemfile.lock update

cd659f3

exclude test/data/dictionary.txt from git commits

873d829

update version of solr-ut

419c518

Gemfile.lock update

94eed8d

Gemfile.lock update

ab43cdc

syphax-bouazzouni force-pushed the development branch from 3217d72 to 987fba3 Compare December 16, 2023 22:53

mdorf and others added 6 commits December 16, 2023 17:39

implemented the first pass at bmir-radx/radx-project#37

a659415

Gemfile.lock update

ad283a2

implemented the first pass at bmir-radx/radx-project#37

2331631

set bundler version to be comptatible with ruby 2.7

07107b1

+ AG v8

Merge remote-tracking branch 'upstream/master' into merge-to-upstream…

9e5ea99

…-v5.26.0

Merge remote-tracking branch 'upstream/develop' into merge-to-upstrea…

f314da7

…m-v5.26.0

syphax-bouazzouni added 4 commits December 27, 2023 13:31

refactor ontologies analytics job to handle the new google analytics …

bada7d2

…migration

add user analytics fetching the monthly user visits count

08f5e42

add page visits analytics fetching last month most visited pages

6f21df6

Merge branch 'development' into merge-to-upstream-v5.26.0

524abca

syphax-bouazzouni force-pushed the merge-to-upstream-v5.26.0 branch 5 times, most recently from ff4327d to 197a228 Compare December 27, 2023 16:37

extract google analytics UA import code to a script to make current c…

080629e

…ode clean of it

syphax-bouazzouni force-pushed the merge-to-upstream-v5.26.0 branch from 197a228 to 080629e Compare December 27, 2023 16:46

add option to force submission archiving even if already archived

1154c7c

syphax-bouazzouni force-pushed the merge-to-upstream-v5.26.0 branch from d0e7f1e to 1154c7c Compare December 27, 2023 17:15

syphax-bouazzouni changed the title ~~Merge to upstream v5.26.0~~ Feature: Enhance Analaytics module to fetch Ontology visits, User visits, and the Most visited pages in the month Dec 27, 2023

syphax-bouazzouni merged commit 9cb209d into development Dec 27, 2023
4 checks passed

syphax-bouazzouni mentioned this pull request Dec 27, 2023

Fix: Google analytics new version and add users analytics ontoportal-lirmm/ontologies_linked_data#118

Merged

This was referenced Dec 28, 2023

Feauture: Update admin page and fix google analytics ontoportal-lirmm/bioportal_web_ui#422

Merged

Merge to master: Release v2.2.1 #18

Merged

Fix: object analytics fetching after the new year change #19

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: Enhance Analaytics module to fetch Ontology visits, User visits, and the Most visited pages in the month #17

Feature: Enhance Analaytics module to fetch Ontology visits, User visits, and the Most visited pages in the month #17

syphax-bouazzouni commented Oct 9, 2023 •

edited

Loading

codecov bot commented Dec 25, 2023 •

edited

Loading

Feature: Enhance Analaytics module to fetch Ontology visits, User visits, and the Most visited pages in the month #17

Feature: Enhance Analaytics module to fetch Ontology visits, User visits, and the Most visited pages in the month #17

Conversation

syphax-bouazzouni commented Oct 9, 2023 • edited Loading

Changes

codecov bot commented Dec 25, 2023 • edited Loading

Codecov Report

syphax-bouazzouni commented Oct 9, 2023 •

edited

Loading

codecov bot commented Dec 25, 2023 •

edited

Loading