3. Using enKORE processor in Toolforge

3.1. SSH into Toolforge

To support projects in Toolforge, you must create an account. Therefore, to sign up or login: https://toolsadmin.wikimedia.org/auth/login/?next=/

Example: account login: USERNAME123 / UNIX shell username: username123

You must create your SSH key (Read), submit your username to join (Here) and wait for approval (See list).

Regarding your SSH key, copy the content from the file keyname.pub into the repository for keys in Toolforge (see right image), and the file keyname is just repassed during the connection.

Note: You can only connect when your submission to join is approved, otherwise, connection requests will be denied.

Example:

$ ssh -i ~/Downloads/path2key/toolforgekey [email protected]

Note: When using VPN or some external wifi-network, then check for IP blockage. Your username may be fine in the system, but your outside IP may be blocked. Therefore, any attempt at connection is not even possible.

Get your IP: https://whatismyipaddress.com/

Access your IP: https://whatismyipaddress.com/blacklist-check, which will inspect your IP across a few platforms.

Note: Avoid using an institutional email that may be changed. For online support if you have issues, then consider entering the link https://wikitech.wikimedia.org/wiki/Portal:Toolforge and click on (connect), which will bring to https://web.libera.chat/?#wikimedia-cloud. Then you can talk about this matter and see if anyone can help.

3.2. Running enKORE-App in Toolforge

To run the code directly into Toolforge, consider the following: you must be added to the enkore group, ssh into toolforge (See 3.1), and the following commands

$ become enkore

$ cd Data/enKORE-corpus-processor

$ git pull

$ git checkout -b temp-branch-name (optional, makes it easier to discard if messed up)

$ deno task process (/data/project/enkore/.deno/bin/deno task process)

Note: Current settings of ~/.bashrc:

add local bin folder, export PATH="/data/project/enkore/.local/bin/:$PATH"

add Deno - https://deno.land/

export DENO_INSTALL="/data/project/enkore/.deno" export PATH="$DENO_INSTALL/bin:$PATH" $ git diff (to review) $ git add . $ git commit -am "manual corpus process ... " $ git checkout main $ git merge temp-branch-name $ git push

3.3. Creating a Toolforge App for the first time

To deploy a simple Pywikibot tool consider https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Pywikibot_tool#Access_your_Pywikibot_tool

Note: Basic test using Toolforge with "crossref-dump"

(1) host a copy of https://academictorrents.com/details/4dcfdf804775f2d92b7a030305fa0350ebef6f3e

(2) makes it accessible to other Toolforge tools (e.g. enkore, scholia) in order to avoid intensive stress over the server of the CrossrefAPI. Such a tool would have multiple uses in Scholia and related wiki-ecosystems. The size may require some special provisions.