Proposal: Korp frontend plugin facility

[janiemi]

Preface

This issue tries to describe the plugin facility I have implemented for the Korp frontend used in the Language Bank of Finland and that I’d propose as the basis for a plugin facility to be included in main Korp frontend code. All feedback on the proposal is welcome.

Disclaimer: I have virtually no prior experience in plugin architectures, and also my knowledge of the Web technologies used in the Korp frontend – such as AngularJS, Webpack and Pug – is almost solely based on the Korp frontend and on what I have learned when modifying it, so I hope I’m not on a completely wrong track with this plugin facility. I also don’t know how future-proof this plugin approach is if or when Korp is eventually ported to Vue.js. The features of the plugin facility reflect the modifications we’ve made to Korp for the Language Bank of Finland, so something might be implemented in a too specific way or be completely missing. Thus, I’d be glad in particular if you pointed out if something in the plugin facility should definitely be done differently.

The code for the plugin facility is currently on top of Språkbanken’s Korp frontend code at commit 46374ad2 of 2022-02-02, but I’ll rebase it on top of the up-to-date code in the near future. (Mechanical Git rebasing won’t be enough; instead, the code will have to be modified by hand. Some plugins may have to be completely rewritten and some may become obsolete.)

The plugin facility is currently documented only in source code comments, so this issue description serves as documentation for the moment. I’ll eventually add some separate documentation, based on the comments and this issue.

I’m sorry that this is rather long for a GitHub issue description.

Korp frontend plugin facility (proposal)

Overview

The aim of the Korp frontend plugin facility is to make it easier to tailor Korp for different sites without having to modify main Korp code. To make this possible, the main Korp code needs some support for plugins and callback hook points in appropriate places in the JavaScript code.

In addition to JavaScript code, Korp frontend plugins may contain Pug code for content, (S)CSS stylesheets and JSON translation files.

Plugin locations

Plugins are searched from three places, in this order:

1. ${configDir}/plugins/: configuration-specific plugins
2. ${pluginDir}/: separately distributed, general-purpose plugins
3. app/plugins/: plugins possibly distributed along with the main Korp frontend code

configDir is the Korp frontend configuration directory specified in run_config.json, and pluginDir is specified there similarly. pluginDir could typically be top_dir/app/plugins.

Plugin structure

Each plugin should have its own subdirectory under a plugin directory. The subdirectory structure of each plugin should follow that of the Korp frontend code:

• scripts/: JavaScript scripts
• styles/: (S)CSS stylesheets
• includes/: Pug files (snippets) to be included
• translations/: JSON translation files

JavaScript plugins

Plugin objects and callbacks

The implementation of the JavaScript class Plugins for plugins is in plugin.js. Plugins are managed through the global window.plugins object.

Korp frontend JavaScript plugins are objects containing function properties which correspond to plugin callback functions for named hook points. A plugin can be either a simple object or an object created from a prototype. Each plugin must be registered with plugins.register(plugin).

Callback functions in a plugin object may be mapped to hook points in two ways:

1. If the plugin contains property callbacks, the keys of callbacks are regarded as hook point names and their values are the callback functions in the plugin to be registered for the hook point. The values may be either single functions (function references), lists of them or strings (names of function properties in the plugin).
2. If the plugin does not contain property callbacks, all function properties whose names do not begin or end with an underscore in the plugin and its direct prototype are added as callbacks, taking the function name as the name of the hook point.

Calling callbacks

Callback functions for a hook point are called via plugins.callActions (for functions not returning a value or whose return value is discarded) or plugins.callFilters (functions returning a value):

• callActions(hookPoint, ...args): Call plugin callback functions (actions) registered for hookPoint (a string), with the (optional) arguments ...args. The possible return value of the functions is ignored.
• callFilters(hookPoint, arg1, ...rest): Call plugin callback functions (filters) registered for hookPoint, with the argument arg1 and optional ...rest Each callback function gets as its arg1 the return value of the previous function. This function returns the value returned by the last callback function.

Callback call order

Plugin callbacks are called in the order in which the plugins have been registered. A rudimentary way of controlling the registration order is to specify arrays providesFeatures and requiresFeatures in the plugin objects: if a plugin has a value (string) in requiresFeatures that has not yet appeared in the providesFeatures of a registered plugin, registering it is deferred until a plugin providing the feature has been registered.

Enabling and disabling plugins

If a plugin has the property name (a string), it can be explicitly enabled or disabled by including the name in the array settings.pluginsEnabled or settings.pluginsDisabled, respectively. If settings.pluginsEnabled is defined, only the plugins listed in it are enabled.

A plugin can also be disabled by setting its property disabled to true in the plugin class constructor. This can be used to disable a plugin if the configuration does not contain settings required by the plugin, for example.

A plugin may contain method initialize, which is called (without arguments) only for enabled plugins, unlike constructor, which is called at object creation time, regardless of whether the plugin is enabled or not.

JavaScript plugin callback hook points

The following JavaScript plugin hook points are currently defined, but more can be added as needs arise, and some may become obsolete because of changes in Korp:

• Actions (called via plugins.callActions):

  • beforeShowLogin: Act before actually showing the login modal, e.g., to handle a SSO login.
  • finishLogout: Act on logout.
  • onAuthRequestDone: Act based on the data returned by the authentication proxy, giving access to the scope.
  • modifyHeaderController: Modify the header controller, giving access to the scope
  • modifySidebarContent: Modify sidebar content, giving access to the controller.
  • beforeLoadApp: Act before loading the actual application, passing some data (currently jStorage) that may be used or modified.
  • modifyLocation: Modify location URL at the very beginning.
  • modifyCorpusConfigs: Modify settings.corpora and settings.corporafolders.
  • modifyLocationOnDomReady: Modify location URL after the DOM is ready.
  • onDomReady: Act when the DOM is ready.
  • onCorpusChooserChange: Modify the corpus chooser.
  • modifyCorpusConfigsList: Modify settings.corpusListing.corpora.
  • modifyCorpusFolderConfigs: Modify settings.corpusfolders.

• Filters (called via plugins.callFilters):

  • filterLoginNeededFor: Filter (or otherwise act on) the list of the corpora for which login is needed.
  • filterModalParams: Modify modal parameters.
  • formatSidebarCorpusInfo: Format corpus information to be displayed in the sidebar, based on corpus object.
  • filterLoginInfo: Filter login object, possibly modifying it based on the data returned by the authentication proxy.
  • filterCorpusChooserSingleSelectedCorpusName: Filter the name of a single selected corpus, typically to remove HTML formatting or added information.
  • formatPopupFolderTitle: Format the folder title shown in the corpus info popup and in the corpus chooser, based on folder configuration.
  • formatPopupFolderInfo: Format the folder description shown in the popup, based on folder configuration.
  • formatCorpusChooserCorpusTitle: Format the corpus title for the corpus chooser.
  • formatPopupCorpusInfo: Format the corpus description shown in the corpus info popup, based on corpus configuration.
  • formatPopupCorpusTitle: Format the corpus description shown in the popup, based on corpus configuration.

AngularJS module in a plugin

A plugin can also create an AngularJS module by calling registerAngularModule(name, ...args), which creates and returns an AngularJS module name with args and registers it to be added as a depencency to the Korp app. Plugins should typically use registerAngularModule instead of angular.module. registerAngularModule should be called from the initialize method of a plugin to allow enabling and disabling the plugin. This approach was inspired by https://stackoverflow.com/a/17944566

Example

An example of a simple plugin for modifying the template (HTML) of the “About Korp” modal (original code):

// A callback method for modifying the template (HTML) of the "About
// Korp" modal by calling the function settings.aboutTemplateModifier.

// Plugin class
class AboutModifier {
    constructor () {
        // Plugin name
        this.name = "about_modifier"
        // Disable the plugin if settings.aboutTemplateModifier is not defined
        this.disabled = (settings.aboutTemplateModifier == undefined)
    }
    // Callback method called at a hook point
    // For the "about" modal, call settings.aboutTemplateModifier on
    // params.template and return the result
    filterModalParams (params, modalName) {
        if (modalName == "about") {
            params.template = settings.aboutTemplateModifier(params.template)
        }
        return params
    }
}

plugins.register(new AboutModifier())

Pug includes

To allow plugins to modify the HTML content of the Korp frontend, they may contain Pug snippets. To make that work, a Pug plugin is defined in webpack.common.js to extend the include directive to allow searching for include files in multiple directories and to allow ignoring non-existent includes.

To enable the non-default include behaviour of Pug, the filename of the include directive needs to be prefixed with options between two colons, separated by commas; for example: include :search,optional:includes/file. The following options are recognized:

• search: Search all (plugin) paths for the file and include the first one found, unless the option all is also specified.
• all: Include all the files found in the order of paths (only in conjunction with search).
• optional: No error if no file is found.

The paths to be searched for includes are currently the following, in this order (** denotes subdirectories recursively):

• ${configDir}/
• ${configDir}/plugins/**
• ${pluginDir}/**
• app/plugins/**
• app/

The possible directory in the include directive is included in the path. Moreover, if the extended include is used in a file in a subdirectory, the subdirectory is included in the path from which to search includes. For example, if the file app/includes/header.pug contains include :search:main_menu, it is searched for in the includes subdirectories of the paths listed above.

Currently, the code uses the extended include to include two files in app/includes/header.pug:

• top_header (optional, include all): An optional “top header” placed above all other elements; for example, a banner.
• main_menu: Allows overriding the main menu from a plugin or a site configuration.

More uses of this feature are expected: for example, a plugin could add extra search options.

Stylesheets

Plugin stylesheets are loaded in index.js from .css and .scss files in the following directories, in this order:

• app/plugins/**
• ${pluginDir}/**
• ${configDir}/plugins/**

JSON translation files

Plugins also need to support JSON translation files, as they may contain items that should be localized or they may override default translations. The following locations are searched for locale*.json files

• app/translations/
• app/plugins/**/translations/
• ${pluginDir}/**translations/
• ${configDir}/translations/
• ${configDir}/plugins/**/translations/

The files for each locale are then merged, translations from later files overriding those from earlier ones.

Implemented plugins

The plugins currently used in the Korp of the Language Bank of Finland are available in the branch plugins/master of our Korp frontend fork. (This branch is a merge of individual plugin branches.) Please note that some of the plugins may be obsoleted by the move of corpus configurations to the backend, and most if not all plugins will need to be updated for the most recent version of Korp.

The current plugins are the following:

• about_modifier: Modify the template (HTML) of the “About Korp” modal.
• config_augment_info: Augment corpus (and folder) information based on site-configurable properties in the corpus (folder) configuration.
• config_corpus_features: Initialize properties in settings.corpora.* based on their property features and the properties in settings.corpusFeatures.
• config_corpusinfo_copier: Copy (extra) corpus information in the info property of a corpus configuration to top-level properties to propagate the information in a corpus folder to its corpora.
• config_corpus_settings_modifier: Allow corpus and corpus folder modifications to be defined in the configuration in functions settings.modifyCorpusSettings and settings.modifyFolderSettings.
• config_licence_category: Add properties licenceType and limitedAccess to corpus configurations based on licence name.
• config_logical_corpora: Add properties logicalCorpus and corpusType to corpus configurations and info.folderType to corpus folder configurations.
• config_sidebar_order: Initialize the order property of all attributes of all the corpora in settings.corpora based on settings.defaultSidebarDisplayOrder and the corpus-specific sidebarDisplayOrder properties.
• config_url_opts: Add a pattern property for rendering a link to attribute configurations with the appropriate urlOpts property.
• corpus_aliases: Modify the location (parameter corpus), mapping possible corpus id aliases (defined in settings.corpusAliases) to actual corpus ids.
• corpuschooser_item_formatter: Allow formatting corpus and folder titles in the corpus chooser according to the type of corpus or folder by formatting functions defined in settings.formatCorpusChooserItem.
• corpuschooser_prompt_empty: Add a prompt (“Select corpora”) to the top frame of the corpus chooser if no corpora are selected.
• corpusinfo_formatter: Format (extra) corpus information in the corpus information popup and sidebar.
• news_banner: Implement an AngularJS module for a closeable news banner for important news (based on Korp’s newsdesk.js).
• shibboleth_auth: Implement authentication and authorization via Shibboleth.
• sidebar_link_section: Initialize the linkAttributes property in corpus configurations and render it in the sidebar.

Missing features (to-do)

This plugin proposal lacks many features, at least the following:

• Allow specifying the order in which individual callbacks are called, as callback A of plugin X might need to be called before that of plugin Y, but callback B of plugin Y before that of plugin X.

• When including the contents of all found Pug files at a certain point, sort them somehow. A way to do this could be to suffix file names with a double-digit number: include :search,all:file would include file-10.pug and file-20.pug, in this order. Another option could be to list names (or patterns) for paths in the order in which they should be considered as an argument to option all; for example, include :search,all=plugin1+plugin2; non-matching ones would be included in an arbitrary order after the matching ones (or maybe optionally not included).

[janiemi]

[I moved the previous content of this comment to https://github.com/spraakbanken/korp-backend/issues/14#issuecomment-2220687551 as it related to Korp backend plugins, not frontend ones, and I had added it here only mistakenly.]