UI Overview + UI Chat with LLM (#7)

* UI Overview + UI Chat with LLM

Signed-off-by: DominikKawka <[email protected]>

* UI Skills/Knowledge Contribution + .env & OAuth config

Signed-off-by: DominikKawka <[email protected]>

* ui documentation iteration

Signed-off-by: DominikKawka <[email protected]>

* added contributions workshop/ guide by @mingxzhao

Signed-off-by: DominikKawka <[email protected]>

* fixed hyperlink directory

Signed-off-by: DominikKawka <[email protected]>

* minor document structure change

Signed-off-by: DominikKawka <[email protected]>

---------

Signed-off-by: DominikKawka <[email protected]>

Author: dominikkawka

docs/images/user-interface/ui_custom_model_endpoint.png

@@ -0,0 +1,51 @@
+---
+title: InstructLab UI .env & OAuth config.
+description: Setting up github OAuth and .env file
+logo: images/ilab_dog.png
+---
+
+This chapter is dedicated to the settings for running the User Interface locally.
+
+## .env
+
+```bash
+IL_UI_ADMIN_USERNAME=admin
+IL_UI_ADMIN_PASSWORD=password
+IL_UI_DEPLOYMENT=dev
+
+OAUTH_GITHUB_ID=<OAUTH_APP_ID>
+OAUTH_GITHUB_SECRET=<OAUTH_APP_SECRET>
+
+NEXTAUTH_SECRET=your_super_secret_random_string
+NEXTAUTH_URL=http://localhost:3000
+
+IL_GRANITE_API=<GRANITE_HOST>
+IL_GRANITE_MODEL_NAME=<GRANITE_MODEL_NAME>
+IL_MERLINITE_API=<MERLINITE_HOST>
+IL_MERLINITE_MODEL_NAME=<MERLINITE_MODEL_NAME>
+
+GITHUB_TOKEN=<TOKEN FOR OAUTH INSTRUCTLAB MEMBER LOOKUP>
+TAXONOMY_DOCUMENTS_REPO=github.com/instructlab-public/taxonomy-knowledge-docs
+NEXT_PUBLIC_AUTHENTICATION_ORG=<AUTHENTICATION_ORG>
+NEXT_PUBLIC_TAXONOMY_REPO_OWNER=<GITHUB_ACCOUNT>
+NEXT_PUBLIC_TAXONOMY_REPO=<REPO_NAME>
+```
+
+## OAuth 
+
+To set up the github OAuth settings for the UI, please refer to the steps below
+
+`Github settings` -> `Developer settings` -> `OAuth Apps` -> `Create New OAuth App` 
+
+Fill in the fields as shown below:
+
+![UI OAuth Details](../images/user-interface/ui_oauth_details.png)
+
+Set the `Homepage URL` and the `Authorization Callback URL` to `http://localhost:3000`.
+
+Once the app is created, there will be an option to create a secret. Press the Create Secret button and it will generate a secret for you.
+
+Update the .env files with the new ID and secret ID generated, `OAUTH_GITHUB_ID` = Client ID, `OAUTH_GITHUB_SECRET` = Client Secret
+
+!!! note 
+    If you prefer to not set up the OAuth, we recommend for you to reach out to the UI Maintainers in our `#ui` [slack channel](https://join.slack.com/t/instruct-lab/shared_invite/zt-2kieyqiz9-zhXSxGnXk6uL_f3hVbD53g) , where they will provide details for setting up an OAuth app for the instructlab-public org.

docs/images/user-interface/ui_dashboard_with_submission.png

@@ -0,0 +1,80 @@
+---
+title: InstructLab UI Skills & Knowledge contributions
+description: Steps to contribute to the community model via UI.
+logo: images/ilab_dog.png
+---
+
+The UI Simplifies the process for Skills & Knowledge contributions by:
+
+* Minimising risk of human error when writing YAML by using the web form. 
+
+* Directly submit a github pull request with a press of a button.
+
+When the form is filled out, you also are given the option to download the YAML and attribution files to your local machine, and to view the form in its original YAML structure before submission.
+
+You can view all your submissions on the dashboard page.
+
+!!! warning
+    Even when running the UI locally, you must be logged in via github to successfully submit your Knowledge and Skills contributions. You can still fill out the form, and download the YAML and attribution files.
+
+For tips on writing Skills & Knowledge contributions, please visit the documentation under the [Taxonomy](/taxonomy/) heading.
+
+## Knowledge Contributions
+
+Firstly you will need to find a source document for your knowledge. Accepted sources can be found [here](/taxonomy/knowledge/guide).
+
+Navigate to the Contribute section of the sidebar and click Knowledge. Here you will see the form to contribute Knowledge to the open-source taxonomy tree.
+
+### Author Information
+
+Use your GitHub account email address and full name here. This will make sure that this contribution and the data with it is properly signed off and credited to you.
+
+### Knowledge Information
+
+In the box for the submission summary, give a brief description of what your knowledge is. This will be used in the PR description after you submit. Below that you will fill in the domain the knowledge you are adding would fall under. For example, if you want to teach the model the winners at the 2024 Olympics, you might put "Olympic history" as the domain.
+
+!!! note 
+    The task domain is a critical part of the SDG process and will be part of the prompt when generating synthetic data. Ask your self: "What kind of textbook would contain the knowledge I am trying to teach the model?"
+
+The final box in Knowledge Information will contain an outline of the document.
+
+### Taxonomy Directory Path
+
+Using the dropdown menu, you will be able to select where you think your knowledge will fit the best in the taxonomy tree.
+
+!!! note 
+    Proper placement within the taxonomy tree will allow other users to more accurately locate existing leaf nodes. Ask yourself: "What section of the library would I expect to find my skill under?"
+
+### Seed Examples
+
+Here you will begin filling out your QNA examples that represent the knowledge you are trying to teach. There must be exactly 5 seed examples in this section. Each seed example needs one unique piece of context from your source document with no more than 500 characters. From this piece of context, you will need to create 3 QNA pairs that can be answered from the context you have selected.
+
+### Document Information
+
+You must prepare a markdown file version of the document you wish to use for the knowledge submission. By dragging and dropping the markdown file into the box, and clicking the submit files button, a forked version of the taxonomy repository will be automatically created on your github profile. 
+
+![UI Knowledge Document Information](../images/user-interface/ui_knowledge_document_info.png)
+
+![Forked Repository Showcase](../images//user-interface/ui_knowledge_repo_created.png)
+
+If you've already uploaded the markdown file to your github, you can switch to manually adding the document, and entering the `commit sha`.
+
+![UI Knowledge Document Manual Information](../images/user-interface/ui_knowledge_document_manual_info.png)
+
+!!! note 
+    Your knowledge document must be place in a public GitHub repo in order to be accepted. When your submission is being reviewed and merged, this document will need to be accessible publicly.
+
+
+### Attribution Information
+
+Linking the source you used for information. Wikipedia articles change overtime, make sure to add the oid from the wikipedia article, which you can find by clicking on `View history`, and select the relevant version. 
+
+The license for Wikipedia articles will be "CC-BY-SA-4.0" and Creator Name can simply be "Wikipedia Authors"
+
+## Dashboard
+
+Once you have submitted a Skills or Knowledge Contribution, you can view it on your dashboard, and edit the submission via the UI if needed.
+
+![UI Dashboard With Contribution](../images/user-interface/ui_dashboard_with_submission.png)
+
+[Next Steps](/user-interface/skills_contributions/){: .md-button .md-button--primary }

docs/images/user-interface/ui_knowledge_document_info.png

@@ -0,0 +1,31 @@
+---
+title: InstructLab UI Playground
+description: Steps to set up the playground to chat with a model
+logo: images/ilab_dog.png
+---
+
+To run with a locally run model, make sure that iLab model serve is running in a seperate terminal. If you are unsure on how to do this, please visit the [Intro to serve and chat](/getting-started/serve_and_chat/) section of this document.
+
+If you go to `Playground > Chat` by using the side navigation bar, you can interact with the merlinite and granite models. 
+
+![UI No Model Response](../images/user-interface/ui_no_model_response.png)
+
+If you are running the ui within a dev environment, the model won't reply because a granite/merinite model endpoint hasn't been given. In this case, we will create a new custom model endpoint, using our locally hosted quantised model.
+
+To add a custom model endpoint, go to `Playground > Custom Model Endpoints` and press the `Add Endpoint` button on the right side. 
+
+You will have 3 fields to fill out
+
+* The URL, where your customised model is hosted, if hosting locally, the URL would be `http://127.0.0.1:8000/`
+
+* The Model Name, `merlinite-7b-lab-Q4_K_M.gguf`
+
+* API Key, you may put any text in here; in this case I've used`randomCharacters`. If you are setting up an API key, please provide the key in this section.
+
+![UI Custom Model Endpoint](../images/user-interface/ui_custom_model_endpoint.png)
+
+Go back to the playground chat, select newly added model and chat.
+
+![UI Model Response](../images/user-interface/ui_model_response.png)
+
+[Next Steps](/user-interface/knowledge_contributions/){: .md-button .md-button--primary }

docs/images/user-interface/ui_knowledge_document_manual_info.png

@@ -0,0 +1,67 @@
+---
+title: InstructLab UI Skills & Knowledge contributions
+description: Steps to contribute to the community model via UI.
+logo: images/ilab_dog.png
+---
+
+The UI Simplifies the process for Skills & Knowledge contributions by:
+
+* Minimising risk of human error when writing YAML by using the web form. 
+
+* Directly submit a github pull request with a press of a button.
+
+When the form is filled out, you also are given the option to download the YAML and attribution files to your local machine, and to view the form in its original YAML structure before submission.
+
+You can view all your submissions on the dashboard page.
+
+!!! warning
+    Even when running the UI locally, you must be logged in via github to successfully submit your Knowledge and Skills contributions. You can still fill out the form, and download the YAML and attribution files.
+
+For tips on writing Skills & Knowledge contributions, please visit the documentation under the [Taxonomy](/taxonomy/) heading.
+
+## Skill Contributions
+
+If you have found a missing skill in granite chat, you can create a skill contribution easily through the UI. Remember, a guide to help you learn about what a Skill is can be found in the InstructLab Github
+
+Navigate to the Contribute section of the sidebar and click Skill. Here you will see the form to contribute a Skill to the open-source taxonomy tree.
+
+On this page you will find all the necessary pieces to fill in to create you Skill contribution, let's go through each one [here](/taxonomy/skills/skills_guide).
+
+### Author Information
+
+Use your GitHub account email address and full name here. This will make sure that this contribution and the data with it is properly signed off and credited to you.
+
+### Skill Information
+
+In the box for the submission summary, give a brief description of what your skill is. This will be used in the PR description after you submit. Below that you will fill in a detailed description of what your skill is teaching the model to do. For example, if your skill is extracting the date from a bean count ledger, you would fill in "Teaching the model to extract the date from a bean count ledger"
+
+!!! note 
+    The detailed description is very important for quality synthetic data generation. What you fill in for this box will be used as prompting during the SDG process. Imagine you are telling the teacher model: "The task is (your description here)"
+
+### Taxonomy Directory Path
+
+Using the dropdown menu, you will be able to select where you think your skill will fit the best in the taxonomy tree. If your skill is a grounded skill (meaning that it requires context to answer) you will need to navigate into the "grounded" section of the taxonomy tree before selecting the proper subsection. Otherwise, select the section you think best represents the subject of your skill.
+
+!!! note 
+    Proper placement within the taxonomy tree will allow other users to more accurately locate existing leaf nodes. Ask yourself: "What section of the library would I expect to find my skill under?"
+
+### Seed Examples
+
+Here you will begin filling out your QNA examples that represent the skill you are trying to teach. You will need a minimum of 5 seed examples, which each example consisting of a Question and Answer. If your skill is a grounded skill and requires context, you will also add context for each QNA pair here.
+
+!!! note 
+    The QNA pairs that you create here should be diverse. You should try to rephrase questions in different ways and create unique contexts. For example, in the case of date extraction from bean count, you do not want to use the same date repeatedly in your seed examples, as this may have the unintended consequence of teaching the model to simply regurgitate the same date when asked any date extraction question.
+
+### Attribution
+
+The information filled in this section will create the attribution.txt file needed for your submission. Fill in a concise title for your skill. If your skill is entirely self created, you can put your name as the creator and use Apache-2.0 as the license. Otherwise please refer to your source material for creator and licensing information.
+
+Once you have filled in all the necessary information, you can finally submit your skill! If you would like to download or view the qna.yaml or attribution.txt, there are drop down menus provided for both. If you choose to, you can download these files to create a PR in the taxonomy repository yourself. Otherwise, you can press submit and the UI will automatically create a PR under your GitHub namespace in the InstructLab Taxonomy repository.
+
+## Dashboard
+
+Once you have submitted a Skills or Knowledge Contribution, you can view it on your dashboard, and edit the submission via the UI if needed.
+
+![UI Dashboard With Contribution](../images/user-interface/ui_dashboard_with_submission.png)
+
+[Next Steps](/user-interface/env_oauth_config/){: .md-button .md-button--primary }

docs/images/user-interface/ui_knowledge_repo_created.png

@@ -0,0 +1,107 @@
+---
+title: InstructLab UI Overview
+description: The overview of 🐶 InstructLab UI.
+logo: images/ilab_dog.png
+---
+
+The InstructLab User Interface (UI) is an additional component in the InstructLab project to make Knowledge/Skills 
+contributions more accessible to the general user. Instead of having to edit a `qna.yaml` file,
+users can fill out the contribution forms on the web. The user can also view all of their contributions in a
+centralised list. You can also chat with the models directly on the playground.
+
+There are 2 ways to access the UI:
+
+1. Visit the website at [ui.instructlab.ai](https://ui.instructlab.ai/).
+
+2. Install and run the UI locally.
+
+
+!!! note
+    To log into the UI and submit Knowledge and Skills contributions, you must be a member of the [instructlab-public](https://github.com/instructlab-public) github repository.
+
+!!! note
+    If you aren't a member of the repository, but still wish to experiment with the UI, we recommend you to reach out to the UI Maintainers in our `#ui` [slack channel](https://join.slack.com/t/instruct-lab/shared_invite/zt-2kieyqiz9-zhXSxGnXk6uL_f3hVbD53g) for an invitation.
+    
+    If you wish to not join but still wish to experiment, download it locally.
+
+## Prerequisites 
+
+Before you can run the User Interface locally, you first must: 
+
+* Have a [github](https://github.com/) account
+
+* Have Node.js installed
+
+!!! warning
+    This guide was written with `Node.js v20.17.0`. If you are using an older version of Node.js, please be aware that the application
+    may not work as intended. 
+
+## Installation Guide
+
+1) Download the InstructLab UI by running the following command in your desired directory:
+
+`git clone https://github.com/instructlab/ui`
+
+You can also download by visting the [InstructLab UI github page](https://github.com/instructlab/ui) and clicking the code, and then the download button.
+
+2) Set up the .env file. Go into the UI folder by running
+
+`cd ui`
+
+Inside the UI folder, create a new file called `.env` and paste the following content inside:
+
+```bash
+IL_UI_ADMIN_USERNAME=admin
+IL_UI_ADMIN_PASSWORD=password
+IL_UI_DEPLOYMENT=dev
+
+OAUTH_GITHUB_ID=<OAUTH_APP_ID>
+OAUTH_GITHUB_SECRET=<OAUTH_APP_SECRET>
+
+NEXTAUTH_SECRET=your_super_secret_random_string
+NEXTAUTH_URL=http://localhost:3000
+
+IL_GRANITE_API=<GRANITE_HOST>
+IL_GRANITE_MODEL_NAME=<GRANITE_MODEL_NAME>
+IL_MERLINITE_API=<MERLINITE_HOST>
+IL_MERLINITE_MODEL_NAME=<MERLINITE_MODEL_NAME>
+
+GITHUB_TOKEN=<TOKEN FOR OAUTH INSTRUCTLAB MEMBER LOOKUP>
+TAXONOMY_DOCUMENTS_REPO=github.com/instructlab-public/taxonomy-knowledge-docs
+NEXT_PUBLIC_AUTHENTICATION_ORG=<AUTHENTICATION_ORG>
+NEXT_PUBLIC_TAXONOMY_REPO_OWNER=<GITHUB_ACCOUNT>
+NEXT_PUBLIC_TAXONOMY_REPO=<REPO_NAME>
+```
+
+As of now, we are only concerned with `IL_UI_ADMIN_USERNAME`, `IL_UI_ADMIN_PASSWORD` and `IL_UI_DEPLOYMENT`. But throughout this UI guide we will be visiting the .env frequently. 
+
+__Save and close the .env file.__
+
+3) Running Node commands to install and run.
+
+Back inside the UI folder, run this command to install needed files for the UI to run:
+
+`npm install`
+
+!!! note
+    Depending on your internet speed and connectivity, this process may take a few minutes.
+
+When the install is finished, start the UI by running:
+
+`make start-dev-local` 
+
+The UI should now be up and running, you can visit it by entering `localhost:3000` in your browser, where it should bring you to the login screen.
+
+![UI Login Screen](../images/user-interface/ui_login_screen.png)
+
+You may stop the UI at any time by running:
+
+`make stop-dev-local`
+
+4) Logging in 
+
+For now, we will log in by entering `admin` and `password` in the username and password fields respectively. You can change the username and password by editing the `IL_UI_ADMIN_USERNAME` and `IL_UI_ADMIN_PASSWORD` values in the .env file.
+
+We are not able to log in with github right now since we haven't set up the github token in the .env file. When logged in, you may see a popup saying that the UI is fetching your submissions, exit out of this notification. If you wish to set up the OAuth, visit the [.env and OAuth config page](/user-interface/env_oauth_config/).
+
+[Next Steps](/user-interface/playground_chat/){: .md-button .md-button--primary }

docs/images/user-interface/ui_login_screen.png

@@ -34,6 +34,12 @@ nav:
       - Knowledge Overview: taxonomy/knowledge/index.md
       - Knowledge Guide: taxonomy/knowledge/guide.md
       - Knowledge Contribution Details: taxonomy/knowledge/contribution_details.md
+  - User Interface:
+      - UI Overview: user-interface/ui_overview.md
+      - Chat with the model: user-interface/playground_chat.md
+      - Contribute knowledge to community model: user-interface/knowledge_contributions.md
+      - Contribute skills to community model: user-interface/skills_contributions.md
+      - UI Configurations: user-interface/env_oauth_config.md
   - Community Model Build:
       - About Community Model Build: cmb/index.md
       - Community Model Build Process: cmb/build_process.md