spawn Shiny app process as a subprocess rather than a worker

README.md

@@ -16,7 +16,6 @@ Built with simplicy and security in mind, it has some features we love -- and ho
 
 These versions are currently being used to develop Concierge, lower versions may work but are untested. See [Known Issues](#known-issues) for an older Docker Compose command which might work for you if you're unable to upgrade.
 
-- **Python >= 3.12** (note: for all commands in documentation we will call the executable `python`. On your system you may need to use `python3`, you can use the `python-is-python3` package to configure the `python` command on Linux). Check with `python --version`.
 - **Docker >= 25.0.3** (currently the vector database and natural language response engine are running in docker containers). Check with `docker --version`.
 - **Docker Compose >= 2.24.6** (while frequently installed with docker, sometimes it's not. Docker compose files are how the docker containers are setup). Check with `docker compose version`.  
 
@@ -40,28 +39,15 @@ You should perform the following system configuration steps according to your Op
 
 ## Setup
 
-The provided install scripts perform a lot of automated cleanup and ensure that all the components are configured correctly to work together. However as this is an open source project you could create your own installation using the contents of this repository.
+Concierge now has a visual configurator which we hope you will like a lot more than the previous text-based one. Just go to the release you wish to install and download the executable for your Operating System. Please let us know if you're using a different Operating System, it may be possible for us to make a build for it. You can also use the development version.
 
-You no longer need to clone this repository to run Concierge.
+Launch the executable and visit the address indicated in your web browser. Select your desired options and click "Start Installation!".
 
-> [!TIP]
-> Pay attention to the use of dashes and underscores in the commands!
+Note that the installer launches all the requirements for you, you generally won't need to use the "Launch Concierge" button. Docker will keep everything running for you and you will be able to access Concierge without going via the Configurator once installed.
 
-Run the following while not being in a virtual environment. This command will create the virtual environment and run the installer from within it.
-
-`python -m pip install launch-concierge`
-
-`python -m launch_concierge.install`
-
-Answer the questions and then the installer will ask if you are ready to make changes to the system.  
-Answer "yes" and let the installation begin!
-There may be additional questions during the installation depending on the options you selected.
-
-Please note that this package isn't the Concierge app itself, it's just a utility that helps you configure the environment and launch the correct Docker Compose file based on your choices, so it shouldn't be hugely risky to download it without using a virtual environment.
-
-### Reinstalling
+## Usage
 
-The above commands also allow you to remove an existing version of Concierge before installing. At the start of the installer if you're changing any settings or need to remove the existing services, it's generally recommended to remove all services except for Ollama, which you can usually keep as it's configured the same way regardless of your Concierge settings, and redownloading the model can take a long time.
+Once you have completed the installation process, Concierge will be running on localhost:15130 or the host and port you selected during setup. It can take a couple of minutes for the containers to be ready after install or relaunch.
 
 ## Configuring Authentication and Authorization
 
@@ -104,36 +90,44 @@ You will now be able to use the username and password you created to log into th
 
 If you need to switch user you may need to revoke the session of the currently logged in user, which is available in the "Sessions" tab for that user, otherwise clicking the login button may just automatically log you in with the same user again.
 
-## Usage
-
-Once you have completed the installation process above, Concierge will be running on localhost:15130 or the host and port you selected during setup. It can take a couple of minutes for the containers to be ready after install or relaunch.
-
-## Update to new release
-
-If running a version prior to 0.3.0 you should delete the files you cloned from the repository (unless you're installing the development environment), remove the related Docker containers and proceed with a fresh install following the instructions above.
-
-`python -m pip install launch-concierge --upgrade`
+## Setup: development environment
 
-`python -m launch_concierge.install`
+### Dependencies
 
-This will make sure to grab the latest version of Concierge, and re-run the installer
+The ones listed at the top of this page, and:
+    - **Python 3.12** the application code is in Python so unless you intend to only run inside Docker, you'll need this
+    - **Bun 1.2.1** the configurator is written in JavaScript that depends on the Bun runtime. Bun allows us to effortlessly build the executable files for each Operating System.
 
-## Setup: development environment
+### Installation
 
 git clone repo or extract zip. 
 
 `cd concierge` go into the cloned project directory.
 
-You should not create a virtual environment as the script below will handle it for you.
+You should not create a virtual environment as the installer will handle it for you.
+
+`cd bun_installer` to go into the subdirectory containing all the Bun scripts for the configurator.
 
-`python install_dev.py` to launch the installer (same steps as the user installer).
+`bun install` to add all dependencies.
+
+`bun run dev_install` will launch the web server in dev mode which provides additional install and launch options compared to the distributed executable. Once the server is running go to http://localhost:3000 to see the configuration options. Click the "Install Development Configuration" button to install the dependencies in a way which will allow you to run local code with your changes.
 
 ## Usage: development environment
 
-Complete one of the installation methods above.
+Install using the instructions above.
 
 Make sure to read the [Contribution Guide](CONTRIBUTING.md) to find out more about coding style enforcement and commit etiquette!
 
+### Launch from Configurator
+
+As well as installing, the web-based Configurator gives you some options to launch Concierge.
+
+The "Launch Concierge" button shouldn't be used to test local changes to the Python scripts as this uses the image from the Docker Hub.
+
+"Launch Local Code (Docker)" will build the local files into a Docker image and put it up in a container mimicking the production environment. You should always try your code here before submitting a PR as there are some minor differences with how the components communicate with each other within Docker compared to running the code locally. The first build can be slow, but the Python dependencies will be cached making subsequent builds much faster.
+
+"Launch Local Code (Python)" is a straightforward way to 
+
 ### Visual Studio Code
 
 Install Shiny for Python VSCode extension.
@@ -153,20 +147,6 @@ Authentication doesn't play very nicely with VSCode, however it is still possibl
 
 This does offer the advantage of hot reloading and being able to use the debugger while testing authentication.
 
-### Launch script
-
-From the cloned project directory simply run `launch_dev.py`.
-
-You will be given the option to launch with CPU or GPU.
-
-`python launch_local.py` will build and launch the code in a Docker container as if it were the production environment, this allows you to locally test interactions between the containers.
-
-## Update to new release: development environment
-
-Clone the latest version of the repo.
-
-Repeat the process used during install.
-
 ## CLI ##
 
 > [!WARNING]

bun_installer/client/index.ts

@@ -34,8 +34,8 @@ const options = {
 
 zxcvbnOptions.setOptions(options)
 
-const password1El = document.getElementById("keycloak_password_first")! as HTMLInputElement
-const password2El = document.getElementById("keycloak_password")! as HTMLInputElement
+const password1El = document.getElementById("keycloak_password_first") as HTMLInputElement
+const password2El = document.getElementById("keycloak_password") as HTMLInputElement
 const formSubmitEls = document.querySelectorAll(".install_button")
 const loggingEls = document.querySelectorAll(".logging_element")
 const loggingToggle = document.getElementById("activity_logging")! as HTMLInputElement
@@ -56,6 +56,11 @@ const enableSubmit = () => formSubmitEls.forEach(el => (el as HTMLButtonElement)
 const disableSubmit = () => formSubmitEls.forEach(el => (el as HTMLButtonElement).disabled = true)
 
 const checkPasswords = () => {
+    if (!password1El || !password2El) {
+        patchPassword(null)
+        enableSubmit()
+        return
+    }
     const password1 = password1El.value
     const password2 = password2El.value
     if (!password1 && !password2) {
@@ -89,7 +94,7 @@ const checkPasswords = () => {
 }
 
 
-password2El.oninput = checkPasswords
+if (password2El) password2El.oninput = checkPasswords
 
 const setLoggingVisibility = () => {
     if (loggingToggle.checked) {

bun_installer/index.tsx

@@ -8,7 +8,7 @@ import doLaunch from "./server/doLaunch"
 import { ExistingRemover } from "./server/existingRemover"
 import css from "./assets/style.css" with {type: "file"}
 import js from "./assets/index.js" with {type: "file"}
-import { file } from "bun"
+import { file, type Subprocess } from "bun"
 import validateInstallForm from "./server/validateInstallForm.js"
 import { $ } from "bun"
 import streamHtml from "./server/streamHtml.js"
@@ -32,6 +32,8 @@ const { values } = parseArgs({
 
 const devMode = !!values['dev-mode']
 
+const state: {worker?: Subprocess} = {}
+
 const app = new Hono()
 
 app.get('/style.css', async c => c.body(await file(css).text(), 201, {
@@ -59,7 +61,7 @@ app.get('/', async c => {
           <WebUILink></WebUILink>
           <p>If the link above isn't working, try (re)launching using the button below.</p>
           <p>Bear in mind that if you just installed Concierge it can take a few minutes before it's up and running.</p>
-          <RelaunchForm devMode={devMode}></RelaunchForm>
+          <RelaunchForm devMode={devMode} isRunning={!!state.worker}></RelaunchForm>
         </section> : null}
         <ExistingRemover></ExistingRemover>
         <section>
@@ -108,17 +110,8 @@ app.post("/remove", c => c.req.formData()
 }))
 app.post("/launch", c => c.req.formData()
   .then(data => streamHtml(c, "Launching Concierge", async stream => {
-    for await (const message of doLaunch(data)) {
-      if (typeof message != "string" && message.command) {
-        if (message.command == "show_stop") await stream.writeln(await <form action="/" method="get">
-          <button type="submit">Stop Concierge</button>
-          <p>The other containers will remain running so you can run the code from VSCode or another application.</p>
-        </form>)
-      }
-      else {
-        await stream.writeln(await <p>{message}</p>)
-      }
-
+    for await (const message of doLaunch(data, state)) {
+      await stream.writeln(await <p>{message}</p>) 
     }
   }))
 )

bun_installer/pythonTest.ts

@@ -0,0 +1,3 @@
+import path from "node:path"
+
+Bun.spawn(["Scripts\\python", "-m", "dev_launcher"], {cwd: path.resolve("..")})

bun_installer/server/devLauncher.ts

@@ -0,0 +1,3 @@
+import runPython from "./runPython";
+
+await runPython("dev_launcher")

bun_installer/server/doLaunch.ts

@@ -1,15 +1,25 @@
 import * as envfile from "envfile"
 import getEnvPath from "./getEnvPath"
-import { $ } from "bun"
-import runPython from "./runPython"
+import { $, type Subprocess } from "bun"
 import logMessage from "./logMessage"
+import { platform } from "node:os"
+import path from "node:path"
 
-export default async function* (options: FormData) {
+export default async function* (options: FormData, state: {worker?: Subprocess}) {
+   const environment = options.get("environment")
+   if (environment == "stop_development") {
+      yield logMessage("Stopping Python code running process...")
+      if (!state.worker) yield logMessage("no worker!")
+      state.worker?.kill("SIGINT")
+      await state.worker?.exited
+      delete state.worker
+      return
+   }
    const envs = envfile.parse(await Bun.file(getEnvPath()).text()) 
    envs.OLLAMA_SERVICE = options.has("use_gpu") ? "ollama-gpu" : "ollama"
    yield logMessage(`Launching Concierge ${envs.OLLAMA_SERVICE.endsWith("gpu") ? "with" : "without"} GPU acceleration.`)
    await Bun.write(getEnvPath(), envfile.stringify(envs))
-   const environment = options.get("environment")
+
    if (environment == "local") {
       yield logMessage("Building local code files to Docker image. This can take a while depending on your internet connection...")
       await $`docker compose -f ./docker_compose/docker-compose-local.yml build`
@@ -20,8 +30,7 @@ export default async function* (options: FormData) {
       yield logMessage("Launching Docker Compose configuration to run Concierge code locally...")
       await $`docker compose -f ./docker_compose/docker-compose-dev.yml up -d`
       yield logMessage("Running Concierge from local codebase...")
-      yield {command: "show_stop"}
-      await runPython("dev_launcher")
+      state.worker = Bun.spawn([platform() == "win32" ? ".\\Scripts\\python" : "./bin/python", "-m", "dev_launcher"], {cwd: path.resolve("..")})
    } 
    else {
       yield logMessage("Launching Concierge Docker Compose configuration...")

bun_installer/server/installOptionsForm.tsx

@@ -72,11 +72,11 @@ export const InstallOptionsForm = async (props: {devMode: boolean}) => {
                 </p>
                 <p>If you don't enable security anyone who can access the web UI will have full privileges to interact with your Concierge instance!</p>
                 <p>The demo configuration should never be used for production as it is a very insecure configuration designed to show off the different access levels using test users.</p>
-                {keycloakEnabled ? <>
+                {keycloakEnabled ? <div id="keycloak_config">
                     <p><strong>There is an existing Keycloak installation.</strong></p>
                     <p>We strongly recommend removing that and OpenSearch using the buttons above before proceeding.</p>
                     <p>If not the installer will attempt to keep the existing configuration but be aware that this isn't well supported and you may end up with an unusable setup.</p>
-                </> : <p id="keycloak_config">
+                </div> : <p id="keycloak_config">
                     <label for="keycloak_password_first">Keycloak Admin Password</label>
                     <input type="password" id="keycloak_password_first"></input>
                     <label for="keycloak_password">Confirm Keycloak Admin Password</label>

bun_installer/server/relaunchForm.tsx

@@ -1,7 +1,7 @@
 import * as envfile from "envfile"
 import getEnvPath from "./getEnvPath"
 
-export const RelaunchForm = async (props: {devMode: boolean}) => {
+export const RelaunchForm = async (props: {devMode: boolean, isRunning?: boolean}) => {
     const envs = envfile.parse(await Bun.file(getEnvPath()).text())
     return (
         <form action="/launch" method="post">
@@ -12,7 +12,9 @@ export const RelaunchForm = async (props: {devMode: boolean}) => {
             <button type="submit">Launch Concierge</button>
             {props.devMode && <>
                 <button type="submit" name="environment" value="local">Launch Local Code (Docker)</button>
-                <button type="submit" name="environment" value="development">Launch Local Code (Python)</button>
+                {props.isRunning ? 
+                <button type="submit" name="environment" value="stop_development">Stop Local Code (Python)</button> :
+                <button type="submit" name="environment" value="development">Launch Local Code (Python)</button>}
             </>}
         </form>
     )

bun_installer/server/runPython.ts

@@ -1,7 +1,7 @@
 import { $ } from "bun"
 import { platform } from "node:os"
 
-export default async (command: string) => {
-    if (platform() == 'win32') await $`Scripts\\python -m ${{raw: command}}`.cwd("..")
-    else await $`./bin/python -m ${{raw: command}}`.cwd("..")
+export default (command: string) => {
+    if (platform() == 'win32') return $`Scripts\\python -m ${{raw: command}}`.cwd("..")
+    return $`./bin/python -m ${{raw: command}}`.cwd("..")
 }