public OpenGrok demo - autodeploy on push with successful build and test

[vladak]

As asked in #1736 , there should be a public demo server that indexes OpenGrok code, running the latest master.

[vladak]

Need to sort out these things:

• deploy mechanism (some sort of Github hook ? Travis ? https://docs.travis-ci.com/user/deployment/)
• domain name + registration
• hosting: AWS/Google cloud/Oracle cloud/Heroku ? (who is going to sponsor this ?)

[vladak]

I think Heroku is clear winner as it provides:

• webapp deployment via webapp-runner (https://devcenter.heroku.com/articles/java-webapp-runner)
• domain name (.herokuapp.com)
• deploy via Travis (https://docs.travis-ci.com/user/deployment/heroku/)
• way to run commands after deploy (so that indexer can run)

and is quite cheap ($7/month for Hobby version - https://www.heroku.com/pricing).

[vladak]

According to https://atmos.org/github-services/auto-deployment/ it is not necessary to deploy via Travis. One can deploy from Github directly to Heroku. For that 2 Services are needed:

• Autodeploy - automatically creates deployment Github objects (https://developer.github.com/v3/repos/deployments/#create-a-deployment), can be set so that deployments are created only for successful CI runs ('Deploy on status').
• HerokuBeta - performs the actual app deployment upon seeing new deployment object being created

This seems to be cleaner than deploying via Travis (and does not extend the dependency - Travis can be replaced by different CI in the future) and provides tracking via the deployment objects.

[vladak]

There is demo.opengrok.org however it would be nice to have this more seamless, ideally with ping-back via webhook to see if the deploy was successful.

[vladak]

Given that there is Docker image it would be shame not to use it since it wraps most of the complexity in usable package. Initially my idea was to deploy using the existing Travis pipeline however that is not needed - I stumbled upon https://stackoverflow.com/a/38593838/11582827 that recommends to use https://github.com/containrrr/watchtower that polls Docker image status and restarts existing containers with the latest image.

So, this basically boils down to couple of things:

• finding a provider that can run Docker (ideally for free)
• mapping demo.opengrok.org to the IP of the VPS
• adding functionality to the OpenGrok Docker image to detect incompatible index and reindex from scratch (based on environment variable that would not be set by default)
• making the Docker container to index demo repositories (OpenGrok repo would suffice)
  • maybe even create new Docker image based on the existing OpenGrok image that would do it

[Gilwe]

Hi, is this idea on hold?

[vladak]

Well, the Docker container works fine, it's just a matter of finding a place to host it and provision for each release. Any suggestions are welcome.

[vladak]

For the time being I settled on Ubuntu image in Oracle Free Tier OCI (it has only 1 GB RAM so it cannot index anything sizeable due to #3243). It runs the OpenGrok Docker image that is automatically updated by watchtower.

The configuration looks like this:

• ~/docker-compose.yml:

version: "3"

# More info at https://github.com/oracle/opengrok/docker/
services:
  opengrok:
    container_name: opengrok-demo
    image: opengrok/docker:master
    ports:
      - "80:8080/tcp"
    environment:
      SYNC_PERIOD_MINUTES: '10'
      READONLY_CONFIG_FILE: '/opengrok/etc/read-only.xml'
      CHECK_INDEX: 1
    restart: always
    # Volumes store your data between container upgrades
    volumes:
       - '~/opengrok/src/:/opengrok/src/'  # source code
       - '~/opengrok/etc/:/opengrok/etc/'  # folder contains configuration.xml
       - '~/opengrok/data/:/opengrok/data/'  # index and other things for source code
  # Automatically update all images and restart the respective containers.
  watchtower:
    image: containrrr/watchtower
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --interval 3600 --cleanup --include-stopped --revive-stopped --debug

• ~/opengrok/etc/read-only.xml:

<?xml version="1.0" encoding="UTF-8"?>
<java version="1.8.0_121" class="java.beans.XMLDecoder">
 <object class="org.opengrok.indexer.configuration.Configuration" id="Configuration0">

  <!-- enable definitions for historical revisions of files in the Navigate window -->
  <void property="ctags">                                                       
    <string>/usr/local/bin/ctags</string>                                       
  </void>                                                                       
  <void property="webappCtags">                                                 
    <boolean>true</boolean>                                                     
  </void>

 </object>
</java>

• /etc/systemd/system/docker-opengrok-demo.service:

[Unit]
Description=Docker Compose OpenGrok demo
Requires=docker.service
After=docker.service

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu
ExecStart=/usr/bin/docker-compose up -d
ExecStop=/usr/bin/docker-compose down
TimeoutStartSec=0
Restart=on-failure
StartLimitBurst=3
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Need to get the DNS entry set to point to the IP address.

One gotcha about watchtower is that it restarts the container with the same properties it was started with so in order to change e.g. environment variables for the opengrok-demo container it is necessary to perform manual restart.

[vladak]

For testing purposes it would be nice if we could publish new OpenGrok image for each source level commit, say to opengrok/docker:master.

[vladak]

The demo is running on http://demo.opengrok.org/

[renannprado]

@vladak I tried to use it, but I always got an error.

[vladak]

@vladak I tried to use it, but I always got an error.

Yeah, it's running on freebie underpowered VM so is struggling. I need to find it a better home.