Is it possible to reindex a new dataset without restarting the container?

[jonasengelmann]

I would like to load a new and updated n3 file. My dataset is rather small, so reindexing and overwriting is much faster than using sparql update. Reindexing runs without any errors using "--overwrite-existing", but when I query my data I get all sorts of errors that only go away when I restart my qlever server container. I also tried running IndexBuilderMain manually inside the server container but it produced the same errors.

[hannahbast]

@jonasengelmann What speaks against simply doing qlever index --overwrite-existing && qlever start --kill-existing-with-same-port?

A safer alternative would be to have two subdirectories current and previous and build the next version in a subdirectory next, and when it's done do qlever start --kill-existing-with-same-port and remove previous and rename current to previous and next to current. That way you always have the previous version of the index as backup in case something goes wrong with the new build. That's how we do it for all our endpoints that we rebuild from scratch periodically. You can put that as a one-liner in your crontab if you like.

[jonasengelmann]

Thanks for the quick response. In my setup, a containerized update pipeline generates a new set of triples that I want to index. To execute qlever commands and restart the server container from inside that container, I would have to provide access to the host's docker daemon socket, which I would like to avoid if possible. It would be really handy to be able to reindex (and restart) via http, similarly to the clear-cache method. Is something like this planned or do you have any idea for a workaround? We currently use Blazegraph's update feature load <file:///tmp/blazegraph/data.n3>

[hannahbast]

@jonasengelmann There is no need to execute qlever commands from inside the container, they can all be executed from outside the container. They automatically take care of existing containers. In particular, if you do qlever start --kill-existing-with-same-port from outside the container, it will kill the old one and start a new one. Alternatively, you can also do qlever stop && qlever start, which achieves the same thing.