Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker: use jetty and/or newer verssion of Tomcat #5064

Closed
barthanssens opened this issue Jun 28, 2024 · 4 comments
Closed

Docker: use jetty and/or newer verssion of Tomcat #5064

barthanssens opened this issue Jun 28, 2024 · 4 comments
Assignees
Labels
docker 📶 enhancement issue is a new feature or improvement M1 Fixed in milestone 1
Milestone

Comments

@barthanssens
Copy link
Contributor

Problem description

Tomcat 8.5 is EOL 31 March 2024 (https://tomcat.apache.org/tomcat-85-eol.html), so we either need to move to a more recent version or switch to another servlet engine for the server/workbench Docker image

Preferred solution

Use Jetty 9.4, since it's already being used by other dependencies (bonus: Jetty is an Eclipse project) and uses the same Servlet API 3.1 as Tomcat. Probably supported until the end of 2025 (jetty/jetty.project#7958)

Are you interested in contributing a solution yourself?

Yes

Alternatives you've considered

No response

Anything else?

Eventually, we'll need to move to a newer version of the servlet API

@barthanssens barthanssens added 📶 enhancement issue is a new feature or improvement docker labels Jun 28, 2024
@barthanssens barthanssens added this to the 5.1.0 milestone Jun 28, 2024
@barthanssens barthanssens self-assigned this Jun 28, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 4, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 4, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 4, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 5, 2024
@aschwarte10
Copy link
Contributor

Preferred solution

Use Jetty 9.4, since it's already being used by other dependencies (bonus: Jetty is an Eclipse project) and uses the same Servlet API 3.1 as Tomcat. Probably supported until the end of 2025 (jetty/jetty.project#7958

@barthanssens can you explain why you are not considering to go to Jetty 11? Is it because of the Jakarta switch that is technically blocking it?

@barthanssens
Copy link
Contributor Author

barthanssens commented Jul 18, 2024

Well, Jetty 11 (or even 12) should be the next step.

Due to various other dependencies, we already have jetty 9.4 in the list of dependencies, so this should not take much effort and (if all goes well) no code changes, just a different Docker makefile that I'm preparing.

(It's available on hub.docker as jetty-experimental, but I haven't tested it yet)

For Jetty 11, we may need to look into the workbench / rdf4j-server (IIRC they both use Servlet API 3.1, not sure if that API works on Jetty 11 without changes).

barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 18, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 22, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 23, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 23, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 23, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 23, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 24, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 24, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 24, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 24, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 25, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 25, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 25, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 26, 2024
@barthanssens
Copy link
Contributor Author

@aschwarte10 turns out Jetty 12 does indeed still support older Servlet APIs when enabling some modules :-)

barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 26, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 26, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 26, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 26, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Jul 26, 2024
@barthanssens
Copy link
Contributor Author

Problem in jetty 12 : #5092

barthanssens added a commit to Fedict/rdf4j that referenced this issue Aug 1, 2024
@barthanssens barthanssens changed the title Docker: use jetty instead of Tomcat Docker: use jetty and/or newer verssion of Tomcat Aug 1, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Aug 1, 2024
barthanssens added a commit to Fedict/rdf4j that referenced this issue Aug 1, 2024
@hmottestad hmottestad added the M1 Fixed in milestone 1 label Nov 10, 2024
Emskiqq added a commit to Emskiqq/rdf4j that referenced this issue Feb 18, 2025
* eclipse-rdf4jGH-4920 SPARQLConnection.size() method should not fetch every statement in the repository. Just send a count query instead.

Signed-off-by: Jerven Bolleman <[email protected]>

* eclipse-rdf4jGH-4920 When sending the remote size query make sure we don't send null,
or RDF4J nill

* eclipse-rdf4jGH-4920 Make the logic that distinguises between counting from the remote default graph, or a dataset clearer

* eclipse-rdf4jGH-4999: improve UpdateWithModelBuilder to support statement removal

* next minor version

* eclipse-rdf4jGH-4997: Enable SparqlBuilder to create VALUES clauses

* set correct version

Signed-off-by: Håvard Ottestad <[email protected]>

* eclipse-rdf4jGH-5054: added DCAT v3 class and properties (eclipse-rdf4j#5057)

* set correct version

Signed-off-by: Håvard Ottestad <[email protected]>

* eclipse-rdf4jGH-5053: avoid log spam for configurations class with legacy settings

When there is only a value for the legacy setting (i.e. the new config
setting is not used) we now log on level debug. Otherwise on warn.

The rational is: when users still have persisted legacy repository
configurations, the log is easily spammed with warnings.

* eclipse-rdf4jGH-5064: added docker makefile for Tomcat 9 / Jetty 9 and updated e2e tests

Signed-off-by: Bart Hanssens <[email protected]>

* eclipse-rdf4jGH-5095 add tests to verify that startRdf and endRdf are called at the correct time

* eclipse-rdf4jGH-5095 correctly call startRDF() and endRDF() when parsing json-ld

* improve release script to delay calling jenkins until we know that the build went through correctly, also improve the automation by reducing the number of times you need to press any key to continue

* release 5.0.2

Signed-off-by: Håvard Ottestad <[email protected]>

* next development iteration: 5.0.3-SNAPSHOT

Signed-off-by: Håvard Ottestad <[email protected]>

* set correct version

Signed-off-by: Håvard Ottestad <[email protected]>

* javadocs for 5.0.2

Signed-off-by: Håvard Ottestad <[email protected]>

* news item and release-notes if relevant for 5.0.2

Signed-off-by: Håvard Ottestad <[email protected]>

* update release notes, news item and downloads file

* add release notes and javadocs for rdf4j 4.3.13

* update script

* eclipse-rdf4jGH-5105: make MonitoringImpl in FedX thread safe

The monitoring service in FedX is a useful tool for evaluations /
benchmarks to inspect the number of requests sent to the endpoints.

This change makes the implementation thread safe.

* eclipse-rdf4jGH-5108 add a java system property to modify the limit for when we can still use SPARQL validation approach with sh:maxCount

* eclipse-rdf4jGH-5112 add test cases

* eclipse-rdf4jGH-5112 fix for sh:pattern difference between SPARQL and transactional validation

* closes eclipse-rdf4jGH-5113

* closes eclipse-rdf4jGH-5114 fix for SPARQL constraint values being null

* closes eclipse-rdf4jGH-5115 fix for bug where sh:class could cause missing validation results when nested sh:property and sh:node

* optimize imports

* wip

* wip

* Update merge_main_to_develop.yml

* eclipse-rdf4jGH-5124 make the three main HttpClient timeouts configurable with sensible defaults.

* Add link to Jelly to about.md

Added a link to the Jelly serialization format implementation for RDF4J: https://w3id.org/jelly/jelly-jvm

Jelly is a binary RDF format with a strong performance focus. The implementation for RDF4J can be either used as a Maven dependency, or as a JAR plugin that you drop into your classpath.

Jelly was also implemented for Apache Jena. An implementation for rdflib is planned.

* eclipse-rdf4jGH-5121: refactor the bind join logic into a reusable base class

Refactor the existing logic for executing bind joins into a reusable
base class.

This change mostly moves the implementation logic from the existing
ControlledWorkerBindJoin class to a new intermediate implementation
(with the goal to make it reusable in a second step for left joins).

Note that the new bind join implementation no longer uses the
ControlledWorkerJoin as base class, i.e. the decision of which join
implementation to use is moved to the strategy.

For backwards code compatibility the "ControlledWorkerBoundJoin" is
kept, but no longer used. Instead the new code is in
ControlledWorkerBindJoin.

* eclipse-rdf4jGH-5121: prepare execution of left joins in the federation strategy

Prepare to execute a specific implementation of a left join
implementation through the federation strategy.

* eclipse-rdf4jGH-5121: implementation of left bind join operator

This change provides the implementation and activation for the left bind
join operator.

The algorithm is as follows:

- execute left bind join using regular bound join query
- process result iteration similar to BoundJoinVALUESConversionIteration
- remember seen set of bindings (using index) and add original bindings
to those, i.e. put to result return all non-seen bindings directly from
the input

Note that the terminology in literature has changed to "bind joins".
Hence, for new classes and methods I try to follow that.

Change is covered with some unit tests

* eclipse-rdf4jGH-5121: configurability of bind left joins

Bind left joins for OPTIONAL can be disabled using the
"enableOptionalAsBindJoin" flag in the federation config

Integrate the switch between implementations in the unit test as
parameterized test

* eclipse-rdf4jGH-5121: code simplifications in bind join implementation

- use for-each loop for iterating bindingset
- use IntHashSet
- use Literal#intValue instead of Integer#parseInt

* javadocs

* news item and release-notes if relevant for 4.3.14

Signed-off-by: Håvard Ottestad <[email protected]>

* updated download.md

* updated release notes

* Bump commons-io:commons-io from 2.11.0 to 2.14.0

Bumps commons-io:commons-io from 2.11.0 to 2.14.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* eclipse-rdf4jGH-5148 Introduce "soft fail" for corrupt ValueStore

* eclipse-rdf4jGH-5148 Fixed typo

* eclipse-rdf4jGH-5153 benchmarks

* eclipse-rdf4jGH-5153 reduced default threshold for using SPARQL based validation of sh:maxCount to sh:maxCount=1 or below

* eclipse-rdf4jGH-5153 early detection of object equality when comparing Values or getting the effective boolean value

* eclipse-rdf4jGH-5124 introduce number of connections and also reduce timeouts (+1 squashed commit)
Squashed commits:
[9aa87b594c] eclipse-rdf4jGH-5124 introduce number of connections and also reduce timeouts

* eclipse-rdf4jGH-5134 add test

* eclipse-rdf4jGH-5148 fixes based on review

* eclipse-rdf4jGH-5148 add tests and extend corruption handling to more parts of the code

* eclipse-rdf4jGH-5148 corrupt data can be written as NQuads

* eclipse-rdf4jGH-5148 add support for more files in the test cases and add a fix for when the corruption causes large amounts of data to be read

* eclipse-rdf4jGH-5148 improved soft fail on corruption for values.id and values.hash files.

* eclipse-rdf4jGH-5148 improved handling of corrupt spoc/posc/... indexes

* eclipse-rdf4jGH-5148 cleanup naming and docs

* eclipse-rdf4jGH-5080 duplicate all settings so we can change the document loader setting to HASMAC JSON-LD

* eclipse-rdf4jGH-5148 better detection of non-empty b-tree

* code cleanup

* eclipse-rdf4jGH-5149 Add search:numDocs property  and maxQueryDocuments param in LuceneSail query

* eclipse-rdf4jGH-5151: use VALUES clause for FedX bind join with no free vars

For evaluation of bind joins the implementation for quite some time
makes use of a VALUES clause query.

Except for one code-path: for bind joins - where in the join all
arguments are bound - it was still using the old UNION query approach.
This approach is error prone and no longer required, i.e. the check join
can be executed with the same logic as the regular VALUES clause.

Note: an additional unit test for covering bind joins with no free vars
is added.

This change also marks a number of methods and classes used for the old
UNION based approach as deprecated. The implementations are internal to
the FedX engine and can be removed in the next major release.

* eclipse-rdf4jGH-5159 close child of GroupIterator

* eclipse-rdf4jGH-5149 Use ParameterizedTest and Objects.requireNonNull

* eclipse-rdf4jGH-5167 test cases

* eclipse-rdf4jGH-5167 correctly handle rsx:targetShape with complex paths

* eclipse-rdf4jGH-5182 improve performance of SHACL sh:pattern

* javadocs 4.3.15

* news item and release-notes if relevant for 4.3.15

Signed-off-by: Håvard Ottestad <[email protected]>

* update release notes and downloads

* eclipse-rdf4jGH-5189 cache Value objects retrieved from parent sail

* eclipse-rdf4jGH-5148 improved error message

* improve javadocs and make some tests more robust

* javadocs for 5.1.0-M1

Signed-off-by: Håvard Ottestad <[email protected]>

* news item and release-notes if relevant for 5.1.0-M1

Signed-off-by: Håvard Ottestad <[email protected]>

* release 5.0.3

Signed-off-by: Håvard Ottestad <[email protected]>

* next development iteration: 5.0.4-SNAPSHOT

Signed-off-by: Håvard Ottestad <[email protected]>

* set correct version

Signed-off-by: Håvard Ottestad <[email protected]>

* javadocs for 5.0.3

Signed-off-by: Håvard Ottestad <[email protected]>

* news item and release-notes if relevant for 5.0.3

Signed-off-by: Håvard Ottestad <[email protected]>

* update download.md

* update release notes

* eclipse-rdf4jGH-5197: preparation for supporting fair sub-query execution in FedX

This change adds preparational infrastructure for having different
implementations of schedulers. Configuration is here prepared by means
of defining a "SchedulerFactory" interface with a default implementation
aside (which essentially mimics the current behavior).

Note that for ease of development some aspects of
ControlledWorkerScheduler are made accessible to sub-classes. The idea
is that in the end version there is an abstract scheduler class
providing shared functionality and different implementation (e.g. the
current FIFO one and a fair implementation)

* eclipse-rdf4jGH-5197: javadoc refinements + smaller initialization changes

- for minor version compatibility the type of the "_taskQueue" field in
the scheduler cannot be changed (to non-final). Hence, for now we use a
dedicated protected initialization method. In the future (next major
release) the idea is to leave the queue entirely managed by the executor
service.
- refinements and clarifications to the javadoc

* eclipse-rdf4jGH-5121: support empty left bind join (OPTIONAL) in FedX

Previously we introduced support for left bind joins in FedX. The case
of empty left bind joins (i.e. where the clause inside the OPTIONAL does
not provide any statements) was not handled and resulted in an exception

This change now adds support for empty optional joins and passes the
results from the left-handside through.

* eclipse-rdf4jGH-5197: mark extension points as experimental

* eclipse-rdf4jGH-5149 Rename MAX_QUERY_DOCUMENTS_KEY to MAX_DOCUMENTS_KEY and MAX_DOCUMENTS_KEY to DEFAULT_NUM_DOCS_KEY

* eclipse-rdf4jGH-5149 code simplification and cleanup

* eclipse-rdf4jGH-5135 add NS to vocabulary files where NS was missing

* eclipse-rdf4jGH-5130 zeroOrMorePath and oneOrMorePath mostly implemented

* eclipse-rdf4jGH-5136 define that minCount and maxCount will only count the distinct values when applied to the union of multiple graphs

* add more logging to the transaction controller in the server

* eclipse-rdf4jGH-5139 fix bug in sh:qualifiedMaxCount

* optimize imports

* eclipse-rdf4jGH-5130 disable ZeroOrMorePath and OneOrMorePath for the time being

* eclipse-rdf4jGH-5130 fix test

* release 5.1.0

Signed-off-by: Håvard Ottestad <[email protected]>

* next development iteration: 5.1.1-SNAPSHOT

Signed-off-by: Håvard Ottestad <[email protected]>

* javadocs for 5.1.0

Signed-off-by: Håvard Ottestad <[email protected]>

* news item and release-notes if relevant for 5.1.0

Signed-off-by: Håvard Ottestad <[email protected]>

* update download.md

* eclipse-rdf4jGH-5208 Enable request tracing through logs using new associated request id

* eclipse-rdf4jGH-5215 Support apostrophe in local name for SPARQL 1.1 and Turtle/Trig/etc. (eclipse-rdf4j#5216)

* update release notes

* eclipse-rdf4jGH-5221 add test

* eclipse-rdf4jGH-5221 fix bug where added statements from a SPARQL update were not notified and were also not added. This only happens when the exact statement was removed by the DELETE clause.

* add a small cache to the regex implementation

* code cleanup

* fixed bug in Extensible Store

* eclipse-rdf4jGH-5229: fix left bind join in FedX for single binding input

This change fixes a situation that can incorrectly cause empty results.
It happens when the input of the left argument is a single binding set
and for special source selection situations (e.g. the right argument is
marked as ExclusiveStatement while the endpoint does not provide data)

To avoid the issue we also use the regular left join logic also for a
single binding set input, which can handle the situation properly.

Issue is covered with a unit test.

* eclipse-rdf4jGH-5231: fix poor query performance for hasStatements() in FedX

The previous implementation of the FedXConnection was delegating
"hasStatements()" to the implementation of "getStatements()", where the
latter was actually fetching data from the federation members.

For checks hasStatements() checks like {null, rdf:type, null} or even
{null, null, null} the implementation is problematic as it would fetch
all data matching the pattern from the federation members, only to
answer if it actually exists.

We now make use of "existence" check on the federation members, and can
actually rely on the source selection cache for this.

Unit test coverage has been added.

* eclipse-rdf4jGH-5234: fix limited config support of FedX source selection cache

Add an additional constructor accepting a supplier for an initialized
cache.

* eclipse-rdf4jGH-5227: fix binding assigner optimizer in FedX

The federation optimizer was missing to execute the binding assigner
(which injects external bindings into the statement pattern). The
consequence was potentially incorrect results (due to source source
selection with partial knowledge) as well as sub-optimal source
selection

Issue is covered with a unit test, which is failing in two places prior
to this change.

* eclipse-rdf4jGH-5231: refine javadoc, add Experimental annotation

* Setup github pages using hugo

* eclipse-rdf4jGH-5214 lmdb supports linux ppc, updated to latest version and also some cleanup in the bom pom

* wip

* fix issue with testsuites benchmark sometimes causing japicmp to trigger by mistake

* release 5.1.1

Signed-off-by: Håvard Ottestad <[email protected]>

* next development iteration: 5.1.2-SNAPSHOT

Signed-off-by: Håvard Ottestad <[email protected]>

* eclipse-rdf4jGH-5247 add test

* eclipse-rdf4jGH-5247 fix issue with LeftJoinIterator when a condition is used

* release 5.1.2

Signed-off-by: Håvard Ottestad <[email protected]>

* next development iteration: 5.1.3-SNAPSHOT

Signed-off-by: Håvard Ottestad <[email protected]>

---------

Signed-off-by: Jerven Bolleman <[email protected]>
Signed-off-by: Håvard Ottestad <[email protected]>
Signed-off-by: Bart Hanssens <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Jerven Bolleman <[email protected]>
Co-authored-by: Florian Kleedorfer <[email protected]>
Co-authored-by: Håvard Ottestad <[email protected]>
Co-authored-by: Bart Hanssens <[email protected]>
Co-authored-by: Andreas Schwarte <[email protected]>
Co-authored-by: Piotr Sowiński <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Hannes Ebner <[email protected]>
Co-authored-by: qaate47 <[email protected]>
Co-authored-by: Frens Jan Rumph <[email protected]>
Co-authored-by: bherber1 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docker 📶 enhancement issue is a new feature or improvement M1 Fixed in milestone 1
Projects
None yet
Development

No branches or pull requests

3 participants