diff --git a/src/main/asciidoc/reference/elasticsearch-clients.adoc b/src/main/asciidoc/reference/elasticsearch-clients.adoc index 935c601bb..1a4376f4a 100644 --- a/src/main/asciidoc/reference/elasticsearch-clients.adoc +++ b/src/main/asciidoc/reference/elasticsearch-clients.adoc @@ -3,13 +3,14 @@ This chapter illustrates configuration and usage of supported Elasticsearch client implementations. -Spring Data Elasticsearch operates upon an Elasticsearch client that is connected to a single Elasticsearch node or a cluster. Although the Elasticsearch Client can be used to work with the cluster, applications using Spring Data Elasticsearch normally use the higher level abstractions of <> and <>. +Spring Data Elasticsearch operates upon an Elasticsearch client that is connected to a single Elasticsearch node or a cluster. +Although the Elasticsearch Client can be used to work with the cluster, applications using Spring Data Elasticsearch normally use the higher level abstractions of <> and <>. [[elasticsearch.clients.transport]] == Transport Client -WARNING: The `TransportClient` is deprecated as of Elasticsearch 7 and will be removed in Elasticsearch 8. (https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[see the Elasticsearch documentation]). Spring Data Elasticsearch will support the `TransportClient` as long as it is available in the used -Elasticsearch <> but has deprecated the classes using it since version 4.0. +WARNING: The `TransportClient` is deprecated as of Elasticsearch 7 and will be removed in Elasticsearch 8. (https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[see the Elasticsearch documentation]). +Spring Data Elasticsearch will support the `TransportClient` as long as it is available in the used Elasticsearch <> but has deprecated the classes using it since version 4.0. We strongly recommend to use the <> instead of the `TransportClient`. @@ -46,6 +47,7 @@ IndexRequest request = new IndexRequest("spring-data") IndexResponse response = client.index(request); ---- + <.> The `TransportClient` must be configured with the cluster name. <.> The host and port to connect the client to. <.> the RefreshPolicy must be set in the `ElasticsearchTemplate` (override `refreshPolicy()` to not use the default) @@ -54,8 +56,7 @@ IndexResponse response = client.index(request); [[elasticsearch.clients.rest]] == High Level REST Client -The Java High Level REST Client is the default client of Elasticsearch, it provides a straight forward replacement for the `TransportClient` as it accepts and returns -the very same request/response objects and therefore depends on the Elasticsearch core project. +The Java High Level REST Client is the default client of Elasticsearch, it provides a straight forward replacement for the `TransportClient` as it accepts and returns the very same request/response objects and therefore depends on the Elasticsearch core project. Asynchronous calls are operated upon a client managed thread pool and require a callback to be notified when the request is done. .High Level REST Client @@ -93,6 +94,7 @@ IndexRequest request = new IndexRequest("spring-data") IndexResponse response = highLevelClient.index(request,RequestOptions.DEFAULT); ---- + <1> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL. <2> Create the RestHighLevelClient. <3> It is also possible to obtain the `lowLevelRest()` client. @@ -131,6 +133,7 @@ Mono response = client.index(request -> .source(singletonMap("feature", "reactive-client")); ); ---- + <.> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL. ==== @@ -162,25 +165,30 @@ ClientConfiguration clientConfiguration = ClientConfiguration.builder() headers.add("currentTime", LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE_TIME)); return headers; }) - .withWebClientConfigurer(webClient -> { <.> - //... - return webClient; - }) - .withHttpClientConfigurer(clientBuilder -> { <.> - //... + .withClientConfigurer( <.> + (ReactiveRestClients.WebClientConfigurationCallback) webClient -> { + // ... + return webClient; + }) + .withClientConfigurer( <.> + (RestClients.RestClientConfigurationCallback) clientBuilder -> { + // ... return clientBuilder; - }) + }) . // ... other options .build(); ---- + <.> Define default headers, if they need to be customized <.> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL. <.> Optionally enable SSL. <.> Optionally set a proxy. <.> Optionally set a path prefix, mostly used when different clusters a behind some reverse proxy. -<.> Set the connection timeout. Default is 10 sec. -<.> Set the socket timeout. Default is 5 sec. +<.> Set the connection timeout. +Default is 10 sec. +<.> Set the socket timeout. +Default is 5 sec. <.> Optionally set headers. <.> Add basic authentication. <.> A `Supplier
` function can be specified which is called every time before a request is sent to Elasticsearch - here, as an example, the current time is written in a header. @@ -188,13 +196,13 @@ ClientConfiguration clientConfiguration = ClientConfiguration.builder() <.> for non-reactive setup a function configuring the REST client ==== -IMPORTANT: Adding a Header supplier as shown in above example allows to inject headers that may change over the time, like authentication JWT tokens. If this is used in the reactive setup, the supplier function *must not* block! +IMPORTANT: Adding a Header supplier as shown in above example allows to inject headers that may change over the time, like authentication JWT tokens. +If this is used in the reactive setup, the supplier function *must not* block! [[elasticsearch.clients.logging]] == Client Logging -To see what is actually sent to and received from the server `Request` / `Response` logging on the transport level needs -to be turned on as outlined in the snippet below. +To see what is actually sent to and received from the server `Request` / `Response` logging on the transport level needs to be turned on as outlined in the snippet below. .Enable transport layer logging [source,xml] diff --git a/src/main/asciidoc/reference/elasticsearch-migration-guide-4.2-4.3.adoc b/src/main/asciidoc/reference/elasticsearch-migration-guide-4.2-4.3.adoc index 4012e4604..a32cc0133 100644 --- a/src/main/asciidoc/reference/elasticsearch-migration-guide-4.2-4.3.adoc +++ b/src/main/asciidoc/reference/elasticsearch-migration-guide-4.2-4.3.adoc @@ -3,12 +3,53 @@ This section describes breaking changes from version 4.2.x to 4.3.x and how removed features can be replaced by new introduced features. +[NOTE] +==== +Elasticsearch is working on a new Client that will replace the `RestHighLevelClient` because the `RestHighLevelClient` uses code from Elasticsearch core libraries which are not Apache 2 licensed anymore. +Spring Data Elasticsearch is preparing for this change as well. +This means that internally the implementations for the `*Operations` interfaces need to change - which should be no problem if users program against the interfaces like `ElasticsearchOperations` or `ReactiveElasticsearchOperations`. +If you are using the implementation classes like `ElasticsearchRestTemplate` directly, you will need to adapt to these changes. + +Spring Data Elasticsearch also removes or replaces the use of classes from the `org.elasticsearch` packages in it's API classes and methods, only using them in the implementation where the access to Elasticsearch is implemented. +For the user that means, that some enum classes that were used are replaced by enums that live in `org.springframework.data.elasticsearch` with the same values, these are internally mapped onto the Elasticsearch ones. + +Places where classes are used that cannot easily be replaced, this usage is marked as deprecated, we are working on replacements. + +Check the sections on <> and <> for further details. +==== + [[elasticsearch-migration-guide-4.2-4.3.deprecations]] == Deprecations [[elasticsearch-migration-guide-4.2-4.3.breaking-changes]] == Breaking Changes +=== Removal of `org.elasticsearch` classes from the API. + +* In the `org.springframework.data.elasticsearch.annotations.CompletionContext` annotation the property `type()` has changed from `org.elasticsearch.search.suggest.completion.context.ContextMapping.Type` to `org.springframework.data.elasticsearch.annotations.CompletionContext.ContextMappingType`, the available enum values are the same. +* In the `org.springframework.data.elasticsearch.annotations.Document` annotation the `versionType()` property has changed to `org.springframework.data.elasticsearch.annotations.Document.VersionType`, the available enum values are the same. +* In the `org.springframework.data.elasticsearch.core.query.Query` interface the `searchType()` property has changed to `org.springframework.data.elasticsearch.core.query.Query.SearchType`, the available enum values are the same. +* In the `org.springframework.data.elasticsearch.core.query.Query` interface the return value of `timeout()` was changed to `java.time.Duration`. + === Handling of field and sourceFilter properties of Query -Up to version 4.2 the `fields` property of a `Query` was interpreted and added to the include list of the `sourceFilter`. This was not correct, as these are different things for Elasticsearch. This has been corrected. As a consequence code might not work anymore that relies on using `fields` to specify which fields should be returned from the document's `_source' and should be changed to use the `sourceFilter`. +Up to version 4.2 the `fields` property of a `Query` was interpreted and added to the include list of the `sourceFilter`. +This was not correct, as these are different things for Elasticsearch. +This has been corrected. +As a consequence code might not work anymore that relies on using `fields` to specify which fields should be returned from the document's `_source' and should be changed to use the `sourceFilter`. + +=== search_type default value + +The default value for the `search_type` in Elasticsearch is `query_then_fetch`. +This now is also set as default value in the `Query` implementations, it was previously set to `dfs_query_then_fetch`. + +=== BulkOptions changes + +Some properties of the `org.springframework.data.elasticsearch.core.query.BulkOptions` class have changed their type: + +* the type of the `timeout` property has been changed to `java.time.Duration`. +* the type of the`refreshPolicy` property has been changed to `org.springframework.data.elasticsearch.core.RefreshPolicy`. + +=== IndicesOptions change + +Spring Data Elasticsearch now uses `org.springframework.data.elasticsearch.core.query.IndicesOptions` instead of `org.elasticsearch.action.support.IndicesOptions`. diff --git a/src/main/java/org/springframework/data/elasticsearch/annotations/CompletionContext.java b/src/main/java/org/springframework/data/elasticsearch/annotations/CompletionContext.java index fff649dea..f510a3d9b 100644 --- a/src/main/java/org/springframework/data/elasticsearch/annotations/CompletionContext.java +++ b/src/main/java/org/springframework/data/elasticsearch/annotations/CompletionContext.java @@ -1,3 +1,18 @@ +/* + * Copyright 2019-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.springframework.data.elasticsearch.annotations; import java.lang.annotation.Documented; @@ -7,12 +22,11 @@ import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; -import org.elasticsearch.search.suggest.completion.context.ContextMapping; - /** * Based on reference doc - https://www.elastic.co/guide/en/elasticsearch/reference/current/suggester-context.html * * @author Robert Gruendler + * @author Peter-Josef Meisch */ @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.FIELD) @@ -22,9 +36,16 @@ String name(); - ContextMapping.Type type(); + ContextMappingType type(); String precision() default ""; String path() default ""; + + /** + * @since 4.3 + */ + enum ContextMappingType { + CATEGORY, GEO + } } diff --git a/src/main/java/org/springframework/data/elasticsearch/annotations/Document.java b/src/main/java/org/springframework/data/elasticsearch/annotations/Document.java index 4378f0bdf..154b1acd0 100644 --- a/src/main/java/org/springframework/data/elasticsearch/annotations/Document.java +++ b/src/main/java/org/springframework/data/elasticsearch/annotations/Document.java @@ -21,7 +21,6 @@ import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; -import org.elasticsearch.index.VersionType; import org.springframework.data.annotation.Persistent; /** @@ -116,9 +115,15 @@ /** * Controls how Elasticsearch dynamically adds fields to the document. - * + * * @since 4.3 */ Dynamic dynamic() default Dynamic.INHERIT; + /** + * @since 4.3 + */ + enum VersionType { + INTERNAL, EXTERNAL, EXTERNAL_GTE + } } diff --git a/src/main/java/org/springframework/data/elasticsearch/client/ClientConfiguration.java b/src/main/java/org/springframework/data/elasticsearch/client/ClientConfiguration.java index 7c6779dff..184c3dcf8 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/ClientConfiguration.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/ClientConfiguration.java @@ -27,6 +27,7 @@ import javax.net.ssl.SSLContext; import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback; +import org.springframework.data.elasticsearch.client.reactive.ReactiveRestClients; import org.springframework.http.HttpHeaders; import org.springframework.lang.Nullable; import org.springframework.web.reactive.function.client.WebClient; @@ -120,16 +121,16 @@ static ClientConfiguration create(InetSocketAddress socketAddress) { boolean useSsl(); /** - * Returns the {@link SSLContext} to use. Can be {@link Optional#empty()} if unconfigured. + * Returns the {@link SSLContext} to use. Can be {@link Optional#empty()} if not configured. * - * @return the {@link SSLContext} to use. Can be {@link Optional#empty()} if unconfigured. + * @return the {@link SSLContext} to use. Can be {@link Optional#empty()} if not configured. */ Optional getSslContext(); /** - * Returns the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if unconfigured. + * Returns the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if not configured. * - * @return the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if unconfigured. + * @return the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if not configured. */ Optional getHostNameVerifier(); @@ -152,7 +153,7 @@ static ClientConfiguration create(InetSocketAddress socketAddress) { /** * Returns the path prefix that should be prepended to HTTP(s) requests for Elasticsearch behind a proxy. - * + * * @return the path prefix. * @since 4.0 */ @@ -161,7 +162,7 @@ static ClientConfiguration create(InetSocketAddress socketAddress) { /** * returns an optionally set proxy in the form host:port - * + * * @return the optional proxy * @since 4.0 */ @@ -173,11 +174,19 @@ static ClientConfiguration create(InetSocketAddress socketAddress) { Function getWebClientConfigurer(); /** - * @return the client configuration callback. + * @return the Rest Client configuration callback. * @since 4.2 + * @deprecated since 4.3 use {@link #getClientConfigurer()} */ + @Deprecated HttpClientConfigCallback getHttpClientConfigurer(); + /** + * @return the client configuration callback + * @since 4.3 + */ + ClientConfigurationCallback getClientConfigurer(); + /** * @return the supplier for custom headers. */ @@ -274,7 +283,7 @@ interface TerminalClientConfigurationBuilder { TerminalClientConfigurationBuilder withDefaultHeaders(HttpHeaders defaultHeaders); /** - * Configure the {@literal milliseconds} for the connect timeout. + * Configure the {@literal milliseconds} for the connect-timeout. * * @param millis the timeout to use. * @return the {@link TerminalClientConfigurationBuilder} @@ -327,7 +336,7 @@ default TerminalClientConfigurationBuilder withSocketTimeout(long millis) { /** * Configure the path prefix that will be prepended to any HTTP(s) requests - * + * * @param pathPrefix the pathPrefix. * @return the {@link TerminalClientConfigurationBuilder} * @since 4.0 @@ -342,21 +351,36 @@ default TerminalClientConfigurationBuilder withSocketTimeout(long millis) { /** * set customization hook in case of a reactive configuration - * + * * @param webClientConfigurer function to configure the WebClient * @return the {@link TerminalClientConfigurationBuilder}. + * @deprecated since 4.3, use {@link #withClientConfigurer(ClientConfigurationCallback)} with + * {@link ReactiveRestClients.WebClientConfigurationCallback} */ + @Deprecated TerminalClientConfigurationBuilder withWebClientConfigurer(Function webClientConfigurer); /** * Register a {HttpClientConfigCallback} to configure the non-reactive REST client. - * + * * @param httpClientConfigurer configuration callback, must not be null. * @return the {@link TerminalClientConfigurationBuilder}. * @since 4.2 + * @deprecated since 4.3, use {@link #withClientConfigurer(ClientConfigurationCallback)} with + * {@link RestClients.RestClientConfigurationCallback} */ + @Deprecated TerminalClientConfigurationBuilder withHttpClientConfigurer(HttpClientConfigCallback httpClientConfigurer); + /** + * Register a {@link ClientConfigurationCallback} to configure the client. + * + * @param clientConfigurer configuration callback, must not be {@literal null}. + * @return the {@link TerminalClientConfigurationBuilder}. + * @since 4.3 + */ + TerminalClientConfigurationBuilder withClientConfigurer(ClientConfigurationCallback clientConfigurer); + /** * set a supplier for custom headers. This is invoked for every HTTP request to Elasticsearch to retrieve headers * that should be sent with the request. A common use case is passing in authentication headers that may change. @@ -377,4 +401,15 @@ default TerminalClientConfigurationBuilder withSocketTimeout(long millis) { */ ClientConfiguration build(); } + + /** + * Callback to be executed to configure a client. + * + * @param the type of the client configuration class. + * @since 4.3 + */ + @FunctionalInterface + interface ClientConfigurationCallback { + T configure(T clientConfigurer); + } } diff --git a/src/main/java/org/springframework/data/elasticsearch/client/ClientConfigurationBuilder.java b/src/main/java/org/springframework/data/elasticsearch/client/ClientConfigurationBuilder.java index 2f8be5b39..dfbd24afe 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/ClientConfigurationBuilder.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/ClientConfigurationBuilder.java @@ -31,6 +31,7 @@ import org.springframework.data.elasticsearch.client.ClientConfiguration.ClientConfigurationBuilderWithRequiredEndpoint; import org.springframework.data.elasticsearch.client.ClientConfiguration.MaybeSecureClientConfigurationBuilder; import org.springframework.data.elasticsearch.client.ClientConfiguration.TerminalClientConfigurationBuilder; +import org.springframework.data.elasticsearch.client.reactive.ReactiveRestClients; import org.springframework.http.HttpHeaders; import org.springframework.lang.Nullable; import org.springframework.util.Assert; @@ -49,7 +50,7 @@ class ClientConfigurationBuilder implements ClientConfigurationBuilderWithRequiredEndpoint, MaybeSecureClientConfigurationBuilder { - private List hosts = new ArrayList<>(); + private final List hosts = new ArrayList<>(); private HttpHeaders headers = HttpHeaders.EMPTY; private boolean useSsl; private @Nullable SSLContext sslContext; @@ -62,7 +63,8 @@ class ClientConfigurationBuilder private @Nullable String proxy; private Function webClientConfigurer = Function.identity(); private Supplier headersSupplier = () -> HttpHeaders.EMPTY; - private HttpClientConfigCallback httpClientConfigurer = httpClientBuilder -> httpClientBuilder; + @Deprecated private HttpClientConfigCallback httpClientConfigurer = httpClientBuilder -> httpClientBuilder; + private ClientConfiguration.ClientConfigurationCallback clientConfigurer = t -> t; /* * (non-Javadoc) @@ -206,6 +208,9 @@ public TerminalClientConfigurationBuilder withWebClientConfigurer( Assert.notNull(webClientConfigurer, "webClientConfigurer must not be null"); this.webClientConfigurer = webClientConfigurer; + // noinspection NullableProblems + this.clientConfigurer = (ReactiveRestClients.WebClientConfigurationCallback) webClientConfigurer::apply; + return this; } @@ -215,6 +220,19 @@ public TerminalClientConfigurationBuilder withHttpClientConfigurer(HttpClientCon Assert.notNull(httpClientConfigurer, "httpClientConfigurer must not be null"); this.httpClientConfigurer = httpClientConfigurer; + // noinspection NullableProblems + this.clientConfigurer = (RestClients.RestClientConfigurationCallback) httpClientConfigurer::customizeHttpClient; + + return this; + } + + @Override + public TerminalClientConfigurationBuilder withClientConfigurer( + ClientConfiguration.ClientConfigurationCallback clientConfigurer) { + + Assert.notNull(clientConfigurer, "clientConfigurer must not be null"); + + this.clientConfigurer = clientConfigurer; return this; } @@ -242,7 +260,7 @@ public ClientConfiguration build() { } return new DefaultClientConfiguration(hosts, headers, useSsl, sslContext, soTimeout, connectTimeout, pathPrefix, - hostnameVerifier, proxy, webClientConfigurer, httpClientConfigurer, headersSupplier); + hostnameVerifier, proxy, webClientConfigurer, httpClientConfigurer, clientConfigurer, headersSupplier); } private static InetSocketAddress parse(String hostAndPort) { diff --git a/src/main/java/org/springframework/data/elasticsearch/client/ClusterNodes.java b/src/main/java/org/springframework/data/elasticsearch/client/ClusterNodes.java index fdafbf96e..bc556ad34 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/ClusterNodes.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/ClusterNodes.java @@ -32,7 +32,10 @@ * * @author Oliver Gierke * @since 3.1 + * @deprecated only used in {@link TransportClientFactoryBean}. */ +@SuppressWarnings("DeprecatedIsStillUsed") +@Deprecated class ClusterNodes implements Streamable { public static ClusterNodes DEFAULT = ClusterNodes.of("127.0.0.1:9300"); @@ -44,7 +47,7 @@ class ClusterNodes implements Streamable { /** * Creates a new {@link ClusterNodes} by parsing the given source. - * + * * @param source must not be {@literal null} or empty. */ private ClusterNodes(String source) { @@ -74,15 +77,14 @@ private ClusterNodes(String source) { /** * Creates a new {@link ClusterNodes} by parsing the given source. The expected format is a comma separated list of * host-port-combinations separated by a colon: {@code host:port,host:port,…}. - * + * * @param source must not be {@literal null} or empty. - * @return */ public static ClusterNodes of(String source) { return new ClusterNodes(source); } - /* + /* * (non-Javadoc) * @see java.lang.Iterable#iterator() */ diff --git a/src/main/java/org/springframework/data/elasticsearch/client/DefaultClientConfiguration.java b/src/main/java/org/springframework/data/elasticsearch/client/DefaultClientConfiguration.java index 9a598739f..d64df7a5d 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/DefaultClientConfiguration.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/DefaultClientConfiguration.java @@ -55,12 +55,13 @@ class DefaultClientConfiguration implements ClientConfiguration { private final Function webClientConfigurer; private final HttpClientConfigCallback httpClientConfigurer; private final Supplier headersSupplier; + private final ClientConfigurationCallback clientConfigurer; DefaultClientConfiguration(List hosts, HttpHeaders headers, boolean useSsl, @Nullable SSLContext sslContext, Duration soTimeout, Duration connectTimeout, @Nullable String pathPrefix, @Nullable HostnameVerifier hostnameVerifier, @Nullable String proxy, Function webClientConfigurer, HttpClientConfigCallback httpClientConfigurer, - Supplier headersSupplier) { + ClientConfigurationCallback clientConfigurer, Supplier headersSupplier) { this.hosts = Collections.unmodifiableList(new ArrayList<>(hosts)); this.headers = new HttpHeaders(headers); @@ -73,6 +74,7 @@ class DefaultClientConfiguration implements ClientConfiguration { this.proxy = proxy; this.webClientConfigurer = webClientConfigurer; this.httpClientConfigurer = httpClientConfigurer; + this.clientConfigurer = clientConfigurer; this.headersSupplier = headersSupplier; } @@ -132,6 +134,12 @@ public HttpClientConfigCallback getHttpClientConfigurer() { return httpClientConfigurer; } + @SuppressWarnings("unchecked") + @Override + public ClientConfigurationCallback getClientConfigurer() { + return (ClientConfigurationCallback) clientConfigurer; + } + @Override public Supplier getHeadersSupplier() { return headersSupplier; diff --git a/src/main/java/org/springframework/data/elasticsearch/client/RestClients.java b/src/main/java/org/springframework/data/elasticsearch/client/RestClients.java index a4500f15b..85fa8ecae 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/RestClients.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/RestClients.java @@ -36,6 +36,7 @@ import org.apache.http.client.config.RequestConfig; import org.apache.http.client.config.RequestConfig.Builder; import org.apache.http.entity.ByteArrayEntity; +import org.apache.http.impl.nio.client.HttpAsyncClientBuilder; import org.apache.http.message.BasicHeader; import org.apache.http.protocol.HttpContext; import org.elasticsearch.client.RestClient; @@ -119,7 +120,7 @@ public static ElasticsearchRestClient create(ClientConfiguration clientConfigura clientConfiguration.getProxy().map(HttpHost::create).ifPresent(clientBuilder::setProxy); - clientBuilder = clientConfiguration.getHttpClientConfigurer().customizeHttpClient(clientBuilder); + clientBuilder = clientConfiguration. getClientConfigurer().configure(clientBuilder); return clientBuilder; }); @@ -198,7 +199,7 @@ public void process(HttpRequest request, HttpContext context) throws IOException } ClientLogger.logRequest(logId, request.getRequestLine().getMethod(), request.getRequestLine().getUri(), "", - () -> new String(buffer.toByteArray())); + buffer::toString); } else { ClientLogger.logRequest(logId, request.getRequestLine().getMethod(), request.getRequestLine().getUri(), ""); } @@ -213,7 +214,7 @@ public void process(HttpResponse response, HttpContext context) { /** * Interceptor to inject custom supplied headers. - * + * * @since 4.0 */ private static class CustomHeaderInjector implements HttpRequestInterceptor { @@ -233,4 +234,13 @@ public void process(HttpRequest request, HttpContext context) { } } } + + /** + * {@link org.springframework.data.elasticsearch.client.ClientConfiguration.ClientConfigurationCallback} to configure + * the RestClient with a {@link HttpAsyncClientBuilder} + * + * @since 4.3 + */ + public interface RestClientConfigurationCallback + extends ClientConfiguration.ClientConfigurationCallback {} } diff --git a/src/main/java/org/springframework/data/elasticsearch/client/reactive/DefaultReactiveElasticsearchClient.java b/src/main/java/org/springframework/data/elasticsearch/client/reactive/DefaultReactiveElasticsearchClient.java index e91b4443e..6e14bd3e7 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/reactive/DefaultReactiveElasticsearchClient.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/reactive/DefaultReactiveElasticsearchClient.java @@ -111,6 +111,7 @@ import org.springframework.data.elasticsearch.client.reactive.ReactiveElasticsearchClient.Indices; import org.springframework.data.elasticsearch.client.util.NamedXContents; import org.springframework.data.elasticsearch.client.util.ScrollState; +import org.springframework.data.elasticsearch.core.ResponseConverter; import org.springframework.data.elasticsearch.core.query.ByQueryResponse; import org.springframework.data.util.Lazy; import org.springframework.http.HttpHeaders; @@ -289,7 +290,7 @@ private static WebClientProvider getWebClientProvider(ClientConfiguration client provider = provider // .withDefaultHeaders(clientConfiguration.getDefaultHeaders()) // - .withWebClientConfigurer(clientConfiguration.getWebClientConfigurer()) // + .withWebClientConfigurer(clientConfiguration. getClientConfigurer()::configure) // .withRequestConfigurer(requestHeadersSpec -> requestHeadersSpec.headers(httpHeaders -> { HttpHeaders suppliedHeaders = clientConfiguration.getHeadersSupplier().get(); @@ -485,7 +486,7 @@ public Mono deleteBy(HttpHeaders headers, DeleteByQueryReq public Mono updateBy(HttpHeaders headers, UpdateByQueryRequest updateRequest) { return sendRequest(updateRequest, requestCreator.updateByQuery(), BulkByScrollResponse.class, headers) // .next() // - .map(ByQueryResponse::of); + .map(ResponseConverter::byQueryResponseOf); } @Override diff --git a/src/main/java/org/springframework/data/elasticsearch/client/reactive/ReactiveRestClients.java b/src/main/java/org/springframework/data/elasticsearch/client/reactive/ReactiveRestClients.java index 00a5cd6b9..d35112a1c 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/reactive/ReactiveRestClients.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/reactive/ReactiveRestClients.java @@ -17,6 +17,7 @@ import org.springframework.data.elasticsearch.client.ClientConfiguration; import org.springframework.util.Assert; +import org.springframework.web.reactive.function.client.WebClient; /** * Utility class for common access to reactive Elasticsearch clients. {@link ReactiveRestClients} consolidates set up @@ -61,4 +62,12 @@ public static ReactiveElasticsearchClient create(ClientConfiguration clientConfi return DefaultReactiveElasticsearchClient.create(clientConfiguration, requestCreator); } + + /** + * {@link org.springframework.data.elasticsearch.client.ClientConfiguration.ClientConfigurationCallback} to configure + * the ReactiveElasticsearchClient with a {@link WebClient} + * + * @since 4.3 + */ + public interface WebClientConfigurationCallback extends ClientConfiguration.ClientConfigurationCallback {} } diff --git a/src/main/java/org/springframework/data/elasticsearch/client/util/ScrollState.java b/src/main/java/org/springframework/data/elasticsearch/client/util/ScrollState.java index ab7201a77..bf6e09763 100644 --- a/src/main/java/org/springframework/data/elasticsearch/client/util/ScrollState.java +++ b/src/main/java/org/springframework/data/elasticsearch/client/util/ScrollState.java @@ -21,13 +21,11 @@ import java.util.List; import java.util.Set; -import org.elasticsearch.action.search.SearchScrollRequest; -import org.elasticsearch.search.Scroll; import org.springframework.lang.Nullable; import org.springframework.util.StringUtils; /** - * Mutable state object holding scrollId to be used for {@link SearchScrollRequest#scroll(Scroll)} + * Mutable state object holding scrollId to be used for scroll requests. * * @author Christoph Strobl * @author Peter-Josef Meisch diff --git a/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchRestTransportTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchRestTransportTemplate.java new file mode 100644 index 000000000..6796706e9 --- /dev/null +++ b/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchRestTransportTemplate.java @@ -0,0 +1,208 @@ +/* + * Copyright 2021-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.data.elasticsearch.core; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import org.elasticsearch.Version; +import org.elasticsearch.action.DocWriteResponse; +import org.elasticsearch.action.bulk.BulkItemResponse; +import org.elasticsearch.action.bulk.BulkResponse; +import org.elasticsearch.action.search.MultiSearchRequest; +import org.elasticsearch.action.search.MultiSearchResponse; +import org.elasticsearch.action.search.SearchResponse; +import org.elasticsearch.index.query.MoreLikeThisQueryBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.springframework.data.elasticsearch.BulkFailureException; +import org.springframework.data.elasticsearch.core.document.SearchDocumentResponse; +import org.springframework.data.elasticsearch.core.mapping.IndexCoordinates; +import org.springframework.data.elasticsearch.core.query.MoreLikeThisQuery; +import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder; +import org.springframework.data.elasticsearch.core.query.Query; +import org.springframework.util.Assert; + +/** + * This class contains methods that are common the implementations derived from {@link AbstractElasticsearchTemplate} + * using either the {@link org.elasticsearch.client.transport.TransportClient} or the + * {@link org.elasticsearch.client.RestHighLevelClient} and that use Elasticsearch specific libraries. + *

+ * Note: Although this class is public, it is not considered to be part of the official Spring Data + * Elasticsearch API and so might change at any time. + * + * @author Peter-Josef Meisch + */ +public abstract class AbstractElasticsearchRestTransportTemplate extends AbstractElasticsearchTemplate { + + // region DocumentOperations + /** + * @param bulkResponse + * @return the list of the item id's + */ + protected List checkForBulkOperationFailure(BulkResponse bulkResponse) { + + if (bulkResponse.hasFailures()) { + Map failedDocuments = new HashMap<>(); + for (BulkItemResponse item : bulkResponse.getItems()) { + + if (item.isFailed()) + failedDocuments.put(item.getId(), item.getFailureMessage()); + } + throw new BulkFailureException( + "Bulk operation has failures. Use ElasticsearchException.getFailedDocuments() for detailed messages [" + + failedDocuments + ']', + failedDocuments); + } + + return Stream.of(bulkResponse.getItems()).map(bulkItemResponse -> { + DocWriteResponse response = bulkItemResponse.getResponse(); + if (response != null) { + return IndexedObjectInformation.of(response.getId(), response.getSeqNo(), response.getPrimaryTerm(), + response.getVersion()); + } else { + return IndexedObjectInformation.of(bulkItemResponse.getId(), null, null, null); + } + + }).collect(Collectors.toList()); + } + + // endregion + + // region SearchOperations + protected SearchHits doSearch(MoreLikeThisQuery query, Class clazz, IndexCoordinates index) { + MoreLikeThisQueryBuilder moreLikeThisQueryBuilder = requestFactory.moreLikeThisQueryBuilder(query, index); + return search( + new NativeSearchQueryBuilder().withQuery(moreLikeThisQueryBuilder).withPageable(query.getPageable()).build(), + clazz, index); + } + + @Override + public List> multiSearch(List queries, Class clazz, IndexCoordinates index) { + MultiSearchRequest request = new MultiSearchRequest(); + for (Query query : queries) { + request.add(requestFactory.searchRequest(query, clazz, index)); + } + + MultiSearchResponse.Item[] items = getMultiSearchResult(request); + + SearchDocumentResponseCallback> callback = new ReadSearchDocumentResponseCallback<>(clazz, index); + List> res = new ArrayList<>(queries.size()); + int c = 0; + for (Query query : queries) { + res.add(callback.doWith(SearchDocumentResponse.from(items[c++].getResponse()))); + } + return res; + } + + @Override + public List> multiSearch(List queries, List> classes) { + + Assert.notNull(queries, "queries must not be null"); + Assert.notNull(classes, "classes must not be null"); + Assert.isTrue(queries.size() == classes.size(), "queries and classes must have the same size"); + + MultiSearchRequest request = new MultiSearchRequest(); + Iterator> it = classes.iterator(); + for (Query query : queries) { + Class clazz = it.next(); + request.add(requestFactory.searchRequest(query, clazz, getIndexCoordinatesFor(clazz))); + } + + MultiSearchResponse.Item[] items = getMultiSearchResult(request); + + List> res = new ArrayList<>(queries.size()); + int c = 0; + Iterator> it1 = classes.iterator(); + for (Query query : queries) { + Class entityClass = it1.next(); + + SearchDocumentResponseCallback> callback = new ReadSearchDocumentResponseCallback<>(entityClass, + getIndexCoordinatesFor(entityClass)); + + SearchResponse response = items[c++].getResponse(); + res.add(callback.doWith(SearchDocumentResponse.from(response))); + } + return res; + } + + @Override + public List> multiSearch(List queries, List> classes, + IndexCoordinates index) { + + Assert.notNull(queries, "queries must not be null"); + Assert.notNull(classes, "classes must not be null"); + Assert.notNull(index, "index must not be null"); + Assert.isTrue(queries.size() == classes.size(), "queries and classes must have the same size"); + + MultiSearchRequest request = new MultiSearchRequest(); + Iterator> it = classes.iterator(); + for (Query query : queries) { + request.add(requestFactory.searchRequest(query, it.next(), index)); + } + + MultiSearchResponse.Item[] items = getMultiSearchResult(request); + + List> res = new ArrayList<>(queries.size()); + int c = 0; + Iterator> it1 = classes.iterator(); + for (Query query : queries) { + Class entityClass = it1.next(); + + SearchDocumentResponseCallback> callback = new ReadSearchDocumentResponseCallback<>(entityClass, + index); + + SearchResponse response = items[c++].getResponse(); + res.add(callback.doWith(SearchDocumentResponse.from(response))); + } + return res; + } + + abstract protected MultiSearchResponse.Item[] getMultiSearchResult(MultiSearchRequest request); + + // endregion + + // region helper + @Override + public Query matchAllQuery() { + return new NativeSearchQueryBuilder().withQuery(QueryBuilders.matchAllQuery()).build(); + } + + @Override + public Query idsQuery(List ids) { + + Assert.notNull(ids, "ids must not be null"); + + return new NativeSearchQueryBuilder().withQuery(QueryBuilders.idsQuery().addIds(ids.toArray(new String[] {}))) + .build(); + } + + @Override + protected String getVendor() { + return "Elasticsearch"; + } + + @Override + protected String getRuntimeLibraryVersion() { + return Version.CURRENT.toString(); + } + + // endregion +} diff --git a/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchTemplate.java index f42cd0fec..d3381980c 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchTemplate.java @@ -15,32 +15,19 @@ */ package org.springframework.data.elasticsearch.core; -import java.util.ArrayList; +import java.time.Duration; import java.util.Arrays; import java.util.Collections; -import java.util.HashMap; import java.util.Iterator; import java.util.List; -import java.util.Map; import java.util.stream.Collectors; -import java.util.stream.Stream; -import org.elasticsearch.action.DocWriteResponse; -import org.elasticsearch.action.bulk.BulkItemResponse; -import org.elasticsearch.action.bulk.BulkResponse; -import org.elasticsearch.action.search.MultiSearchRequest; -import org.elasticsearch.action.search.MultiSearchResponse; import org.elasticsearch.action.search.SearchResponse; -import org.elasticsearch.action.support.WriteRequest; -import org.elasticsearch.action.support.WriteRequestBuilder; -import org.elasticsearch.common.unit.TimeValue; -import org.elasticsearch.index.query.MoreLikeThisQueryBuilder; import org.elasticsearch.search.suggest.SuggestBuilder; import org.springframework.beans.BeansException; import org.springframework.context.ApplicationContext; import org.springframework.context.ApplicationContextAware; import org.springframework.data.convert.EntityReader; -import org.springframework.data.elasticsearch.BulkFailureException; import org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter; import org.springframework.data.elasticsearch.core.convert.MappingElasticsearchConverter; import org.springframework.data.elasticsearch.core.document.Document; @@ -57,7 +44,6 @@ import org.springframework.data.elasticsearch.core.query.IndexQuery; import org.springframework.data.elasticsearch.core.query.IndexQueryBuilder; import org.springframework.data.elasticsearch.core.query.MoreLikeThisQuery; -import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder; import org.springframework.data.elasticsearch.core.query.Query; import org.springframework.data.elasticsearch.core.query.SeqNoPrimaryTerm; import org.springframework.data.elasticsearch.core.query.UpdateQuery; @@ -73,7 +59,13 @@ import org.springframework.util.Assert; /** - * AbstractElasticsearchTemplate + * This class contains methods that are common to different implementations of the {@link ElasticsearchOperations} + * interface that use different clients, like TransportClient, RestHighLevelClient and the next Java client from + * Elasticsearch or some future implementation that might use an Opensearch client. This class must not contain imports + * or use classes that are specific to one of these implementations. + *

+ * Note: Although this class is public, it is not considered to be part of the official Spring Data + * Elasticsearch API and so might change at any time. * * @author Sascha Woo * @author Peter-Josef Meisch @@ -84,9 +76,9 @@ public abstract class AbstractElasticsearchTemplate implements ElasticsearchOper @Nullable protected ElasticsearchConverter elasticsearchConverter; @Nullable protected RequestFactory requestFactory; - @Nullable private EntityOperations entityOperations; - @Nullable private EntityCallbacks entityCallbacks; - @Nullable private RefreshPolicy refreshPolicy; + @Nullable protected EntityOperations entityOperations; + @Nullable protected EntityCallbacks entityCallbacks; + @Nullable protected RefreshPolicy refreshPolicy; @Nullable protected RoutingResolver routingResolver; // region Initialization @@ -176,7 +168,7 @@ public RefreshPolicy getRefreshPolicy() { * @since 4.3 */ public void logVersions() { - VersionInfo.logVersions(getClusterVersion()); + VersionInfo.logVersions(getVendor(), getRuntimeLibraryVersion(), getClusterVersion()); } // endregion @@ -364,40 +356,6 @@ public List bulkOperation(List queries, BulkOptions public abstract List doBulkOperation(List queries, BulkOptions bulkOptions, IndexCoordinates index); - /** - * Pre process the write request before it is sent to the server, eg. by setting the - * {@link WriteRequest#setRefreshPolicy(String) refresh policy} if applicable. - * - * @param request must not be {@literal null}. - * @param - * @return the processed {@link WriteRequest}. - */ - protected > R prepareWriteRequest(R request) { - - if (refreshPolicy == null) { - return request; - } - - return request.setRefreshPolicy(RequestFactory.toElasticsearchRefreshPolicy(refreshPolicy)); - } - - /** - * Pre process the write request before it is sent to the server, eg. by setting the - * {@link WriteRequest#setRefreshPolicy(String) refresh policy} if applicable. - * - * @param requestBuilder must not be {@literal null}. - * @param - * @return the processed {@link WriteRequest}. - */ - protected > R prepareWriteRequestBuilder(R requestBuilder) { - - if (refreshPolicy == null) { - return requestBuilder; - } - - return requestBuilder.setRefreshPolicy(RequestFactory.toElasticsearchRefreshPolicy(refreshPolicy)); - } - // endregion // region SearchOperations @@ -414,8 +372,7 @@ public SearchHitsIterator searchForStream(Query query, Class clazz) { @Override public SearchHitsIterator searchForStream(Query query, Class clazz, IndexCoordinates index) { - long scrollTimeInMillis = TimeValue.timeValueMinutes(1).millis(); - + long scrollTimeInMillis = Duration.ofMinutes(1).toMillis(); // noinspection ConstantConditions int maxCount = query.isLimiting() ? query.getMaxResults() : 0; @@ -436,98 +393,16 @@ public SearchHits search(MoreLikeThisQuery query, Class clazz, IndexCo Assert.notNull(query.getId(), "No document id defined for MoreLikeThisQuery"); - MoreLikeThisQueryBuilder moreLikeThisQueryBuilder = requestFactory.moreLikeThisQueryBuilder(query, index); - return search( - new NativeSearchQueryBuilder().withQuery(moreLikeThisQueryBuilder).withPageable(query.getPageable()).build(), - clazz, index); + return doSearch(query, clazz, index); } + protected abstract SearchHits doSearch(MoreLikeThisQuery query, Class clazz, IndexCoordinates index); + @Override public List> multiSearch(List queries, Class clazz) { return multiSearch(queries, clazz, getIndexCoordinatesFor(clazz)); } - @Override - public List> multiSearch(List queries, Class clazz, IndexCoordinates index) { - MultiSearchRequest request = new MultiSearchRequest(); - for (Query query : queries) { - request.add(requestFactory.searchRequest(query, clazz, index)); - } - - MultiSearchResponse.Item[] items = getMultiSearchResult(request); - - SearchDocumentResponseCallback> callback = new ReadSearchDocumentResponseCallback<>(clazz, index); - List> res = new ArrayList<>(queries.size()); - int c = 0; - for (Query query : queries) { - res.add(callback.doWith(SearchDocumentResponse.from(items[c++].getResponse()))); - } - return res; - } - - @Override - public List> multiSearch(List queries, List> classes) { - - Assert.notNull(queries, "queries must not be null"); - Assert.notNull(classes, "classes must not be null"); - Assert.isTrue(queries.size() == classes.size(), "queries and classes must have the same size"); - - MultiSearchRequest request = new MultiSearchRequest(); - Iterator> it = classes.iterator(); - for (Query query : queries) { - Class clazz = it.next(); - request.add(requestFactory.searchRequest(query, clazz, getIndexCoordinatesFor(clazz))); - } - - MultiSearchResponse.Item[] items = getMultiSearchResult(request); - - List> res = new ArrayList<>(queries.size()); - int c = 0; - Iterator> it1 = classes.iterator(); - for (Query query : queries) { - Class entityClass = it1.next(); - - SearchDocumentResponseCallback> callback = new ReadSearchDocumentResponseCallback<>(entityClass, - getIndexCoordinatesFor(entityClass)); - - SearchResponse response = items[c++].getResponse(); - res.add(callback.doWith(SearchDocumentResponse.from(response))); - } - return res; - } - - @Override - public List> multiSearch(List queries, List> classes, - IndexCoordinates index) { - - Assert.notNull(queries, "queries must not be null"); - Assert.notNull(classes, "classes must not be null"); - Assert.notNull(index, "index must not be null"); - Assert.isTrue(queries.size() == classes.size(), "queries and classes must have the same size"); - - MultiSearchRequest request = new MultiSearchRequest(); - Iterator> it = classes.iterator(); - for (Query query : queries) { - request.add(requestFactory.searchRequest(query, it.next(), index)); - } - - MultiSearchResponse.Item[] items = getMultiSearchResult(request); - - List> res = new ArrayList<>(queries.size()); - int c = 0; - Iterator> it1 = classes.iterator(); - for (Query query : queries) { - Class entityClass = it1.next(); - - SearchDocumentResponseCallback> callback = new ReadSearchDocumentResponseCallback<>(entityClass, - index); - - SearchResponse response = items[c++].getResponse(); - res.add(callback.doWith(SearchDocumentResponse.from(response))); - } - return res; - } - @Override public SearchHits search(Query query, Class clazz) { return search(query, clazz, getIndexCoordinatesFor(clazz)); @@ -557,8 +432,6 @@ protected void searchScrollClear(String scrollId) { */ abstract protected void searchScrollClear(List scrollIds); - abstract protected MultiSearchResponse.Item[] getMultiSearchResult(MultiSearchRequest request); - @Override public SearchResponse suggest(SuggestBuilder suggestion, Class clazz) { return suggest(suggestion, getIndexCoordinatesFor(clazz)); @@ -600,37 +473,6 @@ public IndexCoordinates getIndexCoordinatesFor(Class clazz) { return getRequiredPersistentEntity(clazz).getIndexCoordinates(); } - /** - * @param bulkResponse - * @return the list of the item id's - */ - protected List checkForBulkOperationFailure(BulkResponse bulkResponse) { - - if (bulkResponse.hasFailures()) { - Map failedDocuments = new HashMap<>(); - for (BulkItemResponse item : bulkResponse.getItems()) { - - if (item.isFailed()) - failedDocuments.put(item.getId(), item.getFailureMessage()); - } - throw new BulkFailureException( - "Bulk operation has failures. Use ElasticsearchException.getFailedDocuments() for detailed messages [" - + failedDocuments + ']', - failedDocuments); - } - - return Stream.of(bulkResponse.getItems()).map(bulkItemResponse -> { - DocWriteResponse response = bulkItemResponse.getResponse(); - if (response != null) { - return IndexedObjectInformation.of(response.getId(), response.getSeqNo(), response.getPrimaryTerm(), - response.getVersion()); - } else { - return IndexedObjectInformation.of(bulkItemResponse.getId(), null, null, null); - } - - }).collect(Collectors.toList()); - } - protected T updateIndexedObject(T entity, IndexedObjectInformation indexedObjectInformation) { ElasticsearchPersistentEntity persistentEntity = elasticsearchConverter.getMappingContext() @@ -749,6 +591,18 @@ private IndexQuery getIndexQuery(T entity) { @Nullable abstract protected String getClusterVersion(); + /** + * @return the vendor name of the used cluster and client library + * @since 4.3 + */ + abstract protected String getVendor(); + + /** + * @return the version of the used client runtime library. + * @since 4.3 + */ + abstract protected String getRuntimeLibraryVersion(); + // endregion // region Entity callbacks diff --git a/src/main/java/org/springframework/data/elasticsearch/core/AbstractIndexTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/AbstractIndexTemplate.java index 95d761e6e..6706cf1b4 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/AbstractIndexTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/AbstractIndexTemplate.java @@ -17,14 +17,10 @@ import static org.springframework.util.StringUtils.*; -import java.util.List; import java.util.Map; import java.util.Objects; import java.util.Set; -import org.elasticsearch.cluster.metadata.AliasMetadata; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; import org.springframework.core.annotation.AnnotatedElementUtils; import org.springframework.dao.InvalidDataAccessApiUsageException; import org.springframework.data.elasticsearch.UncategorizedElasticsearchException; @@ -48,8 +44,6 @@ */ abstract class AbstractIndexTemplate implements IndexOperations { - private static final Logger LOGGER = LoggerFactory.getLogger(AbstractIndexTemplate.class); - protected final ElasticsearchConverter elasticsearchConverter; protected final RequestFactory requestFactory; @@ -175,8 +169,6 @@ public void refresh() { protected abstract void doRefresh(IndexCoordinates indexCoordinates); - protected abstract List doQueryForAlias(IndexCoordinates index); - @Override public Map> getAliases(String... aliasNames) { diff --git a/src/main/java/org/springframework/data/elasticsearch/core/ActiveShardCount.java b/src/main/java/org/springframework/data/elasticsearch/core/ActiveShardCount.java new file mode 100644 index 000000000..60cf7b666 --- /dev/null +++ b/src/main/java/org/springframework/data/elasticsearch/core/ActiveShardCount.java @@ -0,0 +1,41 @@ +/* + * Copyright 2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.data.elasticsearch.core; + +/** + * Class corresponding to the Elasticsearch class, but in the org.springframework.data.elasticsearch package + * + * @author Peter-Josef Meisch + */ +public class ActiveShardCount { + private static final int ACTIVE_SHARD_COUNT_DEFAULT = -2; + private static final int ALL_ACTIVE_SHARDS = -1; + + public static final ActiveShardCount DEFAULT = new ActiveShardCount(ACTIVE_SHARD_COUNT_DEFAULT); + public static final ActiveShardCount ALL = new ActiveShardCount(ALL_ACTIVE_SHARDS); + public static final ActiveShardCount NONE = new ActiveShardCount(0); + public static final ActiveShardCount ONE = new ActiveShardCount(1); + + private final int value; + + public ActiveShardCount(int value) { + this.value = value; + } + + public int getValue() { + return value; + } +} diff --git a/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchRestTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchRestTemplate.java index 6e9d2eb78..bac58dc81 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchRestTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchRestTemplate.java @@ -33,6 +33,7 @@ import org.elasticsearch.action.search.SearchRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.action.search.SearchScrollRequest; +import org.elasticsearch.action.support.WriteRequest; import org.elasticsearch.action.update.UpdateRequest; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; @@ -92,7 +93,7 @@ * @author Massimiliano Poggi * @author Farid Faoudi */ -public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate { +public class ElasticsearchRestTemplate extends AbstractElasticsearchRestTransportTemplate { private static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchRestTemplate.class); @@ -223,7 +224,8 @@ protected String doDelete(String id, @Nullable String routing, IndexCoordinates @Override public ByQueryResponse delete(Query query, Class clazz, IndexCoordinates index) { DeleteByQueryRequest deleteByQueryRequest = requestFactory.deleteByQueryRequest(query, clazz, index); - return ByQueryResponse.of(execute(client -> client.deleteByQuery(deleteByQueryRequest, RequestOptions.DEFAULT))); + return ResponseConverter + .byQueryResponseOf(execute(client -> client.deleteByQuery(deleteByQueryRequest, RequestOptions.DEFAULT))); } @Override @@ -261,7 +263,7 @@ public ByQueryResponse updateByQuery(UpdateQuery query, IndexCoordinates index) final BulkByScrollResponse bulkByScrollResponse = execute( client -> client.updateByQuery(updateByQueryRequest, RequestOptions.DEFAULT)); - return ByQueryResponse.of(bulkByScrollResponse); + return ResponseConverter.byQueryResponseOf(bulkByScrollResponse); } public List doBulkOperation(List queries, BulkOptions bulkOptions, @@ -272,6 +274,24 @@ public List doBulkOperation(List queries, BulkOptio updateIndexedObjectsWithQueries(queries, indexedObjectInformationList); return indexedObjectInformationList; } + + /** + * Pre process the write request before it is sent to the server, eg. by setting the + * {@link WriteRequest#setRefreshPolicy(String) refresh policy} if applicable. + * + * @param request must not be {@literal null}. + * @param + * @return the processed {@link WriteRequest}. + */ + protected > R prepareWriteRequest(R request) { + + if (refreshPolicy == null) { + return request; + } + + return request.setRefreshPolicy(RequestFactory.toElasticsearchRefreshPolicy(refreshPolicy)); + } + // endregion // region SearchOperations diff --git a/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchTemplate.java index 0ae4b14da..64a5bd5ab 100755 --- a/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchTemplate.java @@ -36,6 +36,8 @@ import org.elasticsearch.action.search.MultiSearchResponse; import org.elasticsearch.action.search.SearchRequestBuilder; import org.elasticsearch.action.search.SearchResponse; +import org.elasticsearch.action.support.WriteRequest; +import org.elasticsearch.action.support.WriteRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; import org.elasticsearch.client.Client; import org.elasticsearch.common.unit.TimeValue; @@ -90,7 +92,7 @@ * @deprecated as of 4.0 */ @Deprecated -public class ElasticsearchTemplate extends AbstractElasticsearchTemplate { +public class ElasticsearchTemplate extends AbstractElasticsearchRestTransportTemplate { private static final Logger QUERY_LOGGER = LoggerFactory .getLogger("org.springframework.data.elasticsearch.core.QUERY"); private static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchTemplate.class); @@ -246,7 +248,8 @@ protected String doDelete(String id, @Nullable String routing, IndexCoordinates @Override public ByQueryResponse delete(Query query, Class clazz, IndexCoordinates index) { - return ByQueryResponse.of(requestFactory.deleteByQueryRequestBuilder(client, query, clazz, index).get()); + return ResponseConverter + .byQueryResponseOf(requestFactory.deleteByQueryRequestBuilder(client, query, clazz, index).get()); } @Override @@ -288,7 +291,7 @@ public ByQueryResponse updateByQuery(UpdateQuery query, IndexCoordinates index) // UpdateByQueryRequestBuilder has not parameters to set a routing value final BulkByScrollResponse bulkByScrollResponse = updateByQueryRequestBuilder.execute().actionGet(); - return ByQueryResponse.of(bulkByScrollResponse); + return ResponseConverter.byQueryResponseOf(bulkByScrollResponse); } public List doBulkOperation(List queries, BulkOptions bulkOptions, @@ -309,6 +312,24 @@ public List doBulkOperation(List queries, BulkOptio return allIndexedObjectInformations; } + + /** + * Pre process the write request before it is sent to the server, eg. by setting the + * {@link WriteRequest#setRefreshPolicy(String) refresh policy} if applicable. + * + * @param requestBuilder must not be {@literal null}. + * @param + * @return the processed {@link WriteRequest}. + */ + protected > R prepareWriteRequestBuilder(R requestBuilder) { + + if (refreshPolicy == null) { + return requestBuilder; + } + + return requestBuilder.setRefreshPolicy(RequestFactory.toElasticsearchRefreshPolicy(refreshPolicy)); + } + // endregion // region SearchOperations diff --git a/src/main/java/org/springframework/data/elasticsearch/core/ReactiveElasticsearchTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/ReactiveElasticsearchTemplate.java index 02516564a..2f872b7a1 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/ReactiveElasticsearchTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/ReactiveElasticsearchTemplate.java @@ -25,6 +25,7 @@ import java.util.Map; import java.util.stream.Collectors; +import org.elasticsearch.Version; import org.elasticsearch.action.DocWriteResponse; import org.elasticsearch.action.bulk.BulkItemResponse; import org.elasticsearch.action.bulk.BulkRequest; @@ -39,6 +40,7 @@ import org.elasticsearch.action.support.WriteRequest; import org.elasticsearch.action.update.UpdateRequest; import org.elasticsearch.index.get.GetResult; +import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.elasticsearch.index.reindex.DeleteByQueryRequest; import org.elasticsearch.index.reindex.UpdateByQueryRequest; @@ -75,6 +77,7 @@ import org.springframework.data.elasticsearch.core.query.BulkOptions; import org.springframework.data.elasticsearch.core.query.ByQueryResponse; import org.springframework.data.elasticsearch.core.query.IndexQuery; +import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder; import org.springframework.data.elasticsearch.core.query.Query; import org.springframework.data.elasticsearch.core.query.SeqNoPrimaryTerm; import org.springframework.data.elasticsearch.core.query.UpdateQuery; @@ -164,7 +167,11 @@ private ReactiveElasticsearchTemplate copy() { * @since 4.3 */ public Mono logVersions() { - return getClusterVersion().doOnNext(VersionInfo::logVersions).then(); + return getVendor() // + .doOnNext(vendor -> getRuntimeLibraryVersion() // + .doOnNext(runtimeLibraryVersion -> getClusterVersion() // + .doOnNext(clusterVersion -> VersionInfo.logVersions(vendor, runtimeLibraryVersion, clusterVersion)))) // + .then(); // } @Override @@ -557,7 +564,7 @@ public Mono delete(Query query, Class entityType, IndexCoord Assert.notNull(query, "Query must not be null!"); - return doDeleteBy(query, entityType, index).map(ByQueryResponse::of); + return doDeleteBy(query, entityType, index).map(ResponseConverter::byQueryResponseOf); } @Override @@ -939,6 +946,35 @@ protected Mono getClusterVersion() { return Mono.empty(); } + /** + * @return the vendor name of the used cluster and client library + * @since 4.3 + */ + protected Mono getVendor() { + return Mono.just("Elasticsearch"); + } + + /** + * @return the version of the used client runtime library. + * @since 4.3 + */ + protected Mono getRuntimeLibraryVersion() { + return Mono.just(Version.CURRENT.toString()); + } + + @Override + public Query matchAllQuery() { + return new NativeSearchQueryBuilder().withQuery(QueryBuilders.matchAllQuery()).build(); + } + + @Override + public Query idsQuery(List ids) { + + Assert.notNull(ids, "ids must not be null"); + + return new NativeSearchQueryBuilder().withQuery(QueryBuilders.idsQuery().addIds(ids.toArray(new String[] {}))) + .build(); + } // endregion @Override diff --git a/src/main/java/org/springframework/data/elasticsearch/core/ReactiveSearchOperations.java b/src/main/java/org/springframework/data/elasticsearch/core/ReactiveSearchOperations.java index f3398f79c..ed4a0c507 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/ReactiveSearchOperations.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/ReactiveSearchOperations.java @@ -18,14 +18,14 @@ import reactor.core.publisher.Flux; import reactor.core.publisher.Mono; -import org.elasticsearch.index.query.QueryBuilders; +import java.util.List; + import org.elasticsearch.search.aggregations.Aggregation; import org.elasticsearch.search.suggest.Suggest; import org.elasticsearch.search.suggest.SuggestBuilder; import org.springframework.data.domain.Pageable; import org.springframework.data.elasticsearch.core.mapping.IndexCoordinates; import org.springframework.data.elasticsearch.core.query.Query; -import org.springframework.data.elasticsearch.core.query.StringQuery; /** * The reactive operations for the @@ -45,7 +45,7 @@ public interface ReactiveSearchOperations { * @return a {@link Mono} emitting the nr of matching documents. */ default Mono count(Class entityType) { - return count(new StringQuery(QueryBuilders.matchAllQuery().toString()), entityType); + return count(matchAllQuery(), entityType); } /** @@ -215,4 +215,25 @@ default Mono> searchForPage(Query query, Class entityType, * @return the suggest response */ Flux suggest(SuggestBuilder suggestion, IndexCoordinates index); + + // region helper + /** + * Creates a {@link Query} to find all documents. Must be implemented by the concrete implementations to provide an + * appropriate query using the respective client. + * + * @return a query to find all documents + * @since 4.3 + */ + Query matchAllQuery(); + + /** + * Creates a {@link Query} to find get all documents with given ids. Must be implemented by the concrete + * implementations to provide an appropriate query using the respective client. + * + * @param ids the list of ids must not be {@literal null} + * @return query returning the documents with the given ids + * @since 4.3 + */ + Query idsQuery(List ids); + // endregion } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java b/src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java index 6981d5905..b2cd02595 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java @@ -18,13 +18,17 @@ import static org.elasticsearch.index.query.QueryBuilders.*; import static org.springframework.util.CollectionUtils.*; +import java.time.Duration; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; +import java.util.EnumSet; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.Set; +import java.util.stream.Collectors; import org.elasticsearch.action.DocWriteRequest; import org.elasticsearch.action.admin.indices.alias.Alias; @@ -50,6 +54,7 @@ import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.search.SearchRequest; import org.elasticsearch.action.search.SearchRequestBuilder; +import org.elasticsearch.action.search.SearchType; import org.elasticsearch.action.support.ActiveShardCount; import org.elasticsearch.action.support.WriteRequest; import org.elasticsearch.action.update.UpdateRequest; @@ -223,15 +228,15 @@ public BulkRequest bulkRequest(List queries, BulkOptions bulkOptions, IndexCo BulkRequest bulkRequest = new BulkRequest(); if (bulkOptions.getTimeout() != null) { - bulkRequest.timeout(bulkOptions.getTimeout()); + bulkRequest.timeout(TimeValue.timeValueMillis(bulkOptions.getTimeout().toMillis())); } if (bulkOptions.getRefreshPolicy() != null) { - bulkRequest.setRefreshPolicy(bulkOptions.getRefreshPolicy()); + bulkRequest.setRefreshPolicy(toElasticsearchRefreshPolicy(bulkOptions.getRefreshPolicy())); } if (bulkOptions.getWaitForActiveShards() != null) { - bulkRequest.waitForActiveShards(bulkOptions.getWaitForActiveShards()); + bulkRequest.waitForActiveShards(ActiveShardCount.from(bulkOptions.getWaitForActiveShards().getValue())); } if (bulkOptions.getPipeline() != null) { @@ -258,15 +263,15 @@ public BulkRequestBuilder bulkRequestBuilder(Client client, List queries, Bul BulkRequestBuilder bulkRequestBuilder = client.prepareBulk(); if (bulkOptions.getTimeout() != null) { - bulkRequestBuilder.setTimeout(bulkOptions.getTimeout()); + bulkRequestBuilder.setTimeout(TimeValue.timeValueMillis(bulkOptions.getTimeout().toMillis())); } if (bulkOptions.getRefreshPolicy() != null) { - bulkRequestBuilder.setRefreshPolicy(bulkOptions.getRefreshPolicy()); + bulkRequestBuilder.setRefreshPolicy(toElasticsearchRefreshPolicy(bulkOptions.getRefreshPolicy())); } if (bulkOptions.getWaitForActiveShards() != null) { - bulkRequestBuilder.setWaitForActiveShards(bulkOptions.getWaitForActiveShards()); + bulkRequestBuilder.setWaitForActiveShards(ActiveShardCount.from(bulkOptions.getWaitForActiveShards().getValue())); } if (bulkOptions.getPipeline() != null) { @@ -802,7 +807,10 @@ public IndexRequestBuilder indexRequestBuilder(Client client, IndexQuery query, // region search @Nullable public HighlightBuilder highlightBuilder(Query query) { - HighlightBuilder highlightBuilder = query.getHighlightQuery().map(HighlightQuery::getHighlightBuilder).orElse(null); + HighlightBuilder highlightBuilder = query.getHighlightQuery() + .map(highlightQuery -> new HighlightQueryBuilder(elasticsearchConverter.getMappingContext()) + .getHighlightBuilder(highlightQuery.getHighlight(), highlightQuery.getType())) + .orElse(null); if (highlightBuilder == null) { @@ -954,7 +962,7 @@ private SearchRequest prepareSearchRequest(Query query, @Nullable Class clazz } if (query.getIndicesOptions() != null) { - request.indicesOptions(query.getIndicesOptions()); + request.indicesOptions(toElasticsearchIndicesOptions(query.getIndicesOptions())); } if (query.isLimiting()) { @@ -970,7 +978,7 @@ private SearchRequest prepareSearchRequest(Query query, @Nullable Class clazz request.preference(query.getPreference()); } - request.searchType(query.getSearchType()); + request.searchType(SearchType.fromString(query.getSearchType().name().toLowerCase())); prepareSort(query, sourceBuilder, getPersistentEntity(clazz)); @@ -994,9 +1002,9 @@ private SearchRequest prepareSearchRequest(Query query, @Nullable Class clazz request.routing(query.getRoute()); } - TimeValue timeout = query.getTimeout(); + Duration timeout = query.getTimeout(); if (timeout != null) { - sourceBuilder.timeout(timeout); + sourceBuilder.timeout(new TimeValue(timeout.toMillis())); } sourceBuilder.explain(query.getExplain()); @@ -1023,7 +1031,7 @@ private SearchRequestBuilder prepareSearchRequestBuilder(Query query, Client cli Assert.notEmpty(indexNames, "No index defined for Query"); SearchRequestBuilder searchRequestBuilder = client.prepareSearch(indexNames) // - .setSearchType(query.getSearchType()) // + .setSearchType(SearchType.fromString(query.getSearchType().name().toLowerCase())) // .setVersion(true) // .setTrackScores(query.getTrackScores()); if (hasSeqNoPrimaryTermProperty(clazz)) { @@ -1048,7 +1056,7 @@ private SearchRequestBuilder prepareSearchRequestBuilder(Query query, Client cli } if (query.getIndicesOptions() != null) { - searchRequestBuilder.setIndicesOptions(query.getIndicesOptions()); + searchRequestBuilder.setIndicesOptions(toElasticsearchIndicesOptions(query.getIndicesOptions())); } if (query.isLimiting()) { @@ -1086,9 +1094,9 @@ private SearchRequestBuilder prepareSearchRequestBuilder(Query query, Client cli searchRequestBuilder.setRouting(query.getRoute()); } - TimeValue timeout = query.getTimeout(); + Duration timeout = query.getTimeout(); if (timeout != null) { - searchRequestBuilder.setTimeout(timeout); + searchRequestBuilder.setTimeout(new TimeValue(timeout.toMillis())); } searchRequestBuilder.setExplain(query.getExplain()); @@ -1429,7 +1437,7 @@ public UpdateByQueryRequest updateByQueryRequest(UpdateQuery query, IndexCoordin updateByQueryRequest.setQuery(getQuery(queryQuery)); if (queryQuery.getIndicesOptions() != null) { - updateByQueryRequest.setIndicesOptions(queryQuery.getIndicesOptions()); + updateByQueryRequest.setIndicesOptions(toElasticsearchIndicesOptions(queryQuery.getIndicesOptions())); } if (queryQuery.getScrollTime() != null) { @@ -1504,7 +1512,8 @@ public UpdateByQueryRequestBuilder updateByQueryRequestBuilder(Client client, Up updateByQueryRequestBuilder.filter(getQuery(queryQuery)); if (queryQuery.getIndicesOptions() != null) { - updateByQueryRequestBuilder.source().setIndicesOptions(queryQuery.getIndicesOptions()); + updateByQueryRequestBuilder.source() + .setIndicesOptions(toElasticsearchIndicesOptions(queryQuery.getIndicesOptions())); } if (queryQuery.getScrollTime() != null) { @@ -1618,6 +1627,21 @@ private FetchSourceContext getFetchSourceContext(Query searchQuery) { return null; } + public org.elasticsearch.action.support.IndicesOptions toElasticsearchIndicesOptions(IndicesOptions indicesOptions) { + + Assert.notNull(indicesOptions, "indicesOptions must not be null"); + + Set options = indicesOptions.getOptions().stream() + .map(it -> org.elasticsearch.action.support.IndicesOptions.Option.valueOf(it.name().toUpperCase())) + .collect(Collectors.toSet()); + + Set wildcardStates = indicesOptions + .getExpandWildcards().stream() + .map(it -> org.elasticsearch.action.support.IndicesOptions.WildcardStates.valueOf(it.name().toUpperCase())) + .collect(Collectors.toSet()); + + return new org.elasticsearch.action.support.IndicesOptions(EnumSet.copyOf(options), EnumSet.copyOf(wildcardStates)); + } // endregion @Nullable @@ -1656,7 +1680,12 @@ private VersionType retrieveVersionTypeFromPersistentEntity(@Nullable Class c VersionType versionType = null; if (persistentEntity != null) { - versionType = persistentEntity.getVersionType(); + org.springframework.data.elasticsearch.annotations.Document.VersionType entityVersionType = persistentEntity + .getVersionType(); + + if (entityVersionType != null) { + versionType = VersionType.fromString(entityVersionType.name().toLowerCase()); + } } return versionType != null ? versionType : VersionType.EXTERNAL; diff --git a/src/main/java/org/springframework/data/elasticsearch/core/ResponseConverter.java b/src/main/java/org/springframework/data/elasticsearch/core/ResponseConverter.java index 1b8dfb73b..7d8da4053 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/ResponseConverter.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/ResponseConverter.java @@ -28,6 +28,7 @@ import org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse; import org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse; +import org.elasticsearch.action.bulk.BulkItemResponse; import org.elasticsearch.action.get.MultiGetItemResponse; import org.elasticsearch.action.get.MultiGetResponse; import org.elasticsearch.client.indices.GetIndexResponse; @@ -37,11 +38,14 @@ import org.elasticsearch.cluster.metadata.MappingMetadata; import org.elasticsearch.common.collect.ImmutableOpenMap; import org.elasticsearch.common.compress.CompressedXContent; +import org.elasticsearch.index.reindex.BulkByScrollResponse; +import org.elasticsearch.index.reindex.ScrollableHitSource; import org.springframework.data.elasticsearch.core.cluster.ClusterHealth; import org.springframework.data.elasticsearch.core.document.Document; import org.springframework.data.elasticsearch.core.index.AliasData; import org.springframework.data.elasticsearch.core.index.Settings; import org.springframework.data.elasticsearch.core.index.TemplateData; +import org.springframework.data.elasticsearch.core.query.ByQueryResponse; import org.springframework.lang.Nullable; import org.springframework.util.Assert; @@ -313,4 +317,71 @@ public static ClusterHealth clusterHealth(ClusterHealthResponse clusterHealthRes } // endregion + + // region byQueryResponse + public static ByQueryResponse byQueryResponseOf(BulkByScrollResponse bulkByScrollResponse) { + final List failures = bulkByScrollResponse.getBulkFailures() // + .stream() // + .map(ResponseConverter::byQueryResponseFailureOf) // + .collect(Collectors.toList()); // + + final List searchFailures = bulkByScrollResponse.getSearchFailures() // + .stream() // + .map(ResponseConverter::byQueryResponseSearchFailureOf) // + .collect(Collectors.toList());// + + return ByQueryResponse.builder() // + .withTook(bulkByScrollResponse.getTook().getMillis()) // + .withTimedOut(bulkByScrollResponse.isTimedOut()) // + .withTotal(bulkByScrollResponse.getTotal()) // + .withUpdated(bulkByScrollResponse.getUpdated()) // + .withDeleted(bulkByScrollResponse.getDeleted()) // + .withBatches(bulkByScrollResponse.getBatches()) // + .withVersionConflicts(bulkByScrollResponse.getVersionConflicts()) // + .withNoops(bulkByScrollResponse.getNoops()) // + .withBulkRetries(bulkByScrollResponse.getBulkRetries()) // + .withSearchRetries(bulkByScrollResponse.getSearchRetries()) // + .withReasonCancelled(bulkByScrollResponse.getReasonCancelled()) // + .withFailures(failures) // + .withSearchFailure(searchFailures) // + .build(); // + } + + /** + * Create a new {@link ByQueryResponse.Failure} from {@link BulkItemResponse.Failure} + * + * @param failure {@link BulkItemResponse.Failure} to translate + * @return a new {@link ByQueryResponse.Failure} + */ + public static ByQueryResponse.Failure byQueryResponseFailureOf(BulkItemResponse.Failure failure) { + return ByQueryResponse.Failure.builder() // + .withIndex(failure.getIndex()) // + .withType(failure.getType()) // + .withId(failure.getId()) // + .withStatus(failure.getStatus().getStatus()) // + .withAborted(failure.isAborted()) // + .withCause(failure.getCause()) // + .withSeqNo(failure.getSeqNo()) // + .withTerm(failure.getTerm()) // + .build(); // + } + + /** + * Create a new {@link ByQueryResponse.SearchFailure} from {@link ScrollableHitSource.SearchFailure} + * + * @param searchFailure {@link ScrollableHitSource.SearchFailure} to translate + * @return a new {@link ByQueryResponse.SearchFailure} + */ + public static ByQueryResponse.SearchFailure byQueryResponseSearchFailureOf( + ScrollableHitSource.SearchFailure searchFailure) { + return ByQueryResponse.SearchFailure.builder() // + .withReason(searchFailure.getReason()) // + .withIndex(searchFailure.getIndex()) // + .withNodeId(searchFailure.getNodeId()) // + .withShardId(searchFailure.getShardId()) // + .withStatus(searchFailure.getStatus().getStatus()) // + .build(); // + } + + // endregion } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/RestIndexTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/RestIndexTemplate.java index 04a4382dd..23df5306f 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/RestIndexTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/RestIndexTemplate.java @@ -15,7 +15,6 @@ */ package org.springframework.data.elasticsearch.core; -import java.util.ArrayList; import java.util.Collections; import java.util.List; import java.util.Map; @@ -28,7 +27,6 @@ import org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest; import org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse; import org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest; -import org.elasticsearch.client.GetAliasesResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.indices.CreateIndexRequest; import org.elasticsearch.client.indices.GetIndexRequest; @@ -39,7 +37,6 @@ import org.elasticsearch.client.indices.IndexTemplatesExistRequest; import org.elasticsearch.client.indices.PutIndexTemplateRequest; import org.elasticsearch.client.indices.PutMappingRequest; -import org.elasticsearch.cluster.metadata.AliasMetadata; import org.elasticsearch.cluster.metadata.MappingMetadata; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -140,19 +137,6 @@ protected Map doGetMapping(IndexCoordinates index) { }); } - @Override - protected List doQueryForAlias(IndexCoordinates index) { - - GetAliasesRequest getAliasesRequest = requestFactory.getAliasesRequest(index); - - return restTemplate.execute(client -> { - GetAliasesResponse alias = client.indices().getAlias(getAliasesRequest, RequestOptions.DEFAULT); - // we only return data for the first index name that was requested (always have done so) - String index1 = getAliasesRequest.indices()[0]; - return new ArrayList<>(alias.getAliases().get(index1)); - }); - } - @Override protected Map> doGetAliases(@Nullable String[] aliasNames, @Nullable String[] indexNames) { diff --git a/src/main/java/org/springframework/data/elasticsearch/core/SearchOperations.java b/src/main/java/org/springframework/data/elasticsearch/core/SearchOperations.java index 2de504a38..e6cb50a16 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/SearchOperations.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/SearchOperations.java @@ -220,4 +220,23 @@ default SearchHit searchOne(Query query, Class clazz, IndexCoordinates * are completed. */ SearchHitsIterator searchForStream(Query query, Class clazz, IndexCoordinates index); + + /** + * Creates a {@link Query} to get all documents. Must be implemented by the concrete implementations to provide an + * appropriate query using the respective client. + * + * @return a query to find all documents + * @since 4.3 + */ + Query matchAllQuery(); + + /** + * Creates a {@link Query} to find get all documents with given ids. Must be implemented by the concrete + * implementations to provide an appropriate query using the respective client. + * + * @param ids the list of ids must not be {@literal null} + * @return query returning the documents with the given ids + * @since 4.3 + */ + Query idsQuery(List ids); } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/TransportIndexTemplate.java b/src/main/java/org/springframework/data/elasticsearch/core/TransportIndexTemplate.java index 753221bf7..a08989776 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/TransportIndexTemplate.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/TransportIndexTemplate.java @@ -75,14 +75,13 @@ class TransportIndexTemplate extends AbstractIndexTemplate implements IndexOpera private final Client client; - public TransportIndexTemplate(Client client, ElasticsearchConverter elasticsearchConverter, - Class boundClass) { + public TransportIndexTemplate(Client client, ElasticsearchConverter elasticsearchConverter, Class boundClass) { super(elasticsearchConverter, boundClass); this.client = client; } public TransportIndexTemplate(Client client, ElasticsearchConverter elasticsearchConverter, - IndexCoordinates boundIndex) { + IndexCoordinates boundIndex) { super(elasticsearchConverter, boundIndex); this.client = client; } @@ -144,13 +143,6 @@ protected Map doGetMapping(IndexCoordinates index) { return mappings.iterator().next().value.get(IndexCoordinates.TYPE).getSourceAsMap(); } - @Override - protected List doQueryForAlias(IndexCoordinates index) { - - GetAliasesRequest getAliasesRequest = requestFactory.getAliasesRequest(index); - return client.admin().indices().getAliases(getAliasesRequest).actionGet().getAliases().get(index.getIndexName()); - } - @Override protected Map> doGetAliases(@Nullable String[] aliasNames, @Nullable String[] indexNames) { diff --git a/src/main/java/org/springframework/data/elasticsearch/core/index/GeoShapeMappingParameters.java b/src/main/java/org/springframework/data/elasticsearch/core/index/GeoShapeMappingParameters.java index 35aefa1c0..c79061312 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/index/GeoShapeMappingParameters.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/index/GeoShapeMappingParameters.java @@ -17,11 +17,12 @@ import java.io.IOException; -import org.elasticsearch.common.xcontent.XContentBuilder; import org.springframework.data.elasticsearch.annotations.GeoShapeField; import org.springframework.lang.Nullable; import org.springframework.util.Assert; +import com.fasterxml.jackson.databind.node.ObjectNode; + /** * @author Peter-Josef Meisch */ @@ -41,7 +42,7 @@ final class GeoShapeMappingParameters { /** * Creates a GeoShapeMappingParameters from the given annotation. - * + * * @param annotation if null, default values are set in the returned object * @return a parameters object */ @@ -63,27 +64,42 @@ private GeoShapeMappingParameters(boolean coerce, boolean ignoreMalformed, boole this.orientation = orientation; } - public void writeTypeAndParametersTo(XContentBuilder builder) throws IOException { + public boolean isCoerce() { + return coerce; + } + + public boolean isIgnoreMalformed() { + return ignoreMalformed; + } + + public boolean isIgnoreZValue() { + return ignoreZValue; + } + + public GeoShapeField.Orientation getOrientation() { + return orientation; + } - Assert.notNull(builder, "builder must ot be null"); + public void writeTypeAndParametersTo(ObjectNode objectNode) throws IOException { + + Assert.notNull(objectNode, "objectNode must not be null"); if (coerce) { - builder.field(FIELD_PARAM_COERCE, coerce); + objectNode.put(FIELD_PARAM_COERCE, coerce); } if (ignoreMalformed) { - builder.field(FIELD_PARAM_IGNORE_MALFORMED, ignoreMalformed); + objectNode.put(FIELD_PARAM_IGNORE_MALFORMED, ignoreMalformed); } if (!ignoreZValue) { - builder.field(FIELD_PARAM_IGNORE_Z_VALUE, ignoreZValue); + objectNode.put(FIELD_PARAM_IGNORE_Z_VALUE, ignoreZValue); } if (orientation != GeoShapeField.Orientation.ccw) { - builder.field(FIELD_PARAM_ORIENTATION, orientation.name()); + objectNode.put(FIELD_PARAM_ORIENTATION, orientation.name()); } - builder.field(FIELD_PARAM_TYPE, TYPE_VALUE_GEO_SHAPE); - + objectNode.put(FIELD_PARAM_TYPE, TYPE_VALUE_GEO_SHAPE); } } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/index/MappingBuilder.java b/src/main/java/org/springframework/data/elasticsearch/core/index/MappingBuilder.java index 981b7ec52..7d4cfc4fb 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/index/MappingBuilder.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/index/MappingBuilder.java @@ -15,18 +15,16 @@ */ package org.springframework.data.elasticsearch.core.index; -import static org.elasticsearch.common.xcontent.XContentFactory.*; import static org.springframework.data.elasticsearch.core.index.MappingParameters.*; import static org.springframework.util.StringUtils.*; -import java.io.ByteArrayInputStream; import java.io.IOException; import java.lang.annotation.Annotation; +import java.nio.charset.Charset; import java.util.Arrays; import java.util.Iterator; +import java.util.stream.Collectors; -import org.elasticsearch.common.xcontent.XContentBuilder; -import org.elasticsearch.common.xcontent.XContentType; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.core.io.ClassPathResource; @@ -35,16 +33,22 @@ import org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate; import org.springframework.data.elasticsearch.core.ResourceUtil; import org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter; +import org.springframework.data.elasticsearch.core.document.Document; import org.springframework.data.elasticsearch.core.mapping.ElasticsearchPersistentEntity; import org.springframework.data.elasticsearch.core.mapping.ElasticsearchPersistentProperty; import org.springframework.data.mapping.MappingException; import org.springframework.data.mapping.PropertyHandler; import org.springframework.data.util.TypeInformation; import org.springframework.lang.Nullable; +import org.springframework.util.StreamUtils; import org.springframework.util.StringUtils; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.ObjectNode; +import com.fasterxml.jackson.databind.node.TextNode; +import com.fasterxml.jackson.databind.util.RawValue; /** * @author Rizwan Idrees @@ -75,6 +79,7 @@ public class MappingBuilder { private static final String FIELD_CONTEXT_PATH = "path"; private static final String FIELD_CONTEXT_PRECISION = "precision"; private static final String FIELD_DYNAMIC_TEMPLATES = "dynamic_templates"; + private static final String FIELD_INCLUDE_IN_PARENT = "include_in_parent"; private static final String COMPLETION_PRESERVE_SEPARATORS = "preserve_separators"; private static final String COMPLETION_PRESERVE_POSITION_INCREMENTS = "preserve_position_increments"; @@ -86,6 +91,7 @@ public class MappingBuilder { private static final String TYPE_DYNAMIC = "dynamic"; private static final String TYPE_VALUE_KEYWORD = "keyword"; private static final String TYPE_VALUE_GEO_POINT = "geo_point"; + private static final String TYPE_VALUE_GEO_SHAPE = "geo_shape"; private static final String TYPE_VALUE_JOIN = "join"; private static final String TYPE_VALUE_COMPLETION = "completion"; @@ -98,6 +104,7 @@ public class MappingBuilder { private static final String RUNTIME = "runtime"; protected final ElasticsearchConverter elasticsearchConverter; + private final ObjectMapper objectMapper = new ObjectMapper(); private boolean writeTypeHints = true; @@ -126,61 +133,58 @@ protected String buildPropertyMapping(ElasticsearchPersistentEntity entity, writeTypeHints = entity.writeTypeHints(); - XContentBuilder builder = jsonBuilder().startObject(); + ObjectNode objectNode = objectMapper.createObjectNode(); // Dynamic templates - addDynamicTemplatesMapping(builder, entity); + addDynamicTemplatesMapping(objectNode, entity); - mapEntity(builder, entity, true, "", false, FieldType.Auto, null, entity.findAnnotation(DynamicMapping.class), + mapEntity(objectNode, entity, true, "", false, FieldType.Auto, null, entity.findAnnotation(DynamicMapping.class), runtimeFields); - builder.endObject() // root object - .close(); - - return builder.getOutputStream().toString(); + return objectMapper.writer().writeValueAsString(objectNode); } catch (IOException e) { throw new MappingException("could not build mapping", e); } } - private void writeTypeHintMapping(XContentBuilder builder) throws IOException { + private void writeTypeHintMapping(ObjectNode propertiesNode) throws IOException { if (writeTypeHints) { - builder.startObject(TYPEHINT_PROPERTY) // - .field(FIELD_PARAM_TYPE, TYPE_VALUE_KEYWORD) // - .field(FIELD_PARAM_INDEX, false) // - .field(FIELD_PARAM_DOC_VALUES, false) // - .endObject(); + propertiesNode.set(TYPEHINT_PROPERTY, objectMapper.createObjectNode() // + .put(FIELD_PARAM_TYPE, TYPE_VALUE_KEYWORD) // + .put(FIELD_PARAM_INDEX, false) // + .put(FIELD_PARAM_DOC_VALUES, false)); } } - private void mapEntity(XContentBuilder builder, @Nullable ElasticsearchPersistentEntity entity, - boolean isRootObject, String nestedObjectFieldName, boolean nestedOrObjectField, FieldType fieldType, - @Nullable Field parentFieldAnnotation, @Nullable DynamicMapping dynamicMapping, - @Nullable org.springframework.data.elasticsearch.core.document.Document runtimeFields) throws IOException { + private void mapEntity(ObjectNode objectNode, @Nullable ElasticsearchPersistentEntity entity, boolean isRootObject, + String nestedObjectFieldName, boolean nestedOrObjectField, FieldType fieldType, + @Nullable Field parentFieldAnnotation, @Nullable DynamicMapping dynamicMapping, @Nullable Document runtimeFields) + throws IOException { if (entity != null && entity.isAnnotationPresent(Mapping.class)) { Mapping mappingAnnotation = entity.getRequiredAnnotation(Mapping.class); if (!mappingAnnotation.enabled()) { - builder.field(MAPPING_ENABLED, false); + objectNode.put(MAPPING_ENABLED, false); return; } if (mappingAnnotation.dateDetection() != Mapping.Detection.DEFAULT) { - builder.field(DATE_DETECTION, Boolean.parseBoolean(mappingAnnotation.dateDetection().name())); + objectNode.put(DATE_DETECTION, Boolean.parseBoolean(mappingAnnotation.dateDetection().name())); } if (mappingAnnotation.numericDetection() != Mapping.Detection.DEFAULT) { - builder.field(NUMERIC_DETECTION, Boolean.parseBoolean(mappingAnnotation.numericDetection().name())); + objectNode.put(NUMERIC_DETECTION, Boolean.parseBoolean(mappingAnnotation.numericDetection().name())); } if (mappingAnnotation.dynamicDateFormats().length > 0) { - builder.field(DYNAMIC_DATE_FORMATS, mappingAnnotation.dynamicDateFormats()); + objectNode.putArray(DYNAMIC_DATE_FORMATS).addAll( + Arrays.stream(mappingAnnotation.dynamicDateFormats()).map(TextNode::valueOf).collect(Collectors.toList())); } if (runtimeFields != null) { - builder.field(RUNTIME, runtimeFields); + objectNode.set(RUNTIME, objectMapper.convertValue(runtimeFields, JsonNode.class)); } } @@ -189,23 +193,29 @@ private void mapEntity(XContentBuilder builder, @Nullable ElasticsearchPersisten String type = nestedOrObjectField ? fieldType.toString().toLowerCase() : FieldType.Object.toString().toLowerCase(); - builder.startObject(nestedObjectFieldName).field(FIELD_PARAM_TYPE, type); + + ObjectNode nestedObjectNode = objectMapper.createObjectNode(); + nestedObjectNode.put(FIELD_PARAM_TYPE, type); if (nestedOrObjectField && FieldType.Nested == fieldType && parentFieldAnnotation != null && parentFieldAnnotation.includeInParent()) { - builder.field("include_in_parent", true); + nestedObjectNode.put(FIELD_INCLUDE_IN_PARENT, true); } + + objectNode.set(nestedObjectFieldName, nestedObjectNode); + // now go on with the nested one + objectNode = nestedObjectNode; } if (entity != null && entity.dynamic() != Dynamic.INHERIT) { - builder.field(TYPE_DYNAMIC, entity.dynamic().name().toLowerCase()); + objectNode.put(TYPE_DYNAMIC, entity.dynamic().name().toLowerCase()); } else if (dynamicMapping != null) { - builder.field(TYPE_DYNAMIC, dynamicMapping.value().name().toLowerCase()); + objectNode.put(TYPE_DYNAMIC, dynamicMapping.value().name().toLowerCase()); } - builder.startObject(FIELD_PROPERTIES); + ObjectNode propertiesNode = objectNode.putObject(FIELD_PROPERTIES); - writeTypeHintMapping(builder); + writeTypeHintMapping(propertiesNode); if (entity != null) { entity.doWithProperties((PropertyHandler) property -> { @@ -223,19 +233,12 @@ private void mapEntity(XContentBuilder builder, @Nullable ElasticsearchPersisten return; } - buildPropertyMapping(builder, isRootObject, property); + buildPropertyMapping(propertiesNode, isRootObject, property); } catch (IOException e) { logger.warn("error mapping property with name {}", property.getName(), e); } }); } - - builder.endObject(); - - if (writeNestedProperties) { - builder.endObject(); - } - } @Nullable @@ -256,7 +259,7 @@ private org.springframework.data.elasticsearch.core.document.Document getRuntime return null; } - private void buildPropertyMapping(XContentBuilder builder, boolean isRootObject, + private void buildPropertyMapping(ObjectNode propertiesNode, boolean isRootObject, ElasticsearchPersistentProperty property) throws IOException { if (property.isAnnotationPresent(Mapping.class)) { @@ -270,27 +273,28 @@ private void buildPropertyMapping(XContentBuilder builder, boolean isRootObject, ClassPathResource mappings = new ClassPathResource(mappingPath); if (mappings.exists()) { - builder.rawField(property.getFieldName(), mappings.getInputStream(), XContentType.JSON); + propertiesNode.putRawValue(property.getFieldName(), + new RawValue(StreamUtils.copyToString(mappings.getInputStream(), Charset.defaultCharset()))); return; } } } else { - applyDisabledPropertyMapping(builder, property); + applyDisabledPropertyMapping(propertiesNode, property); return; } } if (property.isGeoPointProperty()) { - applyGeoPointFieldMapping(builder, property); + applyGeoPointFieldMapping(propertiesNode, property); return; } if (property.isGeoShapeProperty()) { - applyGeoShapeMapping(builder, property); + applyGeoShapeMapping(propertiesNode, property); } if (property.isJoinFieldProperty()) { - addJoinFieldMapping(builder, property); + addJoinFieldMapping(propertiesNode, property); } Field fieldAnnotation = property.findAnnotation(Field.class); @@ -310,7 +314,7 @@ private void buildPropertyMapping(XContentBuilder builder, boolean isRootObject, ? elasticsearchConverter.getMappingContext().getPersistentEntity(iterator.next()) : null; - mapEntity(builder, persistentEntity, false, property.getFieldName(), true, fieldAnnotation.type(), + mapEntity(propertiesNode, persistentEntity, false, property.getFieldName(), true, fieldAnnotation.type(), fieldAnnotation, dynamicMapping, null); return; } @@ -320,15 +324,15 @@ private void buildPropertyMapping(XContentBuilder builder, boolean isRootObject, if (isCompletionProperty) { CompletionField completionField = property.findAnnotation(CompletionField.class); - applyCompletionFieldMapping(builder, property, completionField); + applyCompletionFieldMapping(propertiesNode, property, completionField); } if (isRootObject && fieldAnnotation != null && property.isIdProperty()) { - applyDefaultIdFieldMapping(builder, property); + applyDefaultIdFieldMapping(propertiesNode, property); } else if (multiField != null) { - addMultiFieldMapping(builder, property, multiField, isNestedOrObjectProperty, dynamicMapping); + addMultiFieldMapping(propertiesNode, property, multiField, isNestedOrObjectProperty, dynamicMapping); } else if (fieldAnnotation != null) { - addSingleFieldMapping(builder, property, fieldAnnotation, isNestedOrObjectProperty, dynamicMapping); + addSingleFieldMapping(propertiesNode, property, fieldAnnotation, isNestedOrObjectProperty, dynamicMapping); } } @@ -339,73 +343,70 @@ private boolean hasRelevantAnnotation(ElasticsearchPersistentProperty property) || property.findAnnotation(CompletionField.class) != null; } - private void applyGeoPointFieldMapping(XContentBuilder builder, ElasticsearchPersistentProperty property) + private void applyGeoPointFieldMapping(ObjectNode propertiesNode, ElasticsearchPersistentProperty property) throws IOException { - builder.startObject(property.getFieldName()).field(FIELD_PARAM_TYPE, TYPE_VALUE_GEO_POINT).endObject(); + propertiesNode.set(property.getFieldName(), + objectMapper.createObjectNode().put(FIELD_PARAM_TYPE, TYPE_VALUE_GEO_POINT)); } - private void applyGeoShapeMapping(XContentBuilder builder, ElasticsearchPersistentProperty property) + private void applyGeoShapeMapping(ObjectNode propertiesNode, ElasticsearchPersistentProperty property) throws IOException { - builder.startObject(property.getFieldName()); - GeoShapeMappingParameters.from(property.findAnnotation(GeoShapeField.class)).writeTypeAndParametersTo(builder); - builder.endObject(); + ObjectNode shapeNode = propertiesNode.putObject(property.getFieldName()); + GeoShapeMappingParameters mappingParameters = GeoShapeMappingParameters + .from(property.findAnnotation(GeoShapeField.class)); + mappingParameters.writeTypeAndParametersTo(shapeNode); } - private void applyCompletionFieldMapping(XContentBuilder builder, ElasticsearchPersistentProperty property, + private void applyCompletionFieldMapping(ObjectNode propertyNode, ElasticsearchPersistentProperty property, @Nullable CompletionField annotation) throws IOException { - builder.startObject(property.getFieldName()); - builder.field(FIELD_PARAM_TYPE, TYPE_VALUE_COMPLETION); + ObjectNode completionNode = propertyNode.putObject(property.getFieldName()); + completionNode.put(FIELD_PARAM_TYPE, TYPE_VALUE_COMPLETION); if (annotation != null) { + completionNode.put(COMPLETION_MAX_INPUT_LENGTH, annotation.maxInputLength()); + completionNode.put(COMPLETION_PRESERVE_POSITION_INCREMENTS, annotation.preservePositionIncrements()); + completionNode.put(COMPLETION_PRESERVE_SEPARATORS, annotation.preserveSeparators()); - builder.field(COMPLETION_MAX_INPUT_LENGTH, annotation.maxInputLength()); - builder.field(COMPLETION_PRESERVE_POSITION_INCREMENTS, annotation.preservePositionIncrements()); - builder.field(COMPLETION_PRESERVE_SEPARATORS, annotation.preserveSeparators()); - if (!StringUtils.isEmpty(annotation.searchAnalyzer())) { - builder.field(FIELD_PARAM_SEARCH_ANALYZER, annotation.searchAnalyzer()); + if (StringUtils.hasLength(annotation.searchAnalyzer())) { + completionNode.put(FIELD_PARAM_SEARCH_ANALYZER, annotation.searchAnalyzer()); } - if (!StringUtils.isEmpty(annotation.analyzer())) { - builder.field(FIELD_PARAM_INDEX_ANALYZER, annotation.analyzer()); + + if (StringUtils.hasLength(annotation.analyzer())) { + completionNode.put(FIELD_PARAM_INDEX_ANALYZER, annotation.analyzer()); } if (annotation.contexts().length > 0) { - builder.startArray(COMPLETION_CONTEXTS); + ArrayNode contextsNode = completionNode.putArray(COMPLETION_CONTEXTS); for (CompletionContext context : annotation.contexts()) { - builder.startObject(); - builder.field(FIELD_CONTEXT_NAME, context.name()); - builder.field(FIELD_CONTEXT_TYPE, context.type().name().toLowerCase()); + ObjectNode contextNode = contextsNode.addObject(); + contextNode.put(FIELD_CONTEXT_NAME, context.name()); + contextNode.put(FIELD_CONTEXT_TYPE, context.type().name().toLowerCase()); if (context.precision().length() > 0) { - builder.field(FIELD_CONTEXT_PRECISION, context.precision()); + contextNode.put(FIELD_CONTEXT_PRECISION, context.precision()); } if (StringUtils.hasText(context.path())) { - builder.field(FIELD_CONTEXT_PATH, context.path()); + contextNode.put(FIELD_CONTEXT_PATH, context.path()); } - - builder.endObject(); } - builder.endArray(); } - } - builder.endObject(); } - private void applyDefaultIdFieldMapping(XContentBuilder builder, ElasticsearchPersistentProperty property) + private void applyDefaultIdFieldMapping(ObjectNode propertyNode, ElasticsearchPersistentProperty property) throws IOException { - builder.startObject(property.getFieldName()) // - .field(FIELD_PARAM_TYPE, TYPE_VALUE_KEYWORD) // - .field(FIELD_INDEX, true) // - .endObject(); // + propertyNode.set(property.getFieldName(), objectMapper.createObjectNode()// + .put(FIELD_PARAM_TYPE, TYPE_VALUE_KEYWORD) // + .put(FIELD_INDEX, true) // + ); } - private void applyDisabledPropertyMapping(XContentBuilder builder, ElasticsearchPersistentProperty property) - throws IOException { + private void applyDisabledPropertyMapping(ObjectNode propertiesNode, ElasticsearchPersistentProperty property) { try { Field field = property.getRequiredAnnotation(Field.class); @@ -414,10 +415,11 @@ private void applyDisabledPropertyMapping(XContentBuilder builder, Elasticsearch throw new IllegalArgumentException("Field type must be 'object"); } - builder.startObject(property.getFieldName()) // - .field(FIELD_PARAM_TYPE, field.type().name().toLowerCase()) // - .field(MAPPING_ENABLED, false) // - .endObject(); // + propertiesNode.set(property.getFieldName(), objectMapper.createObjectNode() // + .put(FIELD_PARAM_TYPE, field.type().name().toLowerCase()) // + .put(MAPPING_ENABLED, false) // + ); + } catch (Exception e) { throw new MappingException("Could not write enabled: false mapping for " + property.getFieldName(), e); } @@ -428,33 +430,29 @@ private void applyDisabledPropertyMapping(XContentBuilder builder, Elasticsearch * * @throws IOException */ - private void addSingleFieldMapping(XContentBuilder builder, ElasticsearchPersistentProperty property, + private void addSingleFieldMapping(ObjectNode propertiesNode, ElasticsearchPersistentProperty property, Field annotation, boolean nestedOrObjectField, @Nullable DynamicMapping dynamicMapping) throws IOException { // build the property json, if empty skip it as this is no valid mapping - XContentBuilder propertyBuilder = jsonBuilder().startObject(); - addFieldMappingParameters(propertyBuilder, annotation, nestedOrObjectField); - propertyBuilder.endObject().close(); + ObjectNode fieldNode = objectMapper.createObjectNode(); + addFieldMappingParameters(fieldNode, annotation, nestedOrObjectField); - if ("{}".equals(propertyBuilder.getOutputStream().toString())) { + if (fieldNode.isEmpty()) { return; } - builder.startObject(property.getFieldName()); + propertiesNode.set(property.getFieldName(), fieldNode); if (nestedOrObjectField) { if (annotation.dynamic() != Dynamic.INHERIT) { - builder.field(TYPE_DYNAMIC, annotation.dynamic().name().toLowerCase()); + fieldNode.put(TYPE_DYNAMIC, annotation.dynamic().name().toLowerCase()); } else if (dynamicMapping != null) { - builder.field(TYPE_DYNAMIC, dynamicMapping.value().name().toLowerCase()); + fieldNode.put(TYPE_DYNAMIC, dynamicMapping.value().name().toLowerCase()); } } - - addFieldMappingParameters(builder, annotation, nestedOrObjectField); - builder.endObject(); } - private void addJoinFieldMapping(XContentBuilder builder, ElasticsearchPersistentProperty property) + private void addJoinFieldMapping(ObjectNode propertiesNode, ElasticsearchPersistentProperty property) throws IOException { JoinTypeRelation[] joinTypeRelations = property.getRequiredAnnotation(JoinTypeRelations.class).relations(); @@ -464,24 +462,23 @@ private void addJoinFieldMapping(XContentBuilder builder, ElasticsearchPersisten property.getFieldName()); return; } - builder.startObject(property.getFieldName()); - builder.field(FIELD_PARAM_TYPE, TYPE_VALUE_JOIN); + ObjectNode propertyNode = propertiesNode.putObject(property.getFieldName()); + propertyNode.put(FIELD_PARAM_TYPE, TYPE_VALUE_JOIN); - builder.startObject(JOIN_TYPE_RELATIONS); + ObjectNode relationsNode = propertyNode.putObject(JOIN_TYPE_RELATIONS); for (JoinTypeRelation joinTypeRelation : joinTypeRelations) { String parent = joinTypeRelation.parent(); String[] children = joinTypeRelation.children(); if (children.length > 1) { - builder.array(parent, children); + relationsNode.putArray(parent) + .addAll(Arrays.stream(children).map(TextNode::valueOf).collect(Collectors.toList())); } else if (children.length == 1) { - builder.field(parent, children[0]); + relationsNode.put(parent, children[0]); } } - builder.endObject(); - builder.endObject(); } /** @@ -489,43 +486,43 @@ private void addJoinFieldMapping(XContentBuilder builder, ElasticsearchPersisten * * @throws IOException */ - private void addMultiFieldMapping(XContentBuilder builder, ElasticsearchPersistentProperty property, + private void addMultiFieldMapping(ObjectNode propertyNode, ElasticsearchPersistentProperty property, MultiField annotation, boolean nestedOrObjectField, @Nullable DynamicMapping dynamicMapping) throws IOException { // main field - builder.startObject(property.getFieldName()); + ObjectNode mainFieldNode = objectMapper.createObjectNode(); + propertyNode.set(property.getFieldName(), mainFieldNode); if (nestedOrObjectField) { if (annotation.mainField().dynamic() != Dynamic.INHERIT) { - builder.field(TYPE_DYNAMIC, annotation.mainField().dynamic().name().toLowerCase()); + mainFieldNode.put(TYPE_DYNAMIC, annotation.mainField().dynamic().name().toLowerCase()); } else if (dynamicMapping != null) { - builder.field(TYPE_DYNAMIC, dynamicMapping.value().name().toLowerCase()); + mainFieldNode.put(TYPE_DYNAMIC, dynamicMapping.value().name().toLowerCase()); } } - addFieldMappingParameters(builder, annotation.mainField(), nestedOrObjectField); + addFieldMappingParameters(mainFieldNode, annotation.mainField(), nestedOrObjectField); // inner fields - builder.startObject("fields"); + ObjectNode innerFieldsNode = mainFieldNode.putObject("fields"); + for (InnerField innerField : annotation.otherFields()) { - builder.startObject(innerField.suffix()); - addFieldMappingParameters(builder, innerField, false); - builder.endObject(); - } - builder.endObject(); - builder.endObject(); + ObjectNode innerFieldNode = innerFieldsNode.putObject(innerField.suffix()); + addFieldMappingParameters(innerFieldNode, innerField, false); + + } } - private void addFieldMappingParameters(XContentBuilder builder, Annotation annotation, boolean nestedOrObjectField) + private void addFieldMappingParameters(ObjectNode fieldNode, Annotation annotation, boolean nestedOrObjectField) throws IOException { MappingParameters mappingParameters = MappingParameters.from(annotation); if (!nestedOrObjectField && mappingParameters.isStore()) { - builder.field(FIELD_PARAM_STORE, true); + fieldNode.put(FIELD_PARAM_STORE, true); } - mappingParameters.writeTypeAndParametersTo(builder); + mappingParameters.writeTypeAndParametersTo(fieldNode); } /** @@ -533,7 +530,7 @@ private void addFieldMappingParameters(XContentBuilder builder, Annotation annot * * @throws IOException */ - private void addDynamicTemplatesMapping(XContentBuilder builder, ElasticsearchPersistentEntity entity) + private void addDynamicTemplatesMapping(ObjectNode objectNode, ElasticsearchPersistentEntity entity) throws IOException { if (entity.isAnnotationPresent(DynamicTemplates.class)) { @@ -543,11 +540,9 @@ private void addDynamicTemplatesMapping(XContentBuilder builder, ElasticsearchPe String jsonString = ResourceUtil.readFileFromClasspath(mappingPath); if (hasText(jsonString)) { - ObjectMapper objectMapper = new ObjectMapper(); JsonNode jsonNode = objectMapper.readTree(jsonString).get("dynamic_templates"); if (jsonNode != null && jsonNode.isArray()) { - String json = objectMapper.writeValueAsString(jsonNode); - builder.rawField(FIELD_DYNAMIC_TEMPLATES, new ByteArrayInputStream(json.getBytes()), XContentType.JSON); + objectNode.set(FIELD_DYNAMIC_TEMPLATES, jsonNode); } } } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/index/MappingParameters.java b/src/main/java/org/springframework/data/elasticsearch/core/index/MappingParameters.java index 38bdbe7e7..0f6b80795 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/index/MappingParameters.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/index/MappingParameters.java @@ -18,11 +18,11 @@ import java.io.IOException; import java.lang.annotation.Annotation; import java.util.ArrayList; +import java.util.Arrays; import java.util.Collections; import java.util.List; +import java.util.stream.Collectors; -import org.elasticsearch.common.Nullable; -import org.elasticsearch.common.xcontent.XContentBuilder; import org.springframework.data.elasticsearch.annotations.DateFormat; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.annotations.FieldType; @@ -32,9 +32,13 @@ import org.springframework.data.elasticsearch.annotations.NullValueType; import org.springframework.data.elasticsearch.annotations.Similarity; import org.springframework.data.elasticsearch.annotations.TermVector; +import org.springframework.lang.Nullable; import org.springframework.util.Assert; import org.springframework.util.StringUtils; +import com.fasterxml.jackson.databind.node.ObjectNode; +import com.fasterxml.jackson.databind.node.TextNode; + /** * A class to hold the mapping parameters that might be set on * {@link org.springframework.data.elasticsearch.annotations.Field } or @@ -58,6 +62,7 @@ public final class MappingParameters { static final String FIELD_PARAM_FORMAT = "format"; static final String FIELD_PARAM_IGNORE_ABOVE = "ignore_above"; static final String FIELD_PARAM_IGNORE_MALFORMED = "ignore_malformed"; + static final String FIELD_PARAM_IGNORE_Z_VALUE = "ignore_z_value"; static final String FIELD_PARAM_INDEX = "index"; static final String FIELD_PARAM_INDEX_OPTIONS = "index_options"; static final String FIELD_PARAM_INDEX_PHRASES = "index_phrases"; @@ -70,8 +75,9 @@ public final class MappingParameters { static final String FIELD_PARAM_NORMS = "norms"; static final String FIELD_PARAM_NULL_VALUE = "null_value"; static final String FIELD_PARAM_POSITION_INCREMENT_GAP = "position_increment_gap"; + static final String FIELD_PARAM_ORIENTATION = "orientation"; static final String FIELD_PARAM_POSITIVE_SCORE_IMPACT = "positive_score_impact"; - static final String FIELD_PARAM_DIMS = "dims"; + static final String FIELD_PARAM_DIMS = "dims"; static final String FIELD_PARAM_SCALING_FACTOR = "scaling_factor"; static final String FIELD_PARAM_SEARCH_ANALYZER = "search_analyzer"; static final String FIELD_PARAM_STORE = "store"; @@ -101,7 +107,7 @@ public final class MappingParameters { private final NullValueType nullValueType; private final Integer positionIncrementGap; private final boolean positiveScoreImpact; - private final Integer dims; + private final Integer dims; private final String searchAnalyzer; private final double scalingFactor; private final Similarity similarity; @@ -163,7 +169,8 @@ private MappingParameters(Field field) { positiveScoreImpact = field.positiveScoreImpact(); dims = field.dims(); if (type == FieldType.Dense_Vector) { - Assert.isTrue(dims >= 1 && dims <= 2048, "Invalid required parameter! Dense_Vector value \"dims\" must be between 1 and 2048."); + Assert.isTrue(dims >= 1 && dims <= 2048, + "Invalid required parameter! Dense_Vector value \"dims\" must be between 1 and 2048."); } Assert.isTrue(field.enabled() || type == FieldType.Object, "enabled false is only allowed for field type object"); enabled = field.enabled(); @@ -205,7 +212,8 @@ private MappingParameters(InnerField field) { positiveScoreImpact = field.positiveScoreImpact(); dims = field.dims(); if (type == FieldType.Dense_Vector) { - Assert.isTrue(dims >= 1 && dims <= 2048, "Invalid required parameter! Dense_Vector value \"dims\" must be between 1 and 2048."); + Assert.isTrue(dims >= 1 && dims <= 2048, + "Invalid required parameter! Dense_Vector value \"dims\" must be between 1 and 2048."); } enabled = true; eagerGlobalOrdinals = field.eagerGlobalOrdinals(); @@ -216,20 +224,20 @@ public boolean isStore() { } /** - * writes the different fields to the builder. + * writes the different fields to an {@link ObjectNode}. * - * @param builder must not be {@literal null}. + * @param objectNode must not be {@literal null} */ - public void writeTypeAndParametersTo(XContentBuilder builder) throws IOException { + public void writeTypeAndParametersTo(ObjectNode objectNode) throws IOException { - Assert.notNull(builder, "builder must ot be null"); + Assert.notNull(objectNode, "objectNode must not be null"); if (fielddata) { - builder.field(FIELD_PARAM_DATA, fielddata); + objectNode.put(FIELD_PARAM_DATA, fielddata); } if (type != FieldType.Auto) { - builder.field(FIELD_PARAM_TYPE, type.name().toLowerCase()); + objectNode.put(FIELD_PARAM_TYPE, type.toString().toLowerCase()); if (type == FieldType.Date) { List formats = new ArrayList<>(); @@ -246,125 +254,123 @@ public void writeTypeAndParametersTo(XContentBuilder builder) throws IOException Collections.addAll(formats, dateFormatPatterns); if (!formats.isEmpty()) { - builder.field(FIELD_PARAM_FORMAT, String.join("||", formats)); + objectNode.put(FIELD_PARAM_FORMAT, String.join("||", formats)); } } } if (!index) { - builder.field(FIELD_PARAM_INDEX, index); + objectNode.put(FIELD_PARAM_INDEX, index); } - if (!StringUtils.isEmpty(analyzer)) { - builder.field(FIELD_PARAM_INDEX_ANALYZER, analyzer); + if (StringUtils.hasLength(analyzer)) { + objectNode.put(FIELD_PARAM_INDEX_ANALYZER, analyzer); } - if (!StringUtils.isEmpty(searchAnalyzer)) { - builder.field(FIELD_PARAM_SEARCH_ANALYZER, searchAnalyzer); + if (StringUtils.hasLength(searchAnalyzer)) { + objectNode.put(FIELD_PARAM_SEARCH_ANALYZER, searchAnalyzer); } - if (!StringUtils.isEmpty(normalizer)) { - builder.field(FIELD_PARAM_NORMALIZER, normalizer); + if (StringUtils.hasLength(normalizer)) { + objectNode.put(FIELD_PARAM_NORMALIZER, normalizer); } if (copyTo != null && copyTo.length > 0) { - builder.field(FIELD_PARAM_COPY_TO, copyTo); + objectNode.putArray(FIELD_PARAM_COPY_TO) + .addAll(Arrays.stream(copyTo).map(TextNode::valueOf).collect(Collectors.toList())); } if (ignoreAbove != null) { Assert.isTrue(ignoreAbove >= 0, "ignore_above must be a positive value"); - builder.field(FIELD_PARAM_IGNORE_ABOVE, ignoreAbove); + objectNode.put(FIELD_PARAM_IGNORE_ABOVE, ignoreAbove); } if (!coerce) { - builder.field(FIELD_PARAM_COERCE, coerce); + objectNode.put(FIELD_PARAM_COERCE, coerce); } if (!docValues) { - builder.field(FIELD_PARAM_DOC_VALUES, docValues); + objectNode.put(FIELD_PARAM_DOC_VALUES, docValues); } if (ignoreMalformed) { - builder.field(FIELD_PARAM_IGNORE_MALFORMED, ignoreMalformed); + objectNode.put(FIELD_PARAM_IGNORE_MALFORMED, ignoreMalformed); } if (indexOptions != IndexOptions.none) { - builder.field(FIELD_PARAM_INDEX_OPTIONS, indexOptions); + objectNode.put(FIELD_PARAM_INDEX_OPTIONS, indexOptions.toString()); } if (indexPhrases) { - builder.field(FIELD_PARAM_INDEX_PHRASES, indexPhrases); + objectNode.put(FIELD_PARAM_INDEX_PHRASES, indexPhrases); } if (indexPrefixes != null) { - builder.startObject(FIELD_PARAM_INDEX_PREFIXES); + ObjectNode prefixNode = objectNode.putObject(FIELD_PARAM_INDEX_PREFIXES); if (indexPrefixes.minChars() != IndexPrefixes.MIN_DEFAULT) { - builder.field(FIELD_PARAM_INDEX_PREFIXES_MIN_CHARS, indexPrefixes.minChars()); + prefixNode.put(FIELD_PARAM_INDEX_PREFIXES_MIN_CHARS, indexPrefixes.minChars()); } if (indexPrefixes.maxChars() != IndexPrefixes.MAX_DEFAULT) { - builder.field(FIELD_PARAM_INDEX_PREFIXES_MAX_CHARS, indexPrefixes.maxChars()); + prefixNode.put(FIELD_PARAM_INDEX_PREFIXES_MAX_CHARS, indexPrefixes.maxChars()); } - builder.endObject(); } if (!norms) { - builder.field(FIELD_PARAM_NORMS, norms); + objectNode.put(FIELD_PARAM_NORMS, norms); } - if (!StringUtils.isEmpty(nullValue)) { - Object value; + if (StringUtils.hasLength(nullValue)) { switch (nullValueType) { case Integer: - value = Integer.valueOf(nullValue); + objectNode.put(FIELD_PARAM_NULL_VALUE, Integer.valueOf(nullValue)); break; case Long: - value = Long.valueOf(nullValue); + objectNode.put(FIELD_PARAM_NULL_VALUE, Long.valueOf(nullValue)); break; case Double: - value = Double.valueOf(nullValue); + objectNode.put(FIELD_PARAM_NULL_VALUE, Double.valueOf(nullValue)); break; case String: default: - value = nullValue; + objectNode.put(FIELD_PARAM_NULL_VALUE, nullValue); break; } - builder.field(FIELD_PARAM_NULL_VALUE, value); } if (positionIncrementGap != null && positionIncrementGap >= 0) { - builder.field(FIELD_PARAM_POSITION_INCREMENT_GAP, positionIncrementGap); + objectNode.put(FIELD_PARAM_POSITION_INCREMENT_GAP, positionIncrementGap); } if (similarity != Similarity.Default) { - builder.field(FIELD_PARAM_SIMILARITY, similarity); + objectNode.put(FIELD_PARAM_SIMILARITY, similarity.toString()); } if (termVector != TermVector.none) { - builder.field(FIELD_PARAM_TERM_VECTOR, termVector); + objectNode.put(FIELD_PARAM_TERM_VECTOR, termVector.toString()); } if (type == FieldType.Scaled_Float) { - builder.field(FIELD_PARAM_SCALING_FACTOR, scalingFactor); + objectNode.put(FIELD_PARAM_SCALING_FACTOR, scalingFactor); } if (maxShingleSize != null) { - builder.field(FIELD_PARAM_MAX_SHINGLE_SIZE, maxShingleSize); + objectNode.put(FIELD_PARAM_MAX_SHINGLE_SIZE, maxShingleSize); } if (!positiveScoreImpact) { - builder.field(FIELD_PARAM_POSITIVE_SCORE_IMPACT, positiveScoreImpact); + objectNode.put(FIELD_PARAM_POSITIVE_SCORE_IMPACT, positiveScoreImpact); } if (type == FieldType.Dense_Vector) { - builder.field(FIELD_PARAM_DIMS, dims); + objectNode.put(FIELD_PARAM_DIMS, dims); } if (!enabled) { - builder.field(FIELD_PARAM_ENABLED, enabled); + objectNode.put(FIELD_PARAM_ENABLED, enabled); } if (eagerGlobalOrdinals) { - builder.field(FIELD_PARAM_EAGER_GLOBAL_ORDINALS, eagerGlobalOrdinals); + objectNode.put(FIELD_PARAM_EAGER_GLOBAL_ORDINALS, eagerGlobalOrdinals); } } } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/index/ReactiveMappingBuilder.java b/src/main/java/org/springframework/data/elasticsearch/core/index/ReactiveMappingBuilder.java index f06b71494..3902f3955 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/index/ReactiveMappingBuilder.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/index/ReactiveMappingBuilder.java @@ -28,7 +28,7 @@ import org.springframework.lang.Nullable; /** - * Subclass of {@link MappingBuilder} with specialized methods TO inhibit blocking CALLS + * Subclass of {@link MappingBuilder} with specialized methods To inhibit blocking calls * * @author Peter-Josef Meisch * @since 4.3 diff --git a/src/main/java/org/springframework/data/elasticsearch/core/mapping/ElasticsearchPersistentEntity.java b/src/main/java/org/springframework/data/elasticsearch/core/mapping/ElasticsearchPersistentEntity.java index 2bb134cbb..d88c4c21d 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/mapping/ElasticsearchPersistentEntity.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/mapping/ElasticsearchPersistentEntity.java @@ -15,7 +15,7 @@ */ package org.springframework.data.elasticsearch.core.mapping; -import org.elasticsearch.index.VersionType; +import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.Dynamic; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.core.index.Settings; @@ -61,7 +61,7 @@ public interface ElasticsearchPersistentEntity extends PersistentEntity extends BasicPersistentEntit private final Lazy settingsParameter; private @Nullable ElasticsearchPersistentProperty seqNoPrimaryTermProperty; private @Nullable ElasticsearchPersistentProperty joinFieldProperty; - private @Nullable VersionType versionType; + private @Nullable Document.VersionType versionType; private boolean createIndexAndMapping; private final Dynamic dynamic; private final Map fieldNamePropertyCache = new ConcurrentHashMap<>(); @@ -156,7 +155,7 @@ public String getRefreshInterval() { @Nullable @Override - public VersionType getVersionType() { + public Document.VersionType getVersionType() { return versionType; } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/query/AbstractQuery.java b/src/main/java/org/springframework/data/elasticsearch/core/query/AbstractQuery.java index 3e5f1a386..7281da289 100755 --- a/src/main/java/org/springframework/data/elasticsearch/core/query/AbstractQuery.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/query/AbstractQuery.java @@ -23,9 +23,6 @@ import java.util.List; import java.util.Optional; -import org.elasticsearch.action.search.SearchType; -import org.elasticsearch.action.support.IndicesOptions; -import org.elasticsearch.common.unit.TimeValue; import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Sort; import org.springframework.lang.Nullable; @@ -52,7 +49,7 @@ abstract class AbstractQuery implements Query { protected float minScore; @Nullable protected Collection ids; @Nullable protected String route; - protected SearchType searchType = SearchType.DFS_QUERY_THEN_FETCH; + protected SearchType searchType = SearchType.QUERY_THEN_FETCH; @Nullable protected IndicesOptions indicesOptions; protected boolean trackScores; @Nullable protected String preference; @@ -61,7 +58,7 @@ abstract class AbstractQuery implements Query { @Nullable private Boolean trackTotalHits; @Nullable private Integer trackTotalHitsUpTo; @Nullable private Duration scrollTime; - @Nullable private TimeValue timeout; + @Nullable private Duration timeout; private boolean explain = false; @Nullable private List searchAfter; protected List rescorerQueries = new ArrayList<>(); @@ -271,7 +268,7 @@ public void setScrollTime(@Nullable Duration scrollTime) { @Nullable @Override - public TimeValue getTimeout() { + public Duration getTimeout() { return timeout; } @@ -281,7 +278,7 @@ public TimeValue getTimeout() { * @param timeout * @since 4.2 */ - public void setTimeout(@Nullable TimeValue timeout) { + public void setTimeout(@Nullable Duration timeout) { this.timeout = timeout; } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/query/BulkOptions.java b/src/main/java/org/springframework/data/elasticsearch/core/query/BulkOptions.java index 415143be3..c88b24ee2 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/query/BulkOptions.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/query/BulkOptions.java @@ -15,18 +15,20 @@ */ package org.springframework.data.elasticsearch.core.query; +import java.time.Duration; import java.util.List; -import org.elasticsearch.action.support.ActiveShardCount; -import org.elasticsearch.action.support.WriteRequest; -import org.elasticsearch.common.unit.TimeValue; +import org.springframework.data.elasticsearch.core.ActiveShardCount; +import org.springframework.data.elasticsearch.core.RefreshPolicy; import org.springframework.data.elasticsearch.core.mapping.IndexCoordinates; import org.springframework.lang.Nullable; /** * Options that may be passed to an - * {@link org.springframework.data.elasticsearch.core.DocumentOperations#bulkIndex(List, BulkOptions, IndexCoordinates)} or - * {@link org.springframework.data.elasticsearch.core.DocumentOperations#bulkUpdate(List, BulkOptions, IndexCoordinates)} call.
+ * {@link org.springframework.data.elasticsearch.core.DocumentOperations#bulkIndex(List, BulkOptions, IndexCoordinates)} + * or + * {@link org.springframework.data.elasticsearch.core.DocumentOperations#bulkUpdate(List, BulkOptions, IndexCoordinates)} + * call.
* Use {@link BulkOptions#builder()} to obtain a builder, then set the desired properties and call * {@link BulkOptionsBuilder#build()} to get the BulkOptions object. * @@ -38,13 +40,13 @@ public class BulkOptions { private static final BulkOptions defaultOptions = builder().build(); - private final @Nullable TimeValue timeout; - private final @Nullable WriteRequest.RefreshPolicy refreshPolicy; + private final @Nullable Duration timeout; + private final @Nullable RefreshPolicy refreshPolicy; private final @Nullable ActiveShardCount waitForActiveShards; private final @Nullable String pipeline; private final @Nullable String routingId; - private BulkOptions(@Nullable TimeValue timeout, @Nullable WriteRequest.RefreshPolicy refreshPolicy, + private BulkOptions(@Nullable Duration timeout, @Nullable RefreshPolicy refreshPolicy, @Nullable ActiveShardCount waitForActiveShards, @Nullable String pipeline, @Nullable String routingId) { this.timeout = timeout; this.refreshPolicy = refreshPolicy; @@ -54,12 +56,12 @@ private BulkOptions(@Nullable TimeValue timeout, @Nullable WriteRequest.RefreshP } @Nullable - public TimeValue getTimeout() { + public Duration getTimeout() { return timeout; } @Nullable - public WriteRequest.RefreshPolicy getRefreshPolicy() { + public RefreshPolicy getRefreshPolicy() { return refreshPolicy; } @@ -101,20 +103,20 @@ public static BulkOptions defaultOptions() { */ public static class BulkOptionsBuilder { - private @Nullable TimeValue timeout; - private @Nullable WriteRequest.RefreshPolicy refreshPolicy; + private @Nullable Duration timeout; + private @Nullable RefreshPolicy refreshPolicy; private @Nullable ActiveShardCount waitForActiveShards; private @Nullable String pipeline; private @Nullable String routingId; private BulkOptionsBuilder() {} - public BulkOptionsBuilder withTimeout(TimeValue timeout) { + public BulkOptionsBuilder withTimeout(Duration timeout) { this.timeout = timeout; return this; } - public BulkOptionsBuilder withRefreshPolicy(WriteRequest.RefreshPolicy refreshPolicy) { + public BulkOptionsBuilder withRefreshPolicy(RefreshPolicy refreshPolicy) { this.refreshPolicy = refreshPolicy; return this; } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/query/ByQueryResponse.java b/src/main/java/org/springframework/data/elasticsearch/core/query/ByQueryResponse.java index 42476b507..7986815a1 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/query/ByQueryResponse.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/query/ByQueryResponse.java @@ -17,11 +17,7 @@ import java.util.Collections; import java.util.List; -import java.util.stream.Collectors; -import org.elasticsearch.action.bulk.BulkItemResponse; -import org.elasticsearch.index.reindex.BulkByScrollResponse; -import org.elasticsearch.index.reindex.ScrollableHitSource; import org.springframework.lang.Nullable; /** @@ -47,8 +43,8 @@ public class ByQueryResponse { private final List searchFailures; private ByQueryResponse(long took, boolean timedOut, long total, long updated, long deleted, int batches, - long versionConflicts, long noops, long bulkRetries, long searchRetries, - @Nullable String reasonCancelled, List failures, List searchFailures) { + long versionConflicts, long noops, long bulkRetries, long searchRetries, @Nullable String reasonCancelled, + List failures, List searchFailures) { this.took = took; this.timedOut = timedOut; this.total = total; @@ -61,8 +57,8 @@ private ByQueryResponse(long took, boolean timedOut, long total, long updated, l this.searchRetries = searchRetries; this.reasonCancelled = reasonCancelled; this.failures = failures; - this.searchFailures = searchFailures; - } + this.searchFailures = searchFailures; + } /** * The number of milliseconds from start to end of the whole operation. @@ -151,14 +147,14 @@ public List getFailures() { return failures; } - /** - * Failures during search phase - */ - public List getSearchFailures() { - return searchFailures; - } + /** + * Failures during search phase + */ + public List getSearchFailures() { + return searchFailures; + } - /** + /** * Create a new {@link ByQueryResponseBuilder} to build {@link ByQueryResponse} * * @return a new {@link ByQueryResponseBuilder} to build {@link ByQueryResponse} @@ -167,34 +163,6 @@ public static ByQueryResponseBuilder builder() { return new ByQueryResponseBuilder(); } - public static ByQueryResponse of(BulkByScrollResponse bulkByScrollResponse) { - final List failures = bulkByScrollResponse.getBulkFailures() // - .stream() // - .map(Failure::of) // - .collect(Collectors.toList()); // - - final List searchFailures = bulkByScrollResponse.getSearchFailures() // - .stream() // - .map(SearchFailure::of) // - .collect(Collectors.toList());// - - return ByQueryResponse.builder() // - .withTook(bulkByScrollResponse.getTook().getMillis()) // - .withTimedOut(bulkByScrollResponse.isTimedOut()) // - .withTotal(bulkByScrollResponse.getTotal()) // - .withUpdated(bulkByScrollResponse.getUpdated()) // - .withDeleted(bulkByScrollResponse.getDeleted()) // - .withBatches(bulkByScrollResponse.getBatches()) // - .withVersionConflicts(bulkByScrollResponse.getVersionConflicts()) // - .withNoops(bulkByScrollResponse.getNoops()) // - .withBulkRetries(bulkByScrollResponse.getBulkRetries()) // - .withSearchRetries(bulkByScrollResponse.getSearchRetries()) // - .withReasonCancelled(bulkByScrollResponse.getReasonCancelled()) // - .withFailures(failures) // - .withSearchFailure(searchFailures) // - .build(); // - } - public static class Failure { @Nullable private final String index; @@ -267,25 +235,6 @@ public static FailureBuilder builder() { return new FailureBuilder(); } - /** - * Create a new {@link Failure} from {@link BulkItemResponse.Failure} - * - * @param failure {@link BulkItemResponse.Failure} to translate - * @return a new {@link Failure} - */ - public static Failure of(BulkItemResponse.Failure failure) { - return builder() // - .withIndex(failure.getIndex()) // - .withType(failure.getType()) // - .withId(failure.getId()) // - .withStatus(failure.getStatus().getStatus()) // - .withAborted(failure.isAborted()) // - .withCause(failure.getCause()) // - .withSeqNo(failure.getSeqNo()) // - .withTerm(failure.getTerm()) // - .build(); // - } - /** * Builder for {@link Failure} */ @@ -348,113 +297,97 @@ public Failure build() { } public static class SearchFailure { - private final Throwable reason; - @Nullable private final Integer status; - @Nullable private final String index; - @Nullable private final Integer shardId; - @Nullable private final String nodeId; - - private SearchFailure(Throwable reason, @Nullable Integer status, @Nullable String index, - @Nullable Integer shardId, @Nullable String nodeId) { - this.reason = reason; - this.status = status; - this.index = index; - this.shardId = shardId; - this.nodeId = nodeId; - } - - public Throwable getReason() { - return reason; - } - - @Nullable - public Integer getStatus() { - return status; - } - - @Nullable - public String getIndex() { - return index; - } - - @Nullable - public Integer getShardId() { - return shardId; - } - - @Nullable - public String getNodeId() { - return nodeId; - } - - /** - * Create a new {@link SearchFailureBuilder} to build {@link SearchFailure} - * - * @return a new {@link SearchFailureBuilder} to build {@link SearchFailure} - */ - public static SearchFailureBuilder builder() { - return new SearchFailureBuilder(); - } - - /** - * Create a new {@link SearchFailure} from {@link ScrollableHitSource.SearchFailure} - * - * @param searchFailure {@link ScrollableHitSource.SearchFailure} to translate - * @return a new {@link SearchFailure} - */ - public static SearchFailure of(ScrollableHitSource.SearchFailure searchFailure) { - return builder() // - .withReason(searchFailure.getReason()) // - .withIndex(searchFailure.getIndex()) // - .withNodeId(searchFailure.getNodeId()) // - .withShardId(searchFailure.getShardId()) // - .withStatus(searchFailure.getStatus().getStatus()) // - .build(); // - } - - /** - * Builder for {@link SearchFailure} - */ - public static final class SearchFailureBuilder { - private Throwable reason; - @Nullable private Integer status; - @Nullable private String index; - @Nullable private Integer shardId; - @Nullable private String nodeId; - - private SearchFailureBuilder() {} - - public SearchFailureBuilder withReason(Throwable reason) { - this.reason = reason; - return this; - } - - public SearchFailureBuilder withStatus(Integer status) { - this.status = status; - return this; - } - - public SearchFailureBuilder withIndex(String index) { - this.index = index; - return this; - } - - public SearchFailureBuilder withShardId(Integer shardId) { - this.shardId = shardId; - return this; - } - - public SearchFailureBuilder withNodeId(String nodeId) { - this.nodeId = nodeId; - return this; - } - - public SearchFailure build() { - return new SearchFailure(reason, status, index, shardId, nodeId); - } - } - - } + private final Throwable reason; + @Nullable private final Integer status; + @Nullable private final String index; + @Nullable private final Integer shardId; + @Nullable private final String nodeId; + + private SearchFailure(Throwable reason, @Nullable Integer status, @Nullable String index, @Nullable Integer shardId, + @Nullable String nodeId) { + this.reason = reason; + this.status = status; + this.index = index; + this.shardId = shardId; + this.nodeId = nodeId; + } + + public Throwable getReason() { + return reason; + } + + @Nullable + public Integer getStatus() { + return status; + } + + @Nullable + public String getIndex() { + return index; + } + + @Nullable + public Integer getShardId() { + return shardId; + } + + @Nullable + public String getNodeId() { + return nodeId; + } + + /** + * Create a new {@link SearchFailureBuilder} to build {@link SearchFailure} + * + * @return a new {@link SearchFailureBuilder} to build {@link SearchFailure} + */ + public static SearchFailureBuilder builder() { + return new SearchFailureBuilder(); + } + + /** + * Builder for {@link SearchFailure} + */ + public static final class SearchFailureBuilder { + private Throwable reason; + @Nullable private Integer status; + @Nullable private String index; + @Nullable private Integer shardId; + @Nullable private String nodeId; + + private SearchFailureBuilder() {} + + public SearchFailureBuilder withReason(Throwable reason) { + this.reason = reason; + return this; + } + + public SearchFailureBuilder withStatus(Integer status) { + this.status = status; + return this; + } + + public SearchFailureBuilder withIndex(String index) { + this.index = index; + return this; + } + + public SearchFailureBuilder withShardId(Integer shardId) { + this.shardId = shardId; + return this; + } + + public SearchFailureBuilder withNodeId(String nodeId) { + this.nodeId = nodeId; + return this; + } + + public SearchFailure build() { + return new SearchFailure(reason, status, index, shardId, nodeId); + } + } + + } public static final class ByQueryResponseBuilder { private long took; @@ -534,9 +467,9 @@ public ByQueryResponseBuilder withFailures(List failures) { } public ByQueryResponseBuilder withSearchFailure(List searchFailures) { - this.searchFailures = searchFailures; - return this; - } + this.searchFailures = searchFailures; + return this; + } public ByQueryResponse build() { return new ByQueryResponse(took, timedOut, total, updated, deleted, batches, versionConflicts, noops, bulkRetries, diff --git a/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQuery.java b/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQuery.java index 19e927afd..152ca5244 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQuery.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQuery.java @@ -15,23 +15,35 @@ */ package org.springframework.data.elasticsearch.core.query; -import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder; +import org.springframework.data.elasticsearch.core.query.highlight.Highlight; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; /** - * Encapsulates an Elasticsearch {@link org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder} to prevent - * leaking of Elasticsearch classes into the query API. - * + * Combines a {@link Highlight} definition with the type of the entity where it's present on a method. + * * @author Peter-Josef Meisch * @since 4.0 */ public class HighlightQuery { - private final HighlightBuilder highlightBuilder; - public HighlightQuery(HighlightBuilder highlightBuilder) { - this.highlightBuilder = highlightBuilder; + private final Highlight highlight; + @Nullable private final Class type; + + public HighlightQuery(Highlight highlight, @Nullable Class type) { + + Assert.notNull(highlight, "highlight must not be null"); + + this.highlight = highlight; + this.type = type; + } + + public Highlight getHighlight() { + return highlight; } - public HighlightBuilder getHighlightBuilder() { - return highlightBuilder; + @Nullable + public Class getType() { + return type; } } diff --git a/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQueryBuilder.java b/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQueryBuilder.java index 0c94044fd..c5269a39a 100644 --- a/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQueryBuilder.java +++ b/src/main/java/org/springframework/data/elasticsearch/core/query/HighlightQueryBuilder.java @@ -20,19 +20,20 @@ import org.elasticsearch.search.fetch.subphase.highlight.AbstractHighlighterBuilder; import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder; -import org.springframework.data.elasticsearch.annotations.Highlight; -import org.springframework.data.elasticsearch.annotations.HighlightField; -import org.springframework.data.elasticsearch.annotations.HighlightParameters; import org.springframework.data.elasticsearch.core.mapping.ElasticsearchPersistentEntity; import org.springframework.data.elasticsearch.core.mapping.ElasticsearchPersistentProperty; +import org.springframework.data.elasticsearch.core.query.highlight.Highlight; +import org.springframework.data.elasticsearch.core.query.highlight.HighlightCommonParameters; +import org.springframework.data.elasticsearch.core.query.highlight.HighlightField; +import org.springframework.data.elasticsearch.core.query.highlight.HighlightFieldParameters; +import org.springframework.data.elasticsearch.core.query.highlight.HighlightParameters; import org.springframework.data.mapping.context.MappingContext; import org.springframework.lang.Nullable; -import org.springframework.util.Assert; import org.springframework.util.StringUtils; /** * Converts the {@link Highlight} annotation from a method to an Elasticsearch {@link HighlightBuilder}. - * + * * @author Peter-Josef Meisch */ public class HighlightQueryBuilder { @@ -45,115 +46,114 @@ public HighlightQueryBuilder( } /** - * creates a HighlightBuilder from an annotation - * + * creates an Elasticsearch HighlightBuilder from an annotation. + * * @param highlight, must not be {@literal null} * @param type the entity type, used to map field names. If null, field names are not mapped. * @return the builder for the highlight */ - public HighlightQuery getHighlightQuery(Highlight highlight, @Nullable Class type) { - - Assert.notNull(highlight, "highlight must not be null"); - + public HighlightBuilder getHighlightBuilder(Highlight highlight, @Nullable Class type) { HighlightBuilder highlightBuilder = new HighlightBuilder(); - addParameters(highlight.parameters(), highlightBuilder, type); + addParameters(highlight.getParameters(), highlightBuilder, type); - for (HighlightField highlightField : highlight.fields()) { - String mappedName = mapFieldName(highlightField.name(), type); + for (HighlightField highlightField : highlight.getFields()) { + String mappedName = mapFieldName(highlightField.getName(), type); HighlightBuilder.Field field = new HighlightBuilder.Field(mappedName); - addParameters(highlightField.parameters(), field, type); + addParameters(highlightField.getParameters(), field, type); highlightBuilder.field(field); } - - return new HighlightQuery(highlightBuilder); + return highlightBuilder; } - private void addParameters(HighlightParameters parameters, AbstractHighlighterBuilder builder, Class type) { + private

void addParameters(P parameters, AbstractHighlighterBuilder builder, + @Nullable Class type) { - if (StringUtils.hasLength(parameters.boundaryChars())) { - builder.boundaryChars(parameters.boundaryChars().toCharArray()); + if (StringUtils.hasLength(parameters.getBoundaryChars())) { + builder.boundaryChars(parameters.getBoundaryChars().toCharArray()); } - if (parameters.boundaryMaxScan() > -1) { - builder.boundaryMaxScan(parameters.boundaryMaxScan()); + if (parameters.getBoundaryMaxScan() > -1) { + builder.boundaryMaxScan(parameters.getBoundaryMaxScan()); } - if (StringUtils.hasLength(parameters.boundaryScanner())) { - builder.boundaryScannerType(parameters.boundaryScanner()); + if (StringUtils.hasLength(parameters.getBoundaryScanner())) { + builder.boundaryScannerType(parameters.getBoundaryScanner()); } - if (StringUtils.hasLength(parameters.boundaryScannerLocale())) { - builder.boundaryScannerLocale(parameters.boundaryScannerLocale()); + if (StringUtils.hasLength(parameters.getBoundaryScannerLocale())) { + builder.boundaryScannerLocale(parameters.getBoundaryScannerLocale()); } - if (parameters.forceSource()) { // default is false - builder.forceSource(parameters.forceSource()); + if (parameters.getForceSource()) { // default is false + builder.forceSource(true); } - if (StringUtils.hasLength(parameters.fragmenter())) { - builder.fragmenter(parameters.fragmenter()); + if (StringUtils.hasLength(parameters.getFragmenter())) { + builder.fragmenter(parameters.getFragmenter()); } - if (parameters.fragmentSize() > -1) { - builder.fragmentSize(parameters.fragmentSize()); + if (parameters.getFragmentSize() > -1) { + builder.fragmentSize(parameters.getFragmentSize()); } - if (parameters.noMatchSize() > -1) { - builder.noMatchSize(parameters.noMatchSize()); + if (parameters.getNoMatchSize() > -1) { + builder.noMatchSize(parameters.getNoMatchSize()); } - if (parameters.numberOfFragments() > -1) { - builder.numOfFragments(parameters.numberOfFragments()); + if (parameters.getNumberOfFragments() > -1) { + builder.numOfFragments(parameters.getNumberOfFragments()); } - if (StringUtils.hasLength(parameters.order())) { - builder.order(parameters.order()); + if (StringUtils.hasLength(parameters.getOrder())) { + builder.order(parameters.getOrder()); } - if (parameters.phraseLimit() > -1) { - builder.phraseLimit(parameters.phraseLimit()); + if (parameters.getPhraseLimit() > -1) { + builder.phraseLimit(parameters.getPhraseLimit()); } - if (parameters.preTags().length > 0) { - builder.preTags(parameters.preTags()); + if (parameters.getPreTags().length > 0) { + builder.preTags(parameters.getPreTags()); } - if (parameters.postTags().length > 0) { - builder.postTags(parameters.postTags()); + if (parameters.getPostTags().length > 0) { + builder.postTags(parameters.getPostTags()); } - if (!parameters.requireFieldMatch()) { // default is true - builder.requireFieldMatch(parameters.requireFieldMatch()); + if (!parameters.getRequireFieldMatch()) { // default is true + builder.requireFieldMatch(false); } - if (StringUtils.hasLength(parameters.type())) { - builder.highlighterType(parameters.type()); + if (StringUtils.hasLength(parameters.getType())) { + builder.highlighterType(parameters.getType()); } - if (builder instanceof HighlightBuilder) { + if (builder instanceof HighlightBuilder && parameters instanceof HighlightParameters) { HighlightBuilder highlightBuilder = (HighlightBuilder) builder; + HighlightParameters highlightParameters = (HighlightParameters) parameters; - if (StringUtils.hasLength(parameters.encoder())) { - highlightBuilder.encoder(parameters.encoder()); + if (StringUtils.hasLength(highlightParameters.getEncoder())) { + highlightBuilder.encoder(highlightParameters.getEncoder()); } - if (StringUtils.hasLength(parameters.tagsSchema())) { - highlightBuilder.tagsSchema(parameters.tagsSchema()); + if (StringUtils.hasLength(highlightParameters.getTagsSchema())) { + highlightBuilder.tagsSchema(highlightParameters.getTagsSchema()); } } - if (builder instanceof HighlightBuilder.Field) { + if (builder instanceof HighlightBuilder.Field && parameters instanceof HighlightFieldParameters) { HighlightBuilder.Field field = (HighlightBuilder.Field) builder; + HighlightFieldParameters fieldParameters = (HighlightFieldParameters) parameters; - if (parameters.fragmentOffset() > -1) { - field.fragmentOffset(parameters.fragmentOffset()); + if ((fieldParameters).getFragmentOffset() > -1) { + field.fragmentOffset(fieldParameters.getFragmentOffset()); } - if (parameters.matchedFields().length > 0) { - field.matchedFields(Arrays.stream(parameters.matchedFields()) // + if (fieldParameters.getMatchedFields().length > 0) { + field.matchedFields(Arrays.stream(fieldParameters.getMatchedFields()) // .map(fieldName -> mapFieldName(fieldName, type)) // .collect(Collectors.toList()) // .toArray(new String[] {})); // diff --git a/src/main/java/org/springframework/data/elasticsearch/core/query/IndicesOptions.java b/src/main/java/org/springframework/data/elasticsearch/core/query/IndicesOptions.java new file mode 100644 index 000000000..7bcb41d57 --- /dev/null +++ b/src/main/java/org/springframework/data/elasticsearch/core/query/IndicesOptions.java @@ -0,0 +1,89 @@ +/* + * Copyright 2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.data.elasticsearch.core.query; + +import java.util.EnumSet; + +/** + * Class mirroring the IndicesOptions from Elasticsearch in Spring Data Elasticsearch API. + * + * @author Peter-Josef Meisch + * @since 4.3 + */ +public class IndicesOptions { + + private EnumSet