Restructure reference docs (#988)

Author: swallez

docs/docset.yml

@@ -2,14 +2,14 @@ project: 'Java API Client'
 exclude:
   - external-resources.md
   - design/*
-  - local/README.md
 cross_links:
   - docs-content
   - elasticsearch
 toc:
   - toc: reference
   - toc: release-notes
 subs:
-  serverless-full:   "Elastic Cloud Serverless"
-  es3:   "Elasticsearch Serverless"
   es:   "Elasticsearch"
+  version: "9.0.0"
+  # Not used by docs-builder, but present in `% :::{include-bloc}` preprocessing directives (see IncludeExpander.java)
+  doc-tests-src: "../../java-client/src/test/java/co/elastic/clients/documentation"

docs/local/README.md

@@ -0,0 +1 @@
+The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/main/java-client/src/test/java/co/elastic/clients/documentation).

docs/reference/_snippets/doc-tests-blurb.md

@@ -0,0 +1,3 @@
+:::{note}
+This is the legacy RestClient. Please migrate to [Rest5Client](/reference/transport/rest5-client/index.md), a drop-in replacement with an up-to-date http library.
+:::

docs/reference/_snippets/legacy-rest-client.md

@@ -9,13 +9,12 @@ API clients come in two flavors: blocking and asynchronous. All methods on async
 
 Both flavors can be used at the same time depending on your needs, sharing the same transport object:
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=blocking-and-async
 ```java
-ElasticsearchTransport transport = ...
-
 // Synchronous blocking client
-ElasticsearchClient client = new ElasticsearchClient(transport);
+ElasticsearchClient esClient = new ElasticsearchClient(transport);
 
-if (client.exists(b -> b.index("products").id("foo")).value()) {
+if (esClient.exists(b -> b.index("products").id("foo")).value()) {
     logger.info("product exists");
 }
 
@@ -36,5 +35,6 @@ asyncClient
 
 Although we won’t go in deeper details on asynchronous programming in Java, remember to handle failures of asynchronous tasks. It’s easy to overlook them and have errors go unnoticed.
 
-The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation).
+:::{include} /reference/_snippets/doc-tests-blurb.md
+:::
 

docs/reference/api-conventions.md

@@ -10,9 +10,10 @@ mapped_pages:
 
 All data types in the Java API Client are immutable. Object creation uses the [builder pattern](https://www.informit.com/articles/article.aspx?p=1216151&seqNum=2) that was popularized in **Effective Java** in 2008.
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=builders
 ```java
-ElasticsearchClient client = ...
-CreateIndexResponse createResponse = client.indices().create(
+ElasticsearchClient esClient = createClient();
+CreateIndexResponse createResponse = esClient.indices().create(
     new CreateIndexRequest.Builder()
         .index("my-index")
         .aliases("foo",
@@ -29,9 +30,10 @@ Note that a builder should not be reused after its `build()` method has been cal
 
 Although this works nicely, having to instantiate builder classes and call the `build()` method is a bit verbose. So every property setter in the Java API Client also accepts a lambda expression that takes a newly created builder as a parameter and returns a populated builder. The snippet above can also be written as:
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=builder-lambdas
 ```java
-ElasticsearchClient client = ...
-CreateIndexResponse createResponse = client.indices()
+ElasticsearchClient esClient = createClient();
+CreateIndexResponse createResponse = esClient.indices()
     .create(createIndexBuilder -> createIndexBuilder
         .index("my-index")
         .aliases("foo", aliasBuilder -> aliasBuilder
@@ -44,9 +46,10 @@ This approach allows for much more concise code, and also avoids importing class
 
 Note in the above example that builder variables are only used to start a chain of property setters. The names of these variables are therefore unimportant and can be shortened to improve readability:
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=builder-lambdas-short
 ```java
-ElasticsearchClient client = ...
-CreateIndexResponse createResponse = client.indices()
+ElasticsearchClient esClient = createClient();
+CreateIndexResponse createResponse = esClient.indices()
     .create(c -> c
         .index("my-index")
         .aliases("foo", a -> a
@@ -59,9 +62,10 @@ Builder lambdas become particularly useful with complex nested queries like the
 
 This example also highlights a useful naming convention for builder parameters in deeply nested structures. For lambda expressions with a single argument, Kotlin provides the implicit `it` parameter and Scala allows use of `_`. This can be approximated in Java by using an underscore or a single letter prefix followed by a number representing the depth level (i.e. `_0`, `_1`, or `b0`, `b1` and so on). Not only does this remove the need to create throw-away variable names, but it also improves code readability. Correct indentation also allows the structure of the query to stand out.
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=builder-intervals
 ```java
-ElasticsearchClient client = ...
-SearchResponse<SomeApplicationData> results = client
+ElasticsearchClient esClient = createClient();
+SearchResponse<SomeApplicationData> results = esClient
     .search(b0 -> b0
         .query(b1 -> b1
             .intervals(b2 -> b2
@@ -92,12 +96,13 @@ SearchResponse<SomeApplicationData> results = client
                 )
             )
         ),
-    SomeApplicationData.class <1>
+    SomeApplicationData.class // <1>
 );
 ```
 
 1. Search results will be mapped to `SomeApplicationData` instances to be readily available to the application.
 
 
-The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation).
+:::{include} /reference/_snippets/doc-tests-blurb.md
+:::
 

docs/reference/blocking-async.md docs/reference/api-conventions/blocking-async.md

@@ -0,0 +1,28 @@
+---
+mapped_pages:
+  - https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/api-conventions.html
+---
+
+# API conventions [api-conventions]
+
+The Java API Client uses a very consistent code structure, using modern code patterns that make complex requests easier to write and complex responses easier to process. The sections below explain these in details.
+
+* [Package structure and namespace clients](package-structure.md)
+* [Method naming conventions](method-naming.md)
+* [Blocking and asynchronous clients](blocking-async.md)
+* [Building API objects](building-objects.md)
+* [Lists and maps](lists-maps.md)
+* [Variant types](variant-types.md)
+* [Object life cycles and thread safety](object-lifecycles.md)
+* [Creating API objects from JSON data](loading-json.md)
+* [Exceptions](exception-conventions.md)
+
+
+
+
+
+
+
+
+
+

docs/reference/building-objects.md docs/reference/api-conventions/building-objects.md

@@ -12,6 +12,7 @@ Properties of type `List` and `Map` are exposed by object builders as a set of o
 
 Object builders create immutable objects, and this also applies to list and map properties that are made immutable at object construction time.
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=collections
 ```java
 // Prepare a list of index names
 List<String> names = Arrays.asList("idx-a", "idx-b", "idx-c");
@@ -48,7 +49,6 @@ SearchRequest search = SearchRequest.of(r -> r
 );
 ```
 
-
 ## List and map values are never `null` [_list_and_map_values_are_never_null]
 
 The {{es}} API has a lot of optional properties. For single-valued properties, the Java API Client represents missing optional values as `null`. Applications therefore have to null-check optional values before using them.
@@ -57,6 +57,7 @@ For lists and maps however, applications often only care about whether they’re
 
 If you ever need to distinguish between a missing (undefined) optional collection and an effectively-empty collection returned by {{es}}, the `ApiTypeHelper` class provides a utility method to distinguish them:
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/ApiConventionsTest.java tag=optional-collections
 ```java
 NodeStatistics stats = NodeStatistics.of(b -> b
     .total(1)
@@ -73,5 +74,6 @@ assertEquals(0, stats.failures().size());
 assertFalse(ApiTypeHelper.isDefined(stats.failures()));
 ```
 
-The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation).
+:::{include} /reference/_snippets/doc-tests-blurb.md
+:::
 

docs/reference/exception-conventions.md docs/reference/api-conventions/exception-conventions.md

@@ -31,16 +31,17 @@ Consider a resource file `some-index.json` containing an index definition:
 
 You can create an index from that definition as follows:
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/LoadingJsonTest.java tag=load-index
 ```java
 InputStream input = this.getClass()
-    .getResourceAsStream("some-index.json"); <1>
+    .getResourceAsStream("some-index.json"); // <1>
 
 CreateIndexRequest req = CreateIndexRequest.of(b -> b
     .index("some-index")
-    .withJson(input) <2>
+    .withJson(input) // <2>
 );
 
-boolean created = client.indices().create(req).acknowledged();
+boolean created = esClient.indices().create(req).acknowledged();
 ```
 
 1. open an input stream for the JSON resource file.
@@ -52,17 +53,18 @@ boolean created = client.indices().create(req).acknowledged();
 
 Similarly, you can read documents to be stored in {{es}} from data files:
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/LoadingJsonTest.java tag=ingest-data
 ```java
 FileReader file = new FileReader(new File(dataDir, "document-1.json"));
 
-IndexRequest<JsonData> req; <1>
+IndexRequest<JsonData> req; // <1>
 
 req = IndexRequest.of(b -> b
     .index("some-index")
     .withJson(file)
 );
 
-client.index(req);
+esClient.index(req);
 ```
 
 1. when calling `withJson()` on data structures that have generic type parameters, these generic types will be considered to be `JsonData`.
@@ -73,6 +75,7 @@ client.index(req);
 
 You can combine `withJson()` with regular calls to setter methods. The example below loads the query part of a search request from a `String` and programmatically adds an aggregation.
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/LoadingJsonTest.java tag=query
 ```java
 Reader queryJson = new StringReader(
     "{" +
@@ -86,8 +89,8 @@ Reader queryJson = new StringReader(
     "}");
 
 SearchRequest aggRequest = SearchRequest.of(b -> b
-    .withJson(queryJson) <1>
-    .aggregations("max-cpu", a1 -> a1 <2>
+    .withJson(queryJson) // <1>
+    .aggregations("max-cpu", a1 -> a1 // <2>
         .dateHistogram(h -> h
             .field("@timestamp")
             .calendarInterval(CalendarInterval.Hour)
@@ -99,8 +102,8 @@ SearchRequest aggRequest = SearchRequest.of(b -> b
     .size(0)
 );
 
-Map<String, Aggregate> aggs = client
-    .search(aggRequest, Void.class) <3>
+Map<String, Aggregate> aggs = esClient
+    .search(aggRequest, Void.class) // <3>
     .aggregations();
 ```
 
@@ -114,6 +117,7 @@ Map<String, Aggregate> aggs = client
 
 The `withJson()` methods are partial deserializers: the properties loaded from the JSON will set property values or replace the previous ones, but will not reset other properties not found in the JSON input. You can use this to combine multiple JSON snippets to build complex search requests. In the example below, we combine separate definitions of a query that selects some documents and an aggregation that is run on the results of this query.
 
+% :::{include-code} src={{doc-tests-src}}/api_conventions/LoadingJsonTest.java tag=query-and-agg
 ```java
 Reader queryJson = new StringReader(
     "{" +
@@ -124,12 +128,12 @@ Reader queryJson = new StringReader(
     "      }" +
     "    }" +
     "  }," +
-    "  \"size\": 100" + <1>
+    "  \"size\": 100" + // <1>
     "}");
 
 Reader aggregationJson = new StringReader(
     "{" +
-    "  \"size\": 0, " + <2>
+    "  \"size\": 0, " + // <2>
     "  \"aggregations\": {" +
     "    \"hours\": {" +
     "      \"date_histogram\": {" +
@@ -148,12 +152,12 @@ Reader aggregationJson = new StringReader(
     "}");
 
 SearchRequest aggRequest = SearchRequest.of(b -> b
-    .withJson(queryJson) <3>
-    .withJson(aggregationJson) <4>
-    .ignoreUnavailable(true) <5>
+    .withJson(queryJson) // <3>
+    .withJson(aggregationJson) // <4>
+    .ignoreUnavailable(true) // <5>
 );
 
-Map<String, Aggregate> aggs = client
+Map<String, Aggregate> aggs = esClient
     .search(aggRequest, Void.class)
     .aggregations();
 ```
@@ -167,5 +171,6 @@ Map<String, Aggregate> aggs = client
 
 Notice that order matters when the JSON snippets have some common properties: just as when setting property values programmatically, the last value that is set for a property overwrites the previous one.
 
-The source code for the examples above can be found in the [Java API Client tests](https://github.com/elastic/elasticsearch-java/tree/master/java-client/src/test/java/co/elastic/clients/documentation).
+:::{include} /reference/_snippets/doc-tests-blurb.md
+:::
 

docs/reference/api-conventions/index.md

@@ -11,7 +11,7 @@ The Java API Client follows this structure: feature groups are called “namespa
 
 Each of the namespace clients can be accessed from the top level {{es}} client. The only exceptions are the “search” and “document” APIs which are located in the `core` subpackage and can be accessed on the main {{es}} client object.
 
-The snippet below shows how to use the indices namespace client to create an index (the lambda syntax is explained in [Building API objects](/reference/building-objects.md)):
+The snippet below shows how to use the indices namespace client to create an index (the lambda syntax is explained in [Building API objects](building-objects.md)):
 
 ```java
 // Create the "products" index