JAVA-2428: Add developer docs (apache#1339)

Author: olim7t

changelog/README.md

@@ -4,6 +4,7 @@
 
 ### 4.3.0 (in progress)
 
+- [documentation] JAVA-2428: Add developer docs
 - [documentation] JAVA-2503: Migrate Cloud "getting started" page to driver manual
 - [improvement] JAVA-2484: Add errors for cloud misconfiguration
 - [improvement] JAVA-2490: Allow to read the secure bundle from an InputStream

core/src/main/java/com/datastax/oss/driver/internal/core/metadata/NodeInfo.java

@@ -35,7 +35,12 @@
  */
 public interface NodeInfo {
 
-  /** The endpoint that the driver will use to connect to the node. */
+  /**
+   * The endpoint that the driver will use to connect to the node.
+   *
+   * <p>This information is required; the driver will not function properly if this method returns
+   * {@code null}.
+   */
   @NonNull
   EndPoint getEndPoint();
 
@@ -124,6 +129,9 @@ public interface NodeInfo {
   /**
    * An additional map of free-form properties, that can be used by custom implementations. They
    * will be copied as-is into {@link Node#getExtras()}.
+   *
+   * <p>This is not required; if you don't have anything specific to report here, it can be null or
+   * empty.
    */
   @Nullable
   Map<String, Object> getExtras();
@@ -138,7 +146,14 @@ public interface NodeInfo {
   @NonNull
   UUID getHostId();
 
-  /** The current version that is associated with the nodes schema. */
+  /**
+   * The current version that is associated with the node's schema.
+   *
+   * <p>This is not required; the driver reports it in {@link Node#getSchemaVersion()}, but for
+   * informational purposes only. It is not used anywhere internally (schema agreement is checked
+   * with {@link TopologyMonitor#checkSchemaAgreement()}, which by default queries system tables
+   * directly, not this field).
+   */
   @Nullable
   UUID getSchemaVersion();
 }

core/src/main/java/com/datastax/oss/driver/internal/core/metadata/TopologyMonitor.java

@@ -44,7 +44,10 @@ public interface TopologyMonitor extends AsyncAutoCloseable {
    *
    * <p>The completion of the future returned by this method marks the point when the driver
    * considers itself "connected" to the cluster, and proceeds with the rest of the initialization:
-   * refreshing the list of nodes and the metadata, opening connection pools, etc.
+   * refreshing the list of nodes and the metadata, opening connection pools, etc. By then, the
+   * topology monitor should be ready to accept calls to its other methods; in particular, {@link
+   * #refreshNodeList()} will be called shortly after the completion of the future, to load the
+   * initial list of nodes to connect to.
    *
    * <p>If {@code advanced.reconnect-on-init = true} in the configuration, this method is
    * responsible for handling reconnection. That is, if the initial attempt to "connect" to the

manual/.nav

@@ -4,4 +4,5 @@ mapper
 api_conventions
 case_sensitivity
 osgi
-cloud
+cloud
+developer

manual/README.md

@@ -6,6 +6,8 @@ Driver modules:
 * [Query builder](query_builder/): a fluent API to create CQL queries programmatically.
 * [Mapper](mapper/): generates the boilerplate to execute queries and convert the results into
   application-level objects.
+* [Developer docs](developer/): explains the codebase and internal extension points for advanced
+  customization.
 
 Common topics:
 

manual/core/ssl/README.md

@@ -4,8 +4,8 @@
 
 Secure the traffic between the driver and Cassandra.
 
-* `advanced.ssl-engine-factory` in the configuration; defaults to none, also available: JSSE, or
-  write your own.
+* `advanced.ssl-engine-factory` in the configuration; defaults to none, also available:
+  config-based, or write your own.
 * or programmatically:
   [CqlSession.builder().withSslEngineFactory()][SessionBuilder.withSslEngineFactory] or
   [CqlSession.builder().withSslContext()][SessionBuilder.withSslContext].
@@ -178,26 +178,12 @@ CqlSession session = CqlSession.builder()
     .build();
 ```
 
-#### Netty
+#### Netty-tcnative
 
-Netty provides a more efficient SSL implementation based on native OpenSSL support. It's possible to
-customize the driver to use it instead of JSSE.
+Netty supports native integration with OpenSSL / boringssl. The driver does not provide this out of
+the box, but with a bit of custom development it is fairly easy to add. See
+[SslHandlerFactory](../../developer/netty_pipeline/#ssl-handler-factory) in the developer docs.
 
-This is an advanced topic and beyond the scope of this document, but here is an overview:
-
-1. add a dependency to Netty-tcnative: follow
-   [these instructions](http://netty.io/wiki/forked-tomcat-native.html);
-2. write your own implementation of the driver's `SslHandlerFactory`. This is a higher-level
-   abstraction than `SslEngineFactory`, that returns a Netty `SslHandler`. You'll build this handler
-   with Netty's own `SslContext`;
-3. write a subclass of `DefaultDriverContext` that overrides `buildSslHandlerFactory()` to return
-   the custom `SslHandlerFactory` you wrote in step 2. This will cause the driver to completely
-   ignore the `ssl-engine-factory` options in the configuration;
-4. write a subclass of `SessionBuilder` that overrides `buildContext` to return the custom context
-   that you wrote in step 3.
-5. build your session with your custom builder.
-
-Note that this approach relies on the driver's [internal API](../../api_conventions).
 
 [dsClientToNode]: https://docs.datastax.com/en/cassandra/3.0/cassandra/configuration/secureSSLClientToNode.html
 [pickle]: http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-server.html

manual/developer/.nav

@@ -0,0 +1,5 @@
+common
+native_protocol
+netty_pipeline
+request_execution
+admin

manual/developer/README.md

@@ -0,0 +1,19 @@
+## Developer docs
+
+This section explains how driver internals work. The intended audience is:
+
+* driver developers and contributors;
+* framework authors, or architects who want to write advanced customizations and integrations. 
+
+Most of this material will involve "internal" packages; see [API conventions](../api_conventions/)
+for more explanations.
+
+We recommend reading about the [common infrastructure](common/) first. Then the documentation goes
+from lowest to highest level:
+
+* [Native protocol layer](native_protocol/): binary encoding of the TCP payloads;
+* [Netty pipeline](netty_pipeline/): networking and low-level stream management;
+* [Request execution](request_execution/): higher-level handling of user requests and responses;
+* [Administrative tasks](admin/): everything else (cluster state and metadata).
+
+If you're reading this on GitHub, the `.nav` file in each directory contains a suggested order.