Skip to content

Commit a577dae

Browse files
zhicwuzhicwu
authored
Merge pull request #1251 from ClickHouse/docs-to-website
Move docs to the website
2 parents 6ce0d38 + 21776d9 commit a577dae

File tree

5 files changed

+140
-515
lines changed

5 files changed

+140
-515
lines changed

‎CONTRIBUTING.md

+121
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
## Getting started
2+
ClickHouse-JDBC client is an open-source project, and we welcome any contributions from the community. Please share your ideas, contribute to the codebase, and help us maintain up-to-date documentation.
3+
4+
### Create a fork of the repository and clone it
5+
```bash
6+
git clone https://github.com/[YOUR_USERNAME]/clickhouse-java
7+
cd clickhouse-java
8+
```
9+
10+
### Set up environment
11+
You have installed:
12+
- JDK 8 or JDK 17+
13+
- To build a multi-release jar use JDK 17+ with `~/.m2/toolchains.xml`
14+
```xml
15+
<?xml version="1.0" encoding="UTF8"?>
16+
<toolchains>
17+
<toolchain>
18+
<type>jdk</type>
19+
<provides>
20+
<version>17</version>
21+
</provides>
22+
<configuration>
23+
<jdkHome>/usr/lib/jvm/java-17-openjdk</jdkHome>
24+
</configuration>
25+
</toolchain>
26+
</toolchains>
27+
```
28+
29+
### Build modules
30+
- JDK 8 Use `mvn -Dj8 -DskipITs clean verify` to compile and generate packages.
31+
- JDK 17+ Use create a multi-release jar (see [JEP-238](https://openjdk.java.net/jeps/238)) please verify that you added `~/.m2/toolchains.xml` and run `mvn -DskipITs clean verify`
32+
33+
34+
To create a native binary of JDBC driver for evaluation and testing:
35+
36+
- [install GraalVM](https://www.graalvm.org/latest/docs/getting-started/) and optionally [upx](https://upx.github.io/)
37+
38+
- make sure you have [native-image](https://www.graalvm.org/latest/docs/getting-started/#native-image) installed, and then build the native binary
39+
40+
```bash
41+
cd clickhouse-java
42+
mvn -DskipTests clean install
43+
cd clickhouse-jdbc
44+
mvn -DskipTests -Pnative clean package
45+
# only if you have upx installed
46+
upx -7 -k target/clickhouse-jdbc-bin
47+
```
48+
49+
- run native binary
50+
51+
```bash
52+
# print usage
53+
./target/clickhouse-jdbc-bin
54+
Usage: clickhouse-jdbc-bin [PROPERTIES] <URL> [QUERY] [FILE]
55+
...
56+
57+
# test database connection using JDBC driver
58+
./target/clickhouse-jdbc-bin -Dverbose=true 'jdbc:ch:http://localhost'
59+
Arguments:
60+
- url=jdbc:ch:http://localhost
61+
- query=select 500000000
62+
- file=jdbc.out
63+
64+
Options: action=read, batch=1000, samples=500000000, serde=true, type=, verbose=true
65+
Processed 1 rows in 85.56 ms (11.69 rows/s)
66+
67+
# test query performance using Java client
68+
./target/clickhouse-jdbc-bin -Dverbose=true -Dtype=uint64 'http://localhost'
69+
...
70+
Processed 500,000,000 rows in 52,491.38 ms (9,525,373.89 rows/s)
71+
72+
# test same query again using JVM for comparison - don't have GraalVM EE so JIT wins in my case
73+
java -Dverbose=true -Dtype=uint64 -jar target/clickhouse-jdbc-*-http.jar 'http://localhost'
74+
...
75+
Processed 500,000,000 rows in 25,267.89 ms (19,787,963.94 rows/s)
76+
```
77+
78+
## Testing
79+
80+
By default, [docker](https://docs.docker.com/engine/install/) is required to run integration test. Docker image(defaults to `clickhouse/clickhouse-server`) will be pulled from Internet, and containers will be created automatically by [testcontainers](https://www.testcontainers.org/) before testing. To test against specific version of ClickHouse, you can pass parameter like `-DclickhouseVersion=22.8` to Maven.
81+
82+
In the case you don't want to use docker and/or prefer to test against an existing server, please follow instructions below:
83+
84+
- make sure the server can be accessed using default account(user `default` and no password), which has both DDL and DML privileges
85+
- add below two configuration files to the existing server and expose all defaults ports for external access
86+
- [ports.xml](../../blob/main/clickhouse-client/src/test/resources/containers/clickhouse-server/config.d/ports.xml) - enable all ports
87+
- and [users.xml](../../blob/main/clickhouse-client/src/test/resources/containers/clickhouse-server/users.d/users.xml) - accounts used for integration test
88+
Note: you may need to change root element from `clickhouse` to `yandex` when testing old version of ClickHouse.
89+
- make sure ClickHouse binary(usually `/usr/bin/clickhouse`) is available in PATH, as it's required to test `clickhouse-cli-client`
90+
- put `test.properties` under either `~/.clickhouse` or `src/test/resources` of your project, with content like below:
91+
```properties
92+
clickhouseServer=x.x.x.x
93+
# below properties are only useful for test containers
94+
#clickhouseVersion=latest
95+
#clickhouseTimezone=UTC
96+
#clickhouseImage=clickhouse/clickhouse-server
97+
#additionalPackages=
98+
```
99+
100+
### Tooling
101+
We use [TestNG](http://org.testng/doc) as testing framework and for running ClickHouse Local instance [testcontainers](https://www.testcontainers.org/modules/databases/clickhouse/).
102+
103+
### Running unit tests
104+
105+
Does not require a running ClickHouse server.
106+
Running the maven commands above will trigger the test.
107+
108+
## Benchmark
109+
110+
To benchmark JDBC drivers:
111+
112+
```bash
113+
cd clickhouse-benchmark
114+
mvn clean package
115+
# single thread mode
116+
java -DdbHost=localhost -jar target/benchmarks.jar -t 1 \
117+
-p client=clickhouse-jdbc -p connection=reuse \
118+
-p statement=prepared -p type=default Query.selectInt8
119+
```
120+
121+
It's time consuming to run all benchmarks against all drivers using different parameters for comparison. If you just need some numbers to understand performance, please refer to [this](https://github.com/ClickHouse/clickhouse-java/issues/768) and watch [this](https://github.com/ClickHouse/clickhouse-java/issues/928) for more information and updates.

‎README.md

+4-163
Original file line numberDiff line numberDiff line change
@@ -63,170 +63,11 @@ The library can be downloaded from both [Github Releases](../../releases) and [M
6363
```
6464

6565
### Java Client
66-
67-
```xml
68-
<dependency>
69-
<groupId>com.clickhouse</groupId>
70-
<!-- or clickhouse-grpc-client if you prefer gRPC -->
71-
<artifactId>clickhouse-http-client</artifactId>
72-
<version>0.4.1</version>
73-
</dependency>
74-
```
75-
76-
```java
77-
// endpoint: protocol://host[:port][/database][?param[=value][&param[=value]][#tag[,tag]]
78-
ClickHouseNode endpoint = ClickHouseNode.of("https://localhost"); // http://localhost:8443?ssl=true&sslmode=NONE
79-
// endpoints: [defaultProtocol://]endpoint[,endpoint][/defaultDatabase][?defaultParameters][#defaultTags]
80-
ClickHouseNodes endpoints = ClickHouseNodes.of("http://(https://[email protected]:443),localhost,"
81-
+ "(tcp://localhost?!auto_discovery#experimental),(grpc://localhost#experimental)?failover=3#test")
82-
83-
try (ClickHouseClient client = ClickHouseClient.newInstance(ClickHouseProtocol.HTTP);
84-
ClickHouseResponse response = client.connect(endpoint) // or client.connect(endpoints)
85-
// you'll have to parse response manually if using a different format
86-
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
87-
.query("select * from numbers(:limit)")
88-
.params(1000).executeAndWait()) {
89-
// or response.stream() if you prefer stream API
90-
for (ClickHouseRecord r : response.records()) {
91-
int num = r.getValue(0).asInteger();
92-
// type conversion
93-
String str = r.getValue(0).asString();
94-
LocalDate date = r.getValue(0).asDate();
95-
}
96-
97-
ClickHouseResponseSummary summary = response.getSummary();
98-
long totalRows = summary.getTotalRowsToRead();
99-
}
100-
```
66+
See the [client docs on the ClickHouse website](https://clickhouse.com/docs/en/integrations/language-clients/java/client).
10167

10268
### JDBC Driver
10369

104-
```xml
105-
<dependency>
106-
<groupId>com.clickhouse</groupId>
107-
<artifactId>clickhouse-jdbc</artifactId>
108-
<version>0.4.1</version>
109-
<!-- use uber jar with all dependencies included, change classifier to http for smaller jar -->
110-
<classifier>all</classifier>
111-
</dependency>
112-
```
113-
114-
```java
115-
// jdbc:(ch|clickhouse):[defaultProtocol://]endpoint[,endpoint][/defaultDatabase][?defaultParameters][#defaultTags]
116-
String url = "jdbc:ch:https://play.clickhouse.com:443";
117-
Properties properties = new Properties();
118-
properties.setProperty("user", "explorer");
119-
properties.setProperty("password", "");
120-
// optional properties
121-
properties.setProperty("client_name", "Agent #1");
122-
...
123-
124-
ClickHouseDataSource dataSource = new ClickHouseDataSource(url, properties);
125-
try (Connection conn = dataSource.getConnection();
126-
Statement stmt = conn.createStatement();
127-
ResultSet rs = stmt.executeQuery("show databases")) {
128-
...
129-
}
130-
```
131-
132-
More examples can be found at [here](../../tree/main/examples/jdbc).
133-
134-
## Build with Maven
135-
136-
Use `mvn -Dj8 -DskipITs clean verify` to compile and generate packages if you're using JDK 8. To create a multi-release jar (see [JEP-238](https://openjdk.java.net/jeps/238)), please use JDK 17+ with `~/.m2/toolchains.xml` like below, and run `mvn -DskipITs clean verify` instead.
137-
138-
```xml
139-
<?xml version="1.0" encoding="UTF8"?>
140-
<toolchains>
141-
<toolchain>
142-
<type>jdk</type>
143-
<provides>
144-
<version>17</version>
145-
</provides>
146-
<configuration>
147-
<jdkHome>/usr/lib/jvm/java-17-openjdk</jdkHome>
148-
</configuration>
149-
</toolchain>
150-
</toolchains>
151-
```
152-
153-
To create a native binary of JDBC driver for evaluation and testing:
154-
155-
- [install GraalVM](https://www.graalvm.org/latest/docs/getting-started/) and optionally [upx](https://upx.github.io/)
156-
157-
- make sure you have [native-image](https://www.graalvm.org/latest/docs/getting-started/#native-image) installed, and then build the native binary
158-
159-
```bash
160-
cd clickhouse-java
161-
mvn -DskipTests clean install
162-
cd clickhouse-jdbc
163-
mvn -DskipTests -Pnative clean package
164-
# only if you have upx installed
165-
upx -7 -k target/clickhouse-jdbc-bin
166-
```
167-
168-
- run native binary
169-
170-
```bash
171-
# print usage
172-
./target/clickhouse-jdbc-bin
173-
Usage: clickhouse-jdbc-bin [PROPERTIES] <URL> [QUERY] [FILE]
174-
...
175-
176-
# test database connection using JDBC driver
177-
./target/clickhouse-jdbc-bin -Dverbose=true 'jdbc:ch:http://localhost'
178-
Arguments:
179-
- url=jdbc:ch:http://localhost
180-
- query=select 500000000
181-
- file=jdbc.out
182-
183-
Options: action=read, batch=1000, samples=500000000, serde=true, type=, verbose=true
184-
Processed 1 rows in 85.56 ms (11.69 rows/s)
185-
186-
# test query performance using Java client
187-
./target/clickhouse-jdbc-bin -Dverbose=true -Dtype=uint64 'http://localhost'
188-
...
189-
Processed 500,000,000 rows in 52,491.38 ms (9,525,373.89 rows/s)
190-
191-
# test same query again using JVM for comparison - don't have GraalVM EE so JIT wins in my case
192-
java -Dverbose=true -Dtype=uint64 -jar target/clickhouse-jdbc-*-http.jar 'http://localhost'
193-
...
194-
Processed 500,000,000 rows in 25,267.89 ms (19,787,963.94 rows/s)
195-
```
196-
197-
## Testing
198-
199-
By default, [docker](https://docs.docker.com/engine/install/) is required to run integration test. Docker image(defaults to `clickhouse/clickhouse-server`) will be pulled from Internet, and containers will be created automatically by [testcontainers](https://www.testcontainers.org/) before testing. To test against specific version of ClickHouse, you can pass parameter like `-DclickhouseVersion=22.8` to Maven.
200-
201-
In the case you don't want to use docker and/or prefer to test against an existing server, please follow instructions below:
202-
203-
- make sure the server can be accessed using default account(user `default` and no password), which has both DDL and DML privileges
204-
- add below two configuration files to the existing server and expose all defaults ports for external access
205-
- [ports.xml](../../blob/main/clickhouse-client/src/test/resources/containers/clickhouse-server/config.d/ports.xml) - enable all ports
206-
- and [users.xml](../../blob/main/clickhouse-client/src/test/resources/containers/clickhouse-server/users.d/users.xml) - accounts used for integration test
207-
Note: you may need to change root element from `clickhouse` to `yandex` when testing old version of ClickHouse.
208-
- make sure ClickHouse binary(usually `/usr/bin/clickhouse`) is available in PATH, as it's required to test `clickhouse-cli-client`
209-
- put `test.properties` under either `~/.clickhouse` or `src/test/resources` of your project, with content like below:
210-
```properties
211-
clickhouseServer=x.x.x.x
212-
# below properties are only useful for test containers
213-
#clickhouseVersion=latest
214-
#clickhouseTimezone=UTC
215-
#clickhouseImage=clickhouse/clickhouse-server
216-
#additionalPackages=
217-
```
218-
219-
## Benchmark
220-
221-
To benchmark JDBC drivers:
222-
223-
```bash
224-
cd clickhouse-benchmark
225-
mvn clean package
226-
# single thread mode
227-
java -DdbHost=localhost -jar target/benchmarks.jar -t 1 \
228-
-p client=clickhouse-jdbc -p connection=reuse \
229-
-p statement=prepared -p type=default Query.selectInt8
230-
```
70+
See the [jdbc driver docs on the ClickHouse website](https://clickhouse.com/docs/en/integrations/language-clients/java/jdbc).
23171

232-
It's time consuming to run all benchmarks against all drivers using different parameters for comparison. If you just need some numbers to understand performance, please refer to [this](https://github.com/ClickHouse/clickhouse-java/issues/768) and watch [this](https://github.com/ClickHouse/clickhouse-java/issues/928) for more information and updates.
72+
## Contributing
73+
Check out our [contributing guide](./CONTRIBUTING.md).

‎clickhouse-client/README.md

+5-55
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22

33
Async Java client for ClickHouse. `clickhouse-client` is an abstract module, so it does not work by itself until being used together with an implementation like `clickhouse-http-client`, `clickhouse-grpc-client` or `clickhouse-cli-client`.
44

5+
## Documentation
6+
See the [ClickHouse website](https://clickhouse.com/docs/en/integrations/language-clients/java/client) for the full documentation entry.
7+
58
## Configuration
69

710
You can pass any client option([common](https://github.com/ClickHouse/clickhouse-java/blob/main/clickhouse-client/src/main/java/com/clickhouse/client/config/ClickHouseClientOption.java), [http](https://github.com/ClickHouse/clickhouse-java/blob/main/clickhouse-http-client/src/main/java/com/clickhouse/client/http/config/ClickHouseHttpOption.java), [grpc](https://github.com/ClickHouse/clickhouse-java/blob/main/clickhouse-grpc-client/src/main/java/com/clickhouse/client/grpc/config/ClickHouseGrpcOption.java), and [cli](https://github.com/ClickHouse/clickhouse-java/blob/main/clickhouse-cli-client/src/main/java/com/clickhouse/client/cli/config/ClickHouseCommandLineOption.java)) to `ClickHouseRequest.option()` and [server setting](https://clickhouse.com/docs/en/operations/settings/) to `ClickHouseRequest.set()` before execution, for instance:
@@ -32,58 +35,5 @@ client.connect("http://localhost/system")
3235

3336
[Default value](https://github.com/ClickHouse/clickhouse-java/blob/main/clickhouse-client/src/main/java/com/clickhouse/client/config/ClickHouseDefaults.java) can be either configured via system property or environment variable.
3437

35-
## Quick Start
36-
37-
```xml
38-
<dependency>
39-
<groupId>com.clickhouse</groupId>
40-
<artifactId>clickhouse-http-client</artifactId>
41-
<version>0.4.1</version>
42-
</dependency>
43-
```
44-
45-
```java
46-
// declare a list of servers to connect to
47-
ClickHouseNodes servers = ClickHouseNodes.of(
48-
"jdbc:ch:http://server1.domain,server2.domain,server3.domain/my_db"
49-
+ "?load_balancing_policy=random&health_check_interval=5000&failover=2");
50-
51-
// execute multiple queries in a worker thread one after another within same session
52-
CompletableFuture<List<ClickHouseResponseSummary>> future = ClickHouseClient.send(servers.get(),
53-
"create database if not exists test",
54-
"use test", // change current database from my_db to test
55-
"create table if not exists test_table(s String) engine=Memory",
56-
"insert into test_table values('1')('2')('3')",
57-
"select * from test_table limit 1",
58-
"truncate table test_table",
59-
"drop table if exists test_table");
60-
61-
// block current thread until queries completed, and then retrieve summaries
62-
// List<ClickHouseResponseSummary> results = future.get();
63-
64-
try (ClickHouseClient client = ClickHouseClient.newInstance(ClickHouseProtocol.HTTP)) {
65-
ClickHouseRequest<?> request = client.connect(servers).format(ClickHouseFormat.RowBinaryWithNamesAndTypes);
66-
// load data into a table and wait until it's completed
67-
request.write()
68-
.query("insert into my_table select c2, c3 from input('c1 UInt8, c2 String, c3 Int32')")
69-
.data(myInputStream).execute().thenAccept(response -> {
70-
response.close();
71-
});
72-
73-
// query with named parameter
74-
try (ClickHouseResponse response = request.query(ClickHouseParameterizedQuery.of(
75-
request.getConfig(), "select * from numbers(:limit)")).params(100000).executeAndWait()) {
76-
for (ClickHouseRecord r : response.records()) {
77-
// Don't cache ClickHouseValue / ClickHouseRecord as they're reused for
78-
// corresponding column / row
79-
ClickHouseValue v = r.getValue(0);
80-
// converts to DateTime64(6)
81-
LocalDateTime dateTime = v.asDateTime(6);
82-
// converts to long/int/byte if you want to
83-
long l = v.asLong();
84-
int i = v.asInteger();
85-
byte b = v.asByte();
86-
}
87-
}
88-
}
89-
```
38+
## Examples
39+
For more example please check [here](https://github.com/ClickHouse/clickhouse-java/tree/main/examples/client).

0 commit comments

Comments
 (0)