Skip to content

Commit 8cb06b6

Browse files
committedSep 7, 2024·
Adding more instrumentation for thread counts and fixing byte count that was ignoring the preamble. Load testing to ensure the server can handle when tons of clients are junk and leave connections open.
1 parent c8102ed commit 8cb06b6

File tree

12 files changed

+288
-32
lines changed

12 files changed

+288
-32
lines changed
 

‎README.md

+15-18
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22

33
**NOTE:** This project is in progress.
44

5-
The goal of this project is to build a full-featured HTTP server and client in plain Java without the use of any libraries. The client and server will use non-blocking NIO in order to provide the highest performance possible.
5+
The goal of this project is to build a full-featured HTTP server and client in plain Java without the use of any libraries. The client and server will use Project Loom virtual threads and blocking I/O so that the Java VM will handle all the context switching between virtual threads as they block on I/O.
6+
7+
For more information about Project Loom and virtual threads, here is a good article to read: https://blogs.oracle.com/javamagazine/post/java-virtual-threads
68

79
## Installation
810

@@ -12,20 +14,20 @@ To add this library to your project, you can include this dependency in your Mav
1214
<dependency>
1315
<groupId>io.fusionauth</groupId>
1416
<artifactId>java-http</artifactId>
15-
<version>0.3.4</version>
17+
<version>0.4.0-RC.2</version>
1618
</dependency>
1719
```
1820

1921
If you are using Gradle, you can add this to your build file:
2022

2123
```groovy
22-
implementation 'io.fusionauth:java-http:0.3.4'
24+
implementation 'io.fusionauth:java-http:0.4.0-RC.2'
2325
```
2426

2527
If you are using Savant, you can add this to your build file:
2628

2729
```groovy
28-
dependency(id: "io.fusionauth:java-http:0.3.4")
30+
dependency(id: "io.fusionauth:java-http:0.4.0-RC.2")
2931
```
3032

3133
## Examples Usages:
@@ -43,7 +45,8 @@ public class Example {
4345
// Handler code goes here
4446
};
4547

46-
HTTPServer server = new HTTPServer().withHandler(handler).withListener(new HTTPListenerConfiguration(4242));
48+
HTTPServer server = new HTTPServer().withHandler(handler)
49+
.withListener(new HTTPListenerConfiguration(4242));
4750
server.start();
4851
// Use server
4952
server.close();
@@ -64,7 +67,8 @@ public class Example {
6467
// Handler code goes here
6568
};
6669

67-
try (HTTPServer server = new HTTPServer().withHandler(handler).withListener(new HTTPListenerConfiguration(4242))) {
70+
try (HTTPServer server = new HTTPServer().withHandler(handler)
71+
.withListener(new HTTPListenerConfiguration(4242))) {
6872
server.start();
6973
// When this block exits, the server will be shutdown
7074
}
@@ -88,7 +92,6 @@ public class Example {
8892
};
8993

9094
HTTPServer server = new HTTPServer().withHandler(handler)
91-
.withNumberOfWorkerThreads(42)
9295
.withShutdownDuration(Duration.ofSeconds(10L))
9396
.withListener(new HTTPListenerConfiguration(4242));
9497
server.start();
@@ -204,15 +207,9 @@ The general requirements and roadmap are as follows:
204207

205208
## FAQ
206209

207-
### Why no Loom?
208-
209-
Project Loom is an exciting development which brings a lot of great new features to Java, such as fibers, continuations and more.
210-
211-
Loom is currently available in Java 19 as a preview feature. Therefore, you can't use it without compiled code that is difficult to use in future Java releases.
212-
213-
This project is anchored to the Java LTS releases to ensure compatibility. Loom will be evaluated once it is out of preview, and available in an LTS version of Java.
210+
### Why virtual threads and not NIO?
214211

215-
The next scheduled LTS release will be Java 21 set to release in September 2023. We are looking forward to that release and to see if we can leverage the Loom features in this project.
212+
Let's face it, NIO is insanely complex to write and maintain. The first 3 versions of `java-http` used NIO with non-blocking selectors, and we encountered numerous bugs, performance issues, etc. If you compare the `0.3-maintenance` branch with `main` of this project, you'll quickly see that switching to virtual threads and standard blocking I/O made our code **MUCH** simpler.
216213

217214
## Helping out
218215

@@ -225,8 +222,8 @@ We are looking for Java developers that are interested in helping us build the c
225222
```bash
226223
$ mkdir ~/savant
227224
$ cd ~/savant
228-
$ wget http://savant.inversoft.org/org/savantbuild/savant-core/2.0.0-RC.6/savant-2.0.0-RC.6.tar.gz
229-
$ tar xvfz savant-2.0.0-RC.6.tar.gz
230-
$ ln -s ./savant-2.0.0-RC.6 current
225+
$ wget http://savant.inversoft.org/org/savantbuild/savant-core/2.0.0-RC.6/savant-2.0.0-RC.7.tar.gz
226+
$ tar xvfz savant-2.0.0-RC.7.tar.gz
227+
$ ln -s ./savant-2.0.0-RC.7 current
231228
$ export PATH=$PATH:~/savant/current/bin/
232229
```

‎build.savant

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jackson5Version = "3.0.1"
1717
restifyVersion = "4.2.1"
1818
testngVersion = "7.10.2"
1919

20-
project(group: "io.fusionauth", name: "java-http", version: "0.4.0-RC.1", licenses: ["ApacheV2_0"]) {
20+
project(group: "io.fusionauth", name: "java-http", version: "0.4.0-RC.2", licenses: ["ApacheV2_0"]) {
2121
workflow {
2222
fetch {
2323
// Dependency resolution order:

‎load-tests/self/build.savant

+1-1
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ project(group: "io.fusionauth", name: "self", version: "0.1.0", licenses: ["Apac
3232

3333
dependencies {
3434
group(name: "compile") {
35-
dependency(id: "io.fusionauth:java-http:0.4.0-{integration}")
35+
dependency(id: "io.fusionauth:java-http:0.4.0-RC.2.{integration}")
3636
}
3737
}
3838

‎load-tests/self/self.iml

+2-2
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,11 @@
1010
<orderEntry type="module-library">
1111
<library>
1212
<CLASSES>
13-
<root url="jar://$USER_HOME$/.savant/cache/io/fusionauth/java-http/0.4.0-{integration}/java-http-0.4.0-{integration}.jar!/" />
13+
<root url="jar://$USER_HOME$/.savant/cache/io/fusionauth/java-http/0.4.0-RC.2.{integration}/java-http-0.4.0-RC.2.{integration}.jar!/" />
1414
</CLASSES>
1515
<JAVADOC />
1616
<SOURCES>
17-
<root url="jar://$USER_HOME$/.savant/cache/io/fusionauth/java-http/0.4.0-{integration}/java-http-0.4.0-{integration}-src.jar!/" />
17+
<root url="jar://$USER_HOME$/.savant/cache/io/fusionauth/java-http/0.4.0-RC.2.{integration}/java-http-0.4.0-RC.2.{integration}-src.jar!/" />
1818
</SOURCES>
1919
</library>
2020
</orderEntry>

‎load-tests/self/src/main/java/io/fusionauth/http/load/Main.java

+5-7
Original file line numberDiff line numberDiff line change
@@ -15,22 +15,19 @@
1515
*/
1616
package io.fusionauth.http.load;
1717

18-
import java.time.Duration;
19-
2018
import io.fusionauth.http.log.Level;
2119
import io.fusionauth.http.log.SystemOutLoggerFactory;
22-
import io.fusionauth.http.server.CountingInstrumenter;
2320
import io.fusionauth.http.server.HTTPListenerConfiguration;
2421
import io.fusionauth.http.server.HTTPServer;
22+
import io.fusionauth.http.server.ThreadSafeCountingInstrumenter;
2523

2624
public class Main {
2725
public static void main(String[] args) throws Exception {
2826
SystemOutLoggerFactory.FACTORY.getLogger(Object.class).setLevel(Level.Debug);
2927

3028
System.out.println("Starting java-http server");
31-
CountingInstrumenter instrumenter = new CountingInstrumenter();
29+
var instrumenter = new ThreadSafeCountingInstrumenter();
3230
try (HTTPServer ignore = new HTTPServer().withHandler(new LoadHandler())
33-
.withClientTimeout(Duration.ofSeconds(100L))
3431
.withCompressByDefault(false)
3532
.withInstrumenter(instrumenter)
3633
.withListener(new HTTPListenerConfiguration(8080))
@@ -39,9 +36,10 @@ public static void main(String[] args) throws Exception {
3936

4037
for (int i = 0; i < 1_000; i++) {
4138
Thread.sleep(10_000);
42-
System.out.printf("Current stats. Bad requests [%s]. Bytes read [%s]. Bytes written [%s]. Chunked requests [%s]. Chunked responses [%s]. Closed connections [%s]. Connections [%s]. Started [%s].\n",
39+
System.out.printf("Current stats. Bad requests [%s]. Bytes read [%s]. Bytes written [%s]. Chunked requests [%s]. Chunked responses [%s]. Closed connections [%s]. Connections [%s]. Started [%s]. Virtual threads [%s].\n",
4340
instrumenter.getBadRequests(), instrumenter.getBytesRead(), instrumenter.getBytesWritten(), instrumenter.getChunkedRequests(),
44-
instrumenter.getChunkedResponses(), instrumenter.getClosedConnections(), instrumenter.getConnections(), instrumenter.getStartedCount());
41+
instrumenter.getChunkedResponses(), instrumenter.getClosedConnections(), instrumenter.getConnections(), instrumenter.getStartedCount(),
42+
instrumenter.getThreadCount());
4543
}
4644

4745
System.out.println("Shutting down java-http server");
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
/*
2+
* Copyright (c) 2024, FusionAuth, All Rights Reserved
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing,
11+
* software distributed under the License is distributed on an
12+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
13+
* either express or implied. See the License for the specific
14+
* language governing permissions and limitations under the License.
15+
*/
16+
package io.fusionauth.http.load;
17+
18+
import java.io.OutputStream;
19+
import java.net.Socket;
20+
import java.util.ArrayList;
21+
import java.util.List;
22+
23+
public class SocketHammer {
24+
public static void main(String[] args) throws Exception {
25+
List<Socket> sockets = new ArrayList<>();
26+
27+
for (int i = 0; i < 10_000; i++) {
28+
try {
29+
Socket socket = new Socket("localhost", 8080);
30+
sockets.add(socket);
31+
System.out.println(i);
32+
33+
OutputStream outputStream = socket.getOutputStream();
34+
outputStream.write("GET".getBytes());
35+
outputStream.flush();
36+
} catch (Exception e) {
37+
//Smother
38+
// System.out.println("Failed");
39+
}
40+
}
41+
42+
Thread.sleep(10_000);
43+
for (Socket socket : sockets) {
44+
socket.close();
45+
}
46+
47+
System.out.println("Done");
48+
}
49+
}
+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
#!/usr/bin/env bash
2+
3+
#
4+
# Copyright (c) 2022, FusionAuth, All Rights Reserved
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
15+
# either express or implied. See the License for the specific
16+
# language governing permissions and limitations under the License.
17+
#
18+
19+
# Grab the path
20+
if [[ ! -d lib ]]; then
21+
echo "Unable to locate library files needed to run the load tests. [lib]"
22+
exit 1
23+
fi
24+
25+
CLASSPATH=.
26+
for f in lib/*.jar; do
27+
CLASSPATH=${CLASSPATH}:${f}
28+
done
29+
30+
suspend=""
31+
if [[ $# -gt 1 && $1 == "--suspend" ]]; then
32+
suspend="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000"
33+
shift
34+
fi
35+
36+
for i in {1..5} ; do
37+
echo "${i}"
38+
~/dev/java/current21/bin/java ${suspend} -cp "${CLASSPATH}" io.fusionauth.http.load.SocketHammer &
39+
done

‎src/main/java/io/fusionauth/http/server/CountingInstrumenter.java

+19-1
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,12 @@
1616
package io.fusionauth.http.server;
1717

1818
/**
19-
* A simple counting instrumenter for the HTTPServer.
19+
* A simple counting instrumenter for the HTTPServer. This is not thread safe, so if you need accurate data, you'll want to use the
20+
* {@link ThreadSafeCountingInstrumenter}.
2021
*
2122
* @author Brian Pontarelli
2223
*/
24+
@SuppressWarnings("unused")
2325
public class CountingInstrumenter implements Instrumenter {
2426
private long badRequests;
2527

@@ -37,6 +39,8 @@ public class CountingInstrumenter implements Instrumenter {
3739

3840
private long startedCount;
3941

42+
private long threadCount;
43+
4044
@Override
4145
public void acceptedConnection() {
4246
connections++;
@@ -94,6 +98,10 @@ public long getStartedCount() {
9498
return startedCount;
9599
}
96100

101+
public long getThreadCount() {
102+
return threadCount;
103+
}
104+
97105
@Override
98106
public void readFromClient(long bytes) {
99107
bytesRead += bytes;
@@ -104,6 +112,16 @@ public void serverStarted() {
104112
startedCount++;
105113
}
106114

115+
@Override
116+
public void threadExited() {
117+
threadCount--;
118+
}
119+
120+
@Override
121+
public void threadStarted() {
122+
threadCount++;
123+
}
124+
107125
@Override
108126
public void wroteToClient(long bytes) {
109127
bytesWritten += bytes;

‎src/main/java/io/fusionauth/http/server/Instrumenter.java

+10
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,16 @@ public interface Instrumenter {
5858
*/
5959
void serverStarted();
6060

61+
/**
62+
* Signals that a virtual thread has exited.
63+
*/
64+
void threadExited();
65+
66+
/**
67+
* Signals that a virtual thread has started.
68+
*/
69+
void threadStarted();
70+
6171
/**
6272
* Called when bytes are written to a client.
6373
*
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
/*
2+
* Copyright (c) 2022, FusionAuth, All Rights Reserved
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing,
11+
* software distributed under the License is distributed on an
12+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
13+
* either express or implied. See the License for the specific
14+
* language governing permissions and limitations under the License.
15+
*/
16+
package io.fusionauth.http.server;
17+
18+
import java.util.concurrent.atomic.AtomicLong;
19+
20+
/**
21+
* A thread safe counting instrumenter for the HTTPServer, that ensures accurate data but could impact performance.
22+
*
23+
* @author Brian Pontarelli
24+
*/
25+
@SuppressWarnings("unused")
26+
public class ThreadSafeCountingInstrumenter implements Instrumenter {
27+
private final AtomicLong badRequests = new AtomicLong();
28+
29+
private final AtomicLong bytesRead = new AtomicLong();
30+
31+
private final AtomicLong bytesWritten = new AtomicLong();
32+
33+
private final AtomicLong chunkedRequests = new AtomicLong();
34+
35+
private final AtomicLong chunkedResponses = new AtomicLong();
36+
37+
private final AtomicLong closedConnections = new AtomicLong();
38+
39+
private final AtomicLong connections = new AtomicLong();
40+
41+
private final AtomicLong startedCount = new AtomicLong();
42+
43+
private final AtomicLong threadCount = new AtomicLong();
44+
45+
@Override
46+
public void acceptedConnection() {
47+
connections.incrementAndGet();
48+
}
49+
50+
@Override
51+
public void badRequest() {
52+
badRequests.incrementAndGet();
53+
}
54+
55+
@Override
56+
public void chunkedRequest() {
57+
chunkedRequests.incrementAndGet();
58+
}
59+
60+
@Override
61+
public void chunkedResponse() {
62+
chunkedResponses.incrementAndGet();
63+
}
64+
65+
@Override
66+
public void connectionClosed() {
67+
closedConnections.incrementAndGet();
68+
}
69+
70+
public long getBadRequests() {
71+
return badRequests.get();
72+
}
73+
74+
public long getBytesRead() {
75+
return bytesRead.get();
76+
}
77+
78+
public long getBytesWritten() {
79+
return bytesWritten.get();
80+
}
81+
82+
public long getChunkedRequests() {
83+
return chunkedRequests.get();
84+
}
85+
86+
public long getChunkedResponses() {
87+
return chunkedResponses.get();
88+
}
89+
90+
public long getClosedConnections() {
91+
return closedConnections.get();
92+
}
93+
94+
public long getConnections() {
95+
return connections.get();
96+
}
97+
98+
public long getStartedCount() {
99+
return startedCount.get();
100+
}
101+
102+
public long getThreadCount() {
103+
return threadCount.get();
104+
}
105+
106+
@Override
107+
public void readFromClient(long bytes) {
108+
bytesRead.addAndGet(bytes);
109+
}
110+
111+
@Override
112+
public void serverStarted() {
113+
startedCount.incrementAndGet();
114+
}
115+
116+
@Override
117+
public void threadExited() {
118+
threadCount.decrementAndGet();
119+
}
120+
121+
@Override
122+
public void threadStarted() {
123+
threadCount.incrementAndGet();
124+
}
125+
126+
@Override
127+
public void wroteToClient(long bytes) {
128+
bytesWritten.addAndGet(bytes);
129+
}
130+
}

‎src/main/java/io/fusionauth/http/server/internal/HTTPWorker.java

+9-1
Original file line numberDiff line numberDiff line change
@@ -86,14 +86,18 @@ public void run() {
8686
HTTPResponse response = null;
8787
boolean keepAlive = false;
8888
try {
89+
if (instrumenter != null) {
90+
instrumenter.threadStarted();
91+
}
92+
8993
while (true) {
9094
logger.debug("Running HTTP worker and preparing to read preamble");
9195
var request = new HTTPRequest(configuration.getContextPath(), configuration.getMultipartBufferSize(),
9296
listener.getCertificate() != null ? "https" : "http", listener.getPort(), socket.getInetAddress().getHostAddress());
9397

9498
logger.debug("Reading preamble");
9599
var inputStream = new ThroughputInputStream(socket.getInputStream(), throughput);
96-
var bodyBytes = HTTPTools.parseRequestPreamble(inputStream, request, buffers.requestBuffer(), () -> state = State.Read);
100+
var bodyBytes = HTTPTools.parseRequestPreamble(inputStream, request, buffers.requestBuffer(), instrumenter, () -> state = State.Read);
97101
var httpInputStream = new HTTPInputStream(configuration, request, inputStream, bodyBytes);
98102
request.setInputStream(httpInputStream);
99103

@@ -179,6 +183,10 @@ public void run() {
179183
// Log the error and signal a failure
180184
logger.error("HTTP worker threw an exception while processing a request.", t);
181185
close(Result.Failure, response);
186+
} finally {
187+
if (instrumenter != null) {
188+
instrumenter.threadExited();
189+
}
182190
}
183191
}
184192

‎src/main/java/io/fusionauth/http/util/HTTPTools.java

+8-1
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
import io.fusionauth.http.log.LoggerFactory;
4040
import io.fusionauth.http.server.HTTPRequest;
4141
import io.fusionauth.http.server.HTTPResponse;
42+
import io.fusionauth.http.server.Instrumenter;
4243

4344
public final class HTTPTools {
4445
private static Logger logger;
@@ -237,11 +238,13 @@ public static HeaderValue parseHeaderValue(String value) {
237238
* @param inputStream The input stream to read the preamble from.
238239
* @param request The HTTP request to populate.
239240
* @param requestBuffer A buffer used for reading to help reduce memory thrashing.
241+
* @param instrumenter The Instrumenter that is informed of bytes read.
240242
* @param readObserver An observer that is called once one byte has been read.
241243
* @return Any leftover body bytes from the last read from the InputStream.
242244
* @throws IOException If the read fails.
243245
*/
244-
public static byte[] parseRequestPreamble(InputStream inputStream, HTTPRequest request, byte[] requestBuffer, Runnable readObserver)
246+
public static byte[] parseRequestPreamble(InputStream inputStream, HTTPRequest request, byte[] requestBuffer, Instrumenter instrumenter,
247+
Runnable readObserver)
245248
throws IOException {
246249
RequestPreambleState state = RequestPreambleState.RequestMethod;
247250
var valueBuffer = new ByteArrayOutputStream(512);
@@ -258,6 +261,10 @@ public static byte[] parseRequestPreamble(InputStream inputStream, HTTPRequest r
258261

259262
logger.trace("Read [{}] from client for preamble.", read);
260263

264+
if (instrumenter != null) {
265+
instrumenter.readFromClient(read);
266+
}
267+
261268
// Tell the callback that we've read at least one byte
262269
readObserver.run();
263270

0 commit comments

Comments
 (0)
Please sign in to comment.