Update Apache HttpClient 5 to use recommended APIs and remove deprecated code #6312

joviegas · 2025-08-01T16:18:23Z

Motivation and Context

Merge PR from project branch to master
Replace httpclient.execute call with httpclient.executeOpen as mentioned in deprecation notes of httpclient.execute #6298
Move connection timeout configuration from RequestConfig to ConnectionConfig in Apache HttpClient 5 #6293
Replace deprecated SSLConnectionSocketFactory with recommended API #6281

Modifications

Testing

Mentioned ineach of PR , did a Benchmark test after all this changes

JDK 17

    # JMH version: 1.37
    # VM version: JDK 17.0.16, OpenJDK 64-Bit Server VM, 17.0.16+8-LTS
    # VM invoker: /usr/lib/jvm/java-17-amazon-corretto.x86_64/bin/java
    # VM options: -Xms2G -Xmx2G
    # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
    # Warmup: 2 iterations, 15 s each
    # Measurement: 3 iterations, 10 s each
    # Timeout: 10 min per iteration
    # Threads: 1 thread, will synchronize iterations
    # Benchmark mode: Throughput, ops/time
    # Benchmark: software.amazon.awssdk.benchmark.apache5.Apache5Benchmark.simplePut
    # Parameters: (executorType = platform, maxConnections = 50, testDataInMB = 5, threadCount = 10)
    ============================================================================================================================================
    BENCHMARK RESULTS SUMMARY
    ============================================================================================================================================
    Client Type          | Benchmark                                          | Throughput      | Avg Latency     | P99 Latency     | Threads   
    --------------------------------------------------------------------------------------------------------------------------------------------
    Apache4              | multiThreadedGet                                   |          4.57/s |        218.90 ms |        328.35 ms |         10
    Apache4              | multiThreadedPut                                   |          4.68/s |        213.50 ms |        320.26 ms |         10
    Apache4              | simpleGet                                          |         17.30/s |         57.81 ms |         86.71 ms |          1
    Apache4              | simplePut                                          |          6.99/s |        143.14 ms |        214.71 ms |          1
    Apache5              | multiThreadedGet                                   |          4.57/s |        218.95 ms |        328.42 ms |         10
    Apache5              | multiThreadedPut                                   |          3.94/s |        253.70 ms |        380.56 ms |         10
    Apache5              | simpleGet                                          |         16.83/s |         59.42 ms |         89.13 ms |          1
    Apache5              | simplePut                                          |          6.76/s |        147.94 ms |        221.92 ms |          1
    ============================================================================================================================================
    Total benchmark results: 8
    ============================================================================================================================================
     
    PERFORMANCE COMPARISON (Apache5 vs Apache4):
    ================================================================================
     
    multiThreadedPut:
    --------------------------------------------------------------------------------
      Apache5             : -15.8% throughput, -18.8% latency improvement
     
    simpleGet:
    --------------------------------------------------------------------------------
      Apache5             : -2.7% throughput, -2.8% latency improvement
     
    multiThreadedGet:
    --------------------------------------------------------------------------------
      Apache5             : -0.0% throughput, -0.0% latency improvement
     
    simplePut:
    --------------------------------------------------------------------------------
      Apache5             : -3.2% throughput, -3.4% latency improvement
     
    ======================================================================

JDK 21

    # JMH version: 1.37
    # VM version: JDK 21.0.7, OpenJDK 64-Bit Server VM, 21.0.7+6-LTS
    # VM invoker: /opt/java/amazon-corretto-21.0.7.6.1-linux-x64/bin/java
    # VM options: -Xms2G -Xmx2G
    # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
    # Warmup: 2 iterations, 15 s each
    # Measurement: 3 iterations, 10 s each
    # Timeout: 10 min per iteration
    # Threads: 1 thread, will synchronize iterations
    # Benchmark mode: Throughput, ops/time
    # Benchmark: software.amazon.awssdk.benchmark.apache5.Apache5VirtualBenchmark.simplePut
    # Parameters: (maxConnections = 50, testDataInMB = 5, threadCount = 10)
     
    15:37:47.119 [main] INFO  software.amazon.awssdk.benchmark.UnifiedBenchmarkRunner - 
    Benchmark complete! CloudWatch metrics published with run ID: 2025-08-01T15:21:55.111454710Z
     
    ============================================================================================================================================
    BENCHMARK RESULTS SUMMARY
    ============================================================================================================================================
    Client Type          | Benchmark                                          | Throughput      | Avg Latency     | P99 Latency     | Threads   
    --------------------------------------------------------------------------------------------------------------------------------------------
    Apache4              | multiThreadedGet                                   |          4.57/s |        218.96 ms |        328.44 ms |         10
    Apache4              | multiThreadedPut                                   |          4.01/s |        249.38 ms |        374.08 ms |         10
    Apache4              | simpleGet                                          |         17.19/s |         58.18 ms |         87.27 ms |          1
    Apache4              | simplePut                                          |          5.97/s |        167.58 ms |        251.36 ms |          1
    Apache5              | multiThreadedGet                                   |          4.57/s |        218.93 ms |        328.40 ms |         10
    Apache5              | multiThreadedPut                                   |          4.63/s |        216.11 ms |        324.17 ms |         10
    Apache5              | simpleGet                                          |         17.25/s |         57.98 ms |         86.96 ms |          1
    Apache5              | simplePut                                          |          6.60/s |        151.56 ms |        227.34 ms |          1
    Apache5-Virtual      | multiThreadedGet                                   |          4.66/s |        214.40 ms |        321.59 ms |         10
    Apache5-Virtual      | multiThreadedPut                                   |          4.57/s |        218.74 ms |        328.12 ms |         10
    Apache5-Virtual      | simpleGet                                          |         16.90/s |         59.17 ms |         88.76 ms |          1
    Apache5-Virtual      | simplePut                                          |          7.03/s |        142.15 ms |        213.23 ms |          1
    ============================================================================================================================================
    Total benchmark results: 12
    ============================================================================================================================================
     
    PERFORMANCE COMPARISON (Apache5 vs Apache4):
    ================================================================================
     
    multiThreadedPut:
    --------------------------------------------------------------------------------
      Apache5             : +15.4% throughput, +13.3% latency improvement
      Apache5-Virtual     : +14.0% throughput, +12.3% latency improvement
     
    simpleGet:
    --------------------------------------------------------------------------------
      Apache5             : +0.3% throughput, +0.3% latency improvement
      Apache5-Virtual     : -1.7% throughput, -1.7% latency improvement
     
    multiThreadedGet:
    --------------------------------------------------------------------------------
      Apache5             : +0.0% throughput, +0.0% latency improvement
      Apache5-Virtual     : +2.1% throughput, +2.1% latency improvement
     
    simplePut:
    --------------------------------------------------------------------------------
      Apache5             : +10.6% throughput, +9.6% latency improvement
      Apache5-Virtual     : +17.9% throughput, +15.2% latency improvement
     
    ================================================================================

License

I confirm that this pull request can be released under the Apache 2 license

* Add initial module for Apache5x for seting up package * Add based on new module checklist

…pache5SDKHttpClient (#6088) * Add initial module for Apache5x for seting up package * Add based on new module checklist * Baseline all the classes from Apache4 SDK client to the new Apache5 module

…aring Checkstyles and spotbug issues (#6100) * Phase 2 , getting Apache 5 compilation and Junit ready along with clearing Checkstyles and spotbug issues * Handle comments from review * Handle comments from Zoe

…reamRequestEntity repeatability (#6132) * Fix HTTP authentication retry failures by improving RepeatableInputStreamRequestEntity repeatability * Upated test cases * Handled comments

This reverts commit 64f7d33.

* Fix architecture test failures for apache5.x * Checkstyle issues

…ientConnectionManager for Connection Manager (#6147) * Fix architecture test failures for apache5.x * Checkstyle issues * Update to use PoolingHttpClientConnectionManager class reference that is implementation of HttpClientConnectionManager

…6154) * Fix architecture test failures for apache5.x * Checkstyle issues * Update to use PoolingHttpClientConnectionManager class reference that is implementation of HttpClientConnectionManager * Fix stream reset failure in RepeatableInputStreamRequestEntity by storing content reference to avoid multiple ContentStreamProvider.newStream() calls that cause IOException when retrying requests with non-resettable streams * writeTo_ConcurrentWrites_HandlesCorrectly no longer needed since even Apache 4.x doesnot suports this

…zil.json (#6191) * Fix architecture test failures for apache5.x * Checkstyle issues * Update to use PoolingHttpClientConnectionManager class reference that is implementation of HttpClientConnectionManager * Fix stream reset failure in RepeatableInputStreamRequestEntity by storing content reference to avoid multiple ContentStreamProvider.newStream() calls that cause IOException when retrying requests with non-resettable streams * writeTo_ConcurrentWrites_HandlesCorrectly no longer needed since even Apache 4.x doesnot suports this * Fix connectionPoolingWorks by setting skipping setConnectionTimeToLive is value is set to 0 since 0 is treated as Infinite timeToLive in Sdk and Apache 4.x but treated as immediate closeConnection in apache 5.x * disableAutomaticRetries in Apache 5.x since SDK handles retries , also define Apache5 dependencies in .brazil.json * Added Test case for Async , handled review ocmments

…-v2 into feature/master/apache5x

* Fix architecture test failures for apache5.x * Checkstyle issues * Update to use PoolingHttpClientConnectionManager class reference that is implementation of HttpClientConnectionManager * Fix stream reset failure in RepeatableInputStreamRequestEntity by storing content reference to avoid multiple ContentStreamProvider.newStream() calls that cause IOException when retrying requests with non-resettable streams * writeTo_ConcurrentWrites_HandlesCorrectly no longer needed since even Apache 4.x doesnot suports this * Fix connectionPoolingWorks by setting skipping setConnectionTimeToLive is value is set to 0 since 0 is treated as Infinite timeToLive in Sdk and Apache 4.x but treated as immediate closeConnection in apache 5.x * disableAutomaticRetries in Apache 5.x since SDK handles retries , also define Apache5 dependencies in .brazil.json * Added Test case for Async , handled review ocmments * Donot do buffer the response using BufferedHttpEntity since it might cause memory issue, this behaviour is same as Apache4.x * Fix compilation issues * Fix checkstyle issues * Remove test which are specific to apache http

* Fix architecture test failures for apache5.x * Checkstyle issues * Update to use PoolingHttpClientConnectionManager class reference that is implementation of HttpClientConnectionManager * Fix stream reset failure in RepeatableInputStreamRequestEntity by storing content reference to avoid multiple ContentStreamProvider.newStream() calls that cause IOException when retrying requests with non-resettable streams * writeTo_ConcurrentWrites_HandlesCorrectly no longer needed since even Apache 4.x doesnot suports this * Fix connectionPoolingWorks by setting skipping setConnectionTimeToLive is value is set to 0 since 0 is treated as Infinite timeToLive in Sdk and Apache 4.x but treated as immediate closeConnection in apache 5.x * disableAutomaticRetries in Apache 5.x since SDK handles retries , also define Apache5 dependencies in .brazil.json * Added Test case for Async , handled review ocmments * Donot do buffer the response using BufferedHttpEntity since it might cause memory issue, this behaviour is same as Apache4.x * Fix compilation issues * Fix checkstyle issues * Remove test which are specific to apache http * Add benchmark for Apache5 and add Streaming Api test cases

… alternatives (#6211) * Clean up unused APIs and add test to make sure it can be handled with alternatives * Added NTCredentials to keep backward compatibilty with Apache4.x

… version (#6214) * Fix architecture test failures for apache5.x * Checkstyle issues * Update to use PoolingHttpClientConnectionManager class reference that is implementation of HttpClientConnectionManager * Fix stream reset failure in RepeatableInputStreamRequestEntity by storing content reference to avoid multiple ContentStreamProvider.newStream() calls that cause IOException when retrying requests with non-resettable streams * writeTo_ConcurrentWrites_HandlesCorrectly no longer needed since even Apache 4.x doesnot suports this * Fix connectionPoolingWorks by setting skipping setConnectionTimeToLive is value is set to 0 since 0 is treated as Infinite timeToLive in Sdk and Apache 4.x but treated as immediate closeConnection in apache 5.x * disableAutomaticRetries in Apache 5.x since SDK handles retries , also define Apache5 dependencies in .brazil.json * Added Test case for Async , handled review ocmments * Donot do buffer the response using BufferedHttpEntity since it might cause memory issue, this behaviour is same as Apache4.x * Fix compilation issues * Fix checkstyle issues * Remove test which are specific to apache http * Add benchmark for Apache5 and add Streaming Api test cases * Update Apache5 to 5.5

* Handled Surface API review comments * Added a single test for localaddress , handled review comments * Removing internal package name as -preview after internal discussion * Fix transient text case failures * Handled comment for merge to master PR for apache 5.x

…nConfig in Apache HttpClient 5 (#6293) * Replacing deprecated API like connectionTimeout on RequestConfig and passing it via defaultconnectionconfigs * Handle review comments

…ned in deprecation notes of httpclient.execute (#6298) * Replacing deprecated API like connectionTimeout on RequestConfig and passing it via defaultconnectionconfigs * Handle review comments * replace httpclient.execute call with httpclient.executeOpen as mentioned in deprecation notes of httpclient.execute * nit updates * fixed sonar quebe issues

…6281) * Replace deprecated SSLConnectionSocketFactory with recommended API * Fixed checkstyle issues * Changed name tlsSocketStrategy on builder * Removed warning log * added more test cases * updated after review * Added ConnectionSocketFactory to Apache5Client builder same as Apache4 * handled PR comments * Removed unused classes after moving to SSL sockets

sonarqubecloud · 2025-08-04T16:51:39Z

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
87.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2025-08-05T19:04:05Z

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.

joviegas and others added 30 commits April 30, 2025 08:27

Add initial empty module for Apache5x for seting up package (#6075)

f3fc974

* Add initial module for Apache5x for seting up package * Add based on new module checklist

Baseline or Copy all the ApacheSDKHttpClient classes to newly added A…

5092cc4

…pache5SDKHttpClient (#6088) * Add initial module for Apache5x for seting up package * Add based on new module checklist * Baseline all the classes from Apache4 SDK client to the new Apache5 module

Phase 2 , getting Apache 5 compilation and Junit ready along with cle…

6e9939f

…aring Checkstyles and spotbug issues (#6100) * Phase 2 , getting Apache 5 compilation and Junit ready along with clearing Checkstyles and spotbug issues * Handle comments from review * Handle comments from Zoe

Merge branch 'master' into feature/master/apache5x

4dda3f7

Update the snap shot

3053d09

Fix HTTP authentication retry failures by improving RepeatableInputSt…

985ca35

…reamRequestEntity repeatability (#6132) * Fix HTTP authentication retry failures by improving RepeatableInputStreamRequestEntity repeatability * Upated test cases * Handled comments

Merge branch 'master' into feature/master/apache5x

b97c9da

Updated snap shot after merge from master

64f7d33

Revert "Updated snap shot after merge from master"

24438e8

This reverts commit 64f7d33.

Updated snap shot after merge from master

62150d6

Fix architecture test failures for apache5.x (#6140)

fbfb70f

* Fix architecture test failures for apache5.x * Checkstyle issues

Merge branch 'master' into feature/master/apache5x

3c4433c

Updated snap shot after merge from master

6d8b5f4

Merge branch 'master' into feature/master/apache5x

0bdf878

Merge branch 'master' into feature/master/apache5x

4671831

Merge branch 'master' into feature/master/apache5x

2f72282

Merge PR#6165 #6165

79f6f0f

Merge branch 'master' into feature/master/apache5x

716a6db

Update snapshots

09e8ee4

Merge branch 'feature/master/apache5x' of github.com:aws/aws-sdk-java…

ad30c25

…-v2 into feature/master/apache5x

Merge branch 'master' into feature/master/apache5x

959605c

Merge from master

a959581

Clean up unused APIs and add test to make sure it can be handled with…

3cc94bb

… alternatives (#6211) * Clean up unused APIs and add test to make sure it can be handled with alternatives * Added NTCredentials to keep backward compatibilty with Apache4.x

Merge branch 'master' into feature/master/apache5x

45f630d

joviegas and others added 14 commits July 8, 2025 15:18

Merge branch 'master' into feature/master/apache5x

7a7e4cd

Added change logs

6383c44

Merge branch 'master' into feature/master/apache5x

ee93292

Review comment

061b5fb

Handled Review comments

f26f7bb

Merge branch 'master' into feature/master/apache5x

dfcfdcb

merge from master

236c102

Move connection timeout configuration from RequestConfig to Connectio…

2590e6e

…nConfig in Apache HttpClient 5 (#6293) * Replacing deprecated API like connectionTimeout on RequestConfig and passing it via defaultconnectionconfigs * Handle review comments

Merge branch 'master' into feature/master/apache5x

7e329e6

Added change logs

0da1dc3

removed old change logs

a93013f

joviegas requested a review from a team as a code owner August 1, 2025 16:18

joviegas changed the title ~~Feature/master/apache5x~~ Update Apache HttpClient 5 to use recommended APIs and remove deprecated code Aug 1, 2025

joviegas added the api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team label Aug 1, 2025

joviegas added 4 commits August 4, 2025 06:45

Merge branch 'master' into feature/master/apache5x

122e35b

remove unused imports

5b2fc0d

inermittent port used in wiremock fixed

b031aba

update change logs

228caac

dagnir approved these changes Aug 4, 2025

View reviewed changes

Merge branch 'master' into feature/master/apache5x

25a61cb

joviegas enabled auto-merge August 4, 2025 20:45

joviegas added this pull request to the merge queue Aug 5, 2025

Merged via the queue into master with commit 559e770 Aug 5, 2025
35 of 37 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Aug 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update Apache HttpClient 5 to use recommended APIs and remove deprecated code #6312

Update Apache HttpClient 5 to use recommended APIs and remove deprecated code #6312

Uh oh!

joviegas commented Aug 1, 2025

Uh oh!

sonarqubecloud bot commented Aug 4, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

Uh oh!

Update Apache HttpClient 5 to use recommended APIs and remove deprecated code #6312

Update Apache HttpClient 5 to use recommended APIs and remove deprecated code #6312

Uh oh!

Conversation

joviegas commented Aug 1, 2025

Motivation and Context

Modifications

Testing

JDK 17

JDK 21

License

Uh oh!

sonarqubecloud bot commented Aug 4, 2025

Quality Gate passed

Uh oh!

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

Uh oh!