Skip to content

Commit 9c95c57

Browse files
BrunoBonacciBrunoBonacci
committed
adding amazonica-s3
1 parent ccf937a commit 9c95c57

File tree

4 files changed

+82
-89
lines changed

4 files changed

+82
-89
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Here the list of libraries tested:
1919
| :white_check_mark: | [clojure/tools.logging](./tools-logging) | Logging library | |
2020
| :white_check_mark: | [clojure/tools.logging+log4j](./tools-logging-log4j) | Logging library | |
2121
| :white_check_mark: | [aleph](./aleph) | Web server | |
22-
| :x: | [amazonica+s3](./amazonica-s3) | Cloud API wrapper library | *Buildtime and Runtime error* |
22+
| :white_check_mark: | [amazonica+s3](./amazonica-s3) | Cloud API wrapper library | |
2323
| :white_check_mark: | [cheshire](./cheshire) | JSON parser/writer | |
2424
| :white_check_mark: | [clj-http-lite](./clj-http-lite) | Web client | |
2525
| :x: | [clj-sophia](./clj-sophia) | A fast RAM-Disk hybrid storage | *Runtime error/JNA* |

‎amazonica-s3/README.md

Lines changed: 42 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -6,95 +6,55 @@ Testing whether [Amazonica/s3](https://github.com/mcohen01/amazonica) library ca
66

77
Currently testing:
88

9-
[amazonica "0.3.142"]
9+
[amazonica "0.3.152"]
1010

1111
Test with:
1212

13-
lein do clean, uberjar, native, run-native
13+
lein do clean, uberjar, native-config, native, run-native
1414

1515
To be able to run this need to provide AWS credentials:
1616

1717
export AWS_ACCESS_KEY_ID=xxx
1818
export AWS_SECRET_ACCESS_KEY=yyyy
1919
export AWS_DEFAULT_REGION=eu-west-1
2020

21-
However with `clojure-1.10.0` the native image compilation shows the
22-
following warning message.
23-
24-
Build on Server(pid: 77850, port: 49942)
25-
[./target/amazonica-s3:77850] classlist: 2,083.42 ms
26-
[./target/amazonica-s3:77850] (cap): 933.91 ms
27-
[./target/amazonica-s3:77850] setup: 1,203.25 ms
28-
[./target/amazonica-s3:77850] analysis: 26,775.72 ms
29-
Warning: Aborting stand-alone image build. unbalanced monitors: mismatch at monitorexit, 96|LoadField#lockee__5436__auto__ != 3|LoadField#lockee__5436__auto__
30-
Detailed message:
31-
Call path from entry point to clojure.spec.gen.alpha$dynaload$fn__2628.invoke():
32-
at clojure.spec.gen.alpha$dynaload$fn__2628.invoke(alpha.clj:21)
33-
at clojure.lang.AFn.applyToHelper(AFn.java:152)
34-
at clojure.lang.AFn.applyTo(AFn.java:144)
35-
at simple.main.main(Unknown Source)
36-
at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:153)
37-
at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
38-
Warning: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
39-
Build on Server(pid: 77850, port: 49942)
40-
[./target/amazonica-s3:77850] classlist: 271.31 ms
41-
[./target/amazonica-s3:77850] (cap): 1,048.31 ms
42-
[./target/amazonica-s3:77850] setup: 1,342.62 ms
43-
[./target/amazonica-s3:77850] (typeflow): 1,530.36 ms
44-
[./target/amazonica-s3:77850] (objects): 447.20 ms
45-
[./target/amazonica-s3:77850] (features): 87.13 ms
46-
[./target/amazonica-s3:77850] analysis: 2,103.02 ms
47-
[./target/amazonica-s3:77850] (clinit): 69.69 ms
48-
[./target/amazonica-s3:77850] universe: 170.83 ms
49-
[./target/amazonica-s3:77850] (parse): 251.07 ms
50-
[./target/amazonica-s3:77850] (inline): 538.56 ms
51-
[./target/amazonica-s3:77850] (compile): 1,937.14 ms
52-
[./target/amazonica-s3:77850] compile: 2,878.45 ms
53-
[./target/amazonica-s3:77850] image: 229.01 ms
54-
[./target/amazonica-s3:77850] write: 138.33 ms
55-
[./target/amazonica-s3:77850] [total]: 7,191.94 ms
56-
Warning: Image './target/amazonica-s3' is a fallback image that requires a JDK for execution (use --no-fallback to suppress fallback image generation).
57-
58-
And the execution of the native image shows:
59-
60-
Error: Could not find or load main class simple.main
61-
62-
63-
Using `clojure-1.8.0` the native image builds successfully however the native binary runs with the following error.
64-
65-
Exception in thread "main" java.lang.NullPointerException
66-
at amazonica.core$constructor_args.invokeStatic(core.clj:415)
67-
at amazonica.core$constructor_args.invoke(core.clj:413)
68-
at amazonica.core$new_instance.invokeStatic(core.clj:445)
69-
at amazonica.core$new_instance.invoke(core.clj:438)
70-
at amazonica.core$create_bean.invokeStatic(core.clj:644)
71-
at amazonica.core$create_bean.invoke(core.clj:642)
72-
at amazonica.core$get_client_configuration.invokeStatic(core.clj:224)
73-
at amazonica.core$get_client_configuration.invoke(core.clj:221)
74-
at amazonica.core$amazon_client_STAR_.invokeStatic(core.clj:277)
75-
at amazonica.core$amazon_client_STAR_.invoke(core.clj:274)
76-
at clojure.lang.AFn.applyToHelper(AFn.java:160)
77-
at clojure.lang.AFn.applyTo(AFn.java:144)
78-
at clojure.core$apply.invokeStatic(core.clj:646)
79-
at clojure.core$memoize$fn__5708.doInvoke(core.clj:6107)
80-
at clojure.lang.RestFn.invoke(RestFn.java:436)
81-
at amazonica.core$candidate_client$fn__329.invoke(core.clj:858)
82-
at clojure.lang.Delay.deref(Delay.java:37)
83-
at clojure.core$deref.invokeStatic(core.clj:2228)
84-
at clojure.core$deref.invoke(core.clj:2214)
85-
at amazonica.core$candidate_client.invokeStatic(core.clj:859)
86-
at amazonica.core$candidate_client.invoke(core.clj:842)
87-
at amazonica.core$fn_call$fn__337.invoke(core.clj:872)
88-
at clojure.lang.Delay.deref(Delay.java:37)
89-
at clojure.core$deref.invokeStatic(core.clj:2228)
90-
at clojure.core$deref.invoke(core.clj:2214)
91-
at amazonica.core$fn_call$fn__339.invoke(core.clj:875)
92-
at amazonica.core$intern_function$fn__375.doInvoke(core.clj:1031)
93-
at clojure.lang.RestFn.invoke(RestFn.java:408)
94-
at simple.main$_main.invokeStatic(main.clj:8)
95-
at simple.main$_main.invoke(main.clj:7)
96-
at clojure.lang.AFn.applyToHelper(AFn.java:152)
97-
at clojure.lang.AFn.applyTo(AFn.java:144)
98-
at simple.main.main(Unknown Source)
99-
100-
More investigation is required.
21+
Building this image requires a more complex procedure. After building
22+
the uberjar as usual we will run the application with a java-agent
23+
that will capture several parameters such as reflected classes, proxy
24+
instances, and jni config.
25+
26+
It is good idea to cover as much as possible of the run-time
27+
functionality during this step, the more information will be able to
28+
collect, the easier/better will be the compilation step.
29+
30+
``` bash
31+
java -agentlib:native-image-agent=config-output-dir=./target/config/ \
32+
-jar ./target/amazonica-s3-0.1.0-SNAPSHOT.jar
33+
```
34+
35+
If multiple execution are required there are ways to merge the
36+
generated config.
37+
38+
We will then feed this configuration into the native image compilation
39+
by adding the following flag:
40+
41+
``` bash
42+
# addint the following flag to native-image
43+
# will tell the compiler to use the information we
44+
# extracted from the introspection step
45+
46+
-H:ConfigurationFileDirectories=./target/config/
47+
```
48+
49+
**IMPORTANT**: Finally, I've noticed that after the compilation, the
50+
native image was still using the AWS configuration I had at set during
51+
the compilation. So even after changing the account credentials it
52+
was still connecting to the same account I used during the
53+
compilation. To avoid this, I had to add the following flag to force
54+
the initialization of the credential provider to happen at runtime.
55+
56+
``` bash
57+
--initialize-at-run-time=com.amazonaws.auth.DefaultAWSCredentialsProviderChain
58+
```
59+
60+
After rebuilding, everything worked fine.!

‎amazonica-s3/project.clj

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,16 @@
11
(defproject amazonica-s3 "0.1.0-SNAPSHOT"
22

3-
:dependencies [#_[org.clojure/clojure "1.10.0"]
4-
[org.clojure/clojure "1.8.0"]
5-
[amazonica "0.3.142"
3+
:dependencies [[org.clojure/clojure "1.10.1"]
4+
[amazonica "0.3.152"
65
:exclusions [com.taoensso/nippy
76
com.amazonaws/aws-java-sdk
87
com.amazonaws/amazon-kinesis-client]]
9-
[com.amazonaws/aws-java-sdk-core "1.11.546"]
10-
[com.amazonaws/aws-java-sdk-s3 "1.11.546"]]
8+
[com.amazonaws/aws-java-sdk-core "1.11.805"]
9+
[com.amazonaws/aws-java-sdk-s3 "1.11.805"]
10+
[log4j/log4j "1.2.17"]
11+
[org.slf4j/slf4j-log4j12 "1.7.30"]
12+
[org.slf4j/jul-to-slf4j "1.7.30"]
13+
[org.slf4j/jcl-over-slf4j "1.7.30"]]
1114

1215
:main simple.main
1316

@@ -16,10 +19,21 @@
1619
:dev {:plugins [[lein-shell "0.5.0"]]}}
1720

1821
:aliases
19-
{"native"
22+
{"native-config"
23+
["shell"
24+
;; run the application to infer the build configuration
25+
"java" "-agentlib:native-image-agent=config-output-dir=./target/config/"
26+
"-jar" "./target/${:uberjar-name:-${:name}-${:version}-standalone.jar}"]
27+
28+
29+
"native"
2030
["shell"
2131
"native-image" "--report-unsupported-elements-at-runtime" "--no-server"
32+
"-H:+PrintClassInitialization"
33+
"-H:ConfigurationFileDirectories=./target/config/"
2234
"--initialize-at-build-time"
35+
"--initialize-at-run-time=com.amazonaws.auth.DefaultAWSCredentialsProviderChain"
36+
"--enable-http" "--enable-https" "--enable-all-security-services"
2337
"-jar" "./target/${:uberjar-name:-${:name}-${:version}-standalone.jar}"
2438
"-H:Name=./target/${:name}"]
2539

‎amazonica-s3/resources/log4j.properties

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Root logger option
2+
log4j.rootLogger=INFO, stdout
3+
4+
# Direct log messages to stdout
5+
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
6+
log4j.appender.stdout.Target=System.out
7+
log4j.appender.stdout.encoding=UTF-8
8+
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
9+
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
10+
11+
# ingore annoying WARN from AWS SDK like the following one:
12+
# WARN BasicProfileConfigLoader:96 - Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
13+
log4j.category.com.amazonaws.auth.profile.internal.BasicProfileConfigLoader=ERROR
14+
15+
# muting warning for JDK9+ JAXTB
16+
# WARN Base64:113 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant
17+
log4j.category.com.amazonaws.util.Base64=ERROR
18+
# muting "Failed to connect to service endpoint:" errors
19+
log4j.category.com.amazonaws=ERROR

0 commit comments

Comments
 (0)