Benchmark and speed processing of polyglot java imports up

[JaroslavTulach]

Pull Request Description

Introduces new startup "Hello World" benchmark with many imports and a performance improvement by delaying loading classes in registerPolyglotSymbol.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

• All code follows the
  Scala,
  Java,
• Benchmarks results are reasonable
  • Initial stdlib benchmarks run - confirms there is 22% slowdown due to additional imports
  • Final stdlib benchmarks run

Original motivation for this PR is @hubertp profilling.npss that spends 700ms in registering polyglot symbols (which means to load all the Java classes referenced by polyglot import java statements and classloading takes time):

Hubert remeasured the behavior on his slow computer and here is the result:

The registerPolyglotImports method is completely gone and the whole processModule takes just 1.1s (previously 1.9s). That seems like an improvement we want to integrate.

[JaroslavTulach]

We have two commits right now:

• Benchmark measuring startup with import of all libraries - f4a29a1
• Delay loading classes in registerPolyglotSymbol by using Suppliers - f93ee21

now we need to restructure the PR and do new benchmark measurements because of

• Fix std benchmarks run #10902

183ca7a should contain all the necessary fixes as well as Benchmark measuring startup with import of all libraries. Benchmark is running at https://github.com/enso-org/enso/actions/runs/10596232710

44ca279 is a follow up of 183ca7a and contains also fix for Delay loading classes in registerPolyglotSymbol by using Suppliers. Benchmark will run at https://github.com/enso-org/enso/actions/runs/10596279560

[JaroslavTulach]

This example was obtained by running enso --new HelloExample and then the body of main was changed.

[JaroslavTulach]

Startup with all imports is 22% slower than startup with simple from Standard.Base import IO:

• this is the benchmark run
• org_enso_benchmarks_generated_Startup_hello_world_startup is 3453ms
• org_enso_benchmarks_generated_Startup_import_world_startup is 4217ms

E.g. a typical empty IDE project wastes 22% processing modules that aren't needed for the execution.

[JaroslavTulach]

The second stdlib benchmarks run has improved all the startup benchmarks, but only minimally by 4%%, 1%% and 17%% - that's below the measurement precision we have. I was expecting bigger speedup according to the VisualVM graphs.

[JaroslavTulach]

The new version 0fba66b removes any trace of registerPolyglotSymbol from the (10ms interval) sampling. That's a good sign. However there still seems to be getPolyglotSymbol caught in the trace two times:

Maybe we could improving the startup time by using Supplier<Object> where the polyglot symbol is requested by the IrToTruffle. One more benchmarks run scheduled with 6ee9c8f

[JaroslavTulach]

Previous benchmark run failed. Scheduling new one for e2c7bd3

[JaroslavTulach]

There is an ExecCompilerTest failure caused by running with micro-distribution. Fixed by 0fba66b

[JaroslavTulach]

The introduction of error message for DataflowError (done in 6c00db4) causes a WarningsTest failure caused by the fact that now there is ex.getMessage() != null for DataflowError. That's good, but as warnings cannot be attached to DataflowErrors - we need a change in the test. Done in 53eece8

[JaroslavTulach]

Resolving polyglot java imports lazily prevents us to emit error during compilation. Instead we get a (dataflow) Error when we execute the run function and try to resolve Random function. Such an error is then returned as a value and needs to be throwException manually.

[JaroslavTulach]

Special case to handle DataflowErrors that cannot have Warnings.

[hubertp]

You only check if it is a TruffleObject not if it is null which seems to be what javadoc suggests?

[JaroslavTulach]

Do you mean the Javadoc of build method that takes Supplier<TruffleObject> argument?

If typing is on, then check for null is the same as instanceof TruffleObject. But I decided to go for the latter as someone could sneak in raw Supplier and return anything. I would need to perform some complicated checks to verify the value is proper Enso interpreter value. That can be added, but for now instanceof TruffleObject is sufficient.

[JaroslavTulach]

At the end this PR has managed to speed the new Import_Hello benchmark by 4% from 4217ms to 4063ms:

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-08-29):

Progress: .

• Speed polyglot java import by 4% up PR merged: Benchmark and speed processing of polyglot java imports up #10899
• better benchmark output PR created: Fold output of each benchmark #10914
• comments on Radek's review: Method return type checks avoid conversion when the type fits #10882 (comment)
• comparing .npss files: https://discord.com/channels/@me/959434339182067712/1278665370877825035
• bugfix: 03c5bd5
• review of Pending Audit Logs are sent in batches #10918 (review)
• review: Add a way to run a single node in a different context in engine. #10719 (comment)
• Reporting registerTypeDefs issue: Remove 519ms startup overhead caused by registerTypeDefinitions #10923 for Hubert
• thinking about unit test for FramePointer
• standup and tea time It should be finished by 2024-08-29.

Discord

Discord - Group Chat That’s All Fun & Games

Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.