Skip to content

Improvement: reduce global locking in aot_call_function() #4058

Open
@palchikov

Description

@palchikov

Feature

During execution of aot_call_function() there are up to 3 calls to aot_copy_exception() (for the case of exported function has no or single result, which I believe is the most common case). And each aot_copy_exception() internally locks global mutex _exception_lock. This can be improved by:

  1. Remove redundant calls to aot_copy_exception()
  2. Make exception_lock() to lock module's mutex instead of global one

Benefit

In my project in a highly concurrent environment I use hundreds of instantiated modules, and each module processes about 1k requests per second. While each module performance is limited by execution time of exported function, I need to increase the number of modules to gain sufficient throughput. With global locks I have very strict limits to which I can add new executors. The more modules I add, the slower each of them works.
By removing global locks (or at least by reducing the number of locks required to aot_call_function()) we can achieve linear scalability.

Implementation

  1. Reducing the number of aot_copy_exception(). There are 3 calls I mentioned above: inside invoke_native_internal, then one for debugging purpose, and finally just before return. invoke_native_internal() uses exception check as its whole result, so further calls to aot_copy_exception() seems to be redundant. The branch functions with result_count > 1 reuses return result in similar places.
  2. The comment inside exception_lock() body tells that there were some plans to make a mutex belong to a particular module. Probably this requires complex changes, but it will give great performance boost in highly concurrent environments.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions