Skip to content
This repository was archived by the owner on Sep 10, 2024. It is now read-only.
This repository was archived by the owner on Sep 10, 2024. It is now read-only.

Rewrite the async job system #2785

@sandhose

Description

@sandhose

We're currently using apalis for async jobs, which… is probably not mature enough.
A few issues with it:

  • the crates are a bit messy
  • they are doing breaking changes all the time. See Upgrade apalis #2784
  • the database structure is… weird
  • the jobs don't retry correctly
  • the "workers" crash when it looses the connection and don't restart
  • sharing state in the job is not type safe

Requirements:

  • have jobs in the database
  • use triggers to NOTIFY workers for new jobs (optional?)
  • lock rows with PG's FOR UPDATE SKIP LOCKED
  • pass the connection which locked the task row to the handler
  • have a way to do cron-like schedules for maintenance tasks

Related:

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-JobsRelated to asynchronous jobs

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions