What feature would you like to be added?

Currently, when using Autogen's Distributed Agent Runtime, tasks are managed using asyncio's Queue. However, this approach does not persist tasks across service restarts. To ensure that tasks can continue execution even after a restart, we propose introducing an external storage mechanism such as Redis queue.

Proposed Solution

• Replace or extend the existing asyncio.Queue implementation with an external message queue, such as Redis queue.
• Ensure that queued tasks are not lost when the service restarts.
• Provide configuration options to allow users to choose between in-memory and persistent queues.

Why is this needed?

• Improved reliability and fault tolerance.
• Ability to recover and continue executing tasks after unexpected failures or restarts.
• Scalability for distributed agent execution.