-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add support for async operations #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,25 @@ | ||
import threading | ||
from contextvars import ContextVar | ||
from functools import wraps | ||
from uuid import uuid4 | ||
|
||
from django.db import connections | ||
|
||
THREAD_LOCAL = threading.local() | ||
# Define context variables for read and write database settings | ||
# These variables will maintain database preferences per context | ||
DB_FOR_READ_OVERRIDE = ContextVar('DB_FOR_READ_OVERRIDE', default='default') | ||
DB_FOR_WRITE_OVERRIDE = ContextVar('DB_FOR_WRITE_OVERRIDE', default='default') | ||
|
||
|
||
class DynamicDbRouter(object): | ||
"""A router that decides what db to read from based on a variable | ||
local to the current thread. | ||
class DynamicDbRouter: | ||
""" | ||
|
||
A router that dynamically determines which database to perform read and write operations | ||
on based on the current execution context. It supports both synchronous and asynchronous code. | ||
""" | ||
|
||
def db_for_read(self, model, **hints): | ||
return getattr(THREAD_LOCAL, 'DB_FOR_READ_OVERRIDE', ['default'])[-1] | ||
return DB_FOR_READ_OVERRIDE.get() | ||
|
||
def db_for_write(self, model, **hints): | ||
return getattr(THREAD_LOCAL, 'DB_FOR_WRITE_OVERRIDE', ['default'])[-1] | ||
return DB_FOR_WRITE_OVERRIDE.get() | ||
|
||
def allow_relation(self, *args, **kwargs): | ||
return True | ||
|
@@ -27,101 +30,89 @@ def allow_syncdb(self, *args, **kwargs): | |
def allow_migrate(self, *args, **kwargs): | ||
return None | ||
|
||
|
||
class in_database(object): | ||
"""A decorator and context manager to do queries on a given database. | ||
|
||
class in_database: | ||
""" | ||
A decorator and context manager to do queries on a given database. | ||
:type database: str or dict | ||
:param database: The database to run queries on. A string | ||
will route through the matching database in | ||
``django.conf.settings.DATABASES``. A dictionary will set up a | ||
connection with the given configuration and route queries to it. | ||
|
||
:type read: bool, optional | ||
:param read: Controls whether database reads will route through | ||
the provided database. If ``False``, reads will route through | ||
the ``'default'`` database. Defaults to ``True``. | ||
|
||
:type write: bool, optional | ||
:param write: Controls whether database writes will route to | ||
the provided database. If ``False``, writes will route to | ||
the ``'default'`` database. Defaults to ``False``. | ||
|
||
When used as eithe a decorator or a context manager, `in_database` | ||
requires a single argument, which is the name of the database to | ||
route queries to, or a configuration dictionary for a database to | ||
route to. | ||
|
||
Usage as a context manager: | ||
|
||
.. code-block:: python | ||
|
||
from my_django_app.utils import tricky_query | ||
|
||
with in_database('Database_A'): | ||
results = tricky_query() | ||
|
||
Usage as a decorator: | ||
|
||
.. code-block:: python | ||
|
||
from my_django_app.models import Account | ||
|
||
@in_database('Database_B') | ||
def lowest_id_account(): | ||
Account.objects.order_by('-id')[0] | ||
|
||
Used with a configuration dictionary: | ||
|
||
.. code-block:: python | ||
|
||
db_config = {'ENGINE': 'django.db.backends.sqlite3', | ||
'NAME': 'path/to/mydatabase.db'} | ||
with in_database(db_config): | ||
# Run queries | ||
""" | ||
def __init__(self, database, read=True, write=False): | ||
def __init__(self, database: str | dict, read=True, write=False): | ||
self.read = read | ||
self.write = write | ||
self.database = database | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does this code do in the event the database is a string? Is there any reason to remove the check below? Or was this code only checked with database being a dictionary? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If database is a string, the code assumes that it's a valid database alias already configured in Django’s DATABASES setting. No special handling is needed, we just use this string directly to reference the database. If database is a dictionary, we would need a more dynamic setup where you might not have a pre-configured database alias, and instead you would need to provide all necessary connection details. There isn't any reason to remove the check, since i was already defining the databases in the settings file, i was just focused on it working for strings. I'll add the check back in my next push There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, let's add it back. The explicit exception is helpful. Given we now have python 3, we could also do |
||
self.created_db_config = False | ||
|
||
# Handle database parameter either as a string (alias) or as a dict (configuration) | ||
if isinstance(database, str): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @davidbm04 let's restore these other if clauses for opening a PR with the original repo. |
||
self.database = database | ||
elif isinstance(database, dict): | ||
# Note: this invalidates the docs above. Update them | ||
# eventually. | ||
# If it's a dict, create a unique database configuration | ||
self.created_db_config = True | ||
self.unique_db_id = str(uuid4()) | ||
connections.databases[self.unique_db_id] = database | ||
self.database = self.unique_db_id | ||
else: | ||
msg = ("database must be an identifier for an existing db, " | ||
"or a complete configuration.") | ||
raise ValueError(msg) | ||
raise ValueError("database must be an identifier (str) for an existing db, " | ||
"or a complete configuration (dict).") | ||
|
||
def __enter__(self): | ||
if not hasattr(THREAD_LOCAL, 'DB_FOR_READ_OVERRIDE'): | ||
THREAD_LOCAL.DB_FOR_READ_OVERRIDE = ['default'] | ||
if not hasattr(THREAD_LOCAL, 'DB_FOR_WRITE_OVERRIDE'): | ||
THREAD_LOCAL.DB_FOR_WRITE_OVERRIDE = ['default'] | ||
read_db = (self.database if self.read | ||
else THREAD_LOCAL.DB_FOR_READ_OVERRIDE[-1]) | ||
write_db = (self.database if self.write | ||
else THREAD_LOCAL.DB_FOR_WRITE_OVERRIDE[-1]) | ||
THREAD_LOCAL.DB_FOR_READ_OVERRIDE.append(read_db) | ||
THREAD_LOCAL.DB_FOR_WRITE_OVERRIDE.append(write_db) | ||
# Capture the current database settings | ||
self.original_read_db = DB_FOR_READ_OVERRIDE.get() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When are the enter and exit methods called in this context? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not sure if i understand this question... Isn't the enter and exit method called automatically when using with? enter should be called at the beginning and exit at the end: doc There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I should have noted this is for my own education ;) I'd figured but I'd never written a context manager for Python so I wanted to confirm. Thanks for the learning, David! |
||
self.original_write_db = DB_FOR_WRITE_OVERRIDE.get() | ||
|
||
# Override the database settings for the duration of the context | ||
if self.read: | ||
DB_FOR_READ_OVERRIDE.set(self.database) | ||
if self.write: | ||
DB_FOR_WRITE_OVERRIDE.set(self.database) | ||
return self | ||
|
||
def __exit__(self, exc_type, exc_value, traceback): | ||
THREAD_LOCAL.DB_FOR_READ_OVERRIDE.pop() | ||
THREAD_LOCAL.DB_FOR_WRITE_OVERRIDE.pop() | ||
# Restore the original database settings after the context. | ||
DB_FOR_READ_OVERRIDE.set(self.original_read_db) | ||
DB_FOR_WRITE_OVERRIDE.set(self.original_write_db) | ||
|
||
# Close and delete created database configuration | ||
if self.created_db_config: | ||
connections[self.unique_db_id].close() | ||
del connections.databases[self.unique_db_id] | ||
|
||
def __call__(self, querying_func): | ||
# Allow the object to be used as a decorator | ||
@wraps(querying_func) | ||
def inner(*args, **kwargs): | ||
# Call the function in our context manager | ||
with self: | ||
return querying_func(*args, **kwargs) | ||
return inner |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clarification: Does ContextVar still support synchronous threads? Or does this only work for async?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From my understanding, contextvar supports synchronous threads. I'll link the documentation but it seems that contextvar behaves similar to threading.local(), which is what was previously used, when values are assigned to different threads. I also tested this out by using the modifed context manager in a synchronous function and everything worked as expected. Here's the link to doc: contextvar
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fantastic. Great work, David!