Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Databases other than DuckDB #399

Open
domoritz opened this issue May 23, 2024 · 8 comments
Open

Support Databases other than DuckDB #399

domoritz opened this issue May 23, 2024 · 8 comments
Labels
projects Project ideas for Mosaic

Comments

@domoritz
Copy link
Member

Create new connectors, for example to Postgres, Clickhouse, or Snowflake. Identify breakage points for Mosaic’s current SQL query generation and design solutions for translating from an internal "canonical" query language to different DB dialects.

A good starting point would be to leverage multi database support in DuckDB.

@domoritz domoritz added the projects Project ideas for Mosaic label May 23, 2024
@weifanwu
Copy link

weifanwu commented May 24, 2024

working on this issue! and almost done!

@derekperkins
Copy link

We're interested in BigQuery + StarRocks adapters. Once there's an adapter pattern, we'd be happy to help contribute SQL interface translations.

This was referenced May 29, 2024
@MarcSkovMadsen
Copy link

MarcSkovMadsen commented Jun 9, 2024

I'm interested in Azure SQL and Azure Synapse databases.

@domoritz
Copy link
Member Author

https://github.com/tobymao/sqlglot could be an interesting experiment to build a rest server that proxies queries to another database. I'm sure it's still tricky because of data loading but worth a try.

@weifanwu
Copy link

Wrote documentation for how to connect (PostgreSQL, MySQL, SQLite) using DuckDB extensions:

#430

@peter-gy
Copy link
Contributor

peter-gy commented Feb 7, 2025

I drafted PR #681 as an experiment to provide support to multiple databases using https://github.com/ibis-project/ibis which in turn uses https://github.com/tobymao/sqlglot under the hood, as suggested by @domoritz.
What do you think about the approach?

@domoritz
Copy link
Member Author

domoritz commented Feb 7, 2025

You're amazing! I have to dig deeper into your pull request but it looks very exciting.

@jheer
Copy link
Member

jheer commented Feb 7, 2025

Wow, still need to review in detail but this looks really promising!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
projects Project ideas for Mosaic
Projects
None yet
Development

No branches or pull requests

6 participants