Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL Support #1444

Open
norberttech opened this issue Feb 4, 2025 · 0 comments
Open

SQL Support #1444

norberttech opened this issue Feb 4, 2025 · 0 comments
Milestone

Comments

@norberttech
Copy link
Member

Flow (just like other data processing frameworks) that operates on DataFrame abstraction (tabular representation of the dataset) can be naturally handled through SQL.

The goal is to create a SQL Parsers that would let us translate SQL Queries to Flow DataFrame pipeline.

We can start from some simpler dialects like SQLite, most likely we won't be able to support all advanced features from PostgreSQL or MySQL, but that's not the goal.
Our goal is to parse basic SQL syntax and be able to add custom functions like:

SELECT 
  id, name, anything
FROM
  parquet_file('path_to_file.parquet')
WHERE
  active = true
ORDER BY id DESC

The biggest complexity comes from lack of good SQL Parser options in PHP but there are few options:

  1. Antlr

We can use Antlr4 with SQLite grammar (it's compatible with php target)

This way we could create a standalone lib flow-php/sql that would be a cleaner abstraction over Antlr4 auto generated code.

  1. SQLFTW

There is one very promising SQL Parser in PHP, SQLFTW.

The main concern is that it doesn't seem to be actively developed and that it brings hard dependency to dogma/dogma.

We should try to reach out to see if they would be interested in a PR that would drop that dependency (as dogma is just another attempt to build a standard php library in the userland)

@norberttech norberttech moved this to Todo in Roadmap Feb 4, 2025
@norberttech norberttech added this to the 0.12.0 milestone Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

1 participant