Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL parser sqlglot does not support impala dialect, so datasets in IMPALA stopped working. #32143

Open
3 tasks
SBIN2010 opened this issue Feb 5, 2025 · 3 comments
Assignees
Labels
data:connect:impala Related to Impala

Comments

@SBIN2010
Copy link

SBIN2010 commented Feb 5, 2025

Bug description

Hi!
I am using IMPALA DB and after upgrading to version 4.1.1 the most of the datasets not working.
When executing queries in SQL LAB - queries work correctly, but it is impossible to create a dataset from queries. The error: “Fatal error”
Image
appears.
But the dataset is still created - but when we open dataset it I get an error -

Image

and

“Failed to load chart data:
Error parsing near 'Query' at line 2:35”.
Image

In the developer console you can see the following error:
Image

When opening a dataset for editing, you can see next error:

Image

A little researching the problem - found that the SQL analysis was improved (#26786).
In favor of SQL parser sqlglot. Which does not support Impala dialect.

I also encounter a similar error in SQL LAB.
(#30551)

Screenshots/recordings

No response

Superset version

4.1.1

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

  • I have searched Superset docs and Slack and didn't find a solution to my problem.
  • I have searched the GitHub issue tracker and didn't find a similar bug report.
  • I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
@dosubot dosubot bot added the data:connect:impala Related to Impala label Feb 5, 2025
Copy link

dosubot bot commented Feb 5, 2025

The issue you're experiencing is related to the use of sqlglot for SQL parsing in Apache Superset version 4.1.1, which does not support the Impala SQL dialect. This is causing errors when creating datasets from SQL Lab queries, even though the queries themselves execute correctly.

Currently, Superset handles unsupported SQL dialects by using custom implementations. However, for Impala, there doesn't seem to be a built-in workaround or custom implementation in the codebase specifically for this dialect. You might need to consider implementing a custom parser or handling logic for Impala SQL within your Superset setup to address this issue.

For now, you could explore the possibility of extending or customizing sqlglot to support Impala, although this would require significant development effort. Alternatively, you might need to wait for an update from the Superset maintainers that addresses this limitation or consider contributing a solution if you're able to develop one.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

@SBIN2010
Copy link
Author

SBIN2010 commented Feb 5, 2025

In the logs:
ERROR:flask_appbuilder.api:Error parsing near 'Query' at line 2:20 Traceback (most recent call last): File "/app/superset/sql/parse.py", line 256, in _parse return sqlglot.parse(script, dialect=dialect) File "/app/venv/lib64/python3.9/site-packages/sqlglot/__init__.py", line 102, in parse return Dialect.get_or_raise(read or dialect).parse(sql, **opts) File "/app/venv/lib64/python3.9/site-packages/sqlglot/dialects/dialect.py", line 919, in parse return self.parser(**opts).parse(self.tokenize(sql), sql) File "/app/venv/lib64/python3.9/site-packages/sqlglot/parser.py", line 1395, in parse return self._parse( File "/app/venv/lib64/python3.9/site-packages/sqlglot/parser.py", line 1467, in _parse self.raise_error("Invalid expression / Unexpected token") File "/app/venv/lib64/python3.9/site-packages/sqlglot/parser.py", line 1508, in raise_error raise error sqlglot.errors.ParseError: Invalid expression / Unexpected token. Line 2, Col: 20. SELECT * FROM `Untitled Query 20 05.02.2025 15:39:10` LIMIT 100 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/app/venv/lib64/python3.9/site-packages/flask_appbuilder/api/__init__.py", line 111, in wraps return f(self, *args, **kwargs) File "/app/superset/views/base_api.py", line 119, in wraps duration, response = time_function(f, self, *args, **kwargs) File "/app/superset/utils/core.py", line 1364, in time_function response = func(*args, **kwargs) File "/app/superset/utils/log.py", line 303, in wrapper value = f(*args, **kwargs) File "/app/superset/views/base_api.py", line 91, in wraps return f(self, *args, **kwargs) File "/app/superset/datasets/api.py", line 339, in post return self.response(201, id=new_model.id, result=item, data=new_model.data) File "/app/superset/connectors/sqla/models.py", line 1416, in data data_ = super().data File "/app/superset/connectors/sqla/models.py", line 404, in data "select_star": self.select_star, File "/app/superset/connectors/sqla/models.py", line 1395, in select_star return self.database.select_star( File "/app/superset/models/core.py", line 719, in select_star return self.db_engine_spec.select_star( File "/app/superset/db_engine_specs/base.py", line 1689, in select_star sql = SQLScript(sql, engine=cls.engine).format() File "/app/superset/sql/parse.py", line 611, in __init__ self.statements = statement_class.split_script(script, engine) File "/app/superset/sql/parse.py", line 316, in split_script ast = cls._parse(remainder, engine)[0] File "/app/superset/sql/parse.py", line 259, in _parse raise SupersetParseError( superset.exceptions.SupersetParseError: Error parsing near 'Query' at line 2:20

@SBIN2010 SBIN2010 changed the title SQL parser sqlglot not support impala dialect SQL parser sqlglot does not support impala dialect, so datasets in IMPALA stopped working. Feb 5, 2025
@rusackas
Copy link
Member

rusackas commented Feb 6, 2025

@betodealmeida might have an idea what's happening here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data:connect:impala Related to Impala
Projects
None yet
Development

No branches or pull requests

3 participants