Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot use the SQLExecuteQueryOperator with elastic connection type (e.g. ElasticsearchSQLHook ) #46434

Open
2 tasks done
dabla opened this issue Feb 4, 2025 · 0 comments
Open
2 tasks done
Labels
area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet provider:elasticsearch

Comments

@dabla
Copy link
Contributor

dabla commented Feb 4, 2025

Apache Airflow Provider(s)

elasticsearch

Versions of Apache Airflow Providers

apache-airflow-providers-elasticsearch==6.0.0

Apache Airflow version

2.10.4

Operating System

Linux

Deployment

Other 3rd-party Helm chart

Deployment details

No response

What happened

Today we tried to use the Elasticsearch connection type (e.g. ElasticsearchSQLHook ) through the SQLExecuteQueryOperator.
You would expect it to work as the ElasticsearchSQLHook extends the DbApiHook, unfortunately it doesn't as it is missing some implementations.

What you think should happen instead

I would expect the Elasticsearch connection type to work with the SQLExecuteQueryOperator, as it supports executing SQL statements.

How to reproduce

Just use the SQLExecuteQueryOperator with the Elasticsearch connection type. Also in the TestElasticsearchSQLHook unit test you can see the cursor is mocked but the ESConnection wrapper class doesn't actually return a cursor.

Anything else

We've patched the ElasticsearchSQLHook in our Airflow installation to make it work, so I created a PR for this to make it work in elasticsearch provider

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@dabla dabla added area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet provider:elasticsearch
Projects
None yet
Development

No branches or pull requests

1 participant