Flask Application Not Found After Upgrading from Python 3.6 and Apache Superset 0.38.1 to Python 3.10 and Superset 4.1.1

[bondany]

Bug description

Hello,

We are upgrading from a very old version of Superset and have successfully exported and imported datasets.

Older Version Details:

• Superset Version: 0.38.1
• Python Version: 3.6.9
• Flask Version: 1.1.4
• Werkzeug Version: 1.0.1

Export Command on the Old Environment:

$ superset export_datasources -f PROD_export.zip

(Note: I noticed that it is not actually a zip file but a YAML file, contrary to what is explained in the docs.)

Newer Version Details:

• Superset Version: 4.1.1
• Python Version: 3.10.12
• Flask Version: 2.3.3
• Werkzeug Version: 3.1.3

Import Command on the New Environment:

$ superset legacy-import-datasources --path PROD_export.yml

After completing the upgrade and importing the dataset, we encountered the following error when trying to run Superset:

Error: Could not locate a Flask application. Use the 'flask --app' option, 'FLASK_APP' environment variable, or a 'wsgi.py' or 'app.py' file in the current directory.

Additional Info

We need to mention that, before the import of the datasets, we had issues with custom_sso_security_manager.py.
We defined SUPERSET_CONFIG_PATH, but, using only this variable, Python was not able to find the custom_sso_security_manager module from superset_config.py. Strangely, creating a simple test.py, Python was able to import it from with no issues.
Double checking the custom_sso_security_manager.py, we could not find any problems.
Defining PYTHONPATH (initially empty) as export PYTHONPATH="/home/superset/.config/superset:$PYTHONPATH" together with export SUPERSET_CONFIG_PATH="/home/superset/.config/superset/superset_config.py" would create circular imports.
Removing SUPERSET_CONFIG_PATH and having only PYTHONPATH was the solution for this problem.

Current ENV

SHELL=/bin/bash
NVM_INC=/home/superset/.nvm/versions/node/v22.13.0/include/node
HISTTIMEFORMAT=%F %T
PWD=/home/superset
LOGNAME=superset
HOME=/home/superset
LANG=C.UTF-8
LS_COLORS=...
VIRTUAL_ENV=/home/superset/superset_env
SUPERSET_HOME=/home/superset/.data/superset
SUPERSET_SECRET_KEY=REDACTED
NVM_DIR=/home/superset/.nvm
SGX_AESM_ADDR=1
LESSCLOSE=/usr/bin/lesspipe %s %s
PYTHONPATH=/home/superset/.config/superset:
TERM=xterm
LESSOPEN=| /usr/bin/lesspipe %s
USER=superset
FLASK_APP=superset
SHLVL=1
NVM_CD_FLAGS=
VIRTUAL_ENV_PROMPT=superset_env
PS1=...
GCC_COLORS=error=...
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/home/superset/superset_env/bin:/home/superset/.nvm/versions/node/v22.13.0/bin:/usr/bin/:/usr/sbin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
NVM_BIN=/home/superset/.nvm/versions/node/v22.13.0/bin
MAIL=/var/mail/superset
_=/usr/bin/printenv

As a last resort, after a couple of days of debugging, we executed the following commands:

$ superset db upgrade
$ superset init

Additionally, I created an admin user for Flask as follows:

$ superset fab create-admin

(source: https://www.cnblogs.com/qq2483763504/p/16718615.html)

But the problem still persist.

Can someone help us understand if this is a bug or bad configuration.

Thank you!

Screenshots/recordings

No response

Superset version

4.1.1

Python version

3.10

Node version

18 or greater

Browser

Not applicable

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

Hey there, @bondany! I'm here to help you with any bugs, questions, or contributions you have. Let's squash those bugs together! 🐞

I found a similar unsolved discussion that might be relevant to your issue: install apache-superset==3.1.0 on win10 successfully, but superset db upgrade failed.... ^[1]. Although it doesn't directly address your specific problem, it might provide some insights or approaches that could be useful.

Regarding your issue, the error message "Could not locate a Flask application" typically indicates that the FLASK_APP environment variable is not set correctly or that the application entry point is not properly configured. Here are a few suggestions to troubleshoot and potentially resolve the issue:

1. Verify FLASK_APP Environment Variable: Ensure that the FLASK_APP environment variable is set to superset. You can do this by running:

   export FLASK_APP=superset

2. Check for Version Compatibility: Ensure that the versions of Flask and Werkzeug are compatible. Sometimes, mismatched versions can cause issues. You might want to try aligning them to versions known to work well together, such as Flask==2.0.2 and Werkzeug==2.0.2, as mentioned in the source you referred to.

3. Configuration Paths: Double-check your configuration paths to ensure there are no circular imports or misconfigurations. It seems you resolved a similar issue by adjusting PYTHONPATH and SUPERSET_CONFIG_PATH.

4. Application Entry Point: Make sure that your application entry point is correctly set up. If you have a custom entry point, ensure that it is correctly referenced in your configuration.

5. Logs and Debugging: Check the Superset logs for any additional error messages or stack traces that might provide more context on the issue.

If these steps do not resolve the issue, it might be helpful to provide more detailed logs or error messages to further diagnose the problem.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[bondany]

@dosu,

Installing Flask==2.0.2 and Werkzeug==2.0.2 gives me the following error:

$ python -m pip install Flask==2.0.2 Werkzeug==2.0.2
Collecting Flask==2.0.2
  Downloading Flask-2.0.2-py3-none-any.whl.metadata (3.8 kB)
Collecting Werkzeug==2.0.2
  Downloading Werkzeug-2.0.2-py3-none-any.whl.metadata (4.5 kB)
[...]
Installing collected packages: Werkzeug, Flask
'ERROR: pip''s dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
apache-superset 4.1.1 requires flask<3.0.0,>=2.2.5, but you have flask 2.0.2 which is incompatible.
flask-session 0.8.0 requires flask>=2.2, but you have flask 2.0.2 which is incompatible.'

[dosubot]

Apache Superset 4.1.1 requires specific versions of Flask and Werkzeug to function correctly. The required versions are Flask 2.3.3 and Werkzeug 3.1.3 ^[1]. Installing older versions like Flask 2.0.2 and Werkzeug 2.0.2 will lead to dependency conflicts, as you've experienced.

To resolve the issue, ensure that you have the correct versions installed by running:

python -m pip install Flask==2.3.3 Werkzeug==3.1.3

This should align with the requirements of Superset 4.1.1 and help avoid compatibility issues.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[bondany]

Ironically, installing Flask==2.0.2 would solve the Flask issue, despite creating conflicts with packages versions...
Can anyone explain this?

[bondany]

Update

Apparently the message appears only when using superset --help which I was using for testing/check Superset configuration correctness. It does not appear with superset db --help or any other combination of command/option, which is confusing.
In the old version we were using, this was not happening...

[rusackas]

For export/import, I can't remember when it changed from yaml to zip, but you may need to import yaml at some intermediary version while there was still support for that. That's a big leap in upgrades, I think you'll be happy with the current state of the app when you get there!