Major Bug: Docker Image based azurite not connecting with Docker Image based Microsoft "MS SQL Server" Instance after version since "azurite 3.24.0"

[mansoorbk]

Which service(blob, file, queue, table) does this issue concern?

All

Which version of the Azurite was used?

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

Docker Hub: mcr.microsoft.com/azure-storage/azurite:latest

What's the Node.js version?

MS Docker Image mcr.microsoft.com/azure-storage/azurite:latest is using Node.js 14

What problem was encountered?

Docker Image based azurite not connecting with Docker Image based Sql Server Instance

Steps to reproduce the issue?

Use docker-compose based setup:

• Manual creation of DATABASE after database is created
• docker exec -d sql-ubuntu-2204 /opt/mssql-tools/bin/sqlcmd -S . -U sa -P "StrongPass!123" -Q "CREATE DATABASE [azurite_blob]"

---- FILE START------
version: '3.4'

services:

sql-ubuntu-2204:
image: mcr.microsoft.com/mssql/server:2022-CU10-ubuntu-22.04
container_name: "sql-ubuntu-2204"
hostname: sql-ubuntu-2204
restart: always
environment:
- MSSQL_PID=Evaluation
- ACCEPT_EULA=Y
- SA_PASSWORD=StrongPass!123
ports:
- "1533:1433"

azurite:
image: mcr.microsoft.com/azure-storage/azurite:latest
container_name: "azurite"
hostname: azurite
restart: always
environment:
- AZURITE_DB=mssql://sa:StrongPass![email protected]:1533/azurite_blob
ports:
- "10000:10000"
- "10001:10001"
- "10002:10002"
---- FILE END------
If possible, please provide the debug log using the -d parameter, replacing <pathtodebuglog> with an appropriate path for your OS, or review the instructions for docker containers:

Exit due to unhandled error: Unexpected token '??='
Exit due to unhandled error: Unexpected token '??='
Exit due to unhandled error: Unexpected token '??='

Please be sure to remove any PII or sensitive information before sharing!
The debug log will log raw request headers and bodies, so that we can replay these against Azurite using REST and create tests to validate resolution.

Have you found a mitigation/solution?

No. But it works with MySQL

[blueww]

@mansoorbk

Our team owns Azurite, but we are not docker expect. You issue looks a docker issue for communication between 2 docker images.

However, from the above info, I think the 1533 port in the Azurite DB connection point to the 1533 port of the Azurite docker container, but not the 1533 port of the host machine. - AZURITE_DB=mssql://sa:StrongPass![[email protected]](mailto:[email protected]):1533/azurite_blob So on the 1533 port of the Azurite docker container there are no DB, so you can't connect to it.

I find some similar questions for how to connect 2 docker containers , you might can reference them.
https://stackoverflow.com/questions/45481943/connecting-two-docker-containers
https://docs.docker.com/network/network-tutorial-standalone/
https://saggu.medium.com/how-to-connect-nultiple-docker-conatiners-17f7ca72e67f

And another way might be put Sql DB and Azurite into same docker container.

[mansoorbk]

@blueww .
I understand docker is not your teams expertise. But please note that azurite Docker Image is officially provided.
But the simple problem is that
1- This is related to MSSQL Server (Microsoft Product) support that your documentation mentions. In Documentation, example is provided for MySql Server and also tests. But not for MSSQL Server.
2- I have slowly narrowed down the main problem with MSSqlServer to 3.24.0 and above"azurite" Docker Image.
3- The version that works for MSSQLServer is azurite 3.23.0.
4- Now that I have Identified the issue. How soon Can I expect the fix for MSSQLServer support?

Following is a working docker-compose example of MSSQLServer and mysql:

# Manual creation of DATABASE (IMPORTNANT!) MS SQL Server
docker exec -d mssql2202 /opt/mssql-tools/bin/sqlcmd -S . -U sa -P "Strong!123Pass" -Q "CREATE DATABASE [azurite_blob]"

# Manual creation of DATABASE (IMPORTNANT!) MySql
docker exec mysql mysql -u root -pStrong!123Pass -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION; FLUSH PRIVILEGES; create database azurite_blob;"

------YML FILE START ------

version: '3.4'

services:

  mysql:
    image: mysql:latest
    container_name: "mysql"
    hostname: mysql
    restart: always
    environment:
     - MYSQL_ROOT_PASSWORD=Strong!123Pass
    ports:
      - "3306:3306"
    networks:
      - networkazurite


  mssql2202:
    image: mcr.microsoft.com/mssql/server:2022-latest
    container_name: "mssql2202"
    hostname: mssql2202
    restart: always
    environment:
     - MSSQL_PID=Evaluation
     - ACCEPT_EULA=Y
     - MSSQL_SA_PASSWORD=Strong!123Pass
    ports:
      - "1433:1433"
    networks:
      - networkazurite

  azurite:
    image: mcr.microsoft.com/azure-storage/azurite:3.23.0
    container_name: "azurite"
    hostname: azurite
    restart: always
    depends_on:
      - mssql2202
      - mysql
    environment:
      - AZURITE_DB=mssql://sa:Strong!123Pass@mssql2202:1433/azurite_blob
      #- AZURITE_DB=mysql://root:Strong!123Pass@mysql:3306/azurite_blob
    ports:
      - "10000:10000"
      - "10001:10001"
      - "10002:10002"
    networks:
      - networkazurite

networks:
  networkazurite:
    name: networkazurite

[mansoorbk]

@blueww Any update on findings of broken support for MS SQL Server in Azurite after version "azurite 3.23.0".

This is definitely a bug that should be put up for resolution for production related cases.
Can you add the tag of bug at least?

[blueww]

@mansoorbk

We have mysql test for every PR in pipeline.
https://github.com/Azure/Azurite/blob/d544d16f910e490fdd9db5565459df701895308f/azure-pipelines.yml#L146C1-L146C23

And this is Azurite blob test pass on mysql 2 weeks ago:
https://dev.azure.com/azure/Azurite/_build/results?buildId=70901&view=logs&jobId=e8602c86-c0d1-5860-9b4d-3b481418c7ab&j=e8602c86-c0d1-5860-9b4d-3b481418c7ab&t=4998437a-59db-5f46-e4f4-becf42d7217f

I have also manually run mysql in docker with following command, then run Azurite 3.29.0 on my local machine (win11) after set environment variable "set AZURITE_DB=mysql://root:[email protected]:13306/azurite_blob_test". Azurite works well with mysql.

          docker run --name mysql -p 13306:3306 -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql
          // sleep 60 seconds
          docker exec mysql mysql -u root -pmy-secret-pw -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION; FLUSH PRIVILEGES;"
          docker exec mysql mysql -u root -pmy-secret-pw -e "create database azurite_blob_test;"

So Azurite latest version 3.29.0 works with mysql in my test, and we have test to cover it in every PR validation.

[blueww]

@EmmaZhu

Would you please help to follow up this issue when I am on vacation?
Thanks!

[mansoorbk]

@blueww @EmmaZhu , I am not sure what your response regarding "MySql" has to do with my question about Microsoft "MS SQL Server" ?
My problem is regarding Microsoft "MS SQL Server" support being broken. It is clearly mentioned above.
Also, Microsoft "MS SQL Server" is officially supported as per your documentation.

https://github.com/Azure/Azurite?tab=readme-ov-file#customized-metadata-storage-by-external-database-preview
https://github.com/Azure/Azurite?tab=readme-ov-file#customized-metadata-storage-by-external-database-preview:~:text=storage%2C%20like%20MySql%2C-,SqlServer,-.
To Recap the bug. I will write my findings again.

1- This bug is related to Microsoft "MS SQL Server" support that your documentation mentions. In Documentation, example is for Microsoft "MS SQL Server" is not provided. It is officially mentioned that it is supported as per the link above.
2- I have slowly narrowed down the main problem with Microsoft "MS SQL Server" to 3.24.0 and above "azurite" Docker Image.
3- The version that works for Microsoft "MS SQL Server" is azurite 3.23.0.
4- Now that I have Identified the issue. How soon Can I expect the fix for Microsoft "MS SQL Server" support?

# Manual creation of DATABASE (IMPORTNANT!) MS SQL Server
docker exec -d mssql2202 /opt/mssql-tools/bin/sqlcmd -S . -U sa -P "Strong!123Pass" -Q "CREATE DATABASE [azurite_blob]"

------YML FILE START ------

version: '3.4'

services:

  mssql2202:
    image: mcr.microsoft.com/mssql/server:2022-latest
    container_name: "mssql2202"
    hostname: mssql2202
    restart: always
    environment:
     - MSSQL_PID=Evaluation
     - ACCEPT_EULA=Y
     - MSSQL_SA_PASSWORD=Strong!123Pass
    ports:
      - "1433:1433"
    networks:
      - networkazurite

  azurite:
    image: mcr.microsoft.com/azure-storage/azurite:3.29.0
    container_name: "azurite"
    hostname: azurite
    restart: always
    depends_on:
      - mssql2202
      - mysql
    environment:
      - AZURITE_DB=mssql://sa:Strong!123Pass@mssql2202:1433/azurite_blob
    ports:
      - "10000:10000"
      - "10001:10001"
      - "10002:10002"
    networks:
      - networkazurite

networks:
  networkazurite:
    name: networkazurite

[EmmaZhu]

Hi @mansoorbk , I'll look into it and try to repro the issue.
Could you share which version of OS are your using? From one of your command parameter, seems you are trying to run Azurite and mssql on ubuntu?

[mansoorbk]

@EmmaZhu Thank you for your help.
My OS Config are:

• Windows 11 x64 Pro (10.0.22621)
• WSL2 (Ubuntu 22.04.3 LTS)
• Docker Desktop (Linux containers)

[EmmaZhu]

Hi @mansoorbk ,

Seems I can repro this issue. I tried to start docker containers with the YML you shared, it did succeeded to start with Azurite 3.23.0 and mssql2022, but got an error like Exit due to unhandled error: Unexpected token '??=' with Azurite 3.29.0, is it the same error as the one you got?

You shared three OS versions, are you running a Linux Container on Windows? If the error I got is the same with the one you got, I should have been able to reproduce the issue, OS version may be irrelevant. I'll look into the error I got first.

[mansoorbk]

@EmmaZhu .
Yes. This is the same error I am getting.
Yes. I am running Linux Container on Windows.

These are the details of my OS and Dev Environment.

• Windows 11 x64 Pro (10.0.22621)
• WSL2 (Ubuntu 22.04.3 LTS)
• Docker Desktop for windows (Linux containers on Windows)

[mansoorbk]

@EmmaZhu Since you can repro this bug.
Can you mark it as bug?
It still only has label of question.

[mansoorbk]

@EmmaZhu Any update ?

[EmmaZhu]

Hi @mansoorbk ,

For a progress update:
I think it might be this change d98f532 caused the failure.

I'm seeing issue when trying to upgrade to a newer version node to build a new image, also cannot validate whether my theory is correct. I'm working on resolving current upgrade issue.

[mansoorbk]

@EmmaZhu Thanks for update. Looking forward to fix.
Can you mark it as bug label? It is showing up as question label.

[mansoorbk]

@EmmaZhu Any update ? This will help us alot in production

[blueww]

@mansoorbk

Emma already has a fix for this issue: #2370
We need to do more review and test on the fix. If no big issue found, we will merge this fix PR and release it in the next Azurite release which might in 2-3 weeks.

[mansoorbk]

@EmmaZhu @blueww
Thanks for the great news. Look forward to the to being able to use mssql with azurite in prod

[EmmaZhu]

Issue has been fixed. Any further investigation required, please feel free to reopen it.