Image upload failed #6

nh20089 · 2023-05-26T03:23:42Z

Image upload failed
http://cytomine.local/api/imageserver/format.json（Return 500）
http://upload.cytomine.local/upload?cytomine=http://cytomine.local&idStorage=51&idProject=（Return 404）
Do you have any good troubleshooting suggestions

gvincke · 2023-05-26T10:19:35Z

Hi,
On which operating system are you running Cytomine ?
Please give the most detailed context on your installation and your use.
Thank you.
Grégoire

nh20089 · 2023-05-29T01:35:29Z

Server system version Centos 7.9 kernel version 3.10.0-1160. el7.x86_ sixty-four

It has been resolved so far http://cytomine.local/api/imageserver/format.json The problem of returning 500 is due to the Domain Name System configuration of the core container.
Image upload still exists http://upload.cytomine.local/upload?cytomine=http://cytomine.local&idStorage=51&idProject= (Return to 404), via git clone https://github.com/cytomine/Cytomine-community-edition.git Install the version and execute docker compose up - d

thanks

nh20089 · 2023-05-29T03:13:14Z

Found an error in nginx
2023/05/29 02:40:37 [error] 58#58: *334 upstream prematurely closed connection while reading response header from upstream, client: 192.168.46.1, server: upload.local, request: "POST /upload?cytomine=http://core.local&idStorage=51&idProject= HTTP/1.1", upstream: "http://172.19.0.9:5000/upload?cytomine=http://core.local&idStorage=51&idProject=", host: "upload.local", referrer: "http://core.local/"

Attempt local access to pims, curl http://172.19.0.9:5000 Return {"detail": "Method Not Allowed"}, curl http://172.19.0.9:5000/upload?cytomine=http://core.local , return {"detail": "Method Not Allowed"}

Do you have any troubleshooting suggestions

jgillardcytomine · 2023-05-30T14:13:09Z

The nginx error seems to be an issue with containers IP Addresses.

Could you try to restart your Cytomine instance?

docker compose down
docker compose up -d

If the problem persists, could you please try to debug the dns resolutions?

# Enter the container
docker exec -it nginx sh

# Install dig
apk add --update bind-tools

# See DNS resolution of upload.local
dig upload.local

# Same for core 
docker exec -it core bash
apt install dnsutils
dig upload.local

Hope this helps 🤓

nh20089 · 2023-05-31T05:12:33Z

Hello, I have modified the hosts files of the internal core and nginx of the container. Executing the ping command returns the IP address of the domain name mapping in the hosts file. By executing the dig upload.local command, I have returned an NXDOMAIN response, which means that the domain name cannot be resolved in the local DNS.

I plan to access in the intranet environment. For Domain Name System, it seems useless to modify the local hosts file. What other solutions are recommended? Thank you!

EKami · 2023-06-22T08:03:55Z

I'm having the same issue. File upload works fine when the server is run locally but when it's distant I get an error at the end of the file upload. Core is returning 2 exceptions in the console, A json parser exception and a nullpointer:

com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"<!DOCTYPE html><html><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><meta name="version" content="2.6.1"><link rel="icon" href="/favicon.ico?v=2"><title>Cytomine</title><link href="/css/app.102a0df7.css" rel="preload" as="style"><link href="/css/chunk-vendors.44629b2d.css" rel="preload" as="style"><link href="/js/app.cfbc550a.js" rel="preload" as="script"><link href="/js/chunk-vendors.b9534804.js"[truncated 439 chars]; line: 1, column: 2]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2391)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:735)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:659)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:2005)

2023-06-22 07:53:39.056 ERROR 1 --- [io-8080-exec-10] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException: Cannot invoke "be.cytomine.utils.JsonObject.get(Object)" because "jsonObject" is null] with root cause

java.lang.NullPointerException: Cannot invoke "be.cytomine.utils.JsonObject.get(Object)" because "jsonObject" is null
        at be.cytomine.service.middleware.ImageServerService.formats(ImageServerService.java:110)
        at be.cytomine.service.middleware.ImageServerService$$FastClassBySpringCGLIB$$b94584bb.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)

I dunno if it's related.

here are the results for the DNS resolutions:

# Enter the container
docker exec -it nginx sh

# Install dig
apk add --update bind-tools

# See DNS resolution of upload.local
$ dig upload.local

; <<>> DiG 9.18.16 <<>> upload.local
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 1929
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;upload.local.                  IN      A

;; Query time: 4 msec
;; SERVER: 127.0.0.11#53(127.0.0.11) (UDP)
;; WHEN: Thu Jun 22 08:02:15 UTC 2023
;; MSG SIZE  rcvd: 30


# Same for core 
docker exec -it core bash
apt install dnsutils
$ dig upload.local

; <<>> DiG 9.16.37-Debian <<>> upload.local
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 29681
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;upload.local.                  IN      A

;; Query time: 0 msec
;; SERVER: 127.0.0.11#53(127.0.0.11)
;; WHEN: Thu Jun 22 08:03:46 UTC 2023
;; MSG SIZE  rcvd: 30

jgillardcytomine · 2023-06-23T08:33:40Z

Hello folks, right now Cytomine and the final client need to resolve the URLs defined in the cytomine.template (and after the installer has been run, in the .env file) to the IP of the machine running Cytomine.

URLS_CORE=cytomine.local
URLS_IMAGE_SERVER=ims.cytomine.local
URLS_UPLOAD=upload.cytomine.local

In other words, in your LAN, those URLs must resolve to the IP of the machine running Cytomine. That must be true from within the containers and for the end-user browsing those URLs.

Hope this helps.

EKami · 2023-06-26T05:40:35Z

Hello folks, right now Cytomine and the final client need to resolve the URLs defined in the cytomine.template (and after the installer has been run, in the .env file) to the IP of the machine running Cytomine.
URLS_CORE=cytomine.local
URLS_IMAGE_SERVER=ims.cytomine.local
URLS_UPLOAD=upload.cytomine.local
In other words, in your LAN, those URLs must resolve to the IP of the machine running Cytomine. That must be true from within the containers and for the end-user browsing those URLs.

Hope this helps.

Thanks for the help but I believe I've already set the correct values, it still doesn't explain why the upload is working locally but not remotely. I get an HTTP 413 when used remotely. I tried to change the nginx upload size but it didn't work.

The file uploads to the server but then when it's finished uploading this error pop-out.

Is there another way to upload files to Cytomine other than the web UI? By mounting a volume onto the pods for example? Thanks!

jgillardcytomine · 2023-06-26T06:57:44Z

This seems to be a DNS problem.
Our host rewriter tries to set the URLS_CORE, URLS_IMAGE_SERVER, URLS_UPLOAD in your /etc/hosts file to resolve locally.
Now if you want to access your Cytomine instance remotely, your have to setup A or CNAME records at your DNS provider.
But it won't allow you to work with .local TLD, so I assume you want to access remotely within your LAN. To do so, you'll need a local DNS server (e.g. dnsmasq) that can reply such queries:

/ # dig upload.cytomine.local

; <<>> DiG 9.18.16 <<>> upload.cytomine.local
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62412
;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;upload.cytomine.local.         IN      A

;; ANSWER SECTION:
upload.cytomine.local.  0       IN      A       192.168.240.6

;; Query time: 0 msec
;; SERVER: 127.0.0.11#53(127.0.0.11) (UDP)
;; WHEN: Mon Jun 26 06:46:43 UTC 2023
;; MSG SIZE  rcvd: 66

Your previous comment shows no ANSWER SECTION. That my be the root cause of the failing upload.

Is there another way to upload files to Cytomine other than the web UI? By mounting a volume onto the pods for example?

No, Cytomine needs to process the image before you can work with it. The only way to have import images into Cytomine is to upload them trough the web UI.

Hope this helps 🤞

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image upload failed #6

Image upload failed #6

nh20089 commented May 26, 2023

gvincke commented May 26, 2023

nh20089 commented May 29, 2023

nh20089 commented May 29, 2023

jgillardcytomine commented May 30, 2023

nh20089 commented May 31, 2023

EKami commented Jun 22, 2023

jgillardcytomine commented Jun 23, 2023

EKami commented Jun 26, 2023 •

edited

Loading

jgillardcytomine commented Jun 26, 2023

Image upload failed #6

Image upload failed #6

Comments

nh20089 commented May 26, 2023

gvincke commented May 26, 2023

nh20089 commented May 29, 2023

nh20089 commented May 29, 2023

jgillardcytomine commented May 30, 2023

nh20089 commented May 31, 2023

EKami commented Jun 22, 2023

jgillardcytomine commented Jun 23, 2023

EKami commented Jun 26, 2023 • edited Loading

jgillardcytomine commented Jun 26, 2023

EKami commented Jun 26, 2023 •

edited

Loading