Resolve LabelBOT for tiled images #1232

[yannik131]

Resolves Issue #1232 by taking a screenshot of the map if the image is tiled.

[yannik131]

@mzur
kleines Problem: Ich hatte aus Versehen die .onnx-Datei mit auf remote gepusht. Ist nicht so leicht, die aus der History wieder rauszukriegen (man müsste die gesamte History mit git filter-repo o. ä. umschreiben). Also würde ich alles löschen und nur die Codeänderungen wieder reinkopieren mit

cp -r resources ~/backup/resources
git reset --hard <last commit before my changes>
git push --force
cp -r ~/backup/resources resources
git add .
git commit -m "Made labelbot work for tiled images"
git push

Dann geht zwar meine History (5 commits -> 1 commit) verloren, aber das ist nicht so schlimm denke ich. Passt das so? Dann mach ich das morgen und dann kann die PR auch reviewed werden.

[mzur]

Ja das passt so, kein Problem 👍

[mzur]

btw this is our commit message style guide in case you are interested: https://chris.beams.io/git-commit

[yannik131]

@mzur I changed the PR to ready for review, but can't select a reviewer or request a review. It's ready though.

[mzur]

Please resolve the linter errors. Run it locally with npm run lint.

[yannik131]

I resolved them, but the pipeline doesn't seem to run automatically @mzur

[mzur]

This works great already! My comments are mostly about stylistic issues.

[yannik131]

I made the changes as requested @mzur, ready for review again.

[mzur]

I've merged the current master.

I noticed that there was a slight delay when the labelbotImage is generated. I suggested some optimizations below.

[mzur]

Suggested change

	if(!this.image.tiled)
	return this.createLabelbotImageFromRegularImage(points);
	else
	return await this.createLabelbotImageFromTiledImage(points);
	if (!this.image.tiled) {
	return this.createLabelbotImageFromRegularImage(points);
	} else {
	return await this.createLabelbotImageFromTiledImage(points);
	}

[mzur]

Suggested change

	if(width === 0 || height === 0) {
	if (width === 0 || height === 0) {

[mzur]

Suggested change

	if(!this.tempCanvas) {
	if (!this.tempCanvas) {

[mzur]

Suggested change

	if(this.labelbotIsActive)
	{
	try {
	labelbotImage = await this.createLabelbotImage(points);
	}
	catch(error) {
	this.annotationSource.removeFeature(e.feature);
	Messages.danger(error.message);
	return;
	}
	}
	if (this.labelbotIsActive) {
	try {
	labelbotImage = await this.createLabelbotImage(points);
	} catch(error) {
	this.annotationSource.removeFeature(e.feature);
	Messages.danger(error.message);
	return;
	}
	}

[mzur]

Suggested change

	if(this.labelbotRequestsInFlight > 0) {
	if (this.labelbotRequestsInFlight > 0) {

[mzur]

makeBlob and createImageBitmap are quite slow (600-700 ms on my machine). We don't need a blob or an ImageBitmap object, though, and can use the trimmed canvas directly (200 ms on my machine):

Suggested change

	makeBlob(event.context.canvas).then(blob => {
	resolve(createImageBitmap(blob));
	});
	resolve(trimCanvas(event.context.canvas));

makeBlob can be moved back to the screenshotButton then.

Most of the remaining 200 ms was used by trimCanvas. This can be improved with an early return (see below).

[mzur]

Early return:

Suggested change

	let ctx = canvas.getContext('2d');
	let ctx = canvas.getContext('2d');
	let topLeft = ctx.getImageData(0, 0, 1, 1);
	let bottomRight = ctx.getImageData(canvas.width - 1, canvas.height - 1, 1, 1);
	if (topLeft.data[3] !== 0 && bottomRight.data[3] !== 0) {
	return canvas;
	}