Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: Can't send data because WebSocket is not opened #38

Open
Wizzzz opened this issue May 23, 2024 · 7 comments
Open

Error: Can't send data because WebSocket is not opened #38

Wizzzz opened this issue May 23, 2024 · 7 comments

Comments

@Wizzzz
Copy link

Wizzzz commented May 23, 2024

Hello, I'm opening this issue because I have a problem with playwright-with-fingerprint. After a few uses, I get an error when generating a browser and I have to stop my nodejs and restart it.

I've been trying to fix the error myself for a while now, but it's impossible, so I'm relying on you.

Here's the error in question:
Error: Can't send data because WebSocket is not opened. at exports.throwIf (C:\Users\Administrator\Desktop\project\node_modules\websocket-as-promised\src\utils.js:4:11) at WebSocketAsPromised.send (C:\Users\Administrator\Desktop\project\node_modules\websocket-as-promised\src\index.js:252:5) at SocketService.send (C:\Users\Administrator\Desktop\project\node_modules\bas-remote-node\src\services\socket.js:89:14) at BasRemoteClient._send (C:\Users\Administrator\Desktop\project\node_modules\bas-remote-node\src\index.js:223:25) at BasRemoteClient.send (C:\Users\Administrator\Desktop\project\node_modules\bas-remote-node\src\index.js:254:17) at BasRemoteClient._startThread (C:\Users\Administrator\Desktop\project\node_modules\bas-remote-node\src\index.js:275:10) at C:\Users\Administrator\Desktop\project\node_modules\bas-remote-node\src\index.js:190:12 at new Promise (<anonymous>) at BasRemoteClient.runFunction (C:\Users\Administrator\Desktop\project\node_modules\bas-remote-node\src\index.js:189:21) at C:\Users\Administrator\Desktop\project\node_modules\browser-with-fingerprints\src\plugin\connector\index.js:31:16

To solve the problem, I have to close my program and restart it.

I've noticed that this error occurs if I have other programs that use playwright-with-fingerprint, but if I don't have any other nodes that use the plugin, I never get the error.

Here's my class for managing browsers:

`
import { FetchOptions } from "browser-with-fingerprints";
import fs from "fs";
import path from "path";
import { BrowserContext } from "playwright-core";
import { plugin } from "playwright-with-fingerprints";

import { Logger } from "../../logger";
import { Proxy } from "../../proxy";
import { BrowserOptions } from "../../types/browser";
import { capsolverDefaultConfig, CapsolverOptions } from "../../types/capsolver";
import { nopechaDefaultConfig, NopechaOptions } from "../../types/nopecha";
import { sleep } from "../../utils";

type InstanceOptions = {
browserOptions: BrowserOptions;
capsolverOptions?: CapsolverOptions;
c;
nopechaOptions?: NopechaOptions;
proxy?: Proxy;
};

type PageOptions = {
blockImageDomains?: string[];
};

export class BrowserInstance {
private browser: BrowserContext | null = null;
readonly instanceOptions: InstanceOptions;

constructor(instanceOptions: InstanceOptions) {
	this.instanceOptions = instanceOptions;

	if (this.instanceOptions.capsolverOptions) {
		const configPath = path.join(__dirname, "..\\..\\extensions\\capsolver\\assets\\config.json");
		const config = { ...capsolverDefaultConfig, ...this.instanceOptions.capsolverOptions };
		fs.writeFileSync(configPath, JSON.stringify(config));
	} else if (this.instanceOptions.nopechaOptions) {
		const manifestPath = path.join(__dirname, "..\\..\\extensions\\nopecha\\manifest.json");
		const config = { ...nopechaDefaultConfig, ...this.instanceOptions.nopechaOptions };
		const manifest = JSON.parse(fs.readFileSync(manifestPath, "utf8"));
		manifest.nopecha = config;
		fs.writeFileSync(manifestPath, JSON.stringify(manifest));
	}

	// ignore unhandled playwright-with-fingerprint exceptions
	if ((global as any).ignoreUncaughtException !== true) {
		Logger.error("Uncaught exceptions are not being handled");
		(global as any).ignoreUncaughtException = true;
		(global as any).browserError = 0;
		process.on("uncaughtException", (err) => {
			console.error("uncaughtException:", err);
			(global as any).browserError++;
			if ((global as any).browserError > 15) {
				Logger.error("Too many browser exceptions, exiting process");
				process.exit(1);
			}
		});
	}
}

private async startBrowser() {
	// set extensions path
	const capsolverPath = path.join(__dirname, "..\\..\\extensions\\capsolver");
	const nopechaPath = path.join(__dirname, "..\\..\\extensions\\nopecha");

	const device: FetchOptions = {
		tags: this.instanceOptions.browserOptions.tags || ["Microsoft Windows", "Chrome"],
	};
	plugin.setRequestTimeout(90000);

	// if chrome device, set last version
	if (device.tags && device.tags.includes("Chrome")) {
		const versions = await plugin.versions("extended");
		const browserVersion = versions[0]["browser_version"];
		const reducedBrowserVersion = parseInt(browserVersion.split(".")[0]);

		device.minBrowserVersion = reducedBrowserVersion;
		device.maxBrowserVersion = reducedBrowserVersion;
	}

	// fetch fingerprint (use minx/max browser version bc impossible use useBrowserVersion)
	const fingerprint = await plugin.fetch(this.instanceOptions.browserOptions.fingerprintSwitcherKey, device);
	const args = this.instanceOptions.browserOptions.args || [];
	plugin.useFingerprint(fingerprint);

	// loads captcha solvers extensions
	if (this.instanceOptions.capsolverOptions) args.push(`--load-extension=${capsolverPath}`);
	else if (this.instanceOptions.nopechaOptions) args.push(`--load-extension=${nopechaPath}`);

	// set proxy
	if (this.instanceOptions.proxy)
		plugin.useProxy(this.instanceOptions.proxy.fullAddress(), {
			detectExternalIP: true,
			changeGeolocation: true,
			changeWebRTC: true,
			changeTimezone: true,
			changeBrowserLanguage: true,
		});

	// launch browser
	this.browser = (await plugin.launchPersistentContext(this.instanceOptions.browserOptions.profilePath, {
		headless: this.instanceOptions.browserOptions.headless,
		args,
	})) as BrowserContext;
}

public async init() {
	if (this.browser) throw "Browser is already initialized";

	// blocks instance creation if one is already in progress
	let isPossibleToCreateBrowser = false;
	if (!fs.existsSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`)) fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, "0");

	do {
		const lastBrowserGeneration = parseInt(fs.readFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, "utf8"));
		if (Date.now() - lastBrowserGeneration > 90000) {
			fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, Date.now().toString());
			isPossibleToCreateBrowser = true;
		} else await sleep(1000);
	} while (!isPossibleToCreateBrowser);

	try {
		const isBrowserCreated = await Promise.race([
			sleep(90000).then(() => false),
			this.startBrowser()
				.then(() => true)
				.catch(async (error) => {
					console.error(error);
					(global as any).browserError++;
					if ((global as any).browserError > 15) {
						Logger.error("Too many browser exceptions, exiting process");
						process.exit(1);
					}
					await sleep(90000);
					return false;
				}),
		]);

		if (!isBrowserCreated) throw "[BROWSER INSTANCE] Timeout or error during browser creation";

		if (!this.browser) throw "[BROWSER INSTANCE] Browser failed to initialize";
	} catch (error) {
		if (this.browser) await this.close();
		throw error;
	} finally {
		await sleep(10000);
		fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, "0");
	}
}

public async newPage(options: PageOptions = {}) {
	if (!this.browser) throw "[BROWSER INSTANCE] Browser is not initialized";

	const page = await this.browser.newPage();

	if (options.blockImageDomains && options.blockImageDomains.length > 0) {
		await page.route("**/*", async (route) => {
			const request = route.request();

			if (request.resourceType() === "image") {
				const url = request.url();
				for (const domain of options.blockImageDomains!) if (url.includes(domain)) return await route.abort();
			}

			return await route.continue();
		});
	}

	return page;
}

public async close() {
	if (!this.browser) throw "[BROWSER INSTANCE] Browser is not initialized";

	await this.browser.close();
}

}
`

Thank advance !

@Wizzzz
Copy link
Author

Wizzzz commented Jul 17, 2024

Hello, I'm coming back to you because I've implemented what you told me, I have a folder with 15 engines and a database that allows me to make it impossible to take 2 engines simultaneously, unfortunately, I have the following problem:
image

It's important to remember that this isn't multi-threading, but rather several nodes with different tasks, so it's not the same process that launches the different browsers.

Here is my code to launch a browser:
`
private async startBrowser() {
// set extensions path
const capsolverPath = path.join(__dirname, '..\..\extensions\capsolver');
const nopechaPath = path.join(__dirname, '..\..\extensions\nopecha');

	// retrieve engine data
	await axios
		.get('http://localhost:4242/', { params: { duration: this.instanceOptions.taskDuration } })
		.then((response: AxiosResponse) => {
			this.workDirId = response.data.uuid;
			plugin.setWorkingFolder(response.data.path);
		})
		.catch(error => {
			Logger.error('Failed to retrieve engine data,  use default path');
		});

	// set tags
	const device: FetchOptions = {
		tags: this.instanceOptions.browserOptions.tags || ['Microsoft Windows', 'Chrome']
	};

	// if chrome device, get and set last version (use min/max browser version bc impossible use useBrowserVersion)
	if (device.tags && device.tags.includes('Chrome')) {
		const versions = await plugin.versions('extended');
		const browserVersion = versions[0]['browser_version'];
		const reducedBrowserVersion = parseInt(browserVersion.split('.')[0]);

		device.minBrowserVersion = reducedBrowserVersion;
		device.maxBrowserVersion = reducedBrowserVersion;
	}

	// if profile set, use it
	if (
		this.instanceOptions.browserOptions.profilePath &&
		fs.existsSync(this.instanceOptions.browserOptions.profilePath)
	) {
		plugin.useProfile(this.instanceOptions.browserOptions.profilePath, {
			loadFingerprint: true,
			loadProxy: true
		});
	}

	// fetch fingerprint
	const fingerprint = await plugin.fetch(this.instanceOptions.browserOptions.fingerprintSwitcherKey, device);
	const args = this.instanceOptions.browserOptions.args || [];
	plugin.useFingerprint(fingerprint);

	// loads captcha solvers extensions
	if (this.instanceOptions.capsolverOptions) args.push(`--load-extension=${capsolverPath}`);
	else if (this.instanceOptions.nopechaOptions) args.push(`--load-extension=${nopechaPath}`);

	// set proxy
	if (this.instanceOptions.proxy)
		plugin.useProxy(this.instanceOptions.proxy.fullAddress(), {
			detectExternalIP: true,
			changeGeolocation: true,
			changeWebRTC: true,
			changeTimezone: true,
			changeBrowserLanguage: true
		});

	// launch browser
	try {
		let isLaunch = false;
		await Promise.race([
			sleep(20000).then(() => {
				if (!isLaunch) throw '[BROWSER INSTANCE] Timeout during launch';
			}),
			new Promise((resolve, reject) => {
				plugin
					.launchPersistentContext(this.instanceOptions.browserOptions.profilePath, {
						headless: this.instanceOptions.browserOptions.headless,
						args
					})
					.then((browser: BrowserContext) => {
						isLaunch = true;
						this.browser = browser;
						resolve(browser);
					})
					.catch((error: any) => {
						reject(error);
					});
			})
		]);
	} catch (error) {
		(global as any).browserError++;
		if ((global as any).browserError > 50) {
			Logger.error('Too many browser exceptions, exiting process');
			process.exit(1);
		}
		throw error;
	}
}

public async init() {
	if (this.browser) throw 'Browser is already initialized';

	// blocks instance creation if one is already in progress
	let isPossibleToCreateBrowser = false;
	if (!fs.existsSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`))
		fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, '0');

	do {
		const lastBrowserGeneration = parseInt(
			fs.readFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, 'utf8')
		);
		if (Date.now() - lastBrowserGeneration > 90000) {
			fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, Date.now().toString());
			isPossibleToCreateBrowser = true;
		} else await sleep(1000);
	} while (!isPossibleToCreateBrowser);

	try {
		const isBrowserCreated = await Promise.race([
			sleep(90000).then(() => false),
			this.startBrowser()
				.then(() => true)
				.catch(async error => {
					console.error(error);
					await sleep(90000);
					return false;
				})
		]);

		if (!isBrowserCreated) throw '[BROWSER INSTANCE] Timeout or error during browser creation';

		if (!this.browser) throw '[BROWSER INSTANCE] Browser failed to initialize';
	} catch (error) {
		if (this.browser) await this.close();
		throw error;
	} finally {
		await sleep(10000);
		fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, '0');
	}
}

`

@CheshireCaat

@CheshireCaat
Copy link
Owner

@Wizzzz Greetings, try to check the following algorithm, perhaps one of the node processes has not completed and is holding a lock, which causes an error:

  • Manually close all node processes.
  • Launch the single browser instance.
  • Check if the problem has occurred again.

Just in case, I recommend that you ALWAYS close the browser using the browser.close() method after completion, because this also clears some of the overlaid locks.

If the problem does not repeat itself, then the processes can still be open when you run your code - there is a timer in the client to automatically turn off the engine after 5 minutes, you can simply force the current process to end upon completion.

Or do additional management before starting - check if there is another process that used the same folder, and if there is, then kill it.

You can also try importing a list of locks from the proper-lockfile package and try to clear them before/after starting manually.

When I have time for fixes and updates, I will try to find another package for locks inside the plugin instead of the current one, because of it there are too many problems and too often.

Anyway, I hope my advice will help you a little.

@Wizzzz
Copy link
Author

Wizzzz commented Jul 17, 2024

@CheshireCaat
After verification, all engines have a .lock file.

Will deleting the .lock file unblock the situation? Technically, with my DB, it's impossible for 2 processes to use the same engine.

So I can make sure that before starting a browser, I check the existence of the .lock file and if it exists, delete it.

If everything is well managed in my DB, there won't be any problems?

@CheshireCaat
Copy link
Owner

@Wizzzz you can try to remove lock file, but there is no guarantee that it complete release it for the target proccess. At least, if it will not work, you can try other advices from the my previous comment.

If everything is well managed in my DB, there won't be any problems?

In theory, taking into account the rest of the recommendations, everything should be fine.

@Wizzzz
Copy link
Author

Wizzzz commented Jul 17, 2024

@CheshireCaat
Okok I'll give it a try and let you know.

Is there a discord or a telegram group with other users of your packages? It might be cool to have one to discuss how we can implement a common solution to face its problems since we are several to have it.

@Wizzzz
Copy link
Author

Wizzzz commented Jul 17, 2024

Is there an example of the solution you propose? Because I'm not sure I've understood 100% of the different steps. ;(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants