How can i minimize total data usage with crawlee + PlaywrightCrawler + camoufox-js ?

[robert-westenberger]

All I need is just the content from the initial HTML document for every request. I don't need any additional scripts, images, stylesheets, anything.

I am using PlaywrightCrawler with camoufox-js. My crawler config looks similar to the example found here https://crawlee.dev/js/docs/guides/avoid-blocking#camoufox

with the addition of this preNavigationHook

preNavigationHooks: [
  async ({ page }) => {
    await page.route('**/*', (route, request) =>
      request.isNavigationRequest() ? route.continue() : route.abort(),
    );
  },
];

Is there anything more that I could be doing?

I also looked into using CheerioCrawler, but it seemed like PlaywrightCrawler + camoufox had better stealth / bot detection capabilities.

[robert-westenberger]

I was looking at my residential proxy provider's dashboard to see how much data was being used. I was looking at the wrong thing and thought I was using more data than I was. It seems like the above preNavigationHook is doing what I need.