How to run a headless driver with user-data-dir and a running headful browser instance?

[KashitsynArtem]

Can I open the driver in headless mode using user-data-dir, but if the browser is already open in headful mode?

It works for me:

headless driver + user-data-dir, but if the browser is closed

or

user-data-dir + has an open browser open, but a headful driver

or

headless driver + open browser, but without using user-data-dir

[mdmintz]

Chrome has two types of headless modes (https://stackoverflow.com/a/73840130/7058266) which are triggered by:

options.add_argument("--headless") # The original headless mode
options.add_argument("--headless=new") # The newer headless mode

The original headless mode has limited functionality, but is a little faster.
The newer headless mode has the same functionality as regular Chrome.

In SeleniumBase, the original headless mode is triggered by --headless / headless=True.
In SeleniumBase, the newer headless mode is triggered by --headless2 / headless2=True.

Sometimes, the newer headless mode will be used automatically (even if specifying the original one) because some things such as UC Mode require the newer headless mode to work. However, for optimal UC Mode performance, Xvfb should be used in order to allow a headed browser in a headless Linux environment:

--xvfb / xvfb=True (This is generally only available on Linux)

[KashitsynArtem]

I am very grateful to you for a quick response, thank you.

Yes, of course I tried various options with your examples in the SeleniumBase repository, the original Selenium and uc, but I could not run the driver in combination: headless driver + user-data-dir + headful browser.

Also, I tried to run the driver with various chromium-command-line-switches that I could find in issues, such as this one:
ultrafunkamsterdam/undetected-chromedriver#1477
Which you also took part in the discussion.

For example, the minimum code:

from seleniumbase import Driver
import time

user_data_dir = r'C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data'

driver = Driver(uc=True,
headless2=True,
user_data_dir=user_data_dir)

driver.get('https://google.com/')
time.sleep(5)
driver.get_screenshot_as_file('screen.png')
time.sleep(5)
driver.quit()

Trace:

raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:9222
from chrome not reachable
Stacktrace:
GetHandleVerifier [0x00007FF6E78E70C2+63090]
(No symbol) [0x00007FF6E7852D12]
(No symbol) [0x00007FF6E76EEB1D]
(No symbol) [0x00007FF6E76DFB3D]
(No symbol) [0x00007FF6E77246CA]
(No symbol) [0x00007FF6E771BEC5]
(No symbol) [0x00007FF6E7760BFB]
(No symbol) [0x00007FF6E7760830]
(No symbol) [0x00007FF6E7756D83]
(No symbol) [0x00007FF6E77283A8]
(No symbol) [0x00007FF6E7729441]
GetHandleVerifier [0x00007FF6E7CE262D+4238301]
GetHandleVerifier [0x00007FF6E7D1F78D+4488509]
GetHandleVerifier [0x00007FF6E7D17A6F+4456479]
GetHandleVerifier [0x00007FF6E79C0606+953270]
(No symbol) [0x00007FF6E785E5DF]
(No symbol) [0x00007FF6E78592B4]
(No symbol) [0x00007FF6E78593EB]
(No symbol) [0x00007FF6E7849C24]
BaseThreadInitThunk [0x00007FF8C56A7344+20]
RtlUserThreadStart [0x00007FF8C61A26B1+33]

The error will occur if the browser instance is already running manually in headful, but if you close the browser, the driver will start in headless and use user-data-dir.

Also tried using:
--remote-debugging-port=9222
And connect the driver to the running browser via various ports, but the driver will work in headful mode (run in a new tab), even with the headless flag.

I saw that you offered the xvfb option, but I need to run the program on Windows without virtual display.

I wasn't quite sure that it was possible to do this at a high level, because I found this:

https://issues.chromium.org/issues/4027524

Maybe you can point me in the right direction? Maybe something more low-level using: Chrome DevTools Protocol.

[mdmintz]

#2118 (comment)

If using the user_data_dir option together with uc mode, the user_data_dir MUST be one that was created via tests running in uc mode. This means that you cannot use one that comes from your default Chrome browser. A regular Chrome browser would reconfigure the user_data_dir in a way such that uc mode would no longer be compatible with it.