Remote Access (#17580)

#4390 - The initial request for "NVDA Remote functionality

Summary of the issue:
This PR integrates NVDA Remote functionality into core with significant
architectural improvements and modernization. While maintaining protocol compatibility with the existing add-on, it introduces cleaner architecture, type safety, proper event handling, and improved maintainability.

Description of user facing changes
Adds a new "Remote" submenu under NVDA's Tools menu with options to:
Connect to another computer
Disconnect from remote session
Mute remote speech
Push clipboard content
Copy connection link
Send Ctrl+Alt+Del
New remote settings panel in NVDA Settings dialog with options for:
Auto-connect on startup
Host/client configuration
Connection type (control/be controlled)
Server address and port settings
Connection key management
Audio feedback preferences
New keyboard shortcuts for remote control (note these are defaults and can all be changed):
F11 to toggle between local and remote control
NVDA+Alt+C to push clipboard
NVDA+Alt+PageDown to disconnect
NVDA+Alt+M to toggle remote mute
Audio cues and visual feedback for connection events
Comprehensive user documentation added to NVDA User Guide
Description of development approach
The implementation follows a modular architecture:

Core Components:

Remote client with session management
Secure transport layer using SSL/TLS
Protocol serialization for message passing
Relay server infrastructure for connection brokering
Integration Points:

Hooks into NVDA's speech, braille and input subsystems
UI integration via wx menu items and dialog boxes
Extension point handling for event propagation
Secure desktop support for UAC dialogs
Added remaining extension points
Security Considerations:

SSL certificate validation and fingerprint verification
Channel-based authentication
The code has been ported from the nvdaremote/nvdaremote repository and significantly improved from the version most-recently deployed.

Testing strategy:

nvdaHelper/interfaces/nvdaControllerInternal/nvdaControllerInternal.acf

@@ -1,7 +1,7 @@
 /*
 This file is a part of the NVDA project.
 URL: http://www.nvda-project.org/
-Copyright 2006-2010 NVDA contributers.
+Copyright 2006-2025 NVDA contributers.
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License version 2.0, as published by
     the Free Software Foundation.
@@ -23,6 +23,7 @@ interface NvdaControllerInternal {
 	[fault_status,comm_status] logMessage();
 	[fault_status,comm_status] vbufChangeNotify();
 	[fault_status,comm_status] installAddonPackageFromPath();
+	[fault_status,comm_status] handleRemoteURL();
 	[fault_status,comm_status] drawFocusRectNotify();
 	[fault_status,comm_status] reportLiveRegion();
 	[fault_status,comm_status] openConfigDirectory();

nvdaHelper/interfaces/nvdaControllerInternal/nvdaControllerInternal.idl

@@ -1,7 +1,7 @@
 /*
 This file is a part of the NVDA project.
 URL: http://www.nvda-project.org/
-Copyright 2006-2018 NV Access Limited, rui Batista, Google LLC.
+Copyright 2006-2025 NV Access Limited, rui Batista, Google LLC, Christopher Toth.
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License version 2.0, as published by
     the Free Software Foundation.
@@ -102,4 +102,10 @@ interface NvdaControllerInternal {
  * Asks NVDA to open currently used configuration directory.
  */
 	error_status_t __stdcall openConfigDirectory();
+
+/**
+ * Handles a remote URL request from the slave process.
+ * @param url The nvdaremote:// URL to process.
+ */
+	error_status_t __stdcall handleRemoteURL([in,string] const wchar_t* url);
 };

nvdaHelper/local/nvdaControllerInternal.c

@@ -1,7 +1,7 @@
 /*
 This file is a part of the NVDA project.
 URL: http://www.nvda-project.org/
-Copyright 2006-2018 NV Access Limited, rui Batista, Google LLC.
+Copyright 2006-2025 NV Access Limited, rui Batista, Google LLC, Christopher Toth.
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License version 2.0, as published by
     the Free Software Foundation.
@@ -70,6 +70,11 @@ error_status_t __stdcall nvdaControllerInternal_installAddonPackageFromPath(cons
 	return _nvdaControllerInternal_installAddonPackageFromPath(addonPath);
 }
 
+error_status_t(__stdcall *_nvdaControllerInternal_handleRemoteURL)(const wchar_t*);
+error_status_t __stdcall nvdaControllerInternal_handleRemoteURL(const wchar_t* url) {
+	return _nvdaControllerInternal_handleRemoteURL(url);
+}
+
 error_status_t(__stdcall *_nvdaControllerInternal_drawFocusRectNotify)(const long, const long, const long, const long, const long);
 error_status_t __stdcall nvdaControllerInternal_drawFocusRectNotify(const long hwnd, const long left, const long top, const long right, const long bottom) {
 	return _nvdaControllerInternal_drawFocusRectNotify(hwnd,left,top,right,bottom);

nvdaHelper/local/nvdaHelperLocal.def

@@ -46,6 +46,7 @@ EXPORTS
 	_nvdaControllerInternal_logMessage
 	_nvdaControllerInternal_typedCharacterNotify
 	_nvdaControllerInternal_installAddonPackageFromPath
+	_nvdaControllerInternal_handleRemoteURL
 	_nvdaControllerInternal_drawFocusRectNotify
 	_nvdaController_brailleMessage
 	_nvdaController_cancelSpeech

nvdaHelper/remote/nvdaHelperRemote.def

@@ -15,6 +15,7 @@ EXPORTS
 	nvdaControllerInternal_logMessage
 	nvdaControllerInternal_vbufChangeNotify
 	nvdaControllerInternal_installAddonPackageFromPath
+	nvdaControllerInternal_handleRemoteURL
 	nvdaController_testIfRunning
 	nvdaController_speakText
 	nvdaController_cancelSpeech

projectDocs/dev/developerGuide/developerGuide.md

@@ -1384,6 +1384,7 @@ For examples of how to define and use new extension points, please see the code
 |`Action` |`pre_speechCanceled` |Triggered before speech is canceled.|
 |`Action` |`pre_speech` |Triggered before NVDA handles prepared speech.|
 |`Action` |`post_speechPaused` |Triggered when speech is paused or resumed.|
+|`Action` |`pre_speechQueued` |Triggered after speech is processed and normalized and directly before it is enqueued.|
 |`Filter` |`filter_speechSequence` |Allows components or add-ons to filter speech sequence before it passes to the synth driver.|
 
 ### synthDriverHandler {#synthDriverHandlerExtPts}

requirements.txt

@@ -3,6 +3,7 @@ SCons==4.8.1
 
 # NVDA's runtime dependencies
 comtypes==1.4.6
+cryptography==44.0.0
 pyserial==3.5
 wxPython==4.2.2
 configobj @ git+https://github.com/DiffSK/configobj@8be54629ee7c26acb5c865b74c76284e80f3aa31#egg=configobj

source/NVDAHelper.py

@@ -1,5 +1,5 @@
 # A part of NonVisual Desktop Access (NVDA)
-# Copyright (C) 2008-2023 NV Access Limited, Peter Vagner, Davy Kager, Mozilla Corporation, Google LLC,
+# Copyright (C) 2008-2025 NV Access Limited, Peter Vagner, Davy Kager, Mozilla Corporation, Google LLC,
 # Leonard de Ruijter
 # This file is covered by the GNU General Public License.
 # See the file COPYING for more details.
@@ -687,6 +687,31 @@ def nvdaControllerInternal_openConfigDirectory():
 	return 0
 
 
+@WINFUNCTYPE(c_long, c_wchar_p)
+def nvdaControllerInternal_handleRemoteURL(url):
+	"""Handles a remote URL request from the slave process.
+
+	:param url: The nvdaremote:// URL to process
+	:return: 0 on success, -1 on failure
+	"""
+	from remoteClient import connectionInfo, _remoteClient as client
+
+	try:
+		if not client:
+			log.error("No RemoteClient instance available")
+			return -1
+		# Queue the URL handling on the main thread
+		queueHandler.queueFunction(
+			queueHandler.eventQueue,
+			client.verifyAndConnect,
+			connectionInfo.ConnectionInfo.fromURL(url),
+		)
+		return 0
+	except Exception:
+		log.error("Error handling remote URL", exc_info=True)
+		return -1
+
+
 class _RemoteLoader:
 	def __init__(self, loaderDir: str):
 		# Create a pipe so we can write to stdin of the loader process.
@@ -776,6 +801,7 @@ def initialize() -> None:
 		),
 		("nvdaControllerInternal_drawFocusRectNotify", nvdaControllerInternal_drawFocusRectNotify),
 		("nvdaControllerInternal_openConfigDirectory", nvdaControllerInternal_openConfigDirectory),
+		("nvdaControllerInternal_handleRemoteURL", nvdaControllerInternal_handleRemoteURL),
 	]:
 		try:
 			_setDllFuncPointer(localLib, "_%s" % name, func)

source/config/__init__.py

@@ -1,5 +1,5 @@
 # A part of NonVisual Desktop Access (NVDA)
-# Copyright (C) 2006-2024 NV Access Limited, Aleksey Sadovoy, Peter Vágner, Rui Batista, Zahari Yurukov,
+# Copyright (C) 2006-2025 NV Access Limited, Aleksey Sadovoy, Peter Vágner, Rui Batista, Zahari Yurukov,
 # Joseph Lee, Babbage B.V., Łukasz Golonka, Julien Cochuyt, Cyrille Bougot
 # This file is covered by the GNU General Public License.
 # See the file COPYING for more details.
@@ -526,6 +526,7 @@ class ConfigManager(object):
 		"update",
 		"development",
 		"addonStore",
+		"remote",
 	}
 	"""
 	Sections that only apply to the base configuration;

source/config/configSpec.py

@@ -13,7 +13,7 @@
 #: provide an upgrade step (@see profileUpgradeSteps.py). An upgrade step does not need to be added when
 #: just adding a new element to (or removing from) the schema, only when old versions of the config
 #: (conforming to old schema versions) will not work correctly with the new schema.
-latestSchemaVersion = 15
+latestSchemaVersion = 16
 
 #: The configuration specification string
 #: @type: String
@@ -344,6 +344,24 @@
 	# UpdateChannel values:
 	# same channel (default), any channel, do not update, stable, beta & dev, beta, dev
 	defaultUpdateChannel = integer(0, 6, default=0)
+
+# Remote Settings
+[remote]
+	[[connections]]
+		last_connected = list(default=list())
+	[[controlserver]]
+		autoconnect = boolean(default=False)
+		self_hosted = boolean(default=False)
+		connection_type = integer(default=0, min=0, max=1) # 0: follower, 1: leader
+		host = string(default="")
+		port = integer(default=6837)
+		key = string(default="")
+	[[seen_motds]]
+		__many__ = string(default="")
+	[[trusted_certs]]
+		__many__ = string(default="")
+	[[ui]]
+		play_sounds = boolean(default=True)
 """
 
 #: The configuration specification

source/config/profileUpgradeSteps.py

@@ -13,19 +13,22 @@
 that no information is lost, while updating the ConfigObj to meet the requirements of the new schema.
 """
 
+import os
+
+import configobj.validate
+from configobj import ConfigObj
 from logHandler import log
+
 from config.configFlags import (
 	NVDAKey,
-	ShowMessages,
-	TetherTo,
+	OutputMode,
+	ReportCellBorders,
 	ReportLineIndentation,
 	ReportTableHeaders,
-	ReportCellBorders,
-	OutputMode,
+	ShowMessages,
+	TetherTo,
 	TypingEcho,
 )
-import configobj.validate
-from configobj import ConfigObj
 
 
 def upgradeConfigFrom_0_to_1(profile: ConfigObj) -> None:
@@ -500,3 +503,38 @@ def _convertTypingEcho(profile: ConfigObj, key: str) -> None:
 		newValue = TypingEcho.EDIT_CONTROLS.value if oldValue else TypingEcho.OFF.value
 		profile["keyboard"][key] = newValue
 		log.debug(f"Converted '{key}' from {oldValue!r} to {newValue} ({TypingEcho(newValue).name}).")
+
+
+def upgradeConfigFrom_15_to_16(profile: ConfigObj) -> None:
+	"""Migrate remote.ini settings into the main config."""
+	remoteIniPath = os.path.join(os.path.dirname(profile.filename), "remote.ini")
+	if not os.path.isfile(remoteIniPath):
+		log.debug(f"No remote.ini found, no action taken. Checked {remoteIniPath}")
+		return
+
+	try:
+		remoteConfig = ConfigObj(remoteIniPath, encoding="UTF-8")
+		log.debug(f"Loading remote config from {remoteIniPath}")
+	except Exception:
+		log.error("Error loading remote.ini", exc_info=True)
+		return
+
+	# Create remote section if it doesn't exist
+	if "remote" not in profile:
+		profile["remote"] = {}
+
+	# Copy all sections from remote.ini
+	for section in remoteConfig:
+		if section not in profile["remote"]:
+			profile["remote"][section] = {}
+		profile["remote"][section].update(remoteConfig[section])
+
+	try:
+		# Backup the old file just in case
+		backupPath = remoteIniPath + ".old"
+		if os.path.exists(backupPath):
+			os.unlink(backupPath)
+		os.rename(remoteIniPath, backupPath)
+		log.debug(f"Backed up remote.ini to {backupPath}")
+	except Exception:
+		log.error("Error backing up remote.ini after migration", exc_info=True)

source/core.py

@@ -1,5 +1,5 @@
 # A part of NonVisual Desktop Access (NVDA)
-# Copyright (C) 2006-2024 NV Access Limited, Aleksey Sadovoy, Christopher Toth, Joseph Lee, Peter Vágner,
+# Copyright (C) 2006-2025 NV Access Limited, Aleksey Sadovoy, Christopher Toth, Joseph Lee, Peter Vágner,
 # Derek Riemer, Babbage B.V., Zahari Yurukov, Łukasz Golonka, Cyrille Bougot, Julien Cochuyt
 # This file is covered by the GNU General Public License.
 # See the file COPYING for more details.
@@ -897,6 +897,12 @@ def main():
 
 	log.debug("Initializing global plugin handler")
 	globalPluginHandler.initialize()
+
+	log.debug("Initializing remote client")
+	import remoteClient
+
+	remoteClient.initialize()
+
 	if globalVars.appArgs.install or globalVars.appArgs.installSilent:
 		import gui.installerGui
 
@@ -1049,6 +1055,7 @@ def _doPostNvdaStartupAction():
 			" This likely indicates NVDA is exiting due to WM_QUIT.",
 		)
 		queueHandler.pumpAll()
+	_terminate(remoteClient)
 	_terminate(gui)
 	config.saveOnExit()
 

source/globalCommands.py

@@ -71,7 +71,7 @@
 import audio
 from audio import appsVolume
 from utils.displayString import DisplayStringEnum
-
+import remoteClient
 
 #: Script category for text review commands.
 # Translators: The name of a category of NVDA commands.
@@ -121,6 +121,9 @@
 #: Script category for audio streaming commands.
 # Translators: The name of a category of NVDA commands.
 SCRCAT_AUDIO = _("Audio")
+#: Script category for Remote commands.
+# Translators: The name of a category of NVDA commands.
+SCRCAT_REMOTE = _("Remote")
 
 # Translators: Reported when there are no settings to configure in synth settings ring
 # (example: when there is no setting for language).
@@ -4914,6 +4917,71 @@ def script_toggleApplicationsVolumeAdjuster(self, gesture: "inputCore.InputGestu
 	def script_toggleApplicationsMute(self, gesture: "inputCore.InputGesture") -> None:
 		appsVolume._toggleAppsVolumeMute()
 
+	@script(
+		# Translators: Describes a command.
+		description=_("""Mute or unmute the speech coming from the remote computer"""),
+		category=SCRCAT_REMOTE,
+	)
+	def script_toggleRemoteMute(self, gesture: "inputCore.InputGesture"):
+		remoteClient._remoteClient.toggleMute()
+
+	@script(
+		gesture="kb:control+shift+NVDA+c",
+		category=SCRCAT_REMOTE,
+		# Translators: Documentation string for the script that sends the contents of the clipboard to the remote machine.
+		description=_("Sends the contents of the clipboard to the remote machine"),
+	)
+	def script_pushClipboard(self, gesture: "inputCore.InputGesture"):
+		remoteClient._remoteClient.pushClipboard()
+
+	@script(
+		# Translators: Documentation string for the script that copies a link to the remote session to the clipboard.
+		description=_("""Copies a link to the remote session to the clipboard"""),
+		category=SCRCAT_REMOTE,
+	)
+	def script_copyRemoteLink(self, gesture: "inputCore.InputGesture"):
+		remoteClient._remoteClient.copyLink()
+		# Translators: A message indicating that a link has been copied to the clipboard.
+		ui.message(_("Copied link"))
+
+	@script(
+		gesture="kb:alt+NVDA+pageDown",
+		category=SCRCAT_REMOTE,
+		# Translators: Documentation string for the script that disconnects a remote session.
+		description=_("Disconnect a remote session"),
+	)
+	@gui.blockAction.when(gui.blockAction.Context.SECURE_MODE)
+	def script_disconnectFromRemote(self, gesture: "inputCore.InputGesture"):
+		if not remoteClient._remoteClient.isConnected:
+			# Translators: A message indicating that the remote client is not connected.
+			ui.message(_("Not connected"))
+			return
+		remoteClient._remoteClient.disconnect()
+
+	@script(
+		gesture="kb:alt+NVDA+pageUp",
+		# Translators: Documentation string for the script that invokes the remote session.
+		description=_("""Connect to a remote computer"""),
+		category=SCRCAT_REMOTE,
+	)
+	@gui.blockAction.when(gui.blockAction.Context.MODAL_DIALOG_OPEN)
+	@gui.blockAction.when(gui.blockAction.Context.SECURE_MODE)
+	def script_connectToRemote(self, gesture: "inputCore.InputGesture"):
+		if remoteClient._remoteClient.isConnected() or remoteClient._remoteClient.connecting:
+			# Translators: A message indicating that the remote client is already connected.
+			ui.message(_("Already connected"))
+			return
+		remoteClient._remoteClient.doConnect()
+
+	@script(
+		# Translators: Documentation string for the script that toggles the control between guest and host machine.
+		description=_("Toggles the control between guest and host machine"),
+		category=SCRCAT_REMOTE,
+		gesture="kb:NVDA+f11",
+	)
+	def script_sendKeys(self, gesture: "inputCore.InputGesture"):
+		remoteClient._remoteClient.toggleRemoteKeyControl(gesture)
+
 
 #: The single global commands instance.
 #: @type: L{GlobalCommands}