Skip to content

03. UIA interface

Jump to bottom
Descolada edited this page Sep 9, 2022 · 2 revisions

UIA interface is the main object of the UIA_Interface.ahk library. It is the starting point that we use to access elements, interact with some general UIA properties, create conditions, TreeWalkers, add event handlers and more.

Getting the UIA_Interface object

In every new script that uses UIA, the very first thing we need to do (after including the UIA_Interface.ahk library with #include <UIA_Interface> of course) is initiate the UIA interface and get the UIA_Interface object. We can do that by calling the UIA_Interface() function which initiates the connection to UIA and returns a UIA_Interface class object. Usually we assign it to a variable for easier access: UIA := UIA_Interface().

UIA_Interface(maxVersion="") can take as an argument the maximum allowed version of UIA interface. By default the UIA_Interface() function will get us the highest available version (usually version 7), but sometimes this might be undesirable. For example if you need consistency between different computers and different Windows versions, you can limit that version. For example UIA_Interface(1) would return the plain UIA_Interface object, that cannot use UIA_Interface2 and higher properties/methods. To check the current version, either run UIAViewer and look at the status bar, or access __Version with UIA_Interface().__Version UIA_Interface() with version 7 will provide access to UIA_Interface up to UIA_Interface7.

Since Elements and other objects also might have multiple versions available, then these can also be limited by changing UIA_Enum.UIA_CurrentVersion_... values. UIA_Enum.UIA_CurrentVersion_Element := 1 will limit Elements objects versions to regular UIA_Element (UIA_Element2 and later properties/methods cannot be used).

UIA_Interface() creates the UIA connection just once, and with later calls returns the previously created UIA_Interface object. To change that behavious, specify maxVersion, then a new connection is made and a new object returned (though so far I haven't encountered a situation that would require doing this...).

UIA_Interface object

UIAutomation interface exposes methods that enable us to discover, access, and filter UI Automation elements. The members of this interface are not specific to a particular element.

Properties

  • ControlViewWalker : see TreeWalkers section
  • ContentViewWalker : see TreeWalkers section
  • RawViewWalker : see TreeWalkers section
  • RawViewCondition : see Conditions section
  • ControlViewCondition : see Conditions section
  • ContentViewCondition : see Conditions section
  • ProxyFactoryMapping
  • ReservedNotSupportedValue
  • ReservedMixedAttributeValue

Methods to access elements

  • GetRootElement() retrieves the UI Automation element that represents the desktop.
  • ElementFromHandle(hwnd, ByRef activateChromiumAccessibility=False) retrieves a UI Automation element for the specified window. Additionally activateChromiumAccessibility flag can be set to True to send the WM_GETOBJECT message to Chromium-based apps to activate accessibility if it isn't activated, in which case the Chromium element will be returned in the activateChromiumAccessibility variable.
  • ElementFromChromium(winTitle="A", activateChromiumAccessibility=True) retrieves the Chromium element from a Chromium app (web-browser based applications such as Skype, Discord etc).
  • ElementFromPoint(x="", y="", ByRef activateChromiumAccessibility=False) retrieves the UI Automation element at the specified point on the desktop, which isn't necessarily the deepest element under the point. Additionally activateChromiumAccessibility flag can be set to True to send the WM_GETOBJECT message to Chromium-based apps to activate accessibility if it isn't activated, in which case the Chromium element will be returned in the activateChromiumAccessibility variable.
  • SmallestElementFromPoint(x="", y="", activateChromiumAccessibility=False, windowEl="") gets ElementFromPoint and filters out the smallest subelement that is under the specified point. If windowEl (window under the point) is provided, then a deep search is performed for the smallest element (this might be very slow in large trees).
  • GetFocusedElement() retrieves the UI Automation element that has the input focus.
  • ElementFromIAccessible(IAcc, childId=0) is currently not working, but should get an UIA_Element from an MSAA/Acc object
  • GetChromiumContentElement(winTitle="A", fromFocused=True) can be used when a Chromium apps content isn't accessible by normal methods (ElementFromHandle, ElementFromChromium, GetRootElement etc). fromFocused=True uses the focused element as a reference point, fromFocused=False uses ElementFromPoint

For a more complete description for these, see the UIA tree section of this Wiki.

Creation methods

  • CreateTreeWalker(condition) retrieves a UIA_TreeWalker object that can be used to traverse the Microsoft UI Automation tree.
  • CreateCacheRequest()
  • CreateTrueCondition()
  • CreateFalseCondition()
  • CreatePropertyCondition(propertyId, value, type="Variant") creates a condition that selects elements that have a property with the specified value.
  • CreatePropertyConditionEx(propertyId, value, type="Variant", flags=0x1)
  • CreateAndCondition(c1,c2)
  • CreateOrCondition(c1,c2)
  • CreateNotCondition(c)
  • CreateCondition(propertyOrExpr, valueOrFlags="", flags=0) easiest way of creating conditions

For a more complete description for these, see the TreeWalker and Conditions sections in this Wiki.

Event handler methods

  • AddAutomationEventHandler(eventId, element, scope=0x4, cacheRequest=0, handler="") registers a method that handles Microsoft UI Automation events. eventId must be an EventId enum. scope must be a TreeScope enum. cacheRequest can be specified is caching is used. handler is an event handler object, which can be created with UIA_CreateEventHandler function.
  • RemoveAutomationEventHandler(eventId, element, handler)
  • AddPropertyChangedEventHandler(element,scope=0x1,cacheRequest=0,handler="",propertyArray="") registers a method that handles an array of property-changed events
  • RemovePropertyChangedEventHandler(element, handler)
  • AddStructureChangedEventHandler(element, handler)
  • RemoveStructureChangedEventHandler(element, handler)
  • AddFocusChangedEventHandler(handler, cacheRequest=0)
  • RemoveFocusChangedEventHandler(handler)
  • RemoveAllEventHandlers()

Other methods

  • CompareElements(e1,e2) checks if two elements are equal. Doing this the normal way with e1 == e2 will not work
  • CompareRuntimeIds(r1,r2)
  • GetPropertyProgrammaticName(Id)
  • GetPatternProgrammaticName(Id)
  • PollForPotentialSupportedPatterns(e, Byref Ids="", Byref Names="") returns an object where keys are the names and values are the Ids
  • PollForPotentialSupportedProperties(e, Byref Ids="", Byref Names="")

UIA_Interface2

Properties

  • AutoSetFocus : specifies whether calls to UI Automation control pattern methods automatically set focus to the target element. Default is True. This property can be get and set.
  • ConnectionTimeout : specifies the length of time that UI Automation will wait for a provider to respond to a client request for an automation element. Default is 20000ms (20 seconds), minimum seems to be 50ms. This property can be get and set.
  • TransactionTimeout : specifies the length of time that UI Automation will wait for a provider to respond to a client request for information about an automation element. Default is 2000ms (2 seconds), minimum seems to be 50ms. This property can be get and set.

Methods

UIA_Interface2 doesn't have additional methods.

UIA_Interface3

Properties

UIA_Interface3 doesn't have additional properties.

Methods

  • AddTextEditTextChangedEventHandler(element, scope, textEditChangeType, cacheRequest=0, handler="")
  • RemoveTextEditTextChangedEventHandler(element, handler)

UIA_Interface4

Properties

UIA_Interface4 doesn't have additional properties.

Methods

  • AddChangesEventHandler(element, scope, changeTypes, changesCount, cacheRequest=0, handler="")
  • RemoveChangesEventHandler(element, handler)

UIA_Interface5

Properties

UIA_Interface5 doesn't have additional properties.

Methods

  • AddNotificationEventHandler(element, scope, cacheRequest, handler)
  • RemoveNotificationEventHandler(element, handler)

UIA_Interface6

Properties

  • ConnectionRecoveryBehavior : indicates whether an accessible technology client adjusts provider request timeouts when the provider is non-responsive.
  • CoalesceEvents : gets or sets whether an accessible technology client receives all events, or a subset where duplicate events are detected and filtered.

Methods

  • CreateEventHandlerGroup()
  • AddEventHandlerGroup(element, handlerGroup)
  • RemoveEventHandlerGroup(element, handlerGroup)
  • AddActiveTextPositionChangedEventHandler(element,scope=0x4,cacheRequest=0,handler="")
  • RemoveActiveTextPositionChangedEventHandler(element,handler)
Clone this wiki locally