Saturday, January 16, 2021

#1 2020-11-29 06:35:13 pm

scrutinizer82
Member
Registered: 2015-08-04
Posts: 80

SecurityAgent GUI hierarchy is captured sporadically

Hello,

I wrote a GUI script that targets SecurityAgent, a faceless application that's starts every time you must supply your credentials for things like viewing passwords in Keychain Access and granting levels of access, e.g always or only once. Every time it kicks off the fields are filled in automatically.
I have succeeded in achieving that in pre-ElCapitan macOS but in Mojave even if the application "SecurityAgent" is frontmost then retrieving its structure in Script Editor (by referencing its elements like

get window 1

,

get every UI element of window 1

etc.) poses an unexpected difficulty: one moment it's retrievable the next one it errors out with the message

 "Invalid index. Can't get window 1 of application "SecurityAgent" of application "System Events"

. Requesting its objects by calling them with

get every UI element of process "SecurityAgent"

returns an empty list.
Is there a workaround to make it work the old way?

NB. BTW, no problem with unlocking "System Preferences" as well as signing in with Apple ID in relevant applications because those follow a more traditional GUI structure.

My script is

Applescript:

tell application "System Events"
   try
       if application process "SecurityAgent" exists then
           
           tell application process "SecurityAgent"
               set frontmost to true -- debugging. Not needed in the real-life scenario because it's already frontmost.
               tell window 1
                   set TheButtons to buttons
                   set ButtonNames to {"OK", "Unlock", "Always Allow"}
                   
                   set AllElems to entire contents
                   set TxtFlds to {}
                   
                   repeat with Elem in AllElems
                       if Elem's contents's role is "AXTextField" then set end of TxtFlds to Elem's contents
                       
                   end repeat
                   
                   if (count TxtFlds) is equal to 2 then
                       
                       set SbRls to {}
                       repeat with Elem in TxtFlds
                           set end of SbRls to subrole of Elem's contents
                       end repeat
                       
                       
                       set _Classes to {}
                       
                       
                       repeat with Elem in SbRls
                           set end of _Classes to Elem's contents's class
                       end repeat
                       
                       set InputSecureFields to {}
                       repeat with i from 1 to the number of items in SbRls
                           set ElemInfo to {SbrlClass:missing value, GenericClass:missing value, ElemValue:missing value}
                           set {ElemInfo's SbrlClass, ElemInfo's GenericClass} to {_Classes's item i's contents, TxtFlds's contents's class}
                           if _Classes's item i's contents is class then set ElemInfo's ElemValue to TxtFlds's item i's contents's value
                           set end of InputSecureFields to ElemInfo
                       end repeat
                       
                       set IsMacUserNameEmpty to ""
                       
                       repeat with aRecord in InputSecureFields
                           if aRecord's contents's SbrlClass is class then
                               
                               set IsMacUserNameEmpty to aRecord's contents's ElemValue
                               exit repeat
                           end if
                           
                       end repeat
                       
                       
                       
                       -- end tell
                       
                       if IsMacUserNameEmpty is "" then
                           tell (the first text field whose subrole is missing value)
                               
                               set focused to true
                               keystroke MYUSERNAME using shift down
                               
                               key code 48
                           end tell
                       end if
                       
                       
                       
                       delay 1
                       tell (the first UI element whose subrole is "AXSecureTextField")
                           keystroke MYPASSWORD using shift down
                       end tell
                       delay 1
                       
                       
                       repeat with aButton in TheButtons
                           if aButton's contents's name is in ButtonNames then
                               tell aButton's contents to perform action "AXPress"
                               exit repeat
                           end if
                       end repeat
                       
                   else if (count TxtFlds) is equal to 1 then -- only the password field.
                       tell (the first UI element whose subrole is "AXSecureTextField")
                           keystroke MYPASSWORD using shift down
                       end tell
                       repeat with aButton in TheButtons
                           if aButton's contents's name is "Always Allow" then
                               tell aButton's contents to perform action "AXPress"
                               exit repeat
                           end if
                       end repeat
                       
                       
                       
                   else -- no input fields at all, only buttons.
                       repeat with aButton in TheButtons
                           if aButton's contents's name is "Always Allow" then
                               tell aButton's contents to perform action "AXPress"
                               exit repeat
                           end if
                       end repeat
                       
                       
                   end if
                   
               end tell
           end tell
       end if
   end try

Model: MacBook Pro 9,1 (mid-2012 15") Core i7 2.3 GHz, 16 GB RAM, 1TB SSD
AppleScript: 2.7
Browser: Safari 605.1.15
Operating System: macOS 10.14

Last edited by scrutinizer82 (2020-11-29 08:04:18 pm)


Scripting in Mac OS X 10.7.5, OS X 10.9.5, macOS 10.14.6

Offline

 

#2 2020-11-29 07:12:52 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 6549

Re: SecurityAgent GUI hierarchy is captured sporadically

scrutinizer82 wrote:

Is there workaround to make it work in the old way?



My guess would be no. it looks like a deliberate effort to remove a perceived security weakness.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/
latenightsw.com

Offline

 

#3 2020-11-30 02:47:34 pm

scrutinizer82
Member
Registered: 2015-08-04
Posts: 80

Re: SecurityAgent GUI hierarchy is captured sporadically

I discovered something about this process. The most rational and realistic explanation could be that it's a GUI bug and memory leak as the system doesn't properly register the state of UI elements. This point gets additional credibility because even user input elements invoked by user Interaction commands (such as display dialog) in both High Sierra and Mojave show wobbliness and a lagging response in the situations when the dialogue needs to go after accepting a click. Also, it seems it has the problem with bringing windows and other UI elements to the front both manually and programmatically. Very often, even if I activate an application then it would appear as if not being frontmost (grayed out), even if accepting input. Either way, I have to click it one more time before bringing to the front actually.

Based on my finding, I was able to replicate this pattern: I locked the Login Keychain, then clicked on "Unlock", the secure input window popped up. I clicked on it manually to make sure it's at the front, then ran my script and got success. Annoyingly, even using the "frontmost" property doesn't necessarily make it frontmost. A window gets detached from its parent process.
I now recall complaints about the WindowServer process. You have to tie the dots to see the blueprint.
I can simulate clicks on registered and known elements. But how do you simulate that one?

Model: MacBook Pro 9,1 (mid-2012 15") Core i7 2.3 GHz, 16 GB RAM, 1TB SSD
AppleScript: 2.7
Browser: Safari 605.1.15
Operating System: macOS 10.14

Last edited by scrutinizer82 (2020-12-01 01:46:06 am)


Scripting in Mac OS X 10.7.5, OS X 10.9.5, macOS 10.14.6


Filed under: GUI, bugs, memory leaks

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)