UI Scripting Issue / Repeat Loop Task

Scripting Forums AppleScript | Mac OS X
#1

I’m hoping someone can help. I’m trying to perform a scripted UI task on an app that has no dictionary and that has very little in the way of UI properties. The app has an issue with the click command so I’m having to use cli’s cliclick. I can navigate to the button in the main window with


the last button of UI element 1 of UI element 1 of group 4 of list 1 of group 2 of window tracks_window whose accessibility description = "send button"

— I’m storing the button into a variable, setting it’s position to a variable then running cliclick using the x, y variable, see below


tell application "System Events" to tell process "the app"
		set tracks_window to title of first window whose title contains "- Tracks"
		set frontmost to true
		set theImage to the last button of UI element 1 of UI element 1 of group 4 of list 1 of group 2 of window tracks_window whose accessibility description = "send button"
		set {x, y} to position of theImage
		do shell script "/usr/local/Cellar/cliclick/5.1/bin/cliclick rc:" & x + 10 & "," & y + 7
		key code 11
		key code 124

…
— Key code 11
— Key code 124

— … Navigates to what’s labelled as


menu item 1 of menu 1 of menu item "Bus" of menu 1 of UI element 1 of group 4 of list 1 of group 2 of window tracks_window

(My UI properties script couldn’t identify it I got that result using Automator)

— When clicking on menu item 1 in Automator, Automator returned the text content of the menu item. I don’t know if it’s a value, description, name, accessibility description or whatever but “System Events” doesn’t seem to think it exists.

— Any of these menu items that are already assigned contain “→” in the menu item text.

— I want to focus the menu and do some thing like:


set theBus to the last menu item of menu 1 of menu item "Bus" of menu 1 of UI element 1 of group 4 of list 1 of group 2 of window tracks_window  whose value does not contain "→"

— As I’m having an issue with the second UI path is there anyway to search the active/frontmost menu and look for the last item that does not contain “→”? Something like:


set theMenu to frontmost is focused of menu
	set theItem to the last menu item of theMenu whose accessibility description does not contain "→"
	set {x, y} to position of theItem
	do shell script "/usr/local/Cellar/cliclick/5.1/bin/cliclick rc:" & x + 10 & "," & y + 7

Or a repeat loop that says something like if highlighted text of selection contains “ →” then key code 125 else key code 36 end if

Thanks in advance

#2

Your code blocks are easier to read if you enclose them in backticks like so
```
code goes here
```
That also makes it easier to copy them

#3

If there’s no way to exclude menu items I may have found a work around. I have a second monitor which is a mixer window with all of the active busses. I searched for what busses are in use using


use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

tell application "System Events" to tell process "the app"
	set tracks_window to title of first window whose title contains "Mixer"
	set frontmost to true
	
end tell
use application "System Events"
set theScope to UI element 2 of group 1 of window tracks_window 
try
	tell application "System Events"
		set uiEl to every UI element of every UI element of theScope whose accessibility description contains "Bus" -- & space & busNo
		return uiEl
		--set freeBus to {}
		--set busNo to {"Bus 1", "Bus 2", "Bus 3", "Bus 5", "Bus 6", "Bus 7", "Bus 8", "Bus 9", "Bus 19", "Bus 20", "Bus 21", "Bus 22", "Bus 23", "Bus 24", "Bus 25", "Bus 26", "Bus 27", "Bus 28", "Bus 29", "Bus 30", "Bus 31", "Bus 32"} as list
	end tell
	
end try
end

uiEl returns a list of lists, I guess because I wrote to every ui element of every ui element.
So every query will return at least about 20 items but not every item will contain an item
How would I phrase a repeat loop to to find the first item of list busNo that does not exist in the contents of uiEl’s returned list of lists?
So I can use the UI element path in the first post to then navigate through the menu using key codes and keystroke the first item of busNo that doesn’t exist. Hope that makes sense.

#4

I’ve never seen the ’ use application “System Events” ’ command before.
What does it do?

#5

I’ve nearly worked it out. It’s just the last if then statement that doesn’t work and I’m not sure why. The script generates the axdsList and starts to iterate through list busNo but it s not returning the first item that’s not in axdsList, looking at the variable list in SD it doesn’t look like it’s getting passed the item 1 of list busNo. Any idea why? I also need to ensure that “Bus 1” can’t be confused with “Bus 11” or “Bus 2” confused with “Bus 27” etc

tell application "System Events" to tell process "the app"
	set tracks_window to title of first window whose title contains "Mixer"
	set frontmost to true
	
end tell
use application "System Events"
set theScope to UI element 2 of group 1 of window tracks_window of application process "the app"
try
	tell application "System Events"
		set uiEl to every UI element of every UI element of theScope whose accessibility description contains "Bus " -- & space & busNo
		set exBus to {}
		
		repeat with i from 1 to count of uiEl
			set uiEls to item i of uiEl
			repeat with i from 1 to count of uiEls
				set uiItems to item i of uiEls
				
				set end of exBus to uiItems
				
				
			end repeat
			
		end repeat
		set axdsList to {}
		repeat with i from 1 to count of exBus
			set aBus to item i of exBus
			set axds to accessibility description of aBus
			set end of axdsList to axds
			
		end repeat
		
		set busNo to {"Bus 1", "Bus 2", "Bus 3", "Bus 5", "Bus 6", "Bus 7", "Bus 8", "Bus 9", "Bus 19", "Bus 20", "Bus 21", "Bus 22", "Bus 23", "Bus 24", "Bus 25", "Bus 26", "Bus 27", "Bus 28", "Bus 29", "Bus 30", "Bus 31", "Bus 32"} as list
		set newBus to {}
		repeat with i from 1 to count of busNo
			set theBus to item i of busNo
			if item theBus of busNo is not in axdsList then
				set newBus to item theBus of busNo
				exit repeat
			end if
		end repeat
		return newBus
	end tell
end try
#6

Hi Robert.

It’s possible to use applications in the same way as scripting additions, AppleScript versions, and system frameworks. The applications’ terminology gets loaded into the script and you don’t then need to use tell statements to get it to compile. My own view is that this is a sure way to cause terminology conflicts and to confuse others trying to read and/or debug your code. And of course it should be used consistently, with the statement at the top of the script, not in the middle amongst instances of the relevant tell statement being used anyway.

1 Like
#7

item is something that is indexed. You are trying to use item theBus instead of item i. There is no item theBus. Change the two instances to item i or just remove item and of busno as you’ve already stated that theBus is item i. Basically, you’re mashing two approaches together and you need to pick one.

As to the Bus27 issue, that would be the case if you were working with a string but you’re working with items in a list. You should test it out so it makes more sense:

set b1 to "Bus1"
set b2 to "Bus2
set blist to {"Bus1", "Bus21"}

b1 is in blist
--> true
b2 is in list
--> false
#8

I just copy and pasted a result from my ui probe results script which had use system events in it. I know I don’t need all those tell statements either. Was just focusing on getting the script to work. I was testing segments in new documents and copied over the whole tell block into the main script. I’ve removed them now

#9

Thanks Mockman. This script is working fine.

tell application "System Events" to tell process "the app"
	set tracks_window to title of first window whose title contains "Mixer"
	set frontmost to true
	set theScope to UI element 2 of group 1 of window tracks_window
	try
		set uiEl to every UI element of every UI element of theScope whose accessibility description contains "Bus " -- & space & busNo
		set exBus to {}
		repeat with i from 1 to count of uiEl
			set uiEls to item i of uiEl
			repeat with i from 1 to count of uiEls
				set uiItems to item i of uiEls
				set end of exBus to uiItems
			end repeat
		end repeat
		set axdsList to {}
		repeat with i from 1 to count of exBus
			set aBus to item i of exBus
			set axds to accessibility description of aBus
			set end of axdsList to axds
		end repeat
		set busNo to {"Bus 1", "Bus 2", "Bus 3", "Bus 5", "Bus 6", "Bus 7", "Bus 8", "Bus 9", "Bus 19", "Bus 20", "Bus 21", "Bus 22", "Bus 23", "Bus 24", "Bus 25", "Bus 26", "Bus 27", "Bus 28", "Bus 29", "Bus 30", "Bus 31", "Bus 32"} as list
		set newBus to {}
		repeat with i from 1 to count of busNo
			set theBus to item i of busNo
			if theBus is not in axdsList then
				set newBus to theBus
				exit repeat
			end if
		end repeat
		return newBus
	end try
end tell
#10

Does anyone know if there is a way to search the menu item descriptions in a menu like I mentioned in my first post and select an item that does not contain “ →”

#11

Assuming everything else is correct, the code for this would be:

set theBus to the last menu item of menu 1 of menu item "Bus" of menu 1 of UI element 1 of group 4 of list 1 of group 2 of window tracks_window whose name does not contain "→"

You had “value” where I think you need “name”.
It might be that the multiple layers of the “of” term could confuse AppleScript about which object the “whose” is referring to. If the above does not work, try this:

	set parentMenu to menu 1 of menu item "Bus" of menu 1 of UI element 1 of group 4 of list 1 of group 2 of window tracks_window
	set theBus to the last menu item of parentMenu whose name does not contain "→"

That might be useful just to increase legibility for a human viewer/programmer, even if AppleScript interprets the single-line code as you intend.

#12

I haven’t been able to identity if it’s a name, value, description or accessibility description. Automator returned the ui element location But My UI probe and system events can’t find it even though it’s the current menu. I was wondering if there was a way to acknowledge the front most element without needing to enter its location.

I’m pretty sure if you write
Set butT to the first button of … of … of … of ….whose axds is “…”
AS will only look for the axds of buttons that are a child of the first of element. Similarity you could type
Set butT to the first button of every ui element of every ui element of group 1 menu 2 of window …. .whose axds is “…” which would return the quoted axds of a button from all uiel’s of all uiel’s of group… in a list of lists

#13

get properties of… will provide you with the name, description, etc… to the extent that the developer included them. And of course, if any properties contain that arrow symbol, this will reveal which.

I’d recommend removing the ‘whose’ clause and just getting the entire list of menu items and then you might be able to discern which is the correct one and get its properties to better craft your ‘whose’ clause.

#14

I located the element with Automator which returned


Menu item 11 of menu 1 of menu item “Bus” of pop up button 1 UI Element 1  of group 4 of list 1 of group 2 of window

But a property search couldn’t locate pop up button 1 and returned no property results from the parent of pop up button 1 with or without the menu open

I don’t understand why Automator can see this but system events can’t

#15

I have no clue what app you’re using so I’m going stop guessing but two things first…

  1. You are missing some words in your line above (see below)
  2. Menu items are usually part of a menu 
menu item 11 of menu 1 of menu item “Bus” of pop up button 1 of ¬
		UI Element 1 of group 4 of list 1 of group 2 of window 1
#16

If a menu is only revealed by clicking a pop up button, it doesn’t “exist” until you click that pop up button. So, your script would need to click the pop up button, THEN try to access the menu item within that menu.

#17

Hey, thanks for the message. I already know that

#18

Every sub menu or pop up button appears after the click. And, they are like windows. They doesn’t appear immediately. So, you need to wait, as well (better, using automated delays). Try following:

tell application "System Events" to tell process "app"
	set frontmost to true
	repeat until window 1 exists -- appear when become frontmost
		delay 0.2
	end repeat
	tell window 1 to tell UI element 1 of group 4 of list 1 of group 2
		click
		repeat until pop up button 1 exists -- appear after clicking UI element 1
			delay 0.2
		end repeat
		tell pop up button 1 to tell menu item "Bus"
			click
			repeat until menu 1 exists -- appear after clicking menu item "Bus"
				delay 0.2
			end repeat
			click menu item 11 of menu 1
		end tell
	end tell
end tell

NOTE: I don’t have your app and the reference you provided is just the last action recorded by Automator. You need to review each recorded action to understand what to click and where to wait (repeat until…). Should be at least 3 actions in Automator recorded workflow, because you click (act) 3 times.