Click at coordinates "return {xCoordinate, yCoordinate}"

Scripting Forums AppleScript | Mac OS X
#1

Hello,

This Applescript returns coordinates of a specific UI element. Is there anyway to write another line of Applescript to use the returned coordinates for a mouse click? I know I could code to always click at the current location {944.5, 844.0}, but the UI may move around and the UI element my change locations. So the idea is to use the UI element ID to get the coordinates every time I launch the script, and then click.

tell application “System Events”
tell process “DaVinci Resolve”
tell checkbox 13 of group 1 of window “DaVinci Resolve by Blackmagic Design - 17.0.0” of application process “Resolve” of application “System Events”
set p to position
set s to size
end tell
end tell
end tell

set xCoordinate to (item 1 of p) + (item 1 of s) / 2
set yCoordinate to (item 2 of p) + (item 2 of s) / 2
return {xCoordinate, yCoordinate}

Result: {944.5, 844.0}

I’m using Keyboard Maestro to accomplish the last step now:
https://drive.google.com/file/d/1eWDfq2ws78BEiRCaCc5Tf3BWVUpwJ9mk/view?usp=sharing

Thanks!

#3

The UI element may change locations but the reference to it is persistent. And, checkbox should have click action, so clicking at position using mouse tool, no need. This should click your checkbox:


tell application "DaVinci Resolve" to activate

tell application "System Events" to tell process "DaVinci Resolve"
	repeat until window "DaVinci Resolve by Blackmagic Design - 17.0.0" exists
		delay 0.02
	end repeat
	tell checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0"
		click it
	end tell
end tell
#4

Hello everyone, thanks so much for the help!

I think DaVinci Resolve’s API isn’t allowing “checkboxes” to be clicked, even thought UI browser seems to think they may be accessible.

I am able to script the clicking of a “button,” but nothing within Applescript would let me click a “checkbox.” I had to use a MouseTools shell to do the final click.

So this worked:

tell application “System Events”
tell process “DaVinci Resolve”
tell checkbox 13 of group 1 of window “DaVinci Resolve by Blackmagic Design - 17.0.0” of application process “Resolve” of application “System Events”
set p to position
set s to size
set xCoordinate to (item 1 of p) + (item 1 of s) / 2
set yCoordinate to (item 2 of p) + (item 2 of s) / 2
do shell script "/Applications/MouseTools -x " & xCoordinate & " -y " & yCoordinate & " -leftClick "
end tell
end tell
end

Here are the two UI elements I tested
“button” 1 (red square “1” around the home icon)
“checkbox 13” (red box “2” around the eyedropper)

https://drive.google.com/file/d/1vb6dlp4DPRs1em308f4FHl_TUYWlsLUy/view?usp=sharing

These are all the scripts I attempted:

https://drive.google.com/file/d/1dqQNSwRITAy16dkVtSqEwJ1dbl_r09jB/view?usp=sharing

Here is what I think is critical piece of info from UI browser.

For “buttons” all three accessibility options are available:

https://drive.google.com/file/d/1-5JXCKYGQqdwpWpmqpNIBzfUzAy--9Sd/view?usp=sharing

However for “checkbox” direct clicking is not available, and while the other two options, click position and select are available, they fail:

https://drive.google.com/file/d/1isiExSXjf1dXvEYMzC_Bzqut9gCnc5Q6/view?usp=sharing

Here is all the info for buttons and checkboxs:

https://drive.google.com/drive/folders/1Apw7SxNAPjKRRzcKhfyf2Q8NNP3VP1Vo?usp=sharing

Am I right? Checkboxes just can’t be clicked without a command line?

Thanks again guys!

#5

I don’t think. 1) Clicking some checkbox is equivalent to setting its value (to 0 or 1). Some UI elements need be focused and enabled (set its focused to true, set its enabled to true) before changing its value (set its value to 1) or clicking (click it).

I see, the application is big (1,86 GB), and paid. So, I won’t install it to find complete solution.

Script Debugger is more helpfull with GUI scripting. If you can change manually value of checkbox in Variables panel of Script Debugger, then your checkbox is clickable (or settable) without using mouse tools.

#6

Thank you KniazidisR,

Script Debugger is great. I can see so much more.

Still a bit confused by debugger though. Here is a screengrab:

https://drive.google.com/file/d/1c6yRhN6dVQD7JATdgR79HUsj5tP37LM3/view?usp=sharing

Under Checkbox 13 “attributes” it says:

  1. “AXEnabled” “settable” = false.
  2. “AXFocused” “settable” = true
  3. “AXValue” “settable” = false.

So this means I cannot set the value by a script right?

However, if I hover the mouse over “enabled” in the green box, a pop up comes up saying, "Is the UI element enabled? (Does it accept clicks?) = true. So should checkbox 13 be clickable, but not settable (have the value changed by script)?

UI browser has clickable grayed out as an option:

https://drive.google.com/file/d/1isiExSXjf1dXvEYMzC_Bzqut9gCnc5Q6/view?usp=sharing

If I run either of these two scripts:

This one:

tell application “System Events”
tell its application process “Resolve”
tell its window “DaVinci Resolve by Blackmagic Design - 17.0.0”
tell its group 1
tell its checkbox 13
set focus to true
set value to 1
end tell
end tell
end tell
end tell
end tell

Or this one:

tell application “System Events”
tell its application process “Resolve”
tell its window “DaVinci Resolve by Blackmagic Design - 17.0.0”
tell its group 1
tell its checkbox 13
set value to 1
end tell
end tell
end tell
end tell
end tell

Applescript says this Result:

tell application “System Events”
set value of checkbox 13 of group 1 of window “DaVinci Resolve by Blackmagic Design - 17.0.0” of application process “Resolve” to 1
end tell

But nothing happens to checkbox 13, or DaVinci Resolve.

If I run this script:

tell application “System Events”
tell its application process “Resolve”
tell its window “DaVinci Resolve by Blackmagic Design - 17.0.0”
tell its group 1
tell its checkbox 13
set focus to true
set enabled to true
set value to 1
end tell
end tell
end tell
end tell
end tell

I get:

Result:
error “System Events got an error: Can’t set enabled of UI element to true.” number -10006 from enabled of UI element

Thanks again!

#7

No. “AXEnabled” “settable” is false, but value of “AXEnabled” is true. So, checkbox is enabled persistently, and you no need to set enabled property. It is already setted to what you want. Instead, try command set focused to true (not set focus to true), then command click it. Try select it command as well. Your checkbox is clickable. I see on the picture, it has 1 action, but you didn’t reveal it. So, I can’t see what action is this

#8

Hello,

I tried both click it and select it and I get the same resut:

checkbox 13 of group 1 of window “DaVinci Resolve by Blackmagic Design - 17.0.0” of application process “Resolve” of application “System Events”

I’m guessing to GUI checkbox is restricted.

Thanks!

#9

You tried this code or something other? :


tell application "System Events"
	tell its application process "Resolve"
		tell its window "DaVinci Resolve by Blackmagic Design - 17.0.0"
			tell its group 1
				tell its checkbox 13
					set its focused to true
					-- delay 1 -- maybe delay is needed
					click it
				end tell
			end tell
		end tell
	end tell
end tell
#10

I gave it a try. Some result:

checkbox 13 of group 1 of window “DaVinci Resolve by Blackmagic Design - 17.0.0” of application process “Resolve” of application “System Events”

I usually get that result if It’s not click, select, or settable. I just use MouseTools for the final click.

Thank you, I appreciate your time!