Saturday, February 27, 2021

#1 2021-01-21 11:31:00 pm

whitebalance
Member
Registered: 2021-01-18
Posts: 10

Click at coordinates "return {xCoordinate, yCoordinate}"

Hello,

This Applescript returns coordinates of a specific UI element.  Is there anyway to write another line of Applescript to use the returned coordinates for a mouse click?  I know I could code to always click at the current location {944.5, 844.0}, but the UI may move around and the UI element my change locations.  So the idea is to use the UI element ID to get the coordinates every time I launch the script, and then click.

tell application "System Events"
    tell process "DaVinci Resolve"
        tell checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0" of application process "Resolve" of application "System Events"
            set p to position
            set s to size
        end tell
    end tell
end tell

set xCoordinate to (item 1 of p) + (item 1 of s) / 2
set yCoordinate to (item 2 of p) + (item 2 of s) / 2
return {xCoordinate, yCoordinate}

Result: {944.5, 844.0}

I'm using Keyboard Maestro to accomplish the last step now:
https://drive.google.com/file/d/1eWDfq2 … sp=sharing

Thanks!

Offline

 

#2 2021-01-22 03:27:31 am

Fredrik71
Member
Registered: 2019-10-23
Posts: 630

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

You say the window position {x,y} could be different.

if you do:

Applescript:

set {xPos,yPos} to {YOUR_VALUE_FOR_X,YOUR_VALUE_FOR_Y}
set position to {xPos,yPos}

Your x,y position will always be the same.


The purpose to study someone else art is not to add, its to make less more.

Offline

 

#3 2021-01-22 08:58:37 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1637

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

whitebalance wrote:

Hello,

I know I could code to always click at the current location {944.5, 844.0}, but the UI may move around and the UI element my change locations.


The UI element may change locations but the reference to it is persistent. And, checkbox should have click action, so clicking at position using mouse tool, no need. This should click your checkbox:

Applescript:


tell application "DaVinci Resolve" to activate

tell application "System Events" to tell process "DaVinci Resolve"
   repeat until window "DaVinci Resolve by Blackmagic Design - 17.0.0" exists
       delay 0.02
   end repeat
   tell checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0"
       click it
   end tell
end tell

Last edited by KniazidisR (2021-01-22 09:00:31 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#4 2021-01-22 11:22:15 pm

whitebalance
Member
Registered: 2021-01-18
Posts: 10

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

Hello everyone, thanks so much for the help!

I think DaVinci Resolve's API isn't allowing "checkboxes" to be clicked, even thought UI browser seems to think they may be accessible.

I am able to script the clicking of a "button," but nothing within Applescript would let me click a "checkbox."  I had to use a MouseTools shell to do the final click.

So this worked:

tell application "System Events"
    tell process "DaVinci Resolve"
        tell checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0" of application process "Resolve" of application "System Events"
            set p to position
            set s to size
            set xCoordinate to (item 1 of p) + (item 1 of s) / 2
            set yCoordinate to (item 2 of p) + (item 2 of s) / 2
            do shell script "/Applications/MouseTools -x " & xCoordinate & " -y " & yCoordinate & " -leftClick "
        end tell
    end tell
end

Here are the two UI elements I tested
"button" 1 (red square "1" around the home icon)
"checkbox 13" (red box "2" around the eyedropper)

https://drive.google.com/file/d/1vb6dlp … sp=sharing

These are all the scripts I attempted:

https://drive.google.com/file/d/1dqQNSw … sp=sharing

Here is what I think is critical piece of info from UI browser.

For "buttons" all three accessibility options are available:

https://drive.google.com/file/d/1-5JXCK … sp=sharing

However for "checkbox" direct clicking is not available, and while the other two options, click position and select are available, they fail:

https://drive.google.com/file/d/1isiExS … sp=sharing

Here is all the info for buttons and checkboxs:

https://drive.google.com/drive/folders/ … sp=sharing

Am I right?  Checkboxes just can't be clicked without a command line?

Thanks again guys!

Last edited by whitebalance (2021-01-22 11:33:17 pm)

Offline

 

#5 2021-01-23 03:13:09 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1637

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

whitebalance wrote:

Am I right?  Checkboxes just can't be clicked without a command line?


I don't think. 1) Clicking some checkbox is equivalent to setting its value (to 0 or 1). Some UI elements need be focused and enabled (set its focused to true, set its enabled to true) before changing its value (set its value to 1) or clicking (click it).

I see, the application is big (1,86 GB), and paid. So, I won't install it to find complete solution.

Script Debugger is more helpfull  with GUI scripting. If you can change manually value of checkbox in Variables panel of Script Debugger, then your checkbox is clickable (or settable) without using mouse tools.

Last edited by KniazidisR (2021-01-23 03:17:47 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#6 2021-01-25 01:16:25 am

whitebalance
Member
Registered: 2021-01-18
Posts: 10

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

Thank you KniazidisR,

Script Debugger is great.  I can see so much more.

Still a bit confused by debugger though.  Here is a screengrab:

https://drive.google.com/file/d/1c6yRhN … sp=sharing

Under Checkbox 13 "attributes" it says:

1.  "AXEnabled" "settable" = false.
2.  "AXFocused" "settable" = true
3.  "AXValue" "settable" = false.

So this means I cannot set the value by a script right?

However, if I hover the mouse over "enabled" in the green box, a pop up comes up saying, "Is the UI element enabled? (Does it accept clicks?) = true.  So should checkbox 13 be clickable, but not settable (have the value changed by script)?

UI browser has clickable grayed out as an option:

https://drive.google.com/file/d/1isiExS … sp=sharing

If I run either of these two scripts:

This one:

tell application "System Events"
    tell its application process "Resolve"
        tell its window "DaVinci Resolve by Blackmagic Design - 17.0.0"
            tell its group 1
                tell its checkbox 13
                    set focus to true
                    set value to 1
                end tell
            end tell
        end tell
    end tell
end tell

Or this one:

tell application "System Events"
    tell its application process "Resolve"
        tell its window "DaVinci Resolve by Blackmagic Design - 17.0.0"
            tell its group 1
                tell its checkbox 13
                    set value to 1
                end tell
            end tell
        end tell
    end tell
end tell

Applescript says this Result:

tell application "System Events"
    set value of checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0" of application process "Resolve" to 1
end tell

But nothing happens to checkbox 13, or DaVinci Resolve.

If I run this script:

tell application "System Events"
    tell its application process "Resolve"
        tell its window "DaVinci Resolve by Blackmagic Design - 17.0.0"
            tell its group 1
                tell its checkbox 13
                    set focus to true
                    set enabled to true
                    set value to 1
                end tell
            end tell
        end tell
    end tell
end tell

I get:

Result:
error "System Events got an error: Can’t set enabled of UI element to true." number -10006 from enabled of UI element

Thanks again!

Last edited by whitebalance (2021-01-25 02:28:10 am)

Offline

 

#7 2021-01-25 06:11:33 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1637

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

whitebalance wrote:


1.  "AXEnabled" "settable" = false.
2.  "AXFocused" "settable" = true
3.  "AXValue" "settable" = false.

So this means I cannot set the value by a script right?



No. "AXEnabled" "settable" is false, but value of "AXEnabled" is true. So, checkbox is enabled persistently, and you no need to set enabled property. It is already setted to what you want. Instead, try command set focused to true (not set focus to true), then command click it. Try select it  command  as well. Your checkbox is clickable.  I see on the picture, it has 1 action, but you didn't reveal it. So, I can't see what action is this

Last edited by KniazidisR (2021-01-25 06:21:17 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#8 2021-01-26 11:15:57 am

whitebalance
Member
Registered: 2021-01-18
Posts: 10

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

Hello,

I tried both click it and select it and I get the same resut:

checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0" of application process "Resolve" of application "System Events"

I'm guessing to GUI checkbox is restricted.

Thanks!

Offline

 

#9 2021-01-27 12:01:52 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 1637

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

whitebalance wrote:

I tried both click it and select it and I get the same resut:


You tried this code or something other? :

Applescript:


tell application "System Events"
   tell its application process "Resolve"
       tell its window "DaVinci Resolve by Blackmagic Design - 17.0.0"
           tell its group 1
               tell its checkbox 13
                   set its focused to true
                   -- delay 1 -- maybe delay is needed
                   click it
               end tell
           end tell
       end tell
   end tell
end tell

Last edited by KniazidisR (2021-01-27 12:05:02 am)


Model: MacBook Pro
OS X: Catalina 10.15.4
Web Browser: Safari 13.1
Ram: 4 GB

Offline

 

#10 2021-01-29 05:18:14 pm

whitebalance
Member
Registered: 2021-01-18
Posts: 10

Re: Click at coordinates "return {xCoordinate, yCoordinate}"

I gave it a try.  Some result:

checkbox 13 of group 1 of window "DaVinci Resolve by Blackmagic Design - 17.0.0" of application process "Resolve" of application "System Events"

I usually get that result if It's not click, select, or settable.  I just use MouseTools for the final click.

Thank you, I appreciate your time!

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)