Monday, July 26, 2021

#1 2018-03-02 08:27:56 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 497

Click and Drag without Additional Libraries

Not a question - just posting here in case it's useful to someone.

I needed the ability to click and drag for UI scripting.

System Events only lets you click at a location.
Applescript Toolbox can be script-embedded, avoiding user installation, but can't click and drag.
MouseTools and ClickClick should both do what I need, but need to be installed on each user's computer.

So here's a solution that just uses included Python:

Applescript:




on mouseDrag(xDown, yDown, xUp, yUp, delayTime)
-- delayTime because the drag may fail if the UI isn't fast enough without a delay. For what I do, .1 works.
   do shell script "
   
/usr/bin/python <<END


from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventCreate
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap
from Quartz.CoreGraphics import kCGEventLeftMouseDragged
import time

def mouseEvent(type, posx, posy):

theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)

CGEventPost(kCGHIDEventTap, theEvent)

def mousemove(posx,posy):

mouseEvent(kCGEventMouseMoved, posx,posy);

def mousedrag(posx,posy):

mouseEvent(kCGEventLeftMouseDragged, posx,posy);

def mousedown(posxdown,posydown):

mouseEvent(kCGEventLeftMouseDown, posxdown,posydown);

def mouseup(posxup,posyup):

mouseEvent(kCGEventLeftMouseUp, posxup,posyup);

ourEvent = CGEventCreate(None);

mousedown("
& xDown & "," & yDown & ");
time.sleep("
& (delayTime as text) & ");
mousedrag("
& xUp & "," & yUp & ");
time.sleep("
& (delayTime as text) & ");
mouseup("
& xUp & "," & yUp & ");

END"

end mouseDrag


Hackintosh built February, 2012 |  Mac OS Sierra
GIGABYTE GA-Z68X-UD3H-B3 | Core i5 2500k | 16 GB DDR3 | GIGABYTE Geforce 1050 TI 4GB
250 GB Samsung 850 EVO | 4 TB RAID
Dell Ultrasharp U3011 | Dell Ultrasharp 2007FPb

Offline

 

#2 2018-03-02 08:54:42 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 497

Re: Click and Drag without Additional Libraries

Just found:

Applescript:


use framework "Foundation"
set mouseLoc to current application's NSEvent's mouseLocation()

Which makes me suspect there's a succinct ASObjC solution to replace the need for ClickClick, MouseTools, AppleScript Toolbox, or Python for all these mouse-related actions.

For now, I'm using System Event to click, ASObjC to get the mouse position, and Python to drag. Works... doesn't feel very elegant.

- Tom.


Hackintosh built February, 2012 |  Mac OS Sierra
GIGABYTE GA-Z68X-UD3H-B3 | Core i5 2500k | 16 GB DDR3 | GIGABYTE Geforce 1050 TI 4GB
250 GB Samsung 850 EVO | 4 TB RAID
Dell Ultrasharp U3011 | Dell Ultrasharp 2007FPb

Offline

 

#3 2018-03-02 09:48:24 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 497

Re: Click and Drag without Additional Libraries

Just to further note for anyone finding this thread:

System Events clicking has proved unreliable for me
ASObjC's "current application's NSEvent's mouseLocation()" returns the mouse coordinates from the bottom left, instead of the top left. Then I have to get the screen resolution and convert. Which means I also have to detect which display they're on... and how's Retina affect all that? All in all, I'm back to using Applescript Toolbox for getting the mouse location and basic clicks, and Python to drag.


Hackintosh built February, 2012 |  Mac OS Sierra
GIGABYTE GA-Z68X-UD3H-B3 | Core i5 2500k | 16 GB DDR3 | GIGABYTE Geforce 1050 TI 4GB
250 GB Samsung 850 EVO | 4 TB RAID
Dell Ultrasharp U3011 | Dell Ultrasharp 2007FPb

Offline

 

#4 2018-03-02 10:59:41 pm

Shane Stanley
Member
From:: Australia
Registered: 2002-12-07
Posts: 6677

Re: Click and Drag without Additional Libraries

t.spoon wrote:

Which makes me suspect there's a succinct ASObjC solution to replace the need for ClickClick



There's not. The CGEvent stuff is all C-based APIs. Although ASObjC can call C-based code, it can't provide the arguments needed in this case.

ASObjC's "current application's NSEvent's mouseLocation()" returns the mouse coordinates from the bottom left, instead of the top left.



That's how they are calculated internally (mostly). AppleScript was based on the the old coordinate system used pre-OS X. But OS X was built on Display PostScript, and I'm sure you're aware of its coordinate system.

how's Retina affect all that?



it doesn't. The values are in points, not pixels.


Shane Stanley <sstanley@myriad-com.com.au>
www.macosxautomation.com/applescript/apps/
latenightsw.com

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)