Hi all,
I am trying to simulate mouse clicks via my script editor (using macOS High Sierra).
Going through the forums on the internet, nothing really worked.
Can someone help here ?
Thx in advance,
Samuel
Model: MacBook Air
Browser: Safari 537.36
Operating System: macOS 10.13
Hi and welcome to MacScripter
A few suggestions to start:
System events can click, but finding the co-ordinates can be a bit hit and miss, using code something like this:
tell application "System Events"
click at {100, 100}
end tell
Cliclick from https://www.bluem.net/en/projects/cliclick/ is very good, using code
something like this, the “dc” stands for double click.
set cliclick_Path to POSIX path of (((path to desktop) as text) & "cliclick")
set cliclick_Path_Posix to POSIX path of cliclick_Path
do shell script quoted form of cliclick_Path_Posix & space & "dc:100,100" -- change the x & y coodinates
Mouse Tools from here, will help getting the coordinates of the mouse cursor, which you can then use to put into ether of the above
http://www.hamsoftengineering.com/codeSharing/MouseTools/MouseTools.html
Hi Budgie,
Thx for your repy.
I was trying to run your 1st sript;
tell application “System Events”
click at {100, 100}
end tell
But nothing happened.
Also when i was trying to run the 2nd in the following way:
tell application “Google Chrome”
activate
delay 7
end tell
set cliclick_Path to POSIX path of (((path to desktop) as text) & “cliclick”)
set cliclick_Path_Posix to POSIX path of cliclick_Path
do shell script quoted form of cliclick_Path_Posix & space & “dc:100,100” – change the x & y coodinates
I got the following error;
error “sh: /Users/Inon/Desktop/cliclick: No such file or directory” number 127
What do I miss here ?
Thx in advance,
Samuel
hi
this definitely works, you may not have seen anything happen because you don’t have anything sitting at those coordinates, try running mouse tools to get the coordiantes of your mouse cursor so it clicks on say the collapse button in a window.
tell application "System Events"
click at {100, 100}
end tell
for the second issue, once you dowloaded the cliclick dmg and opened it, did you put the downloaded cliclick executable on your desktop?, as the script works fine.
Hi
I was trying this script (by using an excell sheet as a grid);
tell application “Microsoft Excel”
activate
delay 2
end tell
tell application “System Events”
repeat 10 times
click at {400, 100}
end repeat
end tell
tell application “System Events”
repeat 10 times
click at {800, 1300}
end repeat
end tell
tell application “System Events”
repeat 10 times
click at {1400, 1400}
end repeat
end tell
but i got this error;
error “System Events got an error: AppleEvent timed out.” number -1712
can you plz expalin what is wrong ?
thx in advance,
Samuel
Hi
Try telling system events to tell process “Microsoft Excel”
activate application "Microsoft Excel"
delay 2
tell application "System Events" to tell process "Microsoft Excel"
repeat 10 times
click at {400, 100}
end repeat
repeat 10 times
click at {800, 1300}
end repeat
repeat 10 times
click at {1400, 1400}
end repeat
end tell
hi
& sorry for my late reply…for some reason what you suggested is not working for me neither.
Anything else i can try ?
Thx in advance
Things that can click from Applescript you may try:
Ones it looks like you’ve tried:
Things you might still try:
Thx t.spoon,
I have managed to simulate clicks by python, but now i have to integrate the apple script simulation with the python.
Thats a challenge for me since I need to sync those two scripts now…
Any clue how I do that ?
Thx in advance,
Samuel
hi
try something like this
do shell script "python /Users/Samuel_Jr/Desktop/YourScriptName.py"
Samuel_Jr,
Sorry, not sure I’m understanding the problem… the example script I linked to made the Python an Applescript handler for click-and-drag operations. Doesn’t making the python into an Applescript handler integrate the two for you?
Dear t.spoon,
Thank you for your reply, i was sure my inquiry was no longer valid.
Lets make it simple… i need to insert mouse click events to my script below;
script starts;
tell application “TextEdit” --{ ################### Text Editor | NotePad a #########}
activate
delay 9
end tell
tell application “System Events”
repeat 30 times
delay (random number from 0.2 to 0.9)
key code 125 – {arrow down}
end repeat
end tell
tell application “System Events”
repeat 14 times
delay (random number from 0.2 to 0.9)
key code 126 – {arrow up}
end repeat
end tell
tell application “System Events”
repeat 8 times
delay (random number from 1.9 to 3.49)
key code 51 – {delete}
end repeat
end tell
tell application “System Events”
delay (random number from 10 to 60)
set textBuffer to " Hello "
repeat with i from 1 to count characters of textBuffer
keystroke (character i of textBuffer)
delay (random number from 1 to 5)
end repeat
end tell
tell application “System Events”
repeat 8 times
delay (random number from 1.9 to 3.49)
key code 51 – {delete}
end repeat
end tell
script ends
Your kind reply will be much appreciated.
Thank you in advance,
Samuel_jr
No need all that stuff. Press Shift+Command+4 and you have live mouse position coordinates. For exit press Esc
Other note is that: System Events click differs from real mouse click. It just selects clicked object on screen. For real click event ((which activates some window or its object) you must use extra tools as ClickClick or MouseTools. AppleScript without extra tools can’t do that as is modal interpretator
System Events click gives real click behavior only if you click some object which is UI element , for example, some button or some menu item.
You can find which UI Elements has some Window in following way:
tell application "System Events"
tell process "Safari"
tell window 1
UI Elements
end tell
end tell
end tell
If some object you need isn’t UI Element, you must use extra tool. No System Events click at {x,y}!!!. For example, you need mouse click on certain row of window’s list and this row isn’t UI Element
Remember: for System Events, there are only windows, menu bars and buttons. Only these things it understands. And… only visible buttons…
First install MouseTools application in Applications folder
then try this:
set mouseToolsPath to (((path to applications folder) as string) & "MouseTools") as alias
set myString to quoted form of POSIX path of mouseToolsPath & " -x "
tell application "Microsoft Excel" to activate
tell application "System Events" to tell process "Microsoft Excel"
repeat until (window "Open new and recent files" exists)
delay 0.1
end repeat
do shell script myString & "250 -y 250 -doubleLeftClick"
end tell
NOTE: This script auto opens on my Mac NEW BLANK WORKBOOK - (EXCEL window at MAXIMUM size)
UI Elements of window “New and Recent Files” of process “Microsoft Excel” of application “System Events” (you can click them without extra tool, using click at {x,y} of System Events):
{menu button “Sign in” of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, radio group 1 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, group 1 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, text field 1 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, group 2 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, button “Cancel” of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, button “Create” of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, button 3 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, button 4 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, button 5 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”, toolbar 1 of window “Open new and recent files” of application process “Microsoft Excel” of application “System Events”}
Dears,
Your help is much appreciated.
as for your inputs;
when trying to write a simple script of:
Applescript:
tell application “System Events”
click at {100, 100}
end tell
it rund - nothing happens - no clicks.
Cliclick from https://www.bluem.net/en/projects/cliclick/ is like creating a sript via python - which is not useful, since i need to sync the clicks with other commands e.g writing text.
it looks so simple and but i start to wonder if i need to upgrade a version or so
thx
Samuel_jr
Two things:
Excel and Text Edit both have Applescript Dictionaries, so many things can be scripted directly rather than with UI scripting. What does your script actually do? Maybe you don’t need mouse clicks?
I modified the Python “drag” script I’d linked to before to simply click.
on mouseClick(x, y)
-- delayTime because the drag may fail if the UI isn't fast enough without a delay. For what I do, .1 works.
do shell script "
/usr/bin/python <<END
from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventCreate
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap
import time
def mouseEvent(type, posx, posy):
theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
CGEventPost(kCGHIDEventTap, theEvent)
def mousemove(posx,posy):
mouseEvent(kCGEventMouseMoved, posx,posy);
def mousedown(posxdown,posydown):
mouseEvent(kCGEventLeftMouseDown, posxdown,posydown);
def mouseup(posxup,posyup):
mouseEvent(kCGEventLeftMouseUp, posxup,posyup);
ourEvent = CGEventCreate(None);
mousedown(" & x & "," & y & ");
time.sleep(" & (".1") & ");
mouseup(" & x & "," & y & ");
END"
end mouseClick
So, you didn’t say when and where you need to click, but inserting some random clicks into your script it looks like this:
tell application "TextEdit" --{ ################### Text Editor | NotePad a #########}
activate
delay 9
end tell
tell application "System Events"
repeat 30 times
delay (random number from 0.2 to 0.9)
key code 125 -- {arrow down}
my mouseClick(1000, 50)
end repeat
end tell
tell application "System Events"
repeat 14 times
delay (random number from 0.2 to 0.9)
key code 126 -- {arrow up}
my mouseClick(20, 360)
end repeat
end tell
tell application "System Events"
repeat 8 times
delay (random number from 1.9 to 3.49)
key code 51 -- {delete}
my mouseClick(42, 42)
end repeat
end tell
tell application "System Events"
delay (random number from 10 to 60)
set textBuffer to " Hello "
repeat with i from 1 to count characters of textBuffer
keystroke (character i of textBuffer)
delay (random number from 1 to 5)
end repeat
end tell
tell application "System Events"
repeat 8 times
delay (random number from 1.9 to 3.49)
key code 51 -- {delete}
end repeat
end tell
mouseClick(1342, 91)
on mouseClick(x, y)
-- delayTime because the drag may fail if the UI isn't fast enough without a delay. For what I do, .1 works.
do shell script "
/usr/bin/python <<END
from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventCreate
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap
import time
def mouseEvent(type, posx, posy):
theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
CGEventPost(kCGHIDEventTap, theEvent)
def mousemove(posx,posy):
mouseEvent(kCGEventMouseMoved, posx,posy);
def mousedown(posxdown,posydown):
mouseEvent(kCGEventLeftMouseDown, posxdown,posydown);
def mouseup(posxup,posyup):
mouseEvent(kCGEventLeftMouseUp, posxup,posyup);
ourEvent = CGEventCreate(None);
mousedown(" & x & "," & y & ");
time.sleep(" & (".1") & ");
mouseup(" & x & "," & y & ");
END"
end mouseClick
Also, please use the Applescript tags when posting code. Button is just above text box when composing a comment.
If there is no active element at this point of the {100,100} screen, then nothing should happen. Only the mouse pointer should go to this point. In order for a real action to take place, there must be an element at this point on the screen that is activated when you click the mouse manually. If nothing happens when the mouse is clicked manually, then nothing will happen with a programmable click. You have already explained everything quite clearly …
Here, click this WORD. Nothing will happen (either manually or programmatically), as this is an inactive element of this web page. And now, click this WORD. This will show you how I translated to you everything I wrote here, from my Russian to English. This is active element I created on this web page for you and now you can click it manually or programmatically.
Now, as I note above, System Events’s click at {x,y} works directly only for UI elements it can see. For example, menu “History” is one UI Element of “Safari”, so we can click it directly:
tell application "Safari"
activate
open location "https://macscripter.net/viewtopic.php?id=46612"
end tell
tell application "System Events" to tell process "Safari"
-- waiting to open web page completly
repeat until (window "MacScripter / simulating mouse clicks macos" exists)
delay 0.1
end repeat
-- opening menu "History"
click at {294, 10}
end tell
Now, say I want click profile of user Samuel_Jr (is at {200,105} when this page opens in Safari in FULL SCREEN MODE). It is active element of web page, but isn’t UI Element. So here works System Events’s variant to click with waiting for focus (quit before the Safari, only to standartize this test):
tell application "Safari"
activate
open location "https://www.macscripter.net/viewtopic.php?pid=194441#p194441"
end tell
tell application "System Events" to tell process "Safari"
-- wait to open page competly
repeat until (window "MacScripter / simulating mouse clicks macos" exists)
delay 0.1
end repeat
-- open page in FULL SCREEN MODE, to standartize this test
tell window "MacScripter / simulating mouse clicks macos" to click UI element 3
click at {200, 105}
delay 1 -- wait to focus the element
click at {200, 105}
end tell