Okay,
I’m new to applescript, I’ve been reading about it, and playing with it for about a month now. Here’s my basic summary; frustrating. My project is to convert a program that runs in Windows using AutoIt ( AutoIt Scripting Language - AutoIt). The job is about 300 times harder than it was with AutoIt. Am I wrong here? Why can’t I just measure the location of the mouse using x/y coordinates and write a script dictating where each click falls? The GUI scripting is a nightmare, UI Element Inspector is neat, but lacks documentation. UI Browser is neat also, and a bit more useful, but the scripts generated never worked. The more I play with this, the more it seems AppleScript is not simple, easy or for the average mac user.
I want a simple solution to record my mouse’s actions, and generate a log which will run as a script. Is such an option available, and if not, should I hire some monkey to build it and sell it to everyone?
I think UI Scripting is concibed to be used only when you can’t do the work using a more direct approach.
In OS 9 days, we had various utilities for this task, specially Sändi’s Additions, AutoType/AutoClick and the fabulous PreFab Player (and others)… But I think near to nobody used them for serious workflows. I mean, lots of folks used them, but for example in a serious business you allways hire someone to write some C for you.
Anyway, most of times there are different solutions for the same task. Perhaps if you explain what you need, someone will be able to help you.
Philosophically, AppleScript isn’t a GUI language. It’s purpose is to automate repetitive, complex, or time-consuming tasks by telling the application(s) concerned to carry out the necessary actions. It does this directly, through the application’s scripting interface (for want of a better expression), not through the user interface.
The success of this obviously depends on application developers having implemented AppleScriptability in their apps, and with a set of commands and classes that allow scripters to achieve what they want to do. Not all application developers do that, of course.
GUI scripting is a recent addition to the system application System Events and is an attempt to extend the reach of AppleScript a little to unscriptable applications and to unscriptable features of applications. It’s not part of AppleScript itself, but a facility offered by System Events. You can tell System Events (through its scripting interface) to kid another running application (through its user interface) that the user has clicked certain buttons, pressed certain keys, or selected certain menu items. It’s a nightmare because the user interface of each application is different and can vary from version to version.
That’s my answer to the philosophy question. I don’t personally know how to get clicked screen coordinates.
The real difficulty underlying the OP’s philosophical question is that he is trying to repeat something he did in WinXP on an OS X machine as if it was WinXP. The OP is far better off to discover how to make the Mac applications do his bidding than he is to attempt to copy what he did before even if the GUIs of the programs he is using look exactly the same.
[NOTE: This is not a criticism or comparison of either system - it is a bald statement of the fact that they are different, and the original poster will have greatest success if he goes with the flow]
The first link posted above claims to do it (not applescript-based), for example.
As for grabbing such info using AppleScript, we have some stuff: Extra Suites, XTool, Smile…
This is a sample script for Smile, which will grab mouse clicks:
set rec to {}
--> script will be stopped bringing Smile to the front,
--> then using the shortcut apple+dot
try
repeat
if mouse button then
set ml to mouse location
try
if rec's item -1 is not ml then set rec's end to ml
on error --> add first mouse location
set rec's end to ml
end try
end if
end repeat
end try
rec --> {{159, 40}, {172, 80}}
This list of clicks made a new window in Safari, then clicked my toolbar-favorite button for MacScripter.
You can use later this list to re-create the clicks using one of the mentioned tools: System Events, Extra Suites, XTool, etc. But obviously it would be much easier:
tell application "Safari" to make new document with properties {URL:"http://www.macscripter.net"}
While I believe XTool is not currently supported (and its click command seems to be broken), here’s a similar routine for Extra Suites:
To record mouse click positions:
display alert "Mouse Recorder" message ("Click \"OK\" to start recording the position of mouse clicks." as Unicode text) & ¬
return & return & "To return the recorded results, press " & «data utxt2318» & ¬
"-. (command-period) or the Esc key." buttons {"Cancel", "OK"} cancel button 1 default button 2
set locList to {}
tell application "Extra Suites" to repeat
try
if ES mouse down then
set locList's end to ES mouse location
repeat while ES mouse down
delay 0.1
end repeat
end if
on error
return locList
end try
end repeat
To replay recorded mouse clicks:
set locList to {{662, 290}, {894, 278}, {827, 373}, {439, 355}} (* or whatever *)
tell application "Extra Suites" to repeat with mouseLoc in locList
ES move mouse mouseLoc
ES click mouse
delay 0.3 (* modify as appropriate *)
end repeat
On the subject of philosophy, I’m perfectly happy to go along with that of Bill Gates…
Yes, it’s partially broken (its double-click command, though, is sometimes the only tool available in certain impossible clicking tasks → ie, in some Java/Flash apps) and actually it only exists in the Osaxen.com database. There is a chance, though, to resurrect it in a near future (fingers crossed )
)