clicking the mouse using applescript: is it possible?

Scripting Forums AppleScript | Mac OS X
, ,
#1

I would like to be able to click the mouse by voice, and as I understand it, if something can be written in an applescript, it can be made speakable.

So is it possible to write a script to click the mouse?

If it is, pointing me to a source that has done it, or where a very newbie can learn, would be very much appreciated.

If it isn’t, oh well, I tried.

Thanks!

Polly

PS MacDictate doesn’t let you do that; ListenDo for sys 9 did, but of course that’s not usable now.

Model: iMac
AppleScript: standard 2008 iMac
Browser: Firefox 2.0.0.11
Operating System: Mac OS X (10.5)

#2

Welcome to the MacScripter Forum, Polly!

(Wow, I never thought I would be able to say that!)

It is possible, using GUI Scripting, to simulate a click anywhere (I think). However, this will only work with one location, so if you only want your script to click in one place, that should be fine. If you want different places, I suppose you could create a different script for each place you want to work and set those all up. Here is the Apple page on GUI Scripting, however it is not all- inclusive. If you posted what you want to click on specifically, I’m sure I of somebody else could help you out.

#3

I wish I could help you, but I am at a pre-beginner stage!

I was hoping for a means to click the mouse anywhere on the screen.

Polly

#4

Polly:

I’m also interested in this basic capability… and have discovered one way to do it so far.

Extra Suites (a third_party Scripting Addition /Application) allows you to click anywhere on screen via a very simple script:


tell application "Extra Suites"
	ES click mouse
end tell

… by simply saying “Click”

The script must be coupled with entries in the Speech preference pane of System Preferences… and requires some additional finagling… though nothing too arduous.

Extra Suites costs $10. for an individual license… I just bought one. (But I have no affiliation.)

I’ll help with setup if you think it’s worth your time.

Peter B.

#5

I’m sure there’s a simply way to do it via GUI scripting, but I haven’t got my applescript book thing on me at the mo.

#6

And have you found your book thing yet?

Not to seem testy, but if you can show us how this is done via (merely) System Events and UI scripting, I imagine there may be others here interested.

System Events allows you to click at specific coordinates… but as near as I can tell, doesn’t return cursor position in any way.

Tell me I’m wrong.

Peter B.

#7

I would very much appreciate set-up help when I get the app!

Would this work without headphones, just with the built-in microphone, do you think?

Polly

[PS I have to finish class set-up tomorrow, so I will look for ES after that.]

#8

Polly:

Speakable Items (accessible from the Speech panel in System Preferences) requires a little ‘teaching’ for the input level of your voice… but it’s not difficult to dial in. I’m using it with the built-in microphone on my PowerBook.

There is at least one AppleScript in the Speakable Items folder (which, sadly, is of rather little use), but provides a simple example how ‘Speakable’ items are set up. You need only name a script (of any function or utility) to the voice command you want to call it by. You can test the basic capability without buying Extra Suites.

Extra Suites provides the (additional) ability to click wherever the cursor is located at any given moment.

The visible interface for Speakable Items leaves something to be desired… but can be ‘hidden’ in the Dock if you wish.

ES is available here at MacScripter’s ‘sister site’, osaxen.com:

http://osaxen.com/files/extrasuites1.1.html

Again, I’m not in any way affiliated, and while ES has many other useful features, I’m not likely to take advantage of the full set myself. For personal use however, at $10., it’s still quite a deal.

I don’t post often here, but I ‘lurk’ frequently. I’ll watch for your progress.

Peter B.

#9

Peter, I’m almost there, I think. but not quite.

I set up speakable items, and there is a lot there I can use. Everything in the “speakable items” folder works.

Except [sigh!] the reason I started all this. I put the ES script for click mouse in the speakable items folder, renamed it mouse click, and that should be that.

But when I say Computer [my keyword] mouse click, while it recognizes the words, it doesn’t click the mouse, but rather opens the script editor with the script. What am I doing wrong?

And while we’re here: this is the key part of script:

ES click mouse with double click --and option
--ES click mouse with control

I don’t actually understand what it is saying. Is it "clicking the mouse by voice results in a double-click? and should I set up two scripts, one with “ES click mouse with double click” and one with “option–ES click mouse with control” without the hyphens?

Thanks for any help!

#10

Hi Polly:

Glad to hear you’re making progress…

Below are two scripts for what (I think) you most wanted. They need to be saved individually (as scripts… not text files or applications) from Script Editor… suitably named, and placed in the Speakable Items folder (as I think you’ve been doing).

In the Speech pref pane, under Speech Recognition ‘Settings’, I’ve now set the keyword to ‘On’ and made it ‘Required before each command’, in the ‘Keyword is:’ option. (Experience with the keyword as ‘Optional before commands’ led to some surprising results when (e.g.) I coughed or sighed loudly.)

I’ve named the two scripts ‘Click’ and ‘Double’ respectively… and they seem to work pretty reliably.

I haven’t had much luck yet with a control click… but will keep at it if you need it.

Note that there are a lot of Speakable Items available by default. I moved most of the ones I’d never use to a folder within Speakable Items and renamed the ones I might use regularly to more simple spoken commands.

However, ‘phonetics’ are important… similarly named commands or scripts may confuse Speech Recognition, even when using a keyword… and I’ve had to rename some commands to be recognized more readily (e.g. ‘Highed’ works better than ‘Hide’, ‘Male’ works better than ‘Mail’).

Also note that Speech Recognition seems to require a ‘teaching’ session (sometimes) of 2-3 repeated spoken attempts to run a newly named (or renamed) script… and sometimes, some names just don’t work that well.

Finally, (it probably goes without saying) Speech Recognition is not likely to work well (if at all) in a noisy room or listening to music (without headphones).

Hope this moves you along.

I’ll check back soon.

Peter B.


tell application "Extra Suites"
	ES click mouse
end tell



tell application "Extra Suites"
	ES click mouse with double click
end tell



tell application "System Events" to key down control

tell application "Extra Suites" to ES click mouse

tell application "System Events" to key up control

–

(PS - The third script above should effect a control click. It seems to work better than the ‘ES click mouse with option’ variation.)

#11

Thank you so much for answering!

I did as you described, and while all the other speakable items work, and I saved your scripts under those names in the “speakable items folder”, still when I say them, the script editor opens and the command is not done.

Yet they work for you. I must be missing doing something very basic [sigh!].

That’s a good idea about the other items in the folder; I’ll do that, taking into account your warnings.

Polly

#12

Ooops!

the doubleclick version works perfectly; the click version doesn’t!

Polly

#13

Polly:

I don’t have an immediate explanation as to why Script Editor is opening some of your Speakable Items scripts. Indeed, that doesn’t happen here.

The fact that the double click script works suggests you’re getting close… try this:

Duplicate the double click script, open the copy in Script Editor, substitute the single click script contents, save, rename the file and see what happens…

Another (lame) suggestion would be to quit Script Editor… and retest… then logout and login and/or restart and retest… and see if things don’t work a bit more predictably at each ‘fall back’ point.

Speech Recognition is a bit flakey, no doubt about it, but my guess is that you can make it work with some additional persistence.

Once again, I’ll check back… but the sun is shining briefly here and I should go…

Hope you get things working.

Keep me posted.

Peter B.

#14

Polly:

A couple of more thoughts…

Yes, on further testing… if you make the mistake of saving a given script as text from Script Editor, and place the text file in the Speakable Items folder, Script Editor will open the text file… instead of running the script… so be diligent about how you save.

‘Due’ (as in ‘Do’) works better than ‘On’ or ‘Computer’ as a reliable keyword.

Peter B.

#15

YAAAAY!!!

I had saved it as text, without realizing it. I didn’t know the right extension to look for!

Everything works now, and boy is that going to be easier on my arms!

I don’t have many skills to offer that can be handled through a computer, but I do make quilts. Would you like a lap or baby quilt?

Polly

[sending this with a voice click!]

#16

Hey, hey…

Polly:

Congrats !!!

I knew you’d get there…

Over the last few days ‘immersed’ in Speakables, I’ve learned a lot… and put quite a few commonplace scripts and commands to the test, but there are loads more possibilities.

As of now, I can call up Safari, Mail, TextEdit, Script Editor, hide the frontmost app, close the frontmost window, switch to Finder, dismiss a dialog (with an ‘OK’ response), show or hide the Speech Feedback app, open the Speakable Items folder, open the Speech pref pane, click, double click or control click and…

All with spoken commands.

And I wouldn’t have bothered myself to learn how if you hadn’t posted.

Maybe I owe you that quilt.

(My Mom kept scraps for quilts, and was an accomplished seamstress, but I don’t think she ever actually tackled a quilt. She’s now long gone, but I still have the scraps.)

If you run into problems, don’t hesitate to stop back or email.

I have now changed my listening keyword to ‘Dew’ in place of ‘Due’… it’s friendlier somehow.

Have fun.

Peter B.

#17

I will, I will!

and thank you so much for taking all that time [g!].

Polly

PS if you ever want to get rid of those scraps, go to www.projectlinus.org, and look for a chapter coordinator near you. She’ll know a quilter who can turn those scraps into quilts for children in need; that’s where my quilts mostly go.

Thanks again!

Polly

#18

Happy to hear it worked out for you, Polly.

I’ve bookmarked www.projectlinus.org and will definitely have a look. It’s (past) time for some spring cleaning.

Peter B.

#19

Hi all,

I have a script I borrowed from a previous job regards setting up wireless via GUI - I’m planning to adapt it for my home network because my wife’s MacBook keeps forgetting it’s WiFi setting. See below edited version written for Tiger :

By the way I didn’t write this and I can’t say who did.
I’m trying to learn apple scripting from websites and know very little about it at the moment.

set WPAString to "XXXXX"

-- use GUI to set password - this creates an entry in the system keychain, with the airport plist

try
	tell application "System Preferences"
		activate
		set current pane to pane "com.apple.preference.network"
		tell application "System Events"
			tell application process "System Preferences"
				delay 1
				keystroke return
				click pop up button 2 of window "Network"
				click menu item "AirPort" of menu 1 of pop up button 2 of window "Network"
				delay 1
				radio button "AirPort" of tab group 1 of group 1 of window "Network"
				delay 1
				-- Now set the appropriate fields in the popup
				set value of combo box 1 of sheet 1 of window "Network" to "SSID"
				click pop up button 1 of sheet 1 of window "Network"
				click menu item "WPA Password" of menu 1 of pop up button 1 of sheet 1 of window "Network" -- Need to check the menu selection for WPA as it was originally WEP Password!
				delay 1
				set the clipboard to WPAString
				keystroke "v" using {command down}
				delay 1
				click button "OK" of sheet 1 of window "Network"
				delay 1
				click button "Apply Now" of window "Network"
				delay 1
			end tell
			tell application "System Preferences" to close the first window
		end tell
	end tell
on error
		display dialog "Setting Preferences failed"
end try