GUI perform action question in Safari (QuickBooks Online)

I’m writing a script using Safari to automate tasks in QuickBooks Online.

My problem is, I have to click a line in a table to open an invoice. I have identified the table through its UI hierarchy, but clicking it has no effect. Here is my hypothesis as to why:

Before moving the mouse over the table, it looks like this:

As one moves the mouse over the table, it highlights the appropriate row, as shown:

At this point, the mouse icon also changes, indicating a hyperlink. However, absolutely no change occurs in the Accessibility Inspector while doing this.

I learned AppleScript from the “AppleScript 1-2-3” book and some supplemental Google searches. I know how to write perform action statements and how to find a list of actions from the Accessibility Inspector, but I haven’t been able to find a comprehensive list of UI actions anywhere.

The only action the inspector shows for this table is “AXShowMenu,” which is not helpful. I suspect there is also an inherited list of basic actions which contains something like “AXMouseOver.” Can anyone tell me if there is and where to find it? Or, any other clue as to why nothing is happening? (I am assuming Safari has to think I’ve moved my mouse over the table before I can click it.)

My code is below. I’ve tried moving the click command up and down in the hierarchy, and even using the position of the row to send a generic “click at {x,y}” command, all to no avail.

activate application "Safari"
tell application "System Events" to tell process "Safari" to tell front window
	set rowCount to count of rows of table 1 of group 3 of group 8 of UI element 1 of scroll area 1 of group 1 of UI element 1 of scroll area 1 of group 1 of group 1 of group 2
	if rowCount > 1 then
		repeat with i from 2 to rowCount
			tell static text 1 of UI element 3 of row i of table 1 of group 3 of group 8 of UI element 1 of scroll area 1 of group 1 of UI element 1 of scroll area 1 of group 1 of group 1 of group 2
				set invNum to value
				click
			end tell
			--statements to process the invoice and return to customer page go here
		end repeat
	else
		--statements for customers with no invoices go here
	end if
end tell

Model: Mac Mini
AppleScript: 2.1.2
Browser: Safari 534.59.8
Operating System: Mac OS X (10.6)

I’ve been experimenting with other methods of mouse control and finally got one to work. It seemed like cliclick was the best option, but I kept getting errors whenever it ran. Instead I used Extra Suites, which is an outdated application but it got the job done. The updated clicking portion of my code is:

	tell static text 1 of UI element 3 of row i of table 1 of group 3 of group 8 of UI element 1 of scroll area 1 of group 1 of UI element 1 of scroll area 1 of group 1 of group 1 of group 2
		set invNum to value
		copy position to {clickX, clickY}
	end tell
	tell application "Extra Suites"
		ES move mouse {(clickX + 5), (clickY + 5)}
		delay 0.1
		ES click mouse
	end tell

This works for me because this is a 1-run script that only I will see. (I’m doing a mass export of accounting records for a client.) It’s not a good solution if you’re writing for an end user, though, because they’d have to download the application and deal with a registration reminder while the script runs.

Take a look at this: CLICLICK (freebie)

GUI scripting could be really tricky for this, since you are scripting a web view inside Safari. What you probably want to do is use Safari’s ‘do JavaScript’ command. Once you are in the web page, the Accessibility GUI scripting won’t work (as you noticed), but you can reference objects in that page by sending JavaScript commands. Here’s an example of how something like that might work:

tell application "Safari"
    activate
    open location "http://rna.tbi.univie.ac.at/cgi-bin/RNAfold.cgi"
    delay 3
    do JavaScript "document.getElementsByName('proceed')[0].click();" in current tab of first window
end tell

That code and more are at: http://stackoverflow.com/questions/21468058/using-javascript-applescript-to-click-button-in-safari
Relevant search terms to find more examples are: AppleScript Do JavaScript Safari

I agree with ccstone, cliclick will do the job

there’s some info and code here

http://macscripter.net/viewtopic.php?pid=173692#p173692

Krioni is right is the way to go.

The UI-Inspector doesn’t find any UI-Elements in the page in Safari, because there are NO System-UI-Elements. The rendered page is separate and apart from the OSX system.