Coerce GUI scripting information into string?

I looked hard, and never found one. As I mentioned over here http://hints.macworld.com/article.php?story=20111208191312748
Apple’s programmers wrote a nice parser to generate the text in the first place, but its output was never made available externally. Now you could write an app that’d send parameters to a ‘get entire contents’ script running under the Script editor, and grab the output from that, but that’d probably end up being a horrible monster to maintain.

This version has extra code in the listToText() handler which allows it to work independently of a script editor.

on run {}
	set appAlias to choose file of type "com.apple.application-bundle" with prompt "Select an application:"
	tell application "System Events" to set {name:fileName} to appAlias
	set appName to text 1 thru -5 of fileName
	do shell script "open " & quoted form of POSIX path of appAlias
	tell me to activate
	display dialog "Display the window or tab in " & appName & " that you want to analyze and click OK." buttons {"OK", "Cancel"} default button 1
	
	tell application "System Events"
		tell application process appName
			set frontmost to true
			set {windowExists, menuExists} to {front window exists, menu bar 1 exists}
			set {winstuff, menustuff} to {missing value, missing value}
			if (windowExists) then set winstuff to my listToText(entire contents of front window)
			if (menuExists) then set menustuff to my listToText(entire contents of menu bar 1)
		end tell
	end tell
	
	tell application "TextEdit"
		activate
		make new document at the front
		set the text of the front document to winstuff & return & "-----" & return & menustuff
	end tell
	-- 	return {winstuff:winstuff, menustuff:menustuff}
end run

on listToText(entireContents) -- (Handler specialised for lists of System Events references.)
	try
		|| of entireContents -- Deliberate error.
	on error stuff -- Get the error message
	end try
	
	-- Parse the message.
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to {"{", "}"} -- Snow Leopard or later.
	set stuff to text from text item 2 to text item -2 of stuff
	
	-- If the list text isn't in decompiled form, create a script with the list in its source code, store it in a temporary file, and run "osadecompile" on the file.
	if (stuff does not contain "application process \"") then
		try
			set scpt to (run script "script
	tell app \"System Events\"
	{" & stuff & "}
	end
	end")
		on error errMsg
			set AppleScript's text item delimiters to astid
			tell application (path to frontmost application as text) to display dialog errMsg buttons {"OK"} default button 1 with icon caution
			return errMsg
		end try
		set tmpPath to (path to temporary items as text) & "Entire contents.scpt"
		store script scpt in file tmpPath replacing yes
		set stuff to (do shell script "osadecompile " & quoted form of POSIX path of tmpPath)
		set stuff to text from text item 2 to text item -2 of stuff
	end if
	
	set AppleScript's text item delimiters to "\"System Events\", "
	set stuff to stuff's text items
	set AppleScript's text item delimiters to "\"System Events\"" & linefeed
	set stuff to stuff as text
	set AppleScript's text item delimiters to astid
	
	return stuff
end listToText

Edit: There’s an upper limit to how much data the script can handle. ‘run script’ can only take so much text and the same appears to be true for the alternative, "osacompile’, even when the input’s a file. Even AppleScript Editor seems to seize up when running the script with this page in Safari.
Edit 2: I’ve made a cleaner exit for when ‘run script’ hits an error. The error message is now returned as the result of the listToText() handler and will appear in what’s passed to TextEdit.

Nigel,

You are indeed a virtuoso scripter! Thank you for this extraordinary code.

EDIT: And thank you for introducing me (and, I expect, many others) to the osadecompile command. Every time I ask even a question on this forum, I end up knowing far more about AppleScript than I did when I started.

I had to introduce myself to it! :wink: I knew of its existence, but had never had cause to use it before.

By the way, I’ve made a small edit in the script since you posted.

Great stuff, Nigel – I’ve used it already :smiley:

OK. Before knocking this subject on the head altogether, I’d like to offer my “director’s cut” version of the script. Assuming that the user’s most likely to have the target application open already, it asks him/her to choose from a list of running applications rather than to navigate to a file. The text output to TextEdit contains tabbed object specifiers rather than full references, which gives a clearer idea of the structure. For people like me who have TextEdit set up to default to “Wrap to Page”, the script makes the new window “Wrap to Window”. I prefer an ordinary handler to an explicit run handler. It keeps all the variables local and thus non-persistent.

-- By squaline (alias partron22?), emendelson, and Nigel Garvey.

on main()
	tell application "System Events" to set appNames to name of application processes whose visible is true
	
	tell application (path to frontmost application as text)
		set appChoice to (choose from list appNames with prompt "Which running application?" with title "Choose an application")
		if (appChoice is false) then error number -128
		set appName to item 1 of appChoice
		
		set setChoice to (choose from list {"Front Window", "Menu Bar"} with prompt "Which UI element set(s)?" with title "Choose an element set" with multiple selections allowed)
		if (setChoice is false) then error number -128
		set setFlag to ((setChoice contains "Front Window") as integer) + (((setChoice contains "Menu Bar") as integer) * 2)
	end tell
	
	tell application "System Events"
		tell application process appName
			set frontmost to true
			set {windowExists, menuExists} to {front window exists, menu bar 1 exists}
		end tell
	end tell
	set {winstuff, menustuff} to {"(No window open)", "(No menu!)"}
	if ((setFlag is not 2) and (windowExists)) then set winstuff to elementListing(appName, "front window")
	if ((setFlag > 1) and (menuExists)) then set menustuff to elementListing(appName, "menu bar 1")
	
	tell application "TextEdit"
		activate
		make new document at the front
		set the text of the front document to item setFlag of {winstuff, menustuff, winstuff & linefeed & "-----" & linefeed & menustuff}
		set WrapToWindow to text 2 thru -1 of (localized string "&Wrap to Window")
	end tell
	
	tell application "System Events"
		tell application process "TextEdit"
			tell menu item WrapToWindow of menu 1 of menu bar item 5 of menu bar 1
				if ((it exists) and (it is enabled)) then perform action "AXPress"
			end tell
		end tell
	end tell
	
	return -- nothing.
end main

on elementListing(appName, elementSet)
	-- Get a textual representation of the entire contents of the element set
	set stuff to text 2 thru -2 of (do shell script "osascript -e 'tell app \"System Events\" to tell application process \"" & appName & "\" to get entire contents of " & elementSet & "' -s s")
	
	-- If the representation contains chevron codes instead of keywords, create a script containing the representation in its source code, store it in the Temporary Items folder, and run "osadecompile" on it. (This may not be necessary now the representation's obtained with "osascript" instead of the error message hack.)
	if (stuff does not contain "application process \"") then
		try
			set scpt to (run script "script
	tell app \"System Events\"
	{" & stuff & "}
	end
	end")
		on error errMsg
			tell application (path to frontmost application as text) to display dialog errMsg buttons {"OK"} default button 1 with icon caution
			return errMsg
		end try
		set tmpPath to (path to temporary items as text) & "Entire contents.scpt"
		store script scpt in file tmpPath replacing yes
		set stuff to (do shell script "osadecompile " & quoted form of POSIX path of tmpPath)
		set stuff to text from text item 2 to text item -2 of stuff
	end if
	
	-- Break up the text, using "\"System Events\", " as a delimiter.
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to "\"System Events\", "
	set stuff to stuff's text items
	-- Insert a textual "reference" to the root object at the beginning of the resulting list.
	set AppleScript's text item delimiters to " of "
	set beginning of stuff to (text from text item 2 to -1 of item 1 of stuff) & "\"System Events\""
	-- Reduce the remaining "reference" fragments to object _specifiers, tabbed according to the number of elements in the references.
	set tabs to tab & tab & tab & tab & tab & tab & tab & tab & tab & tab & tab & tab
	set tabs to tabs & tabs
	repeat with i from 2 to (count stuff)
		set thisLine to item i of stuff
		set lineBits to thisLine's text items
		-- Make sure any " of "s in element names aren't mistaken for those of the reference!
		set elementCount to 0
		set nameContinuation to false
		repeat with j from 1 to (count lineBits)
			set thisBit to item j of lineBits
			if ((not ((nameContinuation) or (thisBit contains "\""))) or ((thisBit ends with "\"") and (thisBit does not end with "\\\"")) or (thisBit ends with "\\\\\"")) then
				-- thisBit is either a complete nameless-element _specifier or it ends the right way to be either a complete named one or the completion of a name.
				set nameContinuation to false
				set elementCount to elementCount + 1
				if (elementCount is 1) then set _specifier to text 1 thru text item j of thisLine
			else
				-- The next "bit" will be the continuation of a name containing " of ".
				set nameContinuation to true
			end if
		end repeat
		set item i of stuff to (text 1 thru (elementCount - 3) of tabs) & _specifier
	end repeat
	-- Coerce back to a single text, inserting line feeds between the items.
	set AppleScript's text item delimiters to linefeed
	set stuff to stuff as text
	set AppleScript's text item delimiters to astid
	
	return stuff
end elementListing

main()

Edit: A couple of minor changes in the light of the comments below.
Edit 2: Text representations of the UI element lists now obtained using “osascript” with the -s option instead of the error-message hack. Subsidiary handler renamed to reflect the modified script layout. Length of tab string doubled. McUsr’s choice-of-element-set suggestion incorporated.
Edit 3: ‘linefeed’ instead of ‘return’ in the TextEdit code, to match the linefeeds inserted by the other handler. It makes a difference if you decide to use TextWrangler instead of TextEdit for the display.
Edit 4: Here’s an alternative elementListing() handler which uses “sed” instead of vanilla AppleScript to produce the indented text. It’s very much shorter, though not very much faster. The line endings in the shell script should all be linefeeds.

on elementListing(appName, elementSet)
	return text 1 thru -2 of (do shell script ("osascript -e 'tell app \"System Events\" to tell application process \"" & appName & "\" to get entire contents of " & elementSet & "' -s s |
sed -E 's/\\\"System Events\\\", /\\\"System Events\\\"\\'$'\\n''/g ; # Put each ''list item'' on its own line.' |
sed -E '# Make two lines from the first: the first with the root reference and the second with the indented first child specifier.
	1 s/^{(.+) of ((window |menu bar [^i]).+)$/\\2\\'$'\\n'$'\\t''\\1/ ;
	1 !{ # Extract and indent the specifiers from the remaining lines:
	s/ of ((window |menu bar [^i]).+$)?/'$'\\t''/g ; # Substitute tabs for the root reference and all '' of ''s.
	:tillDone
		s/(\\\"[^'$'\\t''\\\"]+)'$'\\t''([^\\\"]*)/\\1 of \\2/g ; # Reinstate '' of ''s between quotes.
		t tillDone
	s/'$'\\t''[^'$'\\t'']+/'$'\\t''/g ; # Zap the stuff after each tab, leaving the current specifier & trailing tabs.
	s/^([^'$'\\t'']+)(['$'\\t'']+)$/\\2\\1/ ; # Swap round the specifier and the tabs.
	} ;'") without altering line endings)
end elementListing

Nigel,

That is really superb. The tabbed output is beautifully clear. The whole script is a model of intelligence and efficiency. Thank you again.

For reasons I haven’t sorted, the final script compiles beautifully in the Script Editor (under Lion), but will not compile in Script Debugger 4.5 giving several “access not allowed” errors. It will not compile in SD4.5 in Snow Leopard either, btw, so I don’t think it’s Lion; it’s SD4.5 that’s baulking. In either case when I run it in the Script Editor is says “s” is undefined

Hi, Adam.

There is no variable ‘s’, so maybe one of the apostrophe-s-es has come adrift. Just guessing, but do you have an OSAX or something with the keyword ‘stuff’? I seem to remember having one on my old 4400 which used the StuffIt engine.

On the other hand, the ‘stuff’ variable is in the previous version of the script and didn’t cause you any trouble. :confused:

The conflict with Script Debugger 4.5 is the word specifier as a variable – it’s a dictionary word in SD4.5. Changing it to “spec” does the trick for both compilation AND operation. Works a treat, Nigel.

This is a very impressive script and will help me greatly!

I did have an execution error and needed to add several ‘& tab’ pairs to the ‘set tabs’ line.

Bill

Hi. Welcome to MacScripter.

Thanks for the feedback. I’ve edited the script above in the light of your and Adam’s comments. Twelve tabs weren’t enough?! :o

Hello! :slight_smile:

I got some Internal Table overflow errors, so I tinkered the Brilliant script to just give one chunk of output at a time, either window or menu.

I hope you forgive me! :smiley:


property ScriptTitle : "UI Properties"
-- By squaline (alias partron22?), emendelson, and Nigel Garvey.
-- http://macscripter.net/viewtopic.php?id=37674
on main()
	try
		tell application "System Events"
			set appNames to name of application processes whose visible is true
			set curApp to name of first application process whose frontmost is true
		end tell
		tell application curApp
			
			set appChoice to (choose from list appNames with prompt "Which running application?" with title "Choose an application")
			
			if (appChoice is false) then error number -128
			try
				set outputType to button returned of (display dialog "Choose output" with title ScriptTitle buttons {"Cancel", "Menu", "Window"} default button 2 cancel button 1 with icon 1)
			on error
				error number -128
			end try
			
		end tell
		set appName to item 1 of appChoice
		
		tell application "System Events"
			tell application process appName
				set frontmost to true
				set {windowExists, menuExists} to {front window exists, menu bar 1 exists}
				
				set {winstuff, menustuff} to {"(No window open)", "(No menu!)"}
				if outputType is "Window" then
					if (windowExists) then
						set winstuff to my listToText(entire contents of front window)
					else
						error number 3000
					end if
				else
					if (menuExists) then
						set menustuff to my listToText(entire contents of menu bar 1)
					else
						error number 3001
					end if
				end if
			end tell
		end tell
	on error e number n
		tell application "System Events" to set frontmost of application process curApp to true
		tell application curApp to activate
		if n = 3000 then
			tell application curApp to display alert "No windows to gather data from!"
			
		else if n = 3001 then
			tell application curApp to display alert "No menu to gather data from!"
		end if
		error number -128
	end try
	
	tell application "TextEdit"
		activate
		make new document at the front
		if outputType is "Window" then
			set the text of the front document to winstuff
		else
			set the text of the front document to menustuff
		end if
		set WrapToWindow to text 2 thru -1 of (localized string "&Wrap to Window")
	end tell
	
	tell application "System Events"
		tell application process "TextEdit"
			tell menu item WrapToWindow of menu 1 of menu bar item 5 of menu bar 1
				if ((it exists) and (it is enabled)) then perform action "AXPress"
			end tell
		end tell
	end tell
	tell application curApp to activate
	do shell script "open -b \"com.apple.textedit\""
	
end main

on listToText(entireContents) -- (Handler specialised for lists of System Events references.)
	try
		|| of entireContents -- Deliberate error.
	on error stuff -- Get the error message
	end try
	
	-- Parse the message.
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to {"{", "}"} -- Snow Leopard or later.
	set stuff to text from text item 2 to text item -2 of stuff
	
	-- If the list text isn't in decompiled form, create a script containing the list in its source code, store it in the Temporary Items folder, and run "osadecompile" on it.
	if (stuff does not contain "application process \"") then
		try
			set scpt to (run script "script
	tell app \"System Events\"
	{" & stuff & "}
	end
	end")
		on error errMsg
			set AppleScript's text item delimiters to astid
			tell application (path to frontmost application as text) to display dialog errMsg buttons {"OK"} default button 1 with icon caution
			return errMsg
		end try
		set tmpPath to (path to temporary items as text) & "Entire contents.scpt"
		store script scpt in file tmpPath replacing yes
		set stuff to (do shell script "osadecompile " & quoted form of POSIX path of tmpPath)
		set stuff to text from text item 2 to text item -2 of stuff
	end if
	
	-- Break up the text, using "\"System Events\", " as a delimiter.
	set AppleScript's text item delimiters to "\"System Events\", "
	set stuff to stuff's text items
	-- Insert a textual "reference" to the root object at the beginning of the resulting list.
	set AppleScript's text item delimiters to " of "
	set beginning of stuff to (text from text item 2 to -1 of item 1 of stuff) & "\"System Events\""
	-- Reduce the remaining "reference" fragments to object specifiers, tabbed according to the number of elements in the references.
	set tabs to tab & tab & tab & tab & tab & tab & tab & tab
	set tabs to tabs & tabs
	set tabs to tabs & tabs -- 32 tabs should be enough!
	repeat with i from 2 to (count stuff)
		set thisLine to item i of stuff
		set lineBits to thisLine's text items
		-- Make sure any " of "s in element names aren't mistaken for those of the reference!
		set elementCount to 0
		set nameContinuation to false
		repeat with j from 1 to (count lineBits)
			set thisBit to item j of lineBits
			if ((not ((nameContinuation) or (thisBit contains "\""))) or ((thisBit ends with "\"") and (thisBit does not end with "\\\"")) or (thisBit ends with "\\\\\"")) then
				-- thisBit is either a complete nameless-element specifier or it ends the right way to be either a complete named one or the completion of a name.
				set nameContinuation to false
				set elementCount to elementCount + 1
				if (elementCount is 1) then set spec to text 1 thru text item j of thisLine
			else
				-- The next "bit" will be the continuation of a name containing " of ".
				set nameContinuation to true
			end if
		end repeat
		set item i of stuff to (text 1 thru (elementCount - 3) of tabs) & spec
	end repeat
	-- Coerce back to a single text, inserting line feeds between the items.
	set AppleScript's text item delimiters to linefeed
	set stuff to stuff as text
	set AppleScript's text item delimiters to astid
	
	return stuff
end listToText

main()

I’ve made a few improvements to the script in post #17. Details at the bottom of the post.

In the latest version of Lion (10.7.5) and Script Debugger 5.0.4 the line (which is fine in AppleScript Editor):


if (elementCount is 1) then set specifier to text 1 thru text item j of thisLine

fails because specifier is a key word in SD’s dictionary. Changing specifier to _specifier everywhere solves that problem for SD users.

Further of course, users of this script must have a copy of Nigel’s Insertion Sort script or the line loading it will fail. Here’s my copy:

(* Insertion sort
Algorithm: unknown author.
AppleScript implementation: Arthur J. Knapp and Nigel Garvey, 2003.

Parameters: (list, range index 1, range index 2)
*)

on insertionSort(theList, l, r)
	script o
		property lst : theList
	end script
	
	-- Process the input parmeters.
	set listLen to (count theList)
	if (listLen > 1) then
		-- Negative and/or transposed range indices.
		if (l < 0) then set l to listLen + l + 1
		if (r < 0) then set r to listLen + r + 1
		if (l > r) then set {l, r} to {r, l}
		
		-- Do the sort.
		set u to item l of o's lst -- The highest value sorted so far!
		repeat with j from (l + 1) to r
			-- Get the next unsorted value.
			set v to item j of o's lst
			if (v < u) then
				-- If it's less than highest sorted value, initialise the insertion location to the beginning of the range. This will usually be changed below when a more suitable slot's found.
				set here to l
				-- Work back through the already sorted items, moving up those with greater values than this one, until a lesser or equal value or the beginning of the range is reached.
				set item j of o's lst to u
				repeat with i from (j - 2) to l by -1
					tell item i of o's lst
						if (it > v) then
							-- Greater value. Move it up one position.
							set item (i + 1) of o's lst to it
						else
							-- Lesser or equal value. Set the vacated slot after it as the insertion location.
							set here to i + 1
							exit repeat
						end if
					end tell
				end repeat
				-- Insert the value for insertion at the appropriate location.
				set item here of o's lst to v
			else
				-- The value's greater than or equal to the highest sorted so far. It's now the highest itself.
				set u to v
			end if
		end repeat
	end if
	
	return -- nothing.
end insertionSort

property sort : insertionSort

(* Demo:
set l to {}
repeat 1000 times
	set end of my l to (random number 1000)
end repeat

sort(l, 1, -1)
l
*)

Thanks, Adam. I’d regard the ‘specifier’ thing as a fault in Script Debugger. Its own keywords shouldn’t get in the way of the scripts it’s running or editing. But I’ve changed ‘specifier’ to ‘_specifier’ in the posted code.

The sort is just a cosmetic nicety from my own copy of the script and I’ve now cut it from the posted version.

What’s interesting about “specifier” is that it works in OS X 10.8, but not in 10.7 or 10.6. What happens is that the word specifier is changed to the word reference by the compiler. For some reason the AppleScript Editor deals with that, but Script Debugger does not. The author of SD is looking into it.

I assume you meant to include a “not” in there, but what you’re asking for is probably impossible given the way AS works. Here’s a counter example:

set language to 5

That compiles fine in Script Debugger 5, with language as a variable, but it won’t compile in ASE because language is one of its keywords. I don’t think there’s anyway around that sort of thing, other than having a minimal dictionary and using multi-word terms as much as possible. (Earlier versions of Script Debugger used to offer the option of turning the dictionary off to avoid the problem, but I don’t know that that is practical with Cocoa scripting.)

The problem with specifier stems in part from the decision at some stage to change the terminology used in AS dictionaries. Where once, for example, the move command would specify a direct parameter of type reference, it now specifies type specifier. But the specifier type isn’t actually defined anywhere, so it’s a type that’s a bit in no-man’s land.

Script Debugger has actually defined the specifier type in its dictionary, and I suspect that’s a sensible thing to do. Making it a synonym for reference was required to avoid the term reference compiling as specifier because they both point to the same thing (Cocoa’s NSScriptObjectSpecifier).

Probably more than anyone wanted to know…

Oops. Yes. I was concentrating so hard on trying to get “its” and “it’s” right, I missed that. Now corrected.

Thanks for the background. I hadn’t noticed the ‘language’ thing before.

In sd-talk today, Matt Neuburg points out that on p. 331 of his “AppleScript: The Definitive Guide” (which I have) in the chapter on Dictionaries and wildcards, he points out: