Spurious result of the osadecompile command with an ASObjC script

Bash’s osadecompile command is a principal tool used to retrieve the source text of a compiled AppleScript or ASObjC script. I recently discovered that osadecompile strips the mandatory pipes from the ASObjC keyword |description| in the returned source text. In contrast, the Script Editor and Script Debugger script editors preserve |description|'s pipes in their document windows and when saving the script back to disk.

By way of demonstration, save the following ASObjC script as a file named DemoFile.scpt at any arbitrary location on disk:

use framework "Foundation"
use scripting additions
property || : current application

set aliasFileURL to (||'s |NSURL|)'s fileURLWithPath:"/POSIX/path/to/any/Finder/alias/file"
set resolvedFileURL to (||'s |NSURL|)'s URLByResolvingAliasFileAtURL:aliasFileURL options:0 |error|:(missing value)
set fileDescription to resolvedFileURL's |description|()
display dialog (fileDescription as text)

The example was chosen because it contains various forms of piped names. Open the saved DemoFile.scpt file in Script Editor or Script Debugger. The pipes are preserved in all piped names, including the |description| method name.

Now apply the osadecompile command to the saved DemoFile.scpt file. The |description| method name’s pipes are now stripped:

do shell script "osadecompile " & ([/POSIX/path/to/DemoFile.scpt])'s quoted form
--> strips the pipes from the description method name: ...set fileDescription to resolvedFileURL's description()...

When DemoFile.scpt is run as a script from the source text opened in the Script Editor or Script Debugger document window with |description|'s pipes preserved, the script executes properly:

run script "use framework \"Foundation\"
use scripting additions
property || : current application

set aliasFileURL to (||'s |NSURL|)'s fileURLWithPath:\"/POSIX/path/to/any/Finder/alias/file\"
set resolvedFileURL to (||'s |NSURL|)'s URLByResolvingAliasFileAtURL:aliasFileURL options:0 |error|:(missing value)
set fileDescription to resolvedFileURL's |description|()
display dialog (fileDescription as text)"
--> displays the description of an NSURL object of the underlying file of a Finder alias file, if a valid alias file POSIX path was supplied for /POSIX/path/to/any/Finder/alias/file

In contrast, when DemoFile.scpt is run as a script from the source text returned by osadecompile with |description|'s pipes stripped, the script fails:

run script "use framework \"Foundation\"
use scripting additions
property || : current application

set aliasFileURL to (||'s |NSURL|)'s fileURLWithPath:\"/POSIX/path/to/any/Finder/alias/file\"
set resolvedFileURL to (||'s |NSURL|)'s URLByResolvingAliasFileAtURL:aliasFileURL options:0 |error|:(missing value)
set fileDescription to resolvedFileURL's description()
display dialog (fileDescription as text)"
--> fails with an error because of the missing pipes in the description method name

This is the only case I’ve encountered over the years where osadecompile returns source text different from the form in which the script was originally saved, and in this case sufficiently different that it causes the script to fail when run from the source text. It raises the questions: Why does osadecompile strip only |description|'s pipes but not those of any other names (at least that I’m aware of)? Should this be considered a bug?

I suspect that osadecompile doesn’t so much strip the pipes as fail to insert them, but I’m not sure why |description| should be the only term affected (as far as is known.)

The OSAKit equivalent code seems to work OK. This from an experimental script I happen to have lying around:

use AppleScript version "2.4" -- Mac OS 10.11 (Yosemite) or later.
use framework "Foundation"
use framework "OSAKit" -- Reported to be public, but not documented in Xcode.
use scripting additions

main()
display dialog result

on main()
	set savingToNewFiles to true -- Change to false to replace the originals
	
	set |⌘| to current application
	set scriptPath to POSIX path of (choose file of type {"com.apple.applescript.script", "com.apple.applescript.script-bundle", "com.apple.application-bundle", "com.apple.applescript.text"})
	----------
	-- These three lines nicked from Jonas Whale and restyled.
	set scriptURL to (|⌘|'s class "NSURL"'s fileURLWithPath:(scriptPath))
	set {thisScript, theError} to |⌘|'s class "OSAScript"'s alloc()'s initWithContentsOfURL:(scriptURL) |error|:(reference)
	if (thisScript is missing value) then error (theError's localizedDescription()) as text
	----------
	-- Get this script's source code.
	return thisScript's source() as text
end main

However, this only works properly in an editor. If the code’s saved as an application and run independently, the dialog displayed shows description unpiped. It seems editors have some additional means of working out the context for the pipes.

The AppleScript compiler strips out any pipes it considers unnecessary. From its point of view, unnecessary means (a) they wrap a valid variable name all in lowercase, and (b) there is no terminology conflict.

You can see the first point by adding a call to, say, the hash method. Add pipes, and they are stripped when you compile.

In the case of description, it requires pipes in both Script Editor and Script Debugger because they both define the term in their own dictionaries. Copy the script into an editor without a dictionary, like Script Geek, and the pipes around description are stripped on compiling.

When you use a command-line tool like osadecompile, it runs in its own context, so the pipes are stripped. However when you use AppleScript’s run script command, the script is run in the context of the calling application, and subject to potential terminology conflicts. Again, your final run script example runs fine in Script Geek, where there is no terminology conflict.

It’s yet another reason why run script should generally be reserved for cases where it’s required.

Nigel, thank you for your helpful comments and demonstration script.

Shane, thank you for the explanation of when and where pipes come from. I had had the mistaken impression that all piped keywords in Script Editor and Script Debugger were universally required. But now I realize that pipes are required only to resolve terminology conflicts within the specific terminology space of whatever it is that is processing the script source code. The following example demonstrates that the description keyword does not require pipes when processed with the osadecompile and osacompile commands and the AppleScript compiler, presumably because no description keyword terminology conflict exists for those processors. The DemoFile.scpt from the previous example is first decompiled into its source text by means of an osadecompile command. It is then recompiled and written back to the DemoFile.scpt file location by means of an osacompile command, overwriting the original file. When run, the overwritten script file executes properly and displays the NSURL object’s description in a dialog window, all accomplished without description pipes:

do shell script "export LANG='en_US.UTF-8'; osadecompile " & ("/POSIX/path/to/DemoFile.scpt")'s quoted form & " | osacompile -o " & ("/POSIX/path/to/DemoFile.scpt")'s quoted form
run script "/POSIX/path/to/DemoFile.scpt" --> succeeds

I take to heart your advice of confining the use of run script on osadecompile output to cases where it’s required. If that situation did arise though, what would one do? One possibility might be to pass the output of the osadecompile command through a sed command in order to restore pipes to the description keyword, specifically when it appears in the form: description(). The viability of this approach would depend on the description command not appearing in the script in any other form. (Frankly, I don’t have a feel for how much one can depend on this to be the case.) For example:

set processedSourceText to do shell script "export LANG='en_US.UTF-8'; osadecompile " & ("/POSIX/path/to/DemoFile.scpt")'s quoted form & " | sed -E 's/[[:<:]]description\\(\\)/|description|()/g'"
run script processedSourceText --> succeeds

Another approach, again for the case where run script of osadecompile output is required, involves opening the script in Script Editor and retrieving the script’s source text from the opened document’s text content. The following handler, named scriptText, accomplishes this. Execution of the handler is reasonably snappy, generally returning the source code of even large scripts in less than a second.

on scriptText(scriptRef)
	-- Returns the source text of an AppleScript or ASObjC script by opening the script in Script Editor and reading the opened document's text content
	(*
	INPUT:
		POSIX path, HFS path, AppleScript alias, POSIX file («class furl»), or NSURL object pointing to an AppleScript or ASObjC text file (e.g., .applescript), compiled script file (.scpt), compiled script bundle (.scptd), or application bundle (.app)
	OUTPUT:
		the source text of the input script
	*)
	-- Wrap the code in a considering block to prevent spurious behavior, and a try block to capture any errors
	considering diacriticals, hyphens, punctuation and white space but ignoring case and numeric strings
		try
			-- Validate and process the input script reference
			tell scriptRef
				try
					set scriptAlias to it as alias
				on error
					try
						set scriptAlias to it as POSIX file as alias
					on error
						try
							if (its isKindOfClass:((current application's |NSURL|)'s |class|())) then set scriptAlias to its |path|() as text as POSIX file as alias
						on error
							error "The input file was not found."
						end try
					end try
				end try
			end tell
			tell (scriptAlias's POSIX path) to if (it does not end with ".scpt") and (it does not end with ".scpt/") and (it does not end with ".scptd") and (it does not end with ".scptd/") and (it does not end with ".app") and (it does not end with ".app/") and (it does not end with ".applescript") and (it does not end with ".applescript/") then error "The input file must have one of the following file extensions:" & return & return & tab & ".scpt" & return & tab & ".scptd" & return & tab & ".app" & return & tab & ".applescript"
			-- Open the input script in Script Editor, and get its source text from the text content of the document window
			try
				tell application "Script Editor"
					if it is not running then
						launch
						activate
					end if
					-- Handle the case where the script was already open in Script Editor
					set scriptText to missing value
					repeat with d in documents
						try
							if (d's path as POSIX file as alias) = scriptAlias then
								set scriptText to d's text
								exit repeat
							end if
						end try
					end repeat
					-- Handle the case where the script was not open in Script Editor
					if scriptText = missing value then
						tell (open scriptAlias)
							set scriptText to its text
							close saving no
						end tell
					end if
				end tell
			on error m number n
				error ("Could not get the input script's source text because of the following error:" & return & return & "(" & n & ") " & m)
			end try
			-- Return the result
			return scriptText
		on error m number n
			if n = -128 then error number -128
			if n ≠ -2700 then set m to "(" & n & ")  " & m
			error ("Problem with handler scriptText:" & return & return & m) number n
		end try
	end considering
end scriptText

set processedSourceText to my scriptText("/POSIX/path/to/DemoFile.scpt")
run script processedSourceText --> succeeds

Duh! Of course. Thanks, Shane.

If you’re using osadecompile, the safest thing would be to use osacompile to compile the code to a file before running it. Otherwise use either NSAppleScript or OSAScript to decompile (both have a source method), which will use the same context as run script.

But I’m scratching my head for why. My first thought in such a situation would be along the lines of how the heck did I end up here?

The other issue is that you should exercise some caution with the description method, because it can change what is returned. In fact it has changed with NSData recently, catching out several apps that relied on it.

And although you came upon the issue with the description method, there’s quite a bit of other terminology that could result in exactly the same behavior.

That makes perfect sense, since osacompile and osadecompile must share the same terminologies. Much better than trying to transform the osadecompile’s output into a form suitable for run script.

Likewise, that’s more elegant than my clunky approach of extracting source text from a Script Editor document window.

Thank you for both of those helpful pointers.

Thanks for pointing out that nuance about the value returned by the description method.

Red flags and warning bells were going off as I was putting together the sed modification of the osadecompile output. It just seemed too risky that things might get modified that shouldn’t and vice versa. Thank you again for pointing out osadecompile->osacompile and NSAppleScript/OSAScript->run script, which are the much better way to go.

Regarding the latter matter, as an aside, could I trouble you for a brief explanation of the difference between the NSAppleScript and OSAScript classes? Is NSAppleScript effectively a subclass of OSAScript? In what ways do the executable script objects created by the two classes differ from one another?

I’ve worked on a variety of metaprogramming projects over the years, consisting either of code generating and executing code, or code decompiling an existing script into its source code and executing a modified version of that code. Although this risk-prone work is not for general consumption, I’ve been able to achieve some rather creative scripting solutions with these techniques. One spin-off of this work was the idea I posted of passing an argument directly to an AppleScript application at launch time as a single text string containing a serialized form of the argument. Another spin-off was the runScript handler that I recently posted that mimics AppleScript’s run script command but with a 10-15x lower execution overhead (at least for all the script runners that I tested other than the Script Editor application which, as you and Nigel pointed out, does not exhibit any of that excess overhead; I’m curious what Script Editor does under the hood to avoid the overhead of other script runners, but that is the topic of another conversation.) But the project that directly precipitated the current post was work I’ve been doing on a general-purpose script-running handler that incorporates the capabilities of the run, run script, and osascript commands; allows arguments to be passed as standard AppleScript values; and is able to run code in either the foreground or the background. In particular, I wanted the handler to be able to accept as input not only scripts as source text and scripts saved as files but also compiled script objects generated within code but not saved to disk. Running the latter script objects in the foreground is simple enough to do with AppleScript’s run command. But running those script objects in the background is a different matter. One way to run them in the background would be to save the script object to a temporary file and then execute the temporary file as a background osascript. (This, by the way, was the impetus behind the solution I recently posted to the problem of AppleScript’s store script command breaking when in the scope of a use framework “Foundation” command.) I wasn’t thrilled with that approach because of the potential for cluttering up the /tmp or Temporary Items folder with a large number of temporary files if many scripts were run by the handler in a short time frame. One way around that problem would be to save the script object to a temporary file, get the temporary file’s source text via an osadecompile command, delete the temporary file immediately thereafter, and then run the source text as a background osascript. That was the plan, at least, until I ran into the problem of osadecompile’s output potentially being unsuitable for execution by osascript, the very problem that prompted the current post.

NSAppleScript is part of Foundation, and covers the basics of compile/decompile/run. OSAScript is part of its own framework, which is basically a collection of wrappers around more of the main Carbon APIs. It’s what Script Editor uses. Much of it looks like it was originally put together for Script Editor’s use, but presumably it was made public, at least parts of it, so others could also avoid having to dive into the Carbon APIs. Where NSAppleScript and OSAScript have matching methods, I suspect they’re identical.

It’s no big deal – apps do it all the time. Just make your own folder in there and save them all to it.

Not to be too pedantic, but what you’re talking about is running scripts in a separate process, not in the background. The distinction is important, because there can be implications when actually running scripts in the background if they use AppKit.

So sorry for the delayed response…life’s obligations!

Thank you for the cool insights about NSAppleScript and OSAScript (only wish the latter were documented), and also for the neat advice about creating my own temporary folder to keep the main folder from getting cluttered. That will be very useful.

By running an osascript in the background, I mean the following:

do shell script "osascript /path/to/script &>/dev/null &"

Doesn’t the osascript run fully in the background when run in that fashion?

I’ve observed that when GUI-scripting scripts are run in the background using the above do shell script construct, they often run in a jittery fashion or not at all. Is it for this same reason that one should avoid running AppKit methods in the background, or are there some other undesirable side-effects?

The term background has different contexts here. osascript runs as a background app, but that doesn’t mean its scripts are run in the background – they are still run on its main thread. Whereas scripts run in Script Editor or Script Debugger are run in the background, on a separate thread, to kepp their interfaces responsive (and allow things like having more than one script running at a time). (You can force foreground operation in Script Editor by holding the control key and choosing Run in Foreground.)

It’s a separate issue, as above. I suspect it’s because osascript is faceless and therefore passes any display stuff off to some other process.

Ah, yes, I get the difference you are pointing out between main vs background thread execution.

Not to prolong this discussion, but just to say that I am fascinated by the examples of true background AppleScript script execution you demonstrated so nicely in a prior post. I put together a handler that wraps your code. Your comment in that post about it not being practical seemed a bit understated. For instance, I could envision doing things like downloading large files from the net or performing CPU-intensive tasks in a background AppleScript script while simultaneously handling user responses in dialog window prompts, then using the results of the background script minus the down time of those user responses. But perhaps this might be the topic of a future discussion?

I would like to summarize the key lessons learned from this helpful discussion in case they are of benefit to other scripters.

The scenario to which these lessons apply consists of the case where one wishes to programatically (A) retrieve the source text of an AppleScript or ASObjC script saved to disk, (B) modify the retrieved text with a text-editing tool like Bash’s sed command (or many others), and then (3) execute the modified source text using AppleScript’s run script command or Bash’s osascript command.

Either of the following approaches works well:

Using osadecompile:

(A) Retrieve the saved script’s source text with Bash’s osadecompile command.
(B) Modify the retrieved source text in any way desired, for example, with Bash’s sed command.
(C) Save the modified source text to disk as a compiled script file with Bash’s osacompile command. Do not use AppleScript’s store script command to save the file to disk, and do not execute the modified source text directly with AppleScript’s run script command or Bash’s osascript command, unless you are sure there are no terminology conflicts like the one found for ASObjC’s description method that prompted the current post in the first place.
(D) Execute the saved script file with AppleScript’s run script command or Bash’s osascript command.

Using OSAScript:

(A) Retrieve the saved script’s source text with ASObjC’s OSAScript class (apologies, Nigel, that I didn’t at first but now fully appreciate how helpful your suggested script is for this approach):

use framework "Foundation"
use framework "OSAKit"
use scripting additions

set scriptURL to (current application's |NSURL|)'s fileURLWithPath:("/POSIX/path/to/your/saved/script")
set scriptObj to (current application's OSAScript)'s alloc()'s initWithContentsOfURL:(scriptURL) |error|:(missing value) -- I omitted error checking because I could almost never get this method to err!
set scriptSourceText to scriptObj's source() as text

(B) Modify the retrieved source text in any way desired, for example, with Bash’s sed command.
(C) If you wish to execute the modified source text directly:
Execute the modified source text directly with AppleScript’s run script command or Bash’s osascript command (in the latter case, after escaping literal backslash and double-quote characters.)
-OR-
If you wish the save the modified source text to disk before executing it:
Save the modified source text to disk as a compiled script file with AppleScript’s store script command. Do not use Bash’s osacompile command to save the file to disk unless you are sure there are no terminology conflicts like the one found for ASObjC’s description method that prompted the current post in the first place. Then execute the saved script file with AppleScript’s run script command or Bash’s osascript command.

In most cases, the OSAScript approach seems to be the way to go for three reasons: (1) its terminology is shared with AppleScript’s store script and run script commands and Bash’s osascript command; (2) it can therefore be executed directly without fear of terminology conflicts and without having to save it first to disk; and (3) it retrieves the source text more quickly than the osadecompile command because it avoids the overhead of the do shell script command needed for osadecompile.

One scenario where the osadecompile approach might be preferable would be if the decompilation, editing, recompilation, and execution were all done within a single do shell script command, and one didn’t wish to execute the source text directly but rather only after saving it to disk as a script file. This might take a form like the following:

do shell script "" & ¬
	"sourceText=$(osadecompile '/path/to/original/script/file.scpt')" & linefeed & ¬
	"modifiedSourceText=$(sed -E 's/[blah]/[blah blah blah]/' <<<\"$sourceText\")" & linefeed & ¬
	"osacompile -o '/path/to/modified/script/file.scpt' <<<\"$modifiedSourceText\"" & linefeed & ¬
	"osascript '/path/to/modified/script/file.scpt'"

Edit note: I added the final “do shell script” example after initially submitting this post.