Separate stdout and stderr from do shell script

I have a script that calls a shell script in the form of:

[format]set response to do shell script “mycommand -optionA -optionB”[/format]

The shell script can produce output in either or both stdout and stderr. I know I can get both by doing :

[format]set response to do shell script “mycommand -optionA -optionB 2>&1”[/format]

but the response is a single text object and I don’t know how to separate the two parts. Is there a way to make the command insert some delimiting character in-between, so that I can safely parse out the individual components? I am looking for something like:

[format]set response to do shell script “mycommand -optionA -optionB 2>&1”
set stdout to ???
set stderr to ???
[/format]

Is this possible (without saving the response to file/s)?

This discussion:
https://stackoverflow.com/questions/11027679/capture-stdout-and-stderr-into-different-variables
Will get you two shell variables, one containing stdout and the other containing stdin. Then just append another command returning the two, with whatever you want to use a delimiter inserted between them.

Then just use the delimiter to parse them back out into separate AS variables after they’re returned from the shell.

Thanks. The linked page has 16 answers, I am not sure which one to look at.

More importantly, they all seem to be discussing a bash script file. I don’t know how to implement the advice in AppleScript. I have used the ‘do shell script’ command many times, but always with a single command only. Any chance for a more detailed guidance on this?

Can you provide a generic shell command example that generates two streams of text?

Below is an example for separate files using find. If there’s no error, only out is populated.

try
	do shell script "find " & ((path to desktop)'s POSIX path)'s quoted form & " -name '*.pdf' " & "1>" & ((path to desktop as text) & "stdout.txt")'s POSIX path's quoted form & space & "2>" & ((path to desktop as text) & "stderr.txt")'s POSIX path's quoted form
end try

I can do it using the ‘xsltproc’ command - but you would need a stylesheet that sends a message to reproduce this.

Thank you for this. I may end up using this if there is no way to accomplish the task without saving to file/s.

I tested this on “ls Users bob” to generate both STDERR and STDOUT, “ls Users” for just STDOUT, and “ls bob” for just STDERR. (If you have a “bob” folder in your root directory, adjust accordingly.)



set yourShellCommand to "ls Users bob"
set yourSTDERRmarker to "-DELIMITSTDERR-"
set yourSTDOUTmarker to "-DELIMITSTDOUT-"
set shellLine to "/bin/bash -c '" & yourShellCommand & " 2> >(sed '1s/^/" & yourSTDERRmarker & "/') > >(sed '1s/^/" & yourSTDOUTmarker & "/')'"
try
	set shellReturn to do shell script shellLine
on error errorMessage
	set shellReturn to errorMessage
end try

set delimitHolder to AppleScript's text item delimiters
if shellReturn contains yourSTDERRmarker then
	set AppleScript's text item delimiters to yourSTDERRmarker
	set separatedResponse to text items of shellReturn
	if shellReturn contains yourSTDOUTmarker then
		set STDERR to text item 2 of separatedResponse
		set AppleScript's text item delimiters to yourSTDOUTmarker
		set STDOUT to text item 2 of item 1 of separatedResponse
	else
		set STDOUT to missing value
		set STDERR to text item 2 of shellReturn
	end if
else if shellReturn contains yourSTDOUTmarker then
	set AppleScript's text item delimiters to yourSTDOUTmarker
	set STDERR to missing value
	set STDOUT to text item 2 of shellReturn
else
set STDERR to missing value
set STDOUT to missing value
end if

set AppleScript's text item delimiters to delimitHolder

display dialog "Your shell script returned" & return & "STDOUT:" & return & STDOUT & return & "STDERR:" & return & STDERR

EDIT: at first, I didn’t handle the case where the shell returns nothing to both STDERR and STDOUT. So I added that, so it at least still defines the output variables in that case.

SECOND EDIT: although in retrospect, that last edit was probably unnecessary, because even if the command returns a blank to STDOUT, like
echo ‘’
SED will still append the delimiter to the front… so I don’t think it’s possible for this script to return an entirely blank shell response.

Thanks, @t.spoon and @Marc Anthony. Your last posts here are very useful.

Incidentally, I was surprised that when the shell STDERR was redirected to SED, that Applescript still threw an error running the shell script. As is well known, redirecting STDERR to /dev/null stops an error being thrown, so I was surprised that redirecting it to sed still returned it as an error. I guess after SED, the output is still being printed to STDERR, not STDOUT… it’s just being formatted by SED first. Luckily, the contents of STDOUT are still returned as well.

Presumably, there is some way in Shell to perform multiple diversions; first divert STDERR to SED, then redivert the text modified in SED to STDOUT. I couldn’t quickly get syntax for that working, and what I posted works, so I dropped it for now.

@t.spoon

This seems to be working very well, thank you very much.

Except for one snag: my command is in this form:
[format]xsltproc ‘/path/to/stylesheet.xsl’ ‘/path/to/source.xml’[/format]

When the actual path contains folders with space in their name, I get an error like:
[format]warning: failed to load external entity “/path/to/xyz”
cannot parse /path/to/xyx[/format]
where ‘xyz’ is the part of the folder name before space.

The quoted forms of the paths worked well for me before, but it seems they do not work in bash.

How should I escape the spaces when doing this part:
[format]set yourShellCommand to [/format]

Right, the application is already using single quotes in the shell line. “Quoted form of” puts the arguments in single quotes, which ends up with the opening single quote of the path being interpreted as the closing single quote of the command.

Just use double-quotation marks for the paths and escape them yourself. This is working for me for a path with spaces:

set yourShellCommand to "ls \"Applications/AS Timer.app\""
set yourSTDERRmarker to "-DELIMITSTDERR-"
set yourSTDOUTmarker to "-DELIMITSTDOUT-"
set shellLine to "/bin/bash -c '" & yourShellCommand & " 2> >(sed '1s/^/" & yourSTDERRmarker & "/') > >(sed '1s/^/" & yourSTDOUTmarker & "/')'"
try
	set shellReturn to do shell script shellLine
on error errorMessage
	set shellReturn to errorMessage
end try

set delimitHolder to AppleScript's text item delimiters
if shellReturn contains yourSTDERRmarker then
	set AppleScript's text item delimiters to yourSTDERRmarker
	set separatedResponse to text items of shellReturn
	if shellReturn contains yourSTDOUTmarker then
		set STDERR to text item 2 of separatedResponse
		set AppleScript's text item delimiters to yourSTDOUTmarker
		set STDOUT to text item 2 of item 1 of separatedResponse
	else
		set STDOUT to missing value
		set STDERR to text item 2 of shellReturn
	end if
else if shellReturn contains yourSTDOUTmarker then
	set AppleScript's text item delimiters to yourSTDOUTmarker
	set STDERR to missing value
	set STDOUT to text item 2 of shellReturn
else
	set STDERR to missing value
	set STDOUT to missing value
end if

set AppleScript's text item delimiters to delimitHolder

display dialog "Your shell script returned" & return & "STDOUT:" & return & STDOUT & return & "STDERR:" & return & STDERR

Or for your specific scenario, to not have to mess with escaping things as you type:




set yourShellCommand to "xsltproc"
set path1 to "[insertPathHere]"
set path2 to "[insertPathHere]"
set yourSTDERRmarker to "-DELIMITSTDERR-"
set yourSTDOUTmarker to "-DELIMITSTDOUT-"
set shellLine to "/bin/bash -c '" & yourShellCommand & " \"" & path1 & "\" \"" & path2 & "\" 2> >(sed '1s/^/" & yourSTDERRmarker & "/') > >(sed '1s/^/" & yourSTDOUTmarker & "/')'"
try
	set shellReturn to do shell script shellLine
on error errorMessage
	set shellReturn to errorMessage
end try

set delimitHolder to AppleScript's text item delimiters
if shellReturn contains yourSTDERRmarker then
	set AppleScript's text item delimiters to yourSTDERRmarker
	set separatedResponse to text items of shellReturn
	if shellReturn contains yourSTDOUTmarker then
		set STDERR to text item 2 of separatedResponse
		set AppleScript's text item delimiters to yourSTDOUTmarker
		set STDOUT to text item 2 of item 1 of separatedResponse
	else
		set STDOUT to missing value
		set STDERR to text item 2 of shellReturn
	end if
else if shellReturn contains yourSTDOUTmarker then
	set AppleScript's text item delimiters to yourSTDOUTmarker
	set STDERR to missing value
	set STDOUT to text item 2 of shellReturn
else
	set STDERR to missing value
	set STDOUT to missing value
end if

set AppleScript's text item delimiters to delimitHolder

display dialog "Your shell script returned" & return & "STDOUT:" & return & STDOUT & return & "STDERR:" & return & STDERR

I don’t know about that. It did work for me with single quotes, as long as there weren’t any spaces in the folder names.

In any case, changing the single quotes to double solves the problem, so I don’t know.

Just for my education: why is it necessary to switch to bash?

The default shell for “do shell script” if you don’t specify one is sh. I’m not sure why it’s required to change it here; I think it might be that sh doesn’t allow redirects of stdout and stderr to another function? Or perhaps it’s as simple as the “> >” redirect syntax I was using… I don’t know, I didn’t bother getting to the bottom of it.

I crafted a shell line that worked in terminal, then made Applescript that reproduced my shell line. But I got an error when running it from Applescript… I compared my script-produced shell line to what I was running in shell… they were identical. Pasted the line as produced by the script into shell and ran it and it worked… when that happens, but running the identical line via “do shell script” fails, I assume it’s either a problem with running the command in a different shell, or else it’s a problem with running a path that assumes a different starting point. It wasn’t a path issue, so I switched the shell to bash and the error went away.

Incidentally, I feel certain there’s some way in shell to force it to always print something (the delimiter) in between the redirected output for stdout and stderr, but I’m not that great with shell and couldn’t figure it out, which is why my parsing bit has so much logic.

If you want a cleaner script, you could probably post the shell line this generated to a shell forum and get some guru to tell us how to do that, and get a much cleaner script.

But at least it works.

All right, then I will use this for now and ask around for something simpler.

Again, thank you very much.