Wednesday, December 8, 2021

#1 2021-10-18 12:51:46 am

comment
Member
Registered: 2006-11-18
Posts: 24

Separate stdout and stderr from do shell script

I have a script that calls a shell script in the form of:

set response to do shell script "mycommand -optionA -optionB"

The shell script can produce output in either or both stdout and stderr. I know I can  get both by doing :

set response to do shell script "mycommand -optionA -optionB 2>&1"

but the response is a single text object and I don't know how to separate the two parts. Is there a way to make the command insert some delimiting character in-between, so that I can safely parse out the individual components? I am looking for something like:

set response to do shell script "mycommand -optionA -optionB 2>&1"
set stdout to ???
set stderr to ???

Is this possible (without saving the response to file/s)?

Offline

 

#2 2021-10-19 04:26:37 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

This discussion:
https://stackoverflow.com/questions/110 … -variables
Will get you two shell variables, one containing stdout and the other containing stdin. Then just append another command returning the two, with whatever you want to use a delimiter inserted between them.

Then just use the delimiter to parse them back out into separate AS variables after they're returned from the shell.


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#3 2021-10-20 01:32:57 am

comment
Member
Registered: 2006-11-18
Posts: 24

Re: Separate stdout and stderr from do shell script

Thanks. The linked page has 16 answers, I am not sure which one to look at.

More importantly, they all seem to be discussing a bash script file. I don't know how to implement the advice in AppleScript. I have used the 'do shell script' command many times, but always with a single command only. Any chance for a more detailed guidance on this?

Offline

 

#4 2021-10-20 06:00:17 am

Mockman
Member
From:: Toronto
Registered: 2020-05-27
Posts: 156

Re: Separate stdout and stderr from do shell script

Can you provide a generic shell command example that generates two streams of text?

Online

 

#5 2021-10-20 06:16:16 am

Marc Anthony
Member
From:: Dallas, TX
Registered: 2006-04-27
Posts: 1052

Re: Separate stdout and stderr from do shell script

Below is an example for separate files using find. If there's no error, only out is populated.

Applescript:

try
   do shell script "find " & ((path to desktop)'s POSIX path)'s quoted form & " -name '*.pdf' " & "1>" & ((path to desktop as text) & "stdout.txt")'s POSIX path's quoted form & space & "2>" & ((path to desktop as text) & "stderr.txt")'s POSIX path's quoted form
end try

Last edited by Marc Anthony (2021-10-20 06:21:40 am)

Offline

 

#6 2021-10-20 07:10:40 am

comment
Member
Registered: 2006-11-18
Posts: 24

Re: Separate stdout and stderr from do shell script

Mockman wrote:

Can you provide a generic shell command example that generates two streams of text?

I can do it using the 'xsltproc' command - but you would need a stylesheet that sends a message to reproduce this.

Marc Anthony wrote:

Below is an example for separate files using find. If there's no error, only out is populated.

Thank you for this. I may end up using this if there is no way to accomplish the task without saving to file/s.

Last edited by comment (2021-10-20 01:44:00 pm)

Offline

 

#7 2021-10-21 09:43:11 am

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

I tested this on "ls Users bob" to generate both STDERR and STDOUT, "ls Users" for just STDOUT, and "ls bob" for just STDERR. (If you have a "bob" folder in your root directory, adjust accordingly.)

Applescript:



set yourShellCommand to "ls Users bob"
set yourSTDERRmarker to "-DELIMITSTDERR-"
set yourSTDOUTmarker to "-DELIMITSTDOUT-"
set shellLine to "/bin/bash -c '" & yourShellCommand & " 2> >(sed '1s/^/" & yourSTDERRmarker & "/') > >(sed '1s/^/" & yourSTDOUTmarker & "/')'"
try
   set shellReturn to do shell script shellLine
on error errorMessage
   set shellReturn to errorMessage
end try

set delimitHolder to AppleScript's text item delimiters
if shellReturn contains yourSTDERRmarker then
   set AppleScript's text item delimiters to yourSTDERRmarker
   set separatedResponse to text items of shellReturn
   if shellReturn contains yourSTDOUTmarker then
       set STDERR to text item 2 of separatedResponse
       set AppleScript's text item delimiters to yourSTDOUTmarker
       set STDOUT to text item 2 of item 1 of separatedResponse
   else
       set STDOUT to missing value
       set STDERR to text item 2 of shellReturn
   end if
else if shellReturn contains yourSTDOUTmarker then
   set AppleScript's text item delimiters to yourSTDOUTmarker
   set STDERR to missing value
   set STDOUT to text item 2 of shellReturn
else
set STDERR to missing value
set STDOUT to missing value
end if

set AppleScript's text item delimiters to delimitHolder

display dialog "Your shell script returned" & return & "STDOUT:" & return & STDOUT & return & "STDERR:" & return & STDERR

EDIT: at first, I didn't handle the case where the shell returns nothing to both STDERR and STDOUT. So I added that, so it at least still defines the output variables in that case.

SECOND EDIT: although in retrospect, that last edit was probably unnecessary, because even if the command returns a blank to STDOUT, like
echo ''
SED will still append the delimiter to the front... so I don't think it's possible for this script to return an entirely blank shell response.

Last edited by t.spoon (2021-10-21 07:01:50 pm)


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#8 2021-10-21 10:05:36 am

KniazidisR
Member
From:: Greece
Registered: 2019-03-03
Posts: 2182

Re: Separate stdout and stderr from do shell script

Thanks, @t.spoon and @Marc Anthony. Your last posts here are very useful.


Model: MacBook Pro
OS X: Catalina 10.15.7
Web Browser: Safari 14.1
Ram: 4 GB

Offline

 

#9 2021-10-21 10:08:06 am

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

Incidentally, I was surprised that when the shell STDERR was redirected to SED, that Applescript still threw an error running the shell script. As is well known, redirecting STDERR to /dev/null stops an error being thrown, so I was surprised that redirecting it to sed still returned it as an error. I guess after SED, the output is still being printed to STDERR, not STDOUT... it's just being formatted by SED first. Luckily, the contents of STDOUT are still returned as well.

Presumably, there is some way in Shell to perform multiple diversions; first divert STDERR to SED, then redivert the text modified in SED to STDOUT. I couldn't quickly get syntax for that working, and what I posted works, so I dropped it for now.

Last edited by t.spoon (2021-10-21 10:08:36 am)


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#10 2021-10-21 01:19:23 pm

comment
Member
Registered: 2006-11-18
Posts: 24

Re: Separate stdout and stderr from do shell script

@t.spoon

This seems to be working very well, thank you very much.

Except for one snag: my command is in this form:

xsltproc '/path/to/stylesheet.xsl' '/path/to/source.xml'

When the actual path contains folders with space in their name, I get an error like:

warning: failed to load external entity "/path/to/xyz"
cannot parse /path/to/xyx

where 'xyz' is the part of the folder name before space.

The quoted forms of the paths worked well for me before, but it seems they do not work in bash.

How should I escape the spaces when doing this part:

set yourShellCommand to 

Offline

 

#11 2021-10-21 01:53:35 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

Right, the application is already using single quotes in the shell line. "Quoted form of" puts the arguments in single quotes, which ends up with the opening single quote of the path being interpreted as the closing single quote of the command.

Just use double-quotation marks for the paths and escape them yourself. This is working for me for a path with spaces:



Applescript:

set yourShellCommand to "ls \"Applications/AS Timer.app\""
set yourSTDERRmarker to "-DELIMITSTDERR-"
set yourSTDOUTmarker to "-DELIMITSTDOUT-"
set shellLine to "/bin/bash -c '" & yourShellCommand & " 2> >(sed '1s/^/" & yourSTDERRmarker & "/') > >(sed '1s/^/" & yourSTDOUTmarker & "/')'"
try
   set shellReturn to do shell script shellLine
on error errorMessage
   set shellReturn to errorMessage
end try

set delimitHolder to AppleScript's text item delimiters
if shellReturn contains yourSTDERRmarker then
   set AppleScript's text item delimiters to yourSTDERRmarker
   set separatedResponse to text items of shellReturn
   if shellReturn contains yourSTDOUTmarker then
       set STDERR to text item 2 of separatedResponse
       set AppleScript's text item delimiters to yourSTDOUTmarker
       set STDOUT to text item 2 of item 1 of separatedResponse
   else
       set STDOUT to missing value
       set STDERR to text item 2 of shellReturn
   end if
else if shellReturn contains yourSTDOUTmarker then
   set AppleScript's text item delimiters to yourSTDOUTmarker
   set STDERR to missing value
   set STDOUT to text item 2 of shellReturn
else
   set STDERR to missing value
   set STDOUT to missing value
end if

set AppleScript's text item delimiters to delimitHolder

display dialog "Your shell script returned" & return & "STDOUT:" & return & STDOUT & return & "STDERR:" & return & STDERR


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#12 2021-10-21 02:00:29 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

Or for your specific scenario, to not have to mess with escaping things as you type:

Applescript:




set yourShellCommand to "xsltproc"
set path1 to "[insertPathHere]"
set path2 to "[insertPathHere]"
set yourSTDERRmarker to "-DELIMITSTDERR-"
set yourSTDOUTmarker to "-DELIMITSTDOUT-"
set shellLine to "/bin/bash -c '" & yourShellCommand & " \"" & path1 & "\" \"" & path2 & "\" 2> >(sed '1s/^/" & yourSTDERRmarker & "/') > >(sed '1s/^/" & yourSTDOUTmarker & "/')'"
try
   set shellReturn to do shell script shellLine
on error errorMessage
   set shellReturn to errorMessage
end try

set delimitHolder to AppleScript's text item delimiters
if shellReturn contains yourSTDERRmarker then
   set AppleScript's text item delimiters to yourSTDERRmarker
   set separatedResponse to text items of shellReturn
   if shellReturn contains yourSTDOUTmarker then
       set STDERR to text item 2 of separatedResponse
       set AppleScript's text item delimiters to yourSTDOUTmarker
       set STDOUT to text item 2 of item 1 of separatedResponse
   else
       set STDOUT to missing value
       set STDERR to text item 2 of shellReturn
   end if
else if shellReturn contains yourSTDOUTmarker then
   set AppleScript's text item delimiters to yourSTDOUTmarker
   set STDERR to missing value
   set STDOUT to text item 2 of shellReturn
else
   set STDERR to missing value
   set STDOUT to missing value
end if

set AppleScript's text item delimiters to delimitHolder

display dialog "Your shell script returned" & return & "STDOUT:" & return & STDOUT & return & "STDERR:" & return & STDERR


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#13 2021-10-21 02:30:31 pm

comment
Member
Registered: 2006-11-18
Posts: 24

Re: Separate stdout and stderr from do shell script

t.spoon wrote:

"Quoted form of" puts the arguments in single quotes, which ends up with the opening single quote of the path being interpreted as the closing single quote of the command.

I don't know about that. It did work for me with single quotes, as long as there weren't any spaces in the folder names.

In any case, changing the single quotes to double solves the problem, so I don't know.

Just for my education: why is it necessary to switch to bash?

Offline

 

#14 2021-10-21 07:11:40 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

The default shell for "do shell script" if you don't specify one is sh. I'm not sure why it's required to change it here; I think it might be that sh doesn't allow redirects of stdout and stderr to another function? Or perhaps it's as simple as the "> >" redirect syntax I was using... I don't know, I didn't bother getting to the bottom of it.

I crafted a shell line that worked in terminal, then made Applescript that reproduced my shell line. But I got an error when running it from Applescript... I compared my script-produced shell line to what I was running in shell... they were identical. Pasted the line as produced by the script into shell and ran it and it worked... when that happens, but running the identical line via "do shell script" fails, I assume it's either a problem with running the command in a different shell, or else it's a problem with running a path that assumes a different starting point. It wasn't a path issue, so I switched the shell to bash and the error went away.


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#15 2021-10-21 07:28:44 pm

t.spoon
Member
From:: BFE, Massachusetts
Registered: 2013-01-13
Posts: 518

Re: Separate stdout and stderr from do shell script

Incidentally, I feel certain there's some way in shell to force it to always print something (the delimiter) in between the redirected output for stdout and stderr, but I'm not that great with shell and couldn't figure it out, which is why my parsing bit has so much logic.

If you want a cleaner script, you could probably post the shell line this generated to a shell forum and get some guru to tell us how to do that, and get a much cleaner script.

But at least it works.


Ditched the Hackintosh.
Intel Mac Mini i7 and M1 Mac Mini on a KVM.

Offline

 

#16 2021-10-22 07:07:42 am

comment
Member
Registered: 2006-11-18
Posts: 24

Re: Separate stdout and stderr from do shell script

All right, then I will use this for now and ask around for something simpler.

Again, thank you very much.

Offline

 

Board footer

Powered by FluxBB

RSS (new topics) RSS (active topics)