Call Runs in Terminal, but Not As a “do shell script” AppleScript

Greetings, folks!

NOTE: Apparently, as demonstrated with this line of text, the source code tag (“[code] . . . ”) is not working, even though BBCode is turned on. I am using the quote tag in its place. Thus, I cannot stop the html address from being truncated inside a pair of [url] tags in the Terminal code.[/code]

Here’s my challenge:

I have a call (“mercury-parser” via Yarn, using the full path to the call) that converts Web pages to lines of structured text (the page used in the call describes the service):

If I run it in Terminal (using either /bin/bash or /bin/sh on a 2013 Mac Pro, macOS 10.14.5), I get the proper result: fourteen lines of structured text from the Web page.

However, the very same call returns an error when run as a single-line, “do shell script” AppleScript:

do shell script "/Users/me/.yarn/bin/mercury-parser https://postlight.com/trackchanges/mercury-goes-open-source"

The error returned is:

Thoughts?

Thanks!!

I don’t have the tech savvy to explain it fully, but it has to do with the shell’s environment.
Basically, Terminal respects to current locale’s environment, but do shell script does not.
My Mac is running Dutch: running the date command in Terminal gives me “di 2 jul 2019 10:11:52 CEST”', which is the expected result.
But:

do shell script "date"
--> "Tue Jul  2 10:11:36 CEST 2019"

which is (US) english not Dutch!
Getting do shell to behave is possible:

do shell script "LANG=nl_NL date"
--> "di  2 jul 2019 10:11:36 CEST"

or locale-agnostic:

do shell script "LANG=$(defaults read -g AppleLocale) date"

I think you should try adding that to your shell command.

Sorry about that. There’s obviously some setting in the BBS software which adds [url] tags to anything recognised as a URL. I imagine it’s switch-offable, although its persisted all these years despite the complaints. I’ll drop an e-mail to Mark, who recently acquired this site, to see if there’s anything he can do about it. It used to afflict code within [applescript] tags too, but doesn’t seem to be doing so in this thread.

MacScripter doesn’t use [code] tags. It has [format] tags instead. But the situation’s no better with them. :frowning:

One workaround — although it’s a bind to do — is to put [color=black] and [/color] tags round the initial character of whatever’s giving the problem, to break up the text so that the BBS doesn’t recognise it as anything special. But if you want such text to appear in a ‘code’-like box, you have to use [quote] tags instead:

… which gives:

To find a solution to a problem, you first need to determine why this simple command
node runs fine in the Terminal window, but does not want to run in AppleScript:

do shell script "node"

The error message given above is issued by AppleScript due to this command. I don’t know the subtleties of the node utility to help you more. On my machine this tool is here:

/usr/local/Cellar/node/12.5.0/bin/node

Well, solved. This works fine:


set nodePath to "/usr/local/Cellar/node/12.5.0/bin/node "
set mercuryParserPath to "/usr/local/bin/mercury-parser "
set pageURL to "https://postlight.com/trackchanges/mercury-goes-open-source"

do shell script nodePath & mercuryParserPath & pageURL

I know that you asked the same question on Stackoverflow, without success. Although, I respect this site no less than MacScripter :slight_smile:

The reason the “do shell script” command in AppleScript differs from the terminal is because of the $PATH variable.

In terminal the $PATH variable is something like “/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin”

In the “do shell script” command the $PATH variable is “/usr/bin:/bin:/usr/sbin:/sbin”

do shell script "echo $PATH"

Because they differ, sometimes you have to use a full path to the command so your script can find it.

–edit–

Also just found out that the terminal is using “bash” shell,

Applescript “do shell script” uses “sh” shell

KniazidisR!

I’m speechless . . .

How did you know to add the path to “node” in front of the call?! There is no reference to the node command in my one-line script, so the error made no sense to me and thus prepending the line with the path to the node utility doesn’t make a lot of sense either.

(I feel like I’m a child playing checkers in a hall of chess masters . . . )

Finding the location of the node utility was easy; in Terminal just use the “type” command:

[format]type node[/format]

I got:

[format]node is /Users/me/.nvm/versions/node/v10.15.3/bin/node[/format]

Thus my updated AppleScript becomes:


set nodePath to "/Users/me/.nvm/versions/node/v10.15.3/bin/node "
set mercuryParserPath to "/Users/me/.yarn/bin/mercury-parser "
set pageURL to "https://postlight.com/trackchanges/mercury-goes-open-source"

do shell script nodePath & mercuryParserPath & pageURL

Works like a charm!

THANK YOU, KniazidisR!

Richard Fairbanks

P.S.: Nigel, bless you! Thank you for the workaround!

P.P.S.: If anyone would be interested in the script I wrote to convert a Web page to an iPhone version—trimmed extensively by the mercury-parser command—just let me know. (See the pageURL address for the details.)

Greetings, folks!

Thank you, again, to KniazidisR for solving this!

Appended below is the AppleScript that I have been using, followed by the HTML template text file, called by the script. (There is a free version of BBEdit available, for those who don’t have a license.)

The details of installing the Mercury Web Parser are at:

https://postlight.com/trackchanges/mercury-goes-open-source

There is a screen shot of the result of converting that very page, as viewed on my iPhone 7 Plus at:

https://i.stack.imgur.com/EkBy3.jpg

I have found the Mercury Web Parser an invaluable means of saving and reading Web pages on my iPhone when offline, and offer this for others who might find it useful as well.

Blessings, and thank you!, to everyone’s efforts to solve this!

Richard Fairbanks

————————

The AppleScript:


    global thePage
    
    set theTitle to ""
    set theAuthor to "(author not defined)"
    set theDate to ""
    
    tell application "Safari" to set pageURL to the URL of document 1
    
    set nodePath to "/Users/<UserName>/.nvm/versions/node/v10.15.3/bin/node "
    set mercuryParserPath to "/Users/<UserName>/.yarn/bin/mercury-parser "
    
    set thePage to (do shell script nodePath & mercuryParserPath & pageURL)
    
    delay 1
    
    tell application "BBEdit"
    	activate
    	make new text document
    	tell text window 1
    		set the last line to thePage
    		select insertion point before line 1
    		
    		#### Get the Original Page address, Titie, Author, and Date ####
    		
    		set theOriginal to characters 11 through -3 of line 9 as text -- get the address of the original page
    		find "{\"title\":\"(.+)\",\"author" searching in text 1 of text document 1 options {search mode:grep, starting at top:true, case sensitive:true} with selecting match
    		try
    			set theTitle to (characters 13 through -3 of line 2 as text)
    		end try
    		set theAuthor to characters 13 through -2 of line 3 as text
    		if theAuthor is "null" then set theAuthor to ""
    		try
    			set theDate to (characters 22 through 31 of line 4 as text) & " at " & characters 33 through -8 of line 4 as text
    		end try
    		
    		#### Clean up the code for HTML ####
    		
    		find "\"content\": \"" searching in text 1 of text document 1 options {search mode:grep, starting at top:true, case sensitive:true} with selecting match
    		select insertion point after the selection
    		delay 0.1
    		tell application "System Events" to tell process "BBEdit" to key code 126 using {command down, shift down} -- select everything up to the start of the text
    		delete the selection
    		delay 0.1
    		find "\"next_page_url" searching in text 1 of text document 1 options {search mode:grep, starting at top:true, case sensitive:true} with selecting match
    		delay 0.1
    		select insertion point before the selection
    		tell application "System Events" to tell process "BBEdit" to key code 125 using {command down, shift down} -- select everything after the HTML text
    		delay 0.1
    		delete the selection
    		delay 0.1
    		copy characters 1 through -6 of the contents
    		delay 0.1
    		select insertion point before line 1
    	end tell
    	
    	delay 0.1
    	if window "Readability Template.html" exists then revert document "Readability Template.html"
    	open file "Path:To:File:Readability Template.html"
    	delay 0.1
    	
    	tell text window 1
    		replace "<TITLE>Readability Template</TITLE>" using ("<TITLE>" & theTitle & "</TITLE>") searching in text document 1 options {search mode:literal, starting at top:true}
    		delay 0.1
    		set line 19 to (theTitle & " &nbsp; <a href=" & theOriginal & "> <small>[ORIGINAL PAGE]</small></a><br>" & return & theAuthor & "<br>" & return & theDate & "<br>" & "<br>" & return & return) as text
    		select insertion point before line 23
    		paste
    		
    		-- clean up bad text
    		-- replace unprocessed HTML characters
    		-- Not properly processed by Web Parser Code: Unicode Hex Character Code � (called "Replacement Character")
    		--> as shown parsing http://rense.com/general96/psychotronictech.htm
    		replace "^�$" using "—" searching in text document 1 options {search mode:grep, starting at top:true} -- single character line marker
    		replace ">�<" using ">•<" searching in text document 1 options {search mode:literal, starting at top:true} -- single character line marker
    		
    		replace " �" using "“" searching in text document 1 options {search mode:literal, starting at top:true} -- opening double quote preceded by space
    		replace "� " using "”" searching in text document 1 options {search mode:literal, starting at top:true} -- closing double quote followed by space
    		replace "(.)�(.)" using "\\1’\\2" searching in text document 1 options {search mode:grep, starting at top:true} -- apostrophe
    		
    		-- clean up the last bit of errant code
    		replace "\\\\\\\"" using "\"" options {search mode:grep, starting at top:true}
    		replace ">\\\\n" using ">" searching in text 1 of text document "Readability Template.html" options {starting at top:true}
    		
    		-- break up the text to make it readable (currently not working)
    		replace "</p><p>" using "</p>
    
    <p>" searching in text document 1 options {search mode:literal, starting at top:true}
    		
    		select insertion point before line 1
    	end tell
    	preview in browser
    end tell

and the HTML template called in the script: