Piping curl and grep together in a Do Shell Script command

Scripting Forums AppleScript | Mac OS X
#1

I (a total newbie) want to write an applescript that uses the ‘do shell command’ to curl 5 or 6 web pages in finance.yahoo.com (one page for each mutual fund in my portfolio), and then grep the current net asset value of each fund, so it (the script) can then do some multiplication and show me how much I’m worth each day.

Can grep be used this way? In the curled text of the web pages, the NAVs appear in a line near the text, “Net Asset Value” but, for example, the piped command

curl http://finance.yahoo.com/q?s=BUFSX&d=t | grep Value

doesn’t seem to grep anything and only returns the entire text of the web page. Am I on the wrong track? Any ideas? Can curl itself be used to ferret this data out of the page? Know a better forum for this topic?

#2

I finally gave up on using SOAP calls for this project. A better way was suggested by Kirk McElhearn, thank you very much. Applescript’s DO SHELL SCRIPT command returns whatever text the unix shell puts out (or in–I have to read Kirk’s book about shell commands to see what’s going on here), and curl is a unix command that can get a web page into the standard input buffer (I guess). Once in, the text of the page can be searched for the info you need using another command, grep, which finds whole lines of text having the given search string. A third command, cut, is used here to ferret out just the sought characters - the net asset values of my funds.

Thanks also to Bob Y of Bob’s Mutual Funds Page, whose web site turned out to be very searchable and easy to use, both by just browsing it, and by using curl and grep:

http://customer.wcta.net/roberty/index.html

– My Portfolio

set funds to {“BUFSX”, “PRFDX”, “PRSCX”, “SCGEX”, “SMTVX”, “VGHCX”}
set shares to {241.354, 434.764, 293.067, 558.729, 334.741, 181.834}
set NAVs to {“”}
set numFunds to length of funds

set thisDate to date string of (current date)

set NAVs to NAVs & (do shell script “curl -s www.buffalofunds.com/index.html | grep BUFSX | grep '> ’ | cut -c 379-383”)
set NAVs to NAVs & (do shell script “curl -s customer.wcta.net/roberty/XP.HTM | grep PRFDX | cut -c 114-119”)
set NAVs to NAVs & (do shell script “curl -s customer.wcta.net/roberty/XP.HTM | grep PRSCX | cut -c 118-122”)
set NAVs to NAVs & (do shell script “curl -s customer.wcta.net/roberty/XS.HTM | grep SCGEX | cut -c 114-118”)
set NAVs to NAVs & (do shell script “curl -s customer.wcta.net/roberty/XS.HTM | grep SMTVX | cut -c 114-118”)
set NAVs to NAVs & (do shell script “curl -s customer.wcta.net/roberty/XV.HTM | grep VGHCX | cut -c 113-118”)

set NAVs to the rest of NAVs

set value to 0
set Total to 0
set msgString to "John’s Portfolio as of " & thisDate & return & return & “Fund Shares NAV Value” & return & return

repeat with loopVar from 1 to numFunds
set theseShares to item loopVar of shares
set thisNAV to item loopVar of NAVs as real
set msgString to msgString & item loopVar of funds & tab & tab & tab & theseShares & tab & tab & tab & thisNAV & tab & tab & tab
set value to theseShares * thisNAV
set value to value as integer
set Total to Total + value
set msgString to msgString & “$” & value & return
end repeat

set msgString to msgString & return & “Total value now: $” & Total & return
display dialog msgString