Getting text from a webpage

Scripting Forums AppleScript | Mac OS X
,
#1

How do u get text that is displayed on a website, like http://www.apple.com ?

i’ve been trying for some time now, but it’s a little confusing…

I would really appreciate some help

thxs

#2

This link may help:
http://bbs.applescript.net/viewtopic.php?id=13347

#3

H tapman,

You could use this method with Safari:

tell application "Safari"
	make new document with properties {URL:"http://www.apple.com"}
	delay 5 -- to enable page loading
	set myText to text of document 1
end tell

Or you could try cURL and TIDs:

set myHtml to do shell script "curl http://www.apple.com"

set {myDels, AppleScript's text item delimiters} to {AppleScript's text item delimiters, {"</head>"}}
set myText to text item 2 of myHtml
set AppleScript's text item delimiters to myDels
set myText to remove_markup(myText)

on remove_markup(this_text)
	set myAllowed to {"://", "@"}
	set {myTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, {"<"}}
	set these_items to text items of this_text
	set AppleScript's text item delimiters to {">"}
	repeat with my_item in these_items
		if length of text items of my_item is greater than 1 then
			if (text item 1 of my_item) is not in myAllowed then
				set my_item's contents to (text item 2 of my_item)
			else
				set my_item's contents to ("<" & (text items of my_item)) as text
			end if
		end if
	end repeat
	set AppleScript's text item delimiters to myTID
	set clean_text to these_items as text
	return clean_text
end remove_markup

Both methods have their problems.

Best wishes

John M

#4

What exactly do you need to get, tapman90?

#5

well both of the methods John posted worked great! thxs john

that’s what i wanted

#6

John, all of a sudden, your cUrl script isn’t working…

everytime i run it, i get:

"

Site Map | Search Tips Visit the Apple Store online or at retail locations.1-800-MY-APPLE Important Safety Recall — Rechargeable Batteries for 12-inch iBook G4, 12-inch and 15-inch PowerBook G4 iPod Settlement | Expanded iBook Logic Board Repair Extension Program (12/17/2004) Find Job Opportunities at Apple. Visit other Apple sites around the world: Choose... Asia Australia Austria Belgium Brazil Canada China Denmark Europe Finland France Germany Hong Kong India Ireland Italy Japan Korea Latin America Mexico Netherlands Norway Singapore South Africa Spain Sweden Switzerland Taiwan UK United States Contact Us | Terms of Use | Privacy Policy Copyright © 2005 Apple Computer, Inc. All rights reserved. "

it’s very odd, because it worked before…

any idea why?

#7

Hi tapman,

It’s the “AppleScript’s text item delimiters” (these should normally be a list of one empty string {“”} ). These are changed in the script. It’s considered good practice to save the current ones and return them to the same state afterwards.

It looks like the script didn’t complete one time, and stopped on the third line, and the TIDs were left as {“”}.

Replace the second line of the script with:

set {myDels, AppleScript's text item delimiters} to {{""}, {"</head>"}}

Best wishes

John M

#8

thanks!

#9

Hello everyone,

I need help with a similar script, except I would like to limit the text to get from the webpage (with text item delimiters i guess).

the text to get is located on the webpage between ": " and “
”

then I need to put that text into a variable and return it…

Can anyone help?