Cycling through URL's to find download package?

dbrewood · August 23, 2022, 2:53pm

Is is possible to develop a script to loop through URL’s looking for an update?

I’ve got a software link like:

https://software_distributary/dist/v9.1.2109_Mac/setup.pkg

Ideally I want the script to check through incrementing the package number so checking:

https://software_distributary/dist/v9.1.2109_Mac/setup.pkg
https://software_distributary/dist/v9.1.2110_Mac/setup.pkg
https://software_distributary/dist/v9.1.2111_Mac/setup.pkg

through to

https://software_distributary/dist/v9.1.2200_Mac/setup.pkg

It’d be useful I guess if the script actually counted backwards so finding the newest package, and then either indicated the latest viable result so I could then download it.

When the software hits 2200 I’d then reset the start / end numbers in the script to carry on looking forward.

Does that make sense, is it possible?

estockly · August 23, 2022, 6:27pm

Do you want the script to look on the page pointed to by the URL to see if it’s been updated, or just go by the text in the URL?

dbrewood · August 23, 2022, 6:30pm

If the URL has a valid file installer on the server it will be there as a prospective download. So thinking on that maybe it should just download the latest installer and append the version to the end of it?

dbrewood · August 26, 2022, 9:33am

I decided to try and have a go at this myself. I got as far as:

set i to 2120
repeat while i ≥ 2105
	set myString1 to "https://cdn-dist.distribution.com/dist/v9.1."
	set myString2 to "_Mac/setup.pkg"
	
	set MyURL to myString1 & i & myString2
	
	
	try
		# display dialog MyURL
		open location MyURL
	on error number 127
	end try
	
	# display dialog MyURL
	set i to i - 1
end repeat

And with my limited knowledge I think it’s not really viable as:

I can’t see a way of checking the URL is valid without opening the URL being tested in a browser, which would result in many many URLs being opened if the range to be tested was expanded
If a URL does have a valid download found I can’t see a way of exiting the loop
If a URL does have a valid download found I can’t see a way saving the download with the version number attached.

At least I’ve had a go

dbrewood · August 26, 2022, 10:39am

Post above updated…

estockly · August 26, 2022, 6:04pm

None of those URLs would open on my mac, so I can only go so far.

So first, I would test this on URLs that will open in Safari on your mac, and then look at the text they produce and from that you should be able to determine if they have what you want.

This script should give you an idea about how to proceed. It took about 2 hours to run. (I launched it and walked away) Result below.


use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
set theURLS to {}
set progress total steps to 2200 - 2109
set completedSteps to 0
set progress description to "2109 to 2200"

repeat with aNumber from 2109 to 2200
	set the end of theURLS to ("https://software_distributary/dist/v9.1." & aNumber as text) & "_Mac/setup.pkg"
end repeat
set htmlResults to {}
set AppleScript's text item delimiters to {return}

repeat with thisURL in theURLS
	set progress completed steps to completedSteps
	set progress additional description to thisURL as text
	set thisPageText to WebPageHTML(thisURL as text)
	if thisPageText is not "" then
		set the end of htmlResults to thisPageText as text
	end if
	set completedSteps to completedSteps + 1
end repeat
set AppleScript's text item delimiters to {return& return}

return htmlResults as text
on WebPageHTML(pageURLStr)
	tell application "Safari"
		set startTime to current date
		open location pageURLStr
		tell current tab of window 1
			repeat 24 times
				delay 10
				set pageText to its text
				
				if pageText ≠ "" then exit repeat
			end repeat
			close
			set secondsToLoad to (current date) - startTime
			set secondsToLoad to secondsToLoad as text
			
			return {secondsToLoad, pageURLStr, pageText}
		end tell
	end tell
end WebPageHTML

70
https://software_distributary/dist/v9.1.2109_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2109_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
20
https://software_distributary/dist/v9.1.2110_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2110_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
51
https://software_distributary/dist/v9.1.2111_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2111_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
80
https://software_distributary/dist/v9.1.2112_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2112_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2113_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2113_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2114_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2114_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
121
https://software_distributary/dist/v9.1.2115_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2115_Mac/setup.pkg” because the server where this page is located isn’t responding.
10
https://software_distributary/dist/v9.1.2116_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2116_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2117_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2117_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
11
https://software_distributary/dist/v9.1.2118_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2118_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
100
https://software_distributary/dist/v9.1.2119_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2119_Mac/setup.pkg” because the server where this page is located isn’t responding.
140
https://software_distributary/dist/v9.1.2120_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2120_Mac/setup.pkg” because the server where this page is located isn’t responding.
50
https://software_distributary/dist/v9.1.2121_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2121_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
71
https://software_distributary/dist/v9.1.2122_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2122_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2123_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2123_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2124_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2124_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2125_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2125_Mac/setup.pkg” because the server where this page is located isn’t responding.
10
https://software_distributary/dist/v9.1.2126_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2126_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2127_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2127_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2128_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2128_Mac/setup.pkg” because the server where this page is located isn’t responding.
141
https://software_distributary/dist/v9.1.2129_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2129_Mac/setup.pkg” because the server where this page is located isn’t responding.
140
https://software_distributary/dist/v9.1.2130_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2130_Mac/setup.pkg” because the server where this page is located isn’t responding.
140
https://software_distributary/dist/v9.1.2131_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2131_Mac/setup.pkg” because the server where this page is located isn’t responding.
20
https://software_distributary/dist/v9.1.2132_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2132_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
141
https://software_distributary/dist/v9.1.2133_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2133_Mac/setup.pkg” because the server where this page is located isn’t responding.
50
https://software_distributary/dist/v9.1.2134_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2134_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
141
https://software_distributary/dist/v9.1.2135_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2135_Mac/setup.pkg” because the server where this page is located isn’t responding.
100
https://software_distributary/dist/v9.1.2136_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2136_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2137_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2137_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
101
https://software_distributary/dist/v9.1.2138_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2138_Mac/setup.pkg” because the server where this page is located isn’t responding.
140
https://software_distributary/dist/v9.1.2139_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2139_Mac/setup.pkg” because the server where this page is located isn’t responding.
80
https://software_distributary/dist/v9.1.2140_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2140_Mac/setup.pkg” because the server where this page is located isn’t responding.
41
https://software_distributary/dist/v9.1.2141_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2141_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
50
https://software_distributary/dist/v9.1.2142_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2142_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2143_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2143_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
11
https://software_distributary/dist/v9.1.2144_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2144_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
90
https://software_distributary/dist/v9.1.2145_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2145_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2146_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2146_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
20
https://software_distributary/dist/v9.1.2147_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2147_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
71
https://software_distributary/dist/v9.1.2148_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2148_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2149_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2149_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
20
https://software_distributary/dist/v9.1.2150_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2150_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
91
https://software_distributary/dist/v9.1.2151_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2151_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2152_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2152_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2153_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2153_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2154_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2154_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
71
https://software_distributary/dist/v9.1.2155_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2155_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2156_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2156_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2157_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2157_Mac/setup.pkg” because the server where this page is located isn’t responding.
10
https://software_distributary/dist/v9.1.2158_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2158_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
81
https://software_distributary/dist/v9.1.2159_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2159_Mac/setup.pkg” because the server where this page is located isn’t responding.
100
https://software_distributary/dist/v9.1.2160_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2160_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2161_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2161_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
90
https://software_distributary/dist/v9.1.2162_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2162_Mac/setup.pkg” because the server where this page is located isn’t responding.
141
https://software_distributary/dist/v9.1.2163_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2163_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2164_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2164_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
90
https://software_distributary/dist/v9.1.2165_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2165_Mac/setup.pkg” because the server where this page is located isn’t responding.
21
https://software_distributary/dist/v9.1.2166_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2166_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2167_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2167_Mac/setup.pkg” because the server where this page is located isn’t responding.
10
https://software_distributary/dist/v9.1.2168_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2168_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2169_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2169_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
91
https://software_distributary/dist/v9.1.2170_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2170_Mac/setup.pkg” because the server where this page is located isn’t responding.
80
https://software_distributary/dist/v9.1.2171_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2171_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
40
https://software_distributary/dist/v9.1.2172_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2172_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
20
https://software_distributary/dist/v9.1.2173_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2173_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
31
https://software_distributary/dist/v9.1.2174_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2174_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2175_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2175_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
30
https://software_distributary/dist/v9.1.2176_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2176_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2177_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2177_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
141
https://software_distributary/dist/v9.1.2178_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2178_Mac/setup.pkg” because the server where this page is located isn’t responding.
80
https://software_distributary/dist/v9.1.2179_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2179_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2180_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2180_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
141
https://software_distributary/dist/v9.1.2181_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2181_Mac/setup.pkg” because the server where this page is located isn’t responding.
90
https://software_distributary/dist/v9.1.2182_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2182_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2183_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2183_Mac/setup.pkg” because the server where this page is located isn’t responding.
141
https://software_distributary/dist/v9.1.2184_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2184_Mac/setup.pkg” because the server where this page is located isn’t responding.
90
https://software_distributary/dist/v9.1.2185_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2185_Mac/setup.pkg” because the server where this page is located isn’t responding.
90
https://software_distributary/dist/v9.1.2186_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2186_Mac/setup.pkg” because the server where this page is located isn’t responding.
71
https://software_distributary/dist/v9.1.2187_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2187_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
80
https://software_distributary/dist/v9.1.2188_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2188_Mac/setup.pkg” because the server where this page is located isn’t responding.
81
https://software_distributary/dist/v9.1.2189_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2189_Mac/setup.pkg” because the server where this page is located isn’t responding.
10
https://software_distributary/dist/v9.1.2190_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2190_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
10
https://software_distributary/dist/v9.1.2191_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2191_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
140
https://software_distributary/dist/v9.1.2192_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2192_Mac/setup.pkg” because the server where this page is located isn’t responding.
120
https://software_distributary/dist/v9.1.2193_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2193_Mac/setup.pkg” because the server where this page is located isn’t responding.
10
https://software_distributary/dist/v9.1.2194_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2194_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
141
https://software_distributary/dist/v9.1.2195_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2195_Mac/setup.pkg” because the server where this page is located isn’t responding.
100
https://software_distributary/dist/v9.1.2196_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2196_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
70
https://software_distributary/dist/v9.1.2197_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2197_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.
91
https://software_distributary/dist/v9.1.2198_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2198_Mac/setup.pkg” because the server where this page is located isn’t responding.
140
https://software_distributary/dist/v9.1.2199_Mac/setup.pkg
Safari Can’t Open the Page
Safari can’t open the page “https://software_distributary/dist/v9.1.2199_Mac/setup.pkg” because the server where this page is located isn’t responding.
70
https://software_distributary/dist/v9.1.2200_Mac/setup.pkg
Safari Can’t Connect to the Server
Safari can’t open the page “https://software_distributary/dist/v9.1.2200_Mac/setup.pkg” because Safari can’t connect to the server “software_distributary”.

dbrewood · August 26, 2022, 6:15pm

My thanks, I’ve dropped you a PM with the actual CDN URL in there.

I’ve run the script, it does indeed download an installer (which looks to be good) but ends with an error:

is that expected?
Is it possible to append the version number to the file name downloaded?
Heck of an achievement though.

robertfern · August 26, 2022, 7:31pm

Why not post the the actual CDN so we can test it also.
Here is a cleaner version of the script above


use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
set theURLS to {}
set progress total steps to 2200 - 2109 + 1
set completedSteps to 0
set progress description to "2109 to 2200"

set htmlResults to {}
set AppleScript's text item delimiters to {return}

repeat with aNumber from 2109 to 2200
	set thisURL to (("https://cdn-dist.emclient.com/dist/v9.1." & aNumber as text) & "_Mac/setup.pkg")
	set progress completed steps to completedSteps
	set progress additional description to thisURL as text
	set thisPageText to WebPageHTML(thisURL)
	if thisPageText is not "" then
		set the end of htmlResults to thisPageText as text
	end if
	set completedSteps to completedSteps + 1
end repeat
set AppleScript's text item delimiters to {return & return}

return htmlResults as text

on WebPageHTML(pageURLStr)
	set startTime to current date
	tell application "Safari"
		tell current tab of window 1
			set URL to pageURLStr
			repeat 10 times
				delay 2
				set pageText to its text
				if pageText ≠ "" then exit repeat
			end repeat
		end tell
	end tell
	set secondsToLoad to (current date) - startTime
	set secondsToLoad to secondsToLoad as text
	return {secondsToLoad, pageURLStr, pageText}
end WebPageHTML

I might try a version using curl and “do shell script” instead of safari

dbrewood · August 26, 2022, 8:00pm

Okay, I’ll check out the script tomorrow, hitting the sheets with a migraine…

The actual CDN link is as per

 https://cdn-dist.emclient.com/dist/v9.1.2109_Mac/setup.pkg

I guess it does not really matter

estockly · August 26, 2022, 9:19pm

It matters.

robertfern · August 27, 2022, 1:14am

Here is a much better and cleaned-up version which only returns the list of URLs that have a file download.

Before you run this script, go into Preferences for Safari and go to the “WebSites” tab.
Choose “Downloads” on the left and make sure you set allow for the website “cdn-dist.emclient.com”

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

property htmlResults : {}

local pageText, prevText, thisURL, myTab, flag, aNumber, completedSteps
set progress total steps to 2200 - 2109 + 1
set completedSteps to 0
set progress description to "2109 to 2200"

set prevText to ""
tell application "Safari"
	tell window 1
		set myTab to make tab
		set current tab to myTab
		tell myTab --current tab
			repeat with aNumber from 2109 to 2200
				set flag to true
				set thisURL to ("https://cdn-dist.emclient.com/dist/v9.1." & aNumber & "_Mac/setup.pkg")
				set my progress completed steps to completedSteps
				set my progress additional description to thisURL as text
				set URL to thisURL
				repeat 5 times
					delay 1
					set pageText to its text
					if pageText ≠ prevText then
						set flag to false
						set prevText to pageText
						exit repeat
					end if
				end repeat
				if flag then -- didnt error out, so save this url
					set the end of htmlResults to thisURL
				end if
				set completedSteps to completedSteps + 1
			end repeat
			close
		end tell
	end tell
end tell
return htmlResults

dbrewood · August 27, 2022, 7:03am

Okay… I’ve just run that new script and it downloaded 6 setup files (I’m not sure if it should have actually done the downloads??)

However ideally the script should county downwards therefore only downloading the latest installer? Or run as it is now I guess if the downloads don’t take place?

It did return as a result:

{“https://cdn-dist.emclient.com/dist/v9.1.2109_Mac/setup.pkg”, “https://cdn-dist.emclient.com/dist/v9.1.2125_Mac/setup.pkg”, “https://cdn-dist.emclient.com/dist/v9.1.2128_Mac/setup.pkg”, “https://cdn-dist.emclient.com/dist/v9.1.2129_Mac/setup.pkg”, “https://cdn-dist.emclient.com/dist/v9.1.2130_Mac/setup.pkg”, “https://cdn-dist.emclient.com/dist/v9.1.2148_Mac/setup.pkg”}

So yes, geting there

estockly · August 27, 2022, 2:26pm

Easy to go backwords.

 set theURLS to the reverse of theURLs

Are you saying you want it to exit as soon as it successfully downloads its first file?

dbrewood · August 27, 2022, 2:53pm

Yep that’d be perfect. If it could save the download with file name like

emclient-v9.1.2148_Mac.pkg

That would be absolute perfection. So it does need to start scanning at the higher number and count downwards.
Does that make sense?

estockly · August 27, 2022, 3:31pm

OK, forget the script I posted earlier. With those URLs it doesn’t make sense to read the page in a browser to figure out if the URL is valid. To that script, a failure looks the same as a success (an empty string).

There is probably a better way to see if a URL is valid, but this works, and is much faster. (About 30 seconds OMM).

If you just want the first valid page you can unnote that line in the repeat loop.

If you want to then automatically download, you can write a routine to open the URL(s) in Safari.

HTH
Edit: Since this is now faster I removed all the progress stuff

 use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
use framework "Foundation"

set theURLS to {}
repeat with aNumber from 2109 to 2200
	set the beginning of theURLS to ("https://cdn-dist.emclient.com/dist/v9.1." & aNumber as text) & "_Mac/setup.pkg"
end repeat
set pagesFound to {}
set pagesNotFound to {}

set AppleScript's text item delimiters to {tab}
repeat with thisURL in theURLS
	set {theURL, pageStatus} to WebPageHTML(thisURL as text)
	if pageStatus is "page found" then
		set the end of pagesFound to {"Found", theURL} as text
		--exit repeat --if you want to stop at the first one
	else
		set the end of pagesNotFound to {"Not found", theURL} as text
	end if
end repeat
--Add a routine here to open found page(s) in Safari, if you like

set AppleScript's text item delimiters to {return & return}
return pagesFound & pagesNotFound as text

on WebPageHTML(pageURLStr)
	set nsPageURL to current application's |NSURL|'s URLWithString:pageURLStr
	set {nsPageHTML, theError} to current application's NSData's dataWithContentsOfURL:nsPageURL options:0 |error|:(reference)
	if nsPageHTML = missing value then
		return {pageURLStr, "page not found"}
	else
		return {pageURLStr, "page found"}
	end if
end WebPageHTML

dbrewood · August 27, 2022, 3:40pm

That will do nicely … I can do a manual download once I know which is the latest file.
I did comment ‘in’ the ‘exit repeat --if you want to stop at the first one’ line but the script carried on and didn’t stop?
Did I miss something?

estockly · August 27, 2022, 6:08pm

This is what that repeat loop should look like. For me it exists on the first positive result.


repeat with thisURL in theURLS
   set {theURL, pageStatus} to WebPageHTML(thisURL as text)
   if pageStatus is "page found" then
       set the end of pagesFound to {"Found", theURL} as text
       exit repeat 
   else
       set the end of pagesNotFound to {"Not found", theURL} as text
   end if
end repeat

dbrewood · August 27, 2022, 6:28pm

So no:

set completedSteps to completedSteps + 1

between the endif and end repeat?

So like this (loop variables changed):

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
use framework "Foundation"

set theURLS to {}
set progress total steps to 2200 - 2148
set completedSteps to 0
set progress description to "2148 to 2200"

repeat with aNumber from 2148 to 2200
	set the beginning of theURLS to ("https://cdn-dist.emclient.com/dist/v9.1." & aNumber as text) & "_Mac/setup.pkg"
end repeat
set pagesFound to {}
set pagesNotFound to {}

set AppleScript's text item delimiters to {tab}

repeat with thisURL in theURLS
	set {theURL, pageStatus} to WebPageHTML(thisURL as text)
	if pageStatus is "page found" then
		set the end of pagesFound to {"Found", theURL} as text
		exit repeat
	else
		set the end of pagesNotFound to {"Not found", theURL} as text
	end if
end repeat
set AppleScript's text item delimiters to {return & return}

--Add a routine here to open found page(s) in Safari, if you like

return pagesFound & pagesNotFound as text

on WebPageHTML(pageURLStr)
	set nsPageURL to current application's |NSURL|'s URLWithString:pageURLStr
	set {nsPageHTML, theError} to current application's NSData's dataWithContentsOfURL:nsPageURL options:0 |error|:(reference)
	if nsPageHTML = missing value then
		return {pageURLStr, "page not found"}
	else
		return {pageURLStr, "page found"}
	end if
end WebPageHTML

Tht gives me an output of:

estockly · August 27, 2022, 7:16pm

That is the expected output.

The script tested each of those marked “Not Found” before it got to the one that worked, then it exited and returned the result.

if you don’t need to see the “Not Found” pages comment out this line:

 
--     set the end of pagesNotFound to {"Not found", theURL} as text

dbrewood · August 27, 2022, 7:32pm

Brilliant… Working well now I’ve commented that out. Is there any way of showing a progress indicator so I know it’s working… while it scans?