Listing disk/folder hierarchy contents to text file

This works well on my Documents folder, but I don’t know how it’ll cope with a full DVD or how long it’ll take. The text file it produces contains the full path to each item, but if you uncomment the commented code, it’ll give a “folder contents”-style layout, with just the item names indented below the names of the folders they’re in.

I’ve used a Shell sort to keep the script short (!), but there’s a faster sort here if you need it. Alternatively, you could leave the sort out altogether if you didn’t want the indented layout.

The text files are saves as UTF-8 Unicode text with a BOM.

There may be a faster way to do this using shell scripting, but I don’t have time to research it.

-- Shell sort algorithm: Donald Shell 1959. AppleScript implementation: Nigel Garvey 2010.
on ShellSort(theList, l, r)
	script o
		property lst : theList
	end script
	
	-- Process the input parmeters.
	set listLen to (count theList)
	if (listLen > 1) then
		-- Negative and/or transposed range indices.
		if (l < 0) then set l to listLen + l + 1
		if (r < 0) then set r to listLen + r + 1
		if (l > r) then set {l, r} to {r, l}
		
		-- Do the sort.
		set inc to (r - l + 1) div 2
		repeat while (inc > 0)
			repeat with j from (l + inc) to r
				set v to item j of o's lst
				repeat with i from (j - inc) to l by -inc
					tell item i of o's lst
						if (it > v) then
							set item (i + inc) of o's lst to it
						else
							set i to i + inc
							exit repeat
						end if
					end tell
				end repeat
				set item i of o's lst to v
			end repeat
			set inc to (inc / 2.2) as integer
		end repeat
	end if
	
	return -- nothing.
end ShellSort

-- Create the text to be written to the file.
-- (Uncomment the commented code to get names indented by container rather than full paths.)
on createText(rootPath, contentPaths)
	script o
		property paths : contentPaths
	end script
	
	set astid to AppleScript's text item delimiters
	(*
	set AppleScript's text item delimiters to ""
	set tabStr to {tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab} as text -- Hopefully more than needed!
	
	set AppleScript's text item delimiters to ":"
	set nonIndent to (count rootPath's text items) - 1
	
	repeat with i from 1 to (count o's paths)
		set thisPath to item i of o's paths
		set tiCount to (count thisPath's text items)
		if (thisPath ends with ":") then
			set item i of o's paths to text 1 thru (tiCount - nonIndent - 1) of tabStr & text from text item -2 to -1 of thisPath
		else
			set item i of o's paths to text 1 thru (tiCount - nonIndent) of tabStr & text item -1 of thisPath
		end if
	end repeat
	*)
	set AppleScript's text item delimiters to linefeed
	set outText to "Entire contents of " & rootPath & linefeed & linefeed & o's paths
	(*
	set AppleScript's text item delimiters to linefeed & tab
	set outText to outText's text items
	set AppleScript's text item delimiters to linefeed
	set outText to outText as text
	*)
	set AppleScript's text item delimiters to astid
	
	return outText
end createText

-- Write the text to file as UTF-8.
on writeTextFile(txt, defaultLoc)
	set f to (choose file name with prompt "Save the UTF-8 text listing as." default name "Entire contents listing.txt" default location defaultLoc)
	set fRef to (open for access f with write permission)
	try
		set eof fRef to 0
		write «data rdatEFBBBF» to fRef -- UTF-8 BOM.
		write txt as «class utf8» to fRef
	end try
	close access fRef
	
	display dialog "The listing has been saved in file \"" & f & "\"" buttons {"OK"} default button 1
end writeTextFile

-- Get HFS paths to all the items in the hierarchy except the root.
on getPaths(rootFolder)
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to linefeed
	tell application "Finder" to set thePaths to text items of ((entire contents of rootFolder) as text)
	set AppleScript's text item delimiters to astid
	
	return thePaths
end getPaths

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	set rootPath to rootFolder as text
	
	set thePaths to getPaths(rootFolder)
	ShellSort(thePaths, 1, -1)
	
	set outText to createText(rootPath, thePaths)
	writeTextFile(outText, (path to documents folder))
end main

main()

Hello Nigel

(1) Best wishes for this new year

(2) I would edit a handler this way :


on getPaths(rootFolder)
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to linefeed
	with timeout of 60 * 60 seconds
		tell application "Finder" to set thePaths to text items of ((entire contents of rootFolder) as text)
	end timeout
	set AppleScript's text item delimiters to astid
	
	return thePaths
end getPaths

because the default value (2 minutes) is not sufficient for a large volume.

Yvan KOENIG (VALLAURIS, France) samedi 15 janvier 2011 16:03:25

Here is an alternate script which does the entire job with a single do shell script.

I use


on run
run script main
end run

because it’s a trick which fasten dramatically the execution when scripts are saved as application.
In fact, I started inserting it in the original script.


-- Write the text to file as UTF-8.
on writeTextFile(txt, defaultLoc)
	set f to (choose file name with prompt "Save the UTF-8 text listing as." default name "Entire contents listing.txt" default location defaultLoc)
	set fRef to (open for access f with write permission)
	try
		set eof fRef to 0
		write «data rdatEFBBBF» to fRef -- UTF-8 BOM.
		write txt as «class utf8» to fRef
	end try
	close access fRef
	
	display dialog "The listing has been saved in file \"" & f & "\"" buttons {"OK"} default button 1
end writeTextFile

script main
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	set rootPath to rootFolder as text
	set rootPathUnix to quoted form of POSIX path of rootPath
	
	set txt to do shell script "ls -R " & rootPathUnix
	
	set outText to "Entire contents of " & rootPath & linefeed & linefeed & txt
	writeTextFile(outText, (path to documents folder))
end script

on run
	run script main
end run

Yvan KOENIG (VALLAURIS, France) samedi 15 janvier 2011 17:20:35

Fantastic help from all. Thank you Nigel for caring enough to help me with this task. And I ended up using Yvan KOENIG’s script… really fast!

Is it possible to modify the script from default name “Entire contents listing.txt” to the name of the CD/DVD disk I am running the script on?

Best regards and best wishes for the new year,
Dan

(1) The script displays a save dialog in which Entire contents listing.txt is just a default file name.

I used it to create :

Entire contents listing.txt
Entire contents of Aluice 250.txt
Entire contents of Aluice 500.txt
Entire contents of Macintosh HD Maxtor.txt
Entire contents of Western Caviar.txt
Entire contents of Western 2.txt

(2) You may edit the script


(*
This one may build short default name :
Entire contents of Macintosh HD.txt
if it scan the entire named volume.
Entire contents of Données utilisateur AppleWorks.txt
if it scans the named folder.
*)
-- Write the text to file as UTF-8.
on writeTextFile(txt, defaultLoc, root_name)
	set f to (choose file name with prompt "Save the UTF-8 text listing as." default name "Entire contents of " & root_name & ".txt" default location defaultLoc)
	set fRef to (open for access f with write permission)
	try
		set eof fRef to 0
		write «data rdatEFBBBF» to fRef -- UTF-8 BOM.
		write txt as «class utf8» to fRef
	end try
	close access fRef
	
	display dialog "The listing has been saved in file \"" & f & "\"" buttons {"OK"} default button 1
end writeTextFile

--=====

on decoupe(t, d)
	local oTIDs, l
	set oTIDs to AppleScript's text item delimiters
	set AppleScript's text item delimiters to d
	set l to text items of t
	set AppleScript's text item delimiters to oTIDs
	return l
end decoupe

--=====

script main
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	set rootPath to rootFolder as text
	set rootName to item -2 of my decoupe(rootPath, ":")
	set rootPathUnix to quoted form of POSIX path of rootPath
	
	set txt to do shell script "ls -R " & rootPathUnix
	
	set outText to "Entire contents of " & rootPath & linefeed & linefeed & txt
	writeTextFile(outText, (path to documents folder), rootName)
end script

on run
	run script main
end run


(*
This one may build this kind of descriptive default name :
Entire contents of Macintosh HD_Users_yvankoenig_Documents_Données utilisateur AppleWorks.txt
*)
-- Write the text to file as UTF-8.
on writeTextFile(txt, defaultLoc)
	set defaultName to text 1 thru -2 of my remplace(paragraph 1 of txt, ":", "_")
	set f to (choose file name with prompt "Save the UTF-8 text listing as." default name defaultName & ".txt" default location defaultLoc)
	set fRef to (open for access f with write permission)
	try
		set eof fRef to 0
		write «data rdatEFBBBF» to fRef -- UTF-8 BOM.
		write txt as «class utf8» to fRef
	end try
	close access fRef
	
	display dialog "The listing has been saved in file \"" & f & "\"" buttons {"OK"} default button 1
end writeTextFile

--=====

on remplace(t, d1, d2)
	local oTIDs, l
	set oTIDs to AppleScript's text item delimiters
	set AppleScript's text item delimiters to d1
	set l to text items of t
	set AppleScript's text item delimiters to d2
	set t to l as text
	set AppleScript's text item delimiters to oTIDs
	return t
end remplace

--=====

script main
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	set rootPath to rootFolder as text
	set rootPathUnix to quoted form of POSIX path of rootPath
	
	set txt to do shell script "ls -R " & rootPathUnix
	
	set outText to "Entire contents of " & rootPath & linefeed & linefeed & txt
	writeTextFile(outText, (path to documents folder))
end script

on run
	run script main
end run

Yvan KOENIG (VALLAURIS, France) samedi 15 janvier 2011 20:04:00

Where did you learn this or do you have an example which illustrates it? I haven’t seen this comment before but I’d certainly incorporate it into my scripts because I save lots of them as applications.

(1) Thanks, Yvan. A Happy New Year to you too! :slight_smile:

(2) Thanks too for the “heads up” about the timeout and for your other improvements to the script. I played around with “ls” and “find” this afternoon, but wasn’t keen on the resulting layout of the text. However, Dan seems pleased, so problem solved!

I’ll have to look into that. I know it speeds up execution when the script’s run as an application and it contains commands addressed to another application (something to do with communication between OSAX and application being faster than direct communication between applications), but I haven’t heard of it speeding things up anyway.

Cheers, Dan. You too. Glad you got what you wanted. :slight_smile:

Both work excellently Yvan. Thank you very much for your help!

It seems that my poor English was not clear enough.

Your original script which speak to Finder may take benefit from the tip but for sure, it would be more efficient in scripts triggering many times external applications (in fact external app’s dictionaries or OSAX)
For instance I use it daily in a script which store the location of icons on my Desktop to reset them when I boot the machine. The difference of speed is astonishing.
So, I inserted it in most of my script so, it’s now second nature and maybe, sometimes I use it in scripts where it’s useless.
I’m not sure that it’s useful in the script which use “ls”

I posted the script using “ls” because I thought that the layout is sufficient to retrieve the files thru Spotlight (and because I try to remove calls to Finder when I may do. I feel no need to take benefit of all the layers of code which are between Finder and System Events which is in fact the one which does the job.
To do what you wanted, I feel that two schemes where viable:
(1) the one which you coded triggering the Finder
(2) the one which I used which trigger “low level” tools.
What’s fine is that with mine, I may continue to work when the script is scanning a huge volume which is not the case when it’s the Finder which do the job.
I didn’t pay attention to a scheme using System Events because this one doesn’t offer an ‘entire contents’ feature.

Yvan KOENIG (VALLAURIS, France) samedi 15 janvier 2011 23:10:55

Thanks Nigel and Yvan for clarifying that. I’m sure I can make use of it.

Just for my own satisfaction, here’s the version for which I’d been groping. It’s no better than Yvan’s for Dan’s purposes, but it formats the text prettily. :slight_smile:

-- Create the text to be written to the file.
-- Just a heading and the item names, indented according to their positions in the hierarchy.
-- (Uncomment the (* *) comment markers to preserve the full paths.)
on createText(posixPaths)
	script o
		property paths : posixPaths
	end script
	
	set rootPath to beginning of o's paths
	set item 1 of o's paths to "Entire contents of " & rootPath & linefeed
	set astid to AppleScript's text item delimiters
	-- (*
	considering case
		set AppleScript's text item delimiters to ""
		set tabStr to {tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab} as text -- Hopefully more than needed!
		
		set AppleScript's text item delimiters to "/"
		set nonIndent to (count rootPath's text items) -- 1
		
		set len to (count posixPaths)
		repeat with i from 2 to len
			set thisPath to item i of o's paths
			set tiCount to (count thisPath's text items)
			set thisName to text item -1 of thisPath
			-- If the item name contains any colons, restore the original slashes.
			if (thisName contains ":") then
				set AppleScript's text item delimiters to ":"
				set thisName to thisName's text items
				set AppleScript's text item delimiters to "/"
				set thisName to thisName as text
			end if
			-- If this is a folder path, append a colon to the name.
			if ((i < len) and (item (i + 1) of o's paths begins with thisPath)) then set thisName to thisName & ":" -- or "/", if preferred.
			set item i of o's paths to text 1 thru (tiCount - nonIndent) of tabStr & thisName
		end repeat
		-- *)
		set AppleScript's text item delimiters to linefeed
		set outText to o's paths as text
		-- (*
		set AppleScript's text item delimiters to linefeed & tab
		set outText to outText's text items
		set AppleScript's text item delimiters to linefeed
		set outText to outText as text
	end considering
	-- *)
	set AppleScript's text item delimiters to astid
	
	return outText
end createText

-- Write the text to file as UTF-8.
on writeTextFile(txt, defaultLoc)
	set f to (choose file name with prompt "Save the UTF-8 text listing as." default name (paragraph 1 of txt) & ".txt" default location defaultLoc)
	set fRef to (open for access f with write permission)
	try
		set eof fRef to 0
		write «data rdatEFBBBF» to fRef -- UTF-8 BOM.
		write txt as «class utf8» to fRef
	end try
	close access fRef
	
	display dialog "The listing has been saved in file \"" & f & "\"" buttons {"OK"} default button 1
end writeTextFile

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	
	-- List the hierarchy as POSIX paths, omitting any that contain elements beginning with ".".
	set thePaths to paragraphs of (do shell script "find -f " & (quoted form of POSIX path of rootFolder) & " \\! -path \"*/.*\"")
	
	set outText to createText(thePaths)
	writeTextFile(outText, (path to documents folder))
end main

main()

Edit: It occurs to me that it’s possible (although frowned upon in some quarters) for a file or folder name to contain slash (“/”) characters. In the POSIX paths returned by “ls” or “find”, these will have been rendered as colons; so ideally, the script needs to look out for these and convert them back to slashes for the text file. I’ve modified the above to that end.

I agree Nigel, it’s much easier on the eye. Any chance we could save it as a .rtf file and use Optima as the font? :slight_smile:
And perhaps bold the folder names?

See what you started? :slight_smile:

Must leave you something to aspire to. :wink:

Actually, I was thinking the same thing. (It may take a while) but I’ll consider it my first project and post back here when I get it done!

I can assure you I will NEVER understand to the degree you and Yvan do; I don’t have this great a need. If you don’t mind my asking, how long have you been Applescripting, Nigel? How do you typically apply or use your skills?

Dan

Open the text file with TextEdit then apply the font of your choice.

Today I’m too tired to edit the script to do that.

Yvan KOENIG (VALLAURIS, France) dimanche 16 janvier 2011 22:07:16

Amazing and fascinating, as usual. Always impressed by Nigel and Yvan.

I have an idea that dyost, and others, might like: Instead of turning the listings file into RTF and adding formatting, why not use a single text file as a database for all your disks/discs and add a function to search it for the file you are seeking?



set the_task to button returned of (display dialog "Choose a task." buttons {"Cancel", "Catalog", "Search"})

if the_task is "Catalog" then
	my catalog()
else
	my search()
end if


on catalog()
	-- Yvan's or Nigel's script (modified to append to a specified file)
end catalog


on search()
	-- have user enter search string
	-- read in catalog file
	-- search for search string
	-- make a list of results as complete paths
	-- format results as a numbered list of tabbed entries like this:
	
	-- Item 01
	-- 	My Book 02:
	-- 		iTunes - My Book 02:
	--			Music:
	--				Biréli Lagrène:
	--					Live In Marciac:
	--						07 C' est Si Bon.mp3"
	
	-- write results to a TextEdit file
	-- display dialog which asks the user to enter the number for the file to reveal
	-- tell Finder to reveal the file (if the disk/disc is not present, ask for the disk/disc)
end search

I’d be willing to take a shot at writing this. (Stop laughing, I’m serious.)

But it seems to me that it would be better to get the file listings as complete paths like the Finder does:

“My Book 02:iTunes - My Book 02:Music:Biréli Lagrène:Live In Marciac:07 C’ est Si Bon.mp3”

However, after seeing how much faster that shell script is than “tell Finder get entire contents”, I’m wondering if there is a shell script that gets the listings as complete paths.

Thanks,
icta

Humor me, I’m old.

Hi, icta.

The “find” shell script in my script in post #11 returns the full POSIX paths to the items in the hierarchy, sorted (as far as I can make out) lexically by path. This has the effect of sorting the items by hierarchy too, since each folder path is followed immediately by the paths to the items in that folder. It’s the createText() handler which substitutes tabs for the path elements which aren’t the items’ names. You can stop that by cutting out the two sections in it which are delimited by – (* and – *).

Hello Nigel

I’m back here because with your script of message #11 as well as with mines, I get the error :

error “find: /Volumes/baseÆ’ebay//.Trashes: Permission denied” number 1

when I try to scan a volume (here it’s a disk image)

And of course, I don’t guess a way to filter it without loosing the listing.

Even with a bit of practice, sometimes I hit a wall :frowning:

Yvan KOENIG (VALLAURIS, France) dimanche 20 février 2011 17:35:46

Running the script as administrator helps to get around this problem
(quite annoying that you have to give your pw however)

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	set theShellFindScript to "find -f " & (quoted form of POSIX path of rootFolder) & " \\! -path \"*/.*\""
	-- List the hierarchy as POSIX paths, omitting any that contain elements beginning with ".".
	set thePaths to paragraphs of (do shell script theShellFindScript with administrator privileges)
	
	set outText to createText(thePaths)
	writeTextFile(outText, (path to documents folder))
end main

Hi Bramster and Yvan.

Thanks for your comments. Sorry I’ve been so long getting back!

I don’t remember if I saw Yvan’s post at the time or how or why I responded if I did. Possibly I couldn’t reproduce the error. I would certainly have been puzzled by the fact that a path excluded by the “find” parameters was apparently causing it.

It so happens that a couple of days before Bramster posted his work-round, I wrote a similar script again from scratch using “sed” to edit the paths. This script doesn’t throw an error when tested on a couple of mounted disk images, whereas the script in post #11 does. Initially confusing, since they use similar “find” parameters.

It turns out that the error does occur in my more recent script, but is ignored because there are more commands after “find” in the shell script. Putting a dummy command after “find” in the original shell script ” say “echo "\c"” ” stops the error message from appearing and ” as far as I can see ” produces an identical listing to Bramster’s work-round where the shell script’s run with administrator privileges. The same thing’s probably possible by redirecting Standard Error, but I haven’t got that far with my research!

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	-- List the hierarchy as POSIX paths, omitting any that contain elements beginning with ".".
	-- The "echo" at the end is a dummy to sweep errors under the carpet!
	set theShellFindScript to "find -f " & (quoted form of POSIX path of rootFolder) & " \\! -path \"*/.*\" ; echo \"\\c\""
	set thePaths to paragraphs of (do shell script theShellFindScript)
	
	set outText to createText(thePaths)
	writeTextFile(outText, (path to documents folder))
end main

The “sed” script, if you’re interested, opens the listing directly in TextWrangler, but can easily be adapted to write to file instead:

main()

on main()
	-- Parameters set to return the same items as the script in post #11.
	set listing to (listFolderContents of (POSIX path of (choose folder)) with recursion and packageContents without invisibles)
	openInTextWrangler(listing)
end main

on listFolderContents of rootPath given invisibles:invisibles, packageContents:packageContents, recursion:recursion
	set invisiblesInsert to " -not -path '*/.*' -flags nohidden"
	set packageContentsInsert to " -not -path '*.app/Contents*'"
	set recursionInsert to " -maxdepth 1"
	
	if (invisibles) then set invisiblesInsert to ""
	if (packageContents) then set packageContentsInsert to ""
	if (recursion) then set recursionInsert to ""
	
	set parameterInsert to recursionInsert & invisiblesInsert & packageContentsInsert
	
	set shellScript to "# Find all the relevant container paths.
(find " & quoted form of rootPath & " \\( -type d" & parameterInsert & "  \\) |
# Append a slash to the end of each.
sed 's|$|/|' ;
# Find all the relevant file paths.
find " & quoted form of rootPath & " \\( -type f" & parameterInsert & " \\))  |
# Collate the results into hierarchal/lexical order. ˜sort' doesn't satisfactorily sort by fields, so replace all the slashes with backspaces, which are lexically lower than anything else likely to be in the paths and will act like field delimiters in a straight sort.
sed 's|/|'$'\\b''|g' |
# Sort case-insensitively.
sort  -f |
# Further edit the paths into the required forms.
sed -E '
# Replace the root path (line 1) with the HFS equivalent.
1 c\\'$'\\n''" & (POSIX file rootPath) & "
1 !{ # With each of the other paths .
	# Restore the slashes from the backspaces.
	s|'$'\\b''|/|g
	# But again temporarily replace any trailing slash with a backspace.
	s|/$|'$'\\b''|
	# Replace the root segment and any other slash-terminated segments with tabs.
	# The ˜s' delimiter used here is a "bell" character ('$'\\a''), to minimise the chance of clashing with anything in the root path.
	s'$'\\a''" & rootPath & "/|[^/]+/'$'\\a'''$'\\t'''$'\\a''g
	# Replace any colons in the POSIX form of the item name with slashes.
	s|:|/|g
	# Replace any trailing backspace with a colon.
	s|'$'\\b''$|:|
}'"
	
	return (do shell script shellScript)
end listFolderContents

on openInTextWrangler(listing)
	tell application "TextWrangler"
		activate
		tell (make new text window with properties {text:listing}) to set its soft wrap text to false
	end tell
end openInTextWrangler