Listing disk/folder hierarchy contents to text file

(1) Thanks, Yvan. A Happy New Year to you too! :slight_smile:

(2) Thanks too for the “heads up” about the timeout and for your other improvements to the script. I played around with “ls” and “find” this afternoon, but wasn’t keen on the resulting layout of the text. However, Dan seems pleased, so problem solved!

I’ll have to look into that. I know it speeds up execution when the script’s run as an application and it contains commands addressed to another application (something to do with communication between OSAX and application being faster than direct communication between applications), but I haven’t heard of it speeding things up anyway.

Cheers, Dan. You too. Glad you got what you wanted. :slight_smile:

Both work excellently Yvan. Thank you very much for your help!

It seems that my poor English was not clear enough.

Your original script which speak to Finder may take benefit from the tip but for sure, it would be more efficient in scripts triggering many times external applications (in fact external app’s dictionaries or OSAX)
For instance I use it daily in a script which store the location of icons on my Desktop to reset them when I boot the machine. The difference of speed is astonishing.
So, I inserted it in most of my script so, it’s now second nature and maybe, sometimes I use it in scripts where it’s useless.
I’m not sure that it’s useful in the script which use “ls”

I posted the script using “ls” because I thought that the layout is sufficient to retrieve the files thru Spotlight (and because I try to remove calls to Finder when I may do. I feel no need to take benefit of all the layers of code which are between Finder and System Events which is in fact the one which does the job.
To do what you wanted, I feel that two schemes where viable:
(1) the one which you coded triggering the Finder
(2) the one which I used which trigger “low level” tools.
What’s fine is that with mine, I may continue to work when the script is scanning a huge volume which is not the case when it’s the Finder which do the job.
I didn’t pay attention to a scheme using System Events because this one doesn’t offer an ‘entire contents’ feature.

Yvan KOENIG (VALLAURIS, France) samedi 15 janvier 2011 23:10:55

Thanks Nigel and Yvan for clarifying that. I’m sure I can make use of it.

Just for my own satisfaction, here’s the version for which I’d been groping. It’s no better than Yvan’s for Dan’s purposes, but it formats the text prettily. :slight_smile:

-- Create the text to be written to the file.
-- Just a heading and the item names, indented according to their positions in the hierarchy.
-- (Uncomment the (* *) comment markers to preserve the full paths.)
on createText(posixPaths)
	script o
		property paths : posixPaths
	end script
	
	set rootPath to beginning of o's paths
	set item 1 of o's paths to "Entire contents of " & rootPath & linefeed
	set astid to AppleScript's text item delimiters
	-- (*
	considering case
		set AppleScript's text item delimiters to ""
		set tabStr to {tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab, tab} as text -- Hopefully more than needed!
		
		set AppleScript's text item delimiters to "/"
		set nonIndent to (count rootPath's text items) -- 1
		
		set len to (count posixPaths)
		repeat with i from 2 to len
			set thisPath to item i of o's paths
			set tiCount to (count thisPath's text items)
			set thisName to text item -1 of thisPath
			-- If the item name contains any colons, restore the original slashes.
			if (thisName contains ":") then
				set AppleScript's text item delimiters to ":"
				set thisName to thisName's text items
				set AppleScript's text item delimiters to "/"
				set thisName to thisName as text
			end if
			-- If this is a folder path, append a colon to the name.
			if ((i < len) and (item (i + 1) of o's paths begins with thisPath)) then set thisName to thisName & ":" -- or "/", if preferred.
			set item i of o's paths to text 1 thru (tiCount - nonIndent) of tabStr & thisName
		end repeat
		-- *)
		set AppleScript's text item delimiters to linefeed
		set outText to o's paths as text
		-- (*
		set AppleScript's text item delimiters to linefeed & tab
		set outText to outText's text items
		set AppleScript's text item delimiters to linefeed
		set outText to outText as text
	end considering
	-- *)
	set AppleScript's text item delimiters to astid
	
	return outText
end createText

-- Write the text to file as UTF-8.
on writeTextFile(txt, defaultLoc)
	set f to (choose file name with prompt "Save the UTF-8 text listing as." default name (paragraph 1 of txt) & ".txt" default location defaultLoc)
	set fRef to (open for access f with write permission)
	try
		set eof fRef to 0
		write «data rdatEFBBBF» to fRef -- UTF-8 BOM.
		write txt as «class utf8» to fRef
	end try
	close access fRef
	
	display dialog "The listing has been saved in file \"" & f & "\"" buttons {"OK"} default button 1
end writeTextFile

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	
	-- List the hierarchy as POSIX paths, omitting any that contain elements beginning with ".".
	set thePaths to paragraphs of (do shell script "find -f " & (quoted form of POSIX path of rootFolder) & " \\! -path \"*/.*\"")
	
	set outText to createText(thePaths)
	writeTextFile(outText, (path to documents folder))
end main

main()

Edit: It occurs to me that it’s possible (although frowned upon in some quarters) for a file or folder name to contain slash (“/”) characters. In the POSIX paths returned by “ls” or “find”, these will have been rendered as colons; so ideally, the script needs to look out for these and convert them back to slashes for the text file. I’ve modified the above to that end.

I agree Nigel, it’s much easier on the eye. Any chance we could save it as a .rtf file and use Optima as the font? :slight_smile:
And perhaps bold the folder names?

See what you started? :slight_smile:

Must leave you something to aspire to. :wink:

Actually, I was thinking the same thing. (It may take a while) but I’ll consider it my first project and post back here when I get it done!

I can assure you I will NEVER understand to the degree you and Yvan do; I don’t have this great a need. If you don’t mind my asking, how long have you been Applescripting, Nigel? How do you typically apply or use your skills?

Dan

Open the text file with TextEdit then apply the font of your choice.

Today I’m too tired to edit the script to do that.

Yvan KOENIG (VALLAURIS, France) dimanche 16 janvier 2011 22:07:16

Amazing and fascinating, as usual. Always impressed by Nigel and Yvan.

I have an idea that dyost, and others, might like: Instead of turning the listings file into RTF and adding formatting, why not use a single text file as a database for all your disks/discs and add a function to search it for the file you are seeking?



set the_task to button returned of (display dialog "Choose a task." buttons {"Cancel", "Catalog", "Search"})

if the_task is "Catalog" then
	my catalog()
else
	my search()
end if


on catalog()
	-- Yvan's or Nigel's script (modified to append to a specified file)
end catalog


on search()
	-- have user enter search string
	-- read in catalog file
	-- search for search string
	-- make a list of results as complete paths
	-- format results as a numbered list of tabbed entries like this:
	
	-- Item 01
	-- 	My Book 02:
	-- 		iTunes - My Book 02:
	--			Music:
	--				Biréli Lagrène:
	--					Live In Marciac:
	--						07 C' est Si Bon.mp3"
	
	-- write results to a TextEdit file
	-- display dialog which asks the user to enter the number for the file to reveal
	-- tell Finder to reveal the file (if the disk/disc is not present, ask for the disk/disc)
end search

I’d be willing to take a shot at writing this. (Stop laughing, I’m serious.)

But it seems to me that it would be better to get the file listings as complete paths like the Finder does:

“My Book 02:iTunes - My Book 02:Music:Biréli Lagrène:Live In Marciac:07 C’ est Si Bon.mp3”

However, after seeing how much faster that shell script is than “tell Finder get entire contents”, I’m wondering if there is a shell script that gets the listings as complete paths.

Thanks,
icta

Humor me, I’m old.

Hi, icta.

The “find” shell script in my script in post #11 returns the full POSIX paths to the items in the hierarchy, sorted (as far as I can make out) lexically by path. This has the effect of sorting the items by hierarchy too, since each folder path is followed immediately by the paths to the items in that folder. It’s the createText() handler which substitutes tabs for the path elements which aren’t the items’ names. You can stop that by cutting out the two sections in it which are delimited by – (* and – *).

Hello Nigel

I’m back here because with your script of message #11 as well as with mines, I get the error :

error “find: /Volumes/baseÆ’ebay//.Trashes: Permission denied” number 1

when I try to scan a volume (here it’s a disk image)

And of course, I don’t guess a way to filter it without loosing the listing.

Even with a bit of practice, sometimes I hit a wall :frowning:

Yvan KOENIG (VALLAURIS, France) dimanche 20 février 2011 17:35:46

Running the script as administrator helps to get around this problem
(quite annoying that you have to give your pw however)

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	set theShellFindScript to "find -f " & (quoted form of POSIX path of rootFolder) & " \\! -path \"*/.*\""
	-- List the hierarchy as POSIX paths, omitting any that contain elements beginning with ".".
	set thePaths to paragraphs of (do shell script theShellFindScript with administrator privileges)
	
	set outText to createText(thePaths)
	writeTextFile(outText, (path to documents folder))
end main

Hi Bramster and Yvan.

Thanks for your comments. Sorry I’ve been so long getting back!

I don’t remember if I saw Yvan’s post at the time or how or why I responded if I did. Possibly I couldn’t reproduce the error. I would certainly have been puzzled by the fact that a path excluded by the “find” parameters was apparently causing it.

It so happens that a couple of days before Bramster posted his work-round, I wrote a similar script again from scratch using “sed” to edit the paths. This script doesn’t throw an error when tested on a couple of mounted disk images, whereas the script in post #11 does. Initially confusing, since they use similar “find” parameters.

It turns out that the error does occur in my more recent script, but is ignored because there are more commands after “find” in the shell script. Putting a dummy command after “find” in the original shell script ” say “echo "\c"” ” stops the error message from appearing and ” as far as I can see ” produces an identical listing to Bramster’s work-round where the shell script’s run with administrator privileges. The same thing’s probably possible by redirecting Standard Error, but I haven’t got that far with my research!

on main()
	set rootFolder to (choose folder with prompt "Choose a folder or disk to catalogue.")
	-- List the hierarchy as POSIX paths, omitting any that contain elements beginning with ".".
	-- The "echo" at the end is a dummy to sweep errors under the carpet!
	set theShellFindScript to "find -f " & (quoted form of POSIX path of rootFolder) & " \\! -path \"*/.*\" ; echo \"\\c\""
	set thePaths to paragraphs of (do shell script theShellFindScript)
	
	set outText to createText(thePaths)
	writeTextFile(outText, (path to documents folder))
end main

The “sed” script, if you’re interested, opens the listing directly in TextWrangler, but can easily be adapted to write to file instead:

main()

on main()
	-- Parameters set to return the same items as the script in post #11.
	set listing to (listFolderContents of (POSIX path of (choose folder)) with recursion and packageContents without invisibles)
	openInTextWrangler(listing)
end main

on listFolderContents of rootPath given invisibles:invisibles, packageContents:packageContents, recursion:recursion
	set invisiblesInsert to " -not -path '*/.*' -flags nohidden"
	set packageContentsInsert to " -not -path '*.app/Contents*'"
	set recursionInsert to " -maxdepth 1"
	
	if (invisibles) then set invisiblesInsert to ""
	if (packageContents) then set packageContentsInsert to ""
	if (recursion) then set recursionInsert to ""
	
	set parameterInsert to recursionInsert & invisiblesInsert & packageContentsInsert
	
	set shellScript to "# Find all the relevant container paths.
(find " & quoted form of rootPath & " \\( -type d" & parameterInsert & "  \\) |
# Append a slash to the end of each.
sed 's|$|/|' ;
# Find all the relevant file paths.
find " & quoted form of rootPath & " \\( -type f" & parameterInsert & " \\))  |
# Collate the results into hierarchal/lexical order. ˜sort' doesn't satisfactorily sort by fields, so replace all the slashes with backspaces, which are lexically lower than anything else likely to be in the paths and will act like field delimiters in a straight sort.
sed 's|/|'$'\\b''|g' |
# Sort case-insensitively.
sort  -f |
# Further edit the paths into the required forms.
sed -E '
# Replace the root path (line 1) with the HFS equivalent.
1 c\\'$'\\n''" & (POSIX file rootPath) & "
1 !{ # With each of the other paths .
	# Restore the slashes from the backspaces.
	s|'$'\\b''|/|g
	# But again temporarily replace any trailing slash with a backspace.
	s|/$|'$'\\b''|
	# Replace the root segment and any other slash-terminated segments with tabs.
	# The ˜s' delimiter used here is a "bell" character ('$'\\a''), to minimise the chance of clashing with anything in the root path.
	s'$'\\a''" & rootPath & "/|[^/]+/'$'\\a'''$'\\t'''$'\\a''g
	# Replace any colons in the POSIX form of the item name with slashes.
	s|:|/|g
	# Replace any trailing backspace with a colon.
	s|'$'\\b''$|:|
}'"
	
	return (do shell script shellScript)
end listFolderContents

on openInTextWrangler(listing)
	tell application "TextWrangler"
		activate
		tell (make new text window with properties {text:listing}) to set its soft wrap text to false
	end tell
end openInTextWrangler

Read this post. It basically says everything I had in mind to write :slight_smile:

For the lazy ones, we’re looking for something like this:

find . -name "myfile" && grep -v "Permission denied"

Thanks, DJ! “true” seems a bit less contrived that my “echo "\c"” suggestion. :slight_smile:

What a shame. She only abdicated a couple of days ago! :wink:

:slight_smile: it’s an bit of an odd/harsh saying but it’s the only saying or greet we have when the crown is given to the next generation without any time window between them. The title of queen for beatrix is dead, not the person luckely. Does Prince Charles ever going to be a king? No offence but he was, at least I think, the oldest prince at the ceremony.

Hello.

Here is one that is a bit more contrived, the philosophy is that the files is there, even if I don’t currently have the permissions to them.

find -H -L . -name \*.1  2>&1 |sed 's/: Permission denied//g'

(The -H and -L is for returning any info about the link, not what it points to.)

Maybe not when you are going to run a script over the result.

But then again, this should be faster:

 find  . -name \*.1  2>/dev/null

If there is any trouble with the exit code, then we can fake it for good measure:

 find  . -name \*.1  2>/dev/null || true

Good Luck with your new Kings, I am not very royalistic, but I dare say, that if you get something anywhere near ours, then your good! :slight_smile:

You really should test your code and your assertions before posting! :lol:

By itself, this searches the entire startup volume, which takes for ever.
After a cd to something a little shallower, it returns items whose names end with “.1”.
In the context of the script in post #11, it causes errors where there were none before.

:slight_smile:

I’m so sorry; I only meant to give examples and those were intended to find all “section 1” man files, and the snippets all do their job, flawlessly. I recommend using the last version with || true at the end, as it negates an exit code other than zero. (After you have changed the regexp (glob) to suit your needs of course.) :wink: