Appreciate what you are saying so let me provide some added details.
The code that I am running / testing for this purpose is as follows:
-- Pick the folder (and sub-folders) whose filename lengths will be compared / tested against maxCount
set folderTarget to (choose folder with prompt "Choose the disk / directory / folder whose file lengths' will be compared against maxCount" default location (path to home folder))
set folderTarget to POSIX path of folderTarget
set folderTarget to characters 1 through -2 of folderTarget as text -- remove the second "/" between the folder chosen and any nested folders
--set filesAll to get files of folderTarget
set filesAll to paragraphs of (do shell script "/usr/bin/find " & quoted form of folderTarget & " -type f")
set filesAllCount to count of filesAll
log "There are " & filesAllCount & " files and folders"
When I select /User/Username/Documents/ as folderTarget the above code – and, in the particular, the line of code that reads “There are " & filesAllCount " files and folders” – indicates that there are 32,668 files and folders…let’s call this number A.
When I select /User/Username/Documents/ in Finder and then select File | Gent Info, Finder reports that there are 33,658 items…let’s call this number B.
When looking at the results, the first item of note is that A <>B. while the second item of note is that the script – which reports invisible files – value of A is < Finder’s value of B.
Would appreciate confirmation that my script is working properly and/or an explanation for the difference.
Very last point…when I selected a sub-directory of /User/Username/Documents/ [i.e./User/Username/Documents/To Do List/] I was able to confirm / reconcile the script’s number of files and folders with Finder’s number of items.
With this added detail / information hopefully someone will be able to provide an explanation.
The option -type f is actual files”excluding .rtfd”but your count is assuming files and folders. If your subfolder test has no subfolders, there would be a greater chance for parity. I’m not sure what the Finder’s get info is actually counting; if you make a new folder and put just one file in it, it reports two. This may be a DS_Store or an icon file.
As -type f is actual files does this mean that it specifically excludes subfolders?
If the answer to 1. is yes then
a. I need to correct my faulty assumption that the shell script lists / returns “files and folders” to “files”.
b. Is there a way to change the shell script so that it lists BOTH files and folders?
c. is there a way to get Finder to report on the file count and folder count separately rather than items?
If I choose a subfolder that has no subfolders then:
a. The script reports n files which is the correct / proper number of files [i.e. the number of files I manually count]; and
b. Finder reports (n+1) items which presumably is 1 greater because of a hidden file [i.e.ds_, an icon file, the folder, etc.]
Agreed, per the above.
With the above said is it accurate to conclude that the script is working properly because were I to select a folder with subfolders and a “manually countable number of files” that the script returns the “manually counted” number of files?
You wondered in your post above how to make the find command return both files and folders.
The immediate answer to that is to just remove the -type f clause, however, this will make you return all kind of special files as well, as hard and soft links, which is now excluded. A hard link, is just a directory entry that points to a file another place on your file system, and a soft link, is almost the same, but it has the link to the hard file embedded inside, and can therefore point to a file on another mounted file system.
I think the do shell script will work correctly if you just remove the type f, because I don’t think you have many, or any of such special files in your documents folder.
By the way the man commands can be a bit terse, they are really for people who know the basics from before, like a “lookup” if you will. Apart from tutorials that can be found by google, there may be a book at your library, which teaches you the basics of Unix. This doesn’t fix your scripting issues at the moment, but it will be of great advantage in the long run.
The short answer, IMO, is that find with type -f is a particularly bad choice for this sort of thing (in fact for most things Mac).
As Mark pointed out, type -f excludes things like .rtfd files. And .app files, and some older iWorks files, and any other package files (.dvdpro and .abbu, for example) . But worse, it includes any files within the bundles of those package files.
So find type -f is something that works when it works, which is when you test it on a small sample or you happen to have no package files, and sits there like a time-bomb. I reckon that’s the worst kind of code to use.
It’s a file name limit, not a file path limit. The path limit is separate and depends on the file system used on the device you’re copying to, but on Mac OS I think it’s about 512 characters. (It’s defined in bytes, so I’m assuming you’re not using more obscure characters.)
I appreciate the posts and the included explanations/insights.
In terms of next steps what is the best way to proceed:
Keep the script’s use of the find command as is [i.e. use “find -type f”] on the basis that it will capture/list all files and the likelihood of a folder name exceeding the pathname length limit is minimal given it will be shorter than the file names?
Modify the script’s use of find command [i.e. use “find” without “”-type f"] on the basis that it will capture/all all the files I want listed as well as files i don’t want listed [i.e. the hidden files] but in the end will provide the information/results I am looking for?
Modify the script by using Nigel’s code from post #13 or Shane’s code from post #15?
Appreciate the response and understand that there are separate filename limits [i.e. 255 characters] and pathname limits [i.e. some other number that depends on the device I am copying to]…this is crystal clear.
Two related points are:
If a file were named /User/Joelc/Documents/test_file.docx then what is the first character in the pathname limit [i.e. is it the “/” in “/User”, is it the “U” in “User”, etc.?]?
Is it better/correct to work with the HFS nomenclature or the POSIX nomenclature?
I have given some thought to the issue surrounding the pathname and the inherent problem in using find in a shell script…I have landed that it is best to go with a solution that is reliable and verifiable…I tested the code that Nigel provided in post #15 confirmed that it both works and is very fast [apologies here as I wrongly concluded that the use of entire contents would be painfully slow]…the code is as follows:
script o
property allPaths : missing value
property theList : {}
end script
-- Get the paths to all the files in the chosen hierarchy.
set astid to AppleScript's text item delimiters
set AppleScript's text item delimiters to linefeed
tell application "Finder" to set allPaths of o to text items of (files of entire contents of FolderTarget as text)
set AppleScript's text item delimiters to asti
The two follow up that I have with respect to this code are:
How would one modify the script to get the pathnames in POSIX format rather than HFS format as everything I tried failed?
Is it possible to modify the script such that allPaths included the pathnames of the folders in addition to the files?
set folderTarget to path to documents folder
script o
property allPaths : missing value
end script
-- Get the paths to all the files in the chosen hierarchy.
tell application "Finder" to set allPaths of o to (files of entire contents of folderTarget as alias list)
set pxList to {}
repeat with i from 1 to count allPaths of o
set end of pxList to POSIX path of item i of allPaths of o
end repeat
pxList -- shows the contents of the pxlist
Appreciate the help…now all I need is a solution for getting the pathname to the folders and then I should have everything I need to finish the script given that I have worked out the Excel part over the past few days.
Hopefully you wanted a unique list of the folders pathnames.
set folderTarget to path to documents folder
script o
property allPaths : missing value
property folderPaths : missing value
end script
set astid to AppleScript's text item delimiters
-- Get the paths to all the files in the chosen hierarchy.
tell application "Finder" to set allPaths of o to (files of entire contents of folderTarget as alias list)
set {pxList, formerPaths} to {{}, {}}
set AppleScript's text item delimiters to "/"
set firstPath to (text items 1 thru -2 of (POSIX path of item 1 of allPaths of o)) as text
set {end of formerPaths, end of pxList} to {firstPath, firstPath}
repeat with i from 2 to count allPaths of o
set thisPath to (text items 1 thru -2 of (POSIX path of item i of allPaths of o)) as text
if thisPath is not in formerPaths then
set end of pxList to thisPath
set end of formerPaths to thisPath
end if
end repeat
set AppleScript's text item delimiters to astid
set o's folderPaths to pxList
return pxList
I’m afraid I haven’t been following this topic very closely and have rather lost track of what it’s trying to achieve. I posted my ‘entire contents’ script because it produces HFS paths quite quickly, which at the time were what I thought you wanted.
The Finder’s ‘entire contents’ action is usually pretty slow ” especially when there’s a large number of items in the hierarchy. But if the result’s returned ‘as text’, it’s usually a lot faster. (Text is a simpler form of data to put together and return to the script than is an alias list or a list of Finder references.) However, it’s still not as fast as a “find” shell script and it only returns HFS paths. It’s easy to derive POSIX paths individually from the HFS ones later later, but that adds still more to the time taken.
Of course, if the object’s to just test the lengths of the paths, then strictly speaking, there’s no need to convert them. If the root folder’s on the startup volume, you know that all the HFS paths are longer than the corresponding POSIX paths by the length of the startup volume’s name, so you just need to increase the maximum allowable length by that much. However, you do have to remember that HFS directory paths (as returned by the Finder) end with colons, whereas the corresponding POSIX paths (as returned by find) don’t. (By the way, find inserts its own slash between the path to the root folder and the rest of the path to a found item. So if the input path ends with a slash, as in the shell scripts earlier in this thread, there’ll be an extra slash at that point in all the found paths.)
Not entirely sure about what you mean by “intergrate”, but I mocked up a script that returns the folder path side by side with the file path. You can juggle it a little by commenting out, and commenting in some lines, if you prefer a text version. I hope this suits you, one way or another.
set folderTarget to path to documents folder
script o
property allPaths : missing value
property folderPaths : missing value
end script
set astid to AppleScript's text item delimiters
-- Get the paths to all the files in the chosen hierarchy.
tell application "Finder" to set allPaths of o to (files of entire contents of folderTarget as alias list)
set AppleScript's text item delimiters to "/"
set pxList to {}
# set pxList to "" if you are going for the text version for
repeat with i from 1 to count allPaths of o
set thisFile to POSIX path of item i of allPaths of o
set end of pxList to {thisFile, ((text items 1 thru -2 of thisFile) as text)}
# the above returns a list consisting of path to file, and another item having paths to its folder
# aka item 1 of item i of pxList and item 2 of item i of pxList
#
# set end of pxList to thisFile & ", " & ((text items 1 thru -2 of thisFile) as text) & linefeed
# the above returns a comma separated list of full path name to file, and the pathname of its folder
end repeat
set AppleScript's text item delimiters to astid
set o's folderPaths to pxList
return pxList
A huge thanks to everyone’s assistance and time spent helping me…although it has been tough slogging I do think that I am – thanks to all of you help – getting better at this.
As a result of all the help that everyone has provided – both today and in the past – I feel the need to keep the dialog going so have worked diligently on my script [i.e. at least the “file and folder identification” portion].
As far as methodology is concerned I have landed on the following approach:
a) I am identifying the list of all files [thanks to Nigel for pointing me in the right direction] without the use of the find command;
b) I am identifying the list of all folders [thanks to McUsrll for further pointing me in the right direction] without the use of the find command;
c) I am providing the user with the option of working with the HFS file format or the POSIX file format.
The resulting script is as follows:
-- set the parameters / variables
set allPaths to {} -- The list of all files in HFS format
set allPathsPX to {} -- The list of all files in POSIX format
set theList to {} -- The list of all files whose pathname > maxCount in HFS format
set theListPX to {} -- The list of all files whole pathname > maxCount in POSIX format
set firstPath to {} -- The temporary holder for the "current index -1" folder
set secondPath to {} -- The temporary holder for the "current index" folder
set allPathsFolder to {} -- The list of all folders in HFS format
set allPathsFolderPX to {} -- The list of all folders in POSIX format
set maxCount to 200 -- As a temporary / testing maxCount
set FolderTarget to (choose folder with prompt "Choose the disk / directory / folder whose file lengths' will be compared against maxCount" default location (path to documents folder))
-- Get the pathnames of all the files in the chosen disk / directory / folder
set astid to AppleScript's text item delimiters
set AppleScript's text item delimiters to linefeed
tell application "Finder" to set allPaths to text items of (files of entire contents of FolderTarget as text)
-- Get the pathnames of all the folders (as opposod to "files") in the chosen disk / directory / folder
set AppleScript's text item delimiters to ":"
repeat with i from 1 to count allPaths
if ((i = 1) is true) then
set firstPath to (text items 1 through -2 of item i of allPaths)
set secondPath to (text items 1 through -2 of item i of allPaths)
set end of allPathsFolder to firstPath
end if
if ((i > 1) is true) then
set secondPath to (text items 1 through -2 of item i of allPaths) -- Get the folder pathname of the next file
if ((firstPath is not equal to secondPath) is true) then -- The folder pathname of the next file differs from the folder pathname of the current file
****if (((text items 1 through -2 of firstPath) is not equal to (text items 1 through -2 of secondPath)) is true) then -- Test whether a nested (next level) folder exists
set allreadyThere to false -- Test whether the nested (next level) folder previously noted
repeat with j from 1 to count allPathsFolder
if (((text items 1 through -2 of secondPath as text) = item j of allPathsFolder as text) is true) then set allreadyThere to true
end repeat
if allreadyThere is false then set end of allPathsFolder to (text items 1 through -2 of secondPath)
set end of allPathsFolder to secondPath
set firstPath to secondPath****
else
set end of allPathsFolder to secondPath
set firstPath to secondPath
end if
end if
end if
end repeat
-- Provide the user with the option of using the HFS or POSIX file format
display dialog "Are the filename lengths being tested in HFS or POSIX file format?" buttons {"HFS", "POSIX"} default button 2 with icon note
set fileFormat to button returned of result
if ((fileFormat = "POSIX") is true) then
repeat with k from 1 to count allPaths
set end of allPathsPX to POSIX path of item k of allPaths
end repeat
repeat with k from 1 to count allPathsFolder
set end of allPathsFolderPX to POSIX path of ((item k of allPathsFolder as text) & ":")
end repeat
end if
if ((fileFormat = "POSIX") is true) then
set allPaths to {}
set allPathsFolder to {}
repeat with l from 1 to count allPathsPX
set end of allPaths to item l of allPathsPX
end repeat
repeat with l from 1 to count allPathsFolderPX
set end of allPathsFolder to item l of allPathsFolderPX
end repeat
end if
-- Test that all files and folders were processed
log "Total files are " & (count of allPaths)
log "Total folders are " & (count of allPathsFolder)
log "Total files and folders to compare to finder are " & ((count of allPaths) + (count of allPathsFolder) + 1) -- The 1 is for the parent folder
-- Reset / return AppleScript's text item delimiter to its original value
set AppleScript's text item delimiters to astid
-- Create the list of files whose pathname > maxCount and return the list
repeat with n from 1 to (count allPaths)
set thisPath to item n of allPaths
if ((count thisPath) > maxCount) then set end of theList to thisPath -- as alias
end repeat
return theList
The resulting script is – because I am new to scripting – not as efficient, elegant or as short as possible but it “more or less works”.
The script was more complicated to code than suggested in the previous posts because of “nested” / “next level” folders that contained only folders [i.e. no files]. The script – without the portion which is contained within the **** – was not identifying folders that contained only folders [no files].
When I run the script on most of disks /directories / folders that I tested I do get the proper number of files and folders reported [i.e. script metrics match finder metrics]…when I run the script on a heavily nested folder I no longer get the proper number of files and folders reported [i.e. script metrics do NOT match finder metrics]…when, for example, I ran the script on the Documents folder the script reports 33,451 total files and folders while Finder reports 33,660 for a difference of 290.
Would appreciate any assistance / insights people have to help me fix this.
Thanks.
PS. Next and hopefully the final step will occur tomorrow when I fix the Excel portion of the coding…I will then post the script for all to see / use / etc.!
this is a version using find which is a bit faster (even with two shell calls)
script o
property filePaths : ""
property folderPaths : ""
end script
set maxCount to 80 -- As a temporary / testing maxCount
set theList to {} -- The list of all files whose pathname > maxCount
set FolderTarget to POSIX path of (choose folder with prompt "Choose the disk / directory / folder whose file lengths' will be compared against maxCount" default location (path to documents folder))
set o's folderPaths to paragraphs of (do shell script "/usr/bin/find " & quoted form of FolderTarget & " -type d")
set o's filePaths to paragraphs of (do shell script "/usr/bin/find " & quoted form of FolderTarget & " -type f ! -type l ! -name '.*'")
display dialog "Are the filename lengths being tested in HFS or POSIX file format?" buttons {"HFS", "POSIX"} default button 2 with icon note
set fileFormat to button returned of result
if fileFormat = "POSIX" then
repeat with aPath in o's filePaths
if (length of aPath) > maxCount then set end of theList to contents of aPath -- as alias
end repeat
else
repeat with aPath in o's filePaths
set HFSPath to POSIX file aPath as text
if (length of HFSPath) > maxCount then set end of theList to HFSPath -- as alias
end repeat
end if
log "Total files are " & (count of o's filePaths)
log "Total folders are " & (count of o's folderPaths)
log "Total files and folders to compare to finder are " & ((count of o's filePaths) + (count of o's folderPaths) + 1) -- The 1 is for the parent folder
return theList
Appreciate the script and will look at it later in the day…amazing how a few lines of code with two shell calls can replicate the much longer and and more involved code that I wrote.
I still need / want to figure out / fix my code for the additional nested folders to get it working!
