I want to improve the speed when Getting Entire Contents of Folder in FileManagerLib of @Shane Stanley.
I think, following handler can be improved if I have AsObjC value for True constant (it is some NS number as I understand) before entering in the repeat loop.
I want to avoid coercion theValue as boolean. I want to try if theValue is AsObjCTrue.
on filesAndPackagesInURLArray:theURLs
-- set AsObjCTrue to ????? -- how to get THIS
set itemURLs to NSMutableArray's array()
repeat with aURL in theURLs -- is it a directory?
set {theResult, theValue, theError} to (aURL's getResourceValue:(reference) forKey:NSURLIsDirectoryKey |error|:(reference))
if (theValue as boolean) then -- is it a package?
set {theResult, theValue, theError} to (aURL's getResourceValue:(reference) forKey:NSURLIsPackageKey |error|:(reference))
if (theValue as boolean) then (itemURLs's addObject:aURL)
else
(itemURLs's addObject:aURL)
end if
end repeat
return itemURLs
end filesAndPackagesInURLArray:
It gives speed improvment 1.32 times :lol: Tested with folder containing 1645 files.
Thank you very much, @Shane for your help:
-- Get Entire Contents of Folder (AsObjC, @KniazidisR with help from @Shane Stanley)
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
-- classes, constants, and enums used
property NSMutableArray : a reference to current application's NSMutableArray
property NSURLIsDirectoryKey : a reference to current application's NSURLIsDirectoryKey
property |NSURL| : a reference to current application's |NSURL|
property NSFileManager : a reference to current application's NSFileManager
property NSURLIsPackageKey : a reference to current application's NSURLIsPackageKey
property NSURLPathKey : a reference to current application's NSURLPathKey
property NSArray : a reference to current application's NSArray
property NSString : a reference to current application's NSString
set theFolder to POSIX path of (path to scripts folder from user domain)
my entireContentsOf:theFolder includeInvisibles:false includeFiles:true includeFolders:false resultType:"paths"
-- Returns the entire contents of a folder and its subfolders.
-- If includeFolders is false, only files (including packages) will be returned.
-- If includeFiles is false, only folders will be returned.
-- The resultType parameter takes a string:, "paths" to get POSIX paths, "names" to get just the names, "POSIX names" to get POSIX names (the / character appears as a :), "urls" to get an array of NSURLs, or "files" to get AppleScript file references. The latter is only available under macOS 10.11 and later.
on entireContentsOf:aFileOrPath includeInvisibles:incInvis includeFiles:incFiles includeFolders:incFolders resultType:resType
set theURL to my makeURLFromFileOrPath:aFileOrPath
set theOptions to 2 -- NSDirectoryEnumerationSkipsPackageDescendants
if not incInvis then set theOptions to 6 -- NSDirectoryEnumerationSkipsHiddenFiles
set theEnumerator to NSFileManager's |defaultManager|()'s enumeratorAtURL:theURL includingPropertiesForKeys:{} options:theOptions errorHandler:(missing value)
set theURLs to theEnumerator's allObjects()
if not incFolders then
set theURLs to my filesAndPackagesInURLArray:theURLs
else if not incFiles then
set theURLs to my foldersInURLArray:theURLs
end if
return my convertURLs:theURLs resultType:resType
end entireContentsOf:includeInvisibles:includeFiles:includeFolders:resultType:
-- This handler is called by other handlers
on makeURLFromFileOrPath:theFileOrPathInput
-- make it into a Cocoa object for easier comparison
set theFileOrPath to (NSArray's arrayWithObject:theFileOrPathInput)'s firstObject()
if (theFileOrPath's isKindOfClass:(NSString)) as boolean then
if (theFileOrPath's hasPrefix:"/") as boolean then -- full POSIX path
return |NSURL|'s fileURLWithPath:theFileOrPath
else if (theFileOrPath's hasPrefix:"~") as boolean then -- POSIX path needing ~ expansion
return |NSURL|'s fileURLWithPath:(theFileOrPath's |stringByExpandingTildeInPath|())
else -- must be HFS path
return |NSURL|'s fileURLWithPath:(POSIX path of theFileOrPathInput)
end if
else if (theFileOrPath's isKindOfClass:(|NSURL|)) as boolean then -- happens with files and aliases in 10.11
return theFileOrPath
else -- must be a file or alias
return |NSURL|'s fileURLWithPath:(POSIX path of theFileOrPathInput)
end if
end makeURLFromFileOrPath:
-- This handler is called by other handlers
on filesAndPackagesInURLArray:theURLs
set AsObjCTrue to current application's NSNumber's numberWithBool:true
set itemURLs to NSMutableArray's array()
repeat with aURL in theURLs -- is it a directory?
set {theResult, theValue, theError} to (aURL's getResourceValue:(reference) forKey:NSURLIsDirectoryKey |error|:(reference))
if theValue = AsObjCTrue then -- is it a package?
set {theResult, theValue, theError} to (aURL's getResourceValue:(reference) forKey:NSURLIsPackageKey |error|:(reference))
if theValue = AsObjCTrue then (itemURLs's addObject:aURL)
else
(itemURLs's addObject:aURL)
end if
end repeat
return itemURLs
end filesAndPackagesInURLArray:
-- This handler is called by other handlers
on convertURLs:theURLs resultType:resType
considering numeric strings
if resType = "names" then
return ((((theURLs's valueForKey:"lastPathComponent")'s componentsJoinedByString:(character id 0))'s stringByReplacingOccurrencesOfString:":" withString:"/")'s componentsSeparatedByString:(character id 0)) as list
else if resType = "POSIX names" then
return (theURLs's valueForKey:"lastPathComponent") as list
else if resType = "urls" then
return theURLs
else if resType = "files" and AppleScript's version > "2.4" then
return theURLs as list
else
return (theURLs's valueForKey:"path") as list
end if
end considering
end convertURLs:resultType:
.
To compare the speed, here is original version from FileManagerLib:
-- Get Entire Contents of Folder (from FileManagerLib of @Shane Stanley)
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
-- classes, constants, and enums used
property NSMutableArray : a reference to current application's NSMutableArray
property NSURLIsDirectoryKey : a reference to current application's NSURLIsDirectoryKey
property |NSURL| : a reference to current application's |NSURL|
property NSFileManager : a reference to current application's NSFileManager
property NSURLIsPackageKey : a reference to current application's NSURLIsPackageKey
property NSURLPathKey : a reference to current application's NSURLPathKey
property NSArray : a reference to current application's NSArray
property NSString : a reference to current application's NSString
set theFolder to POSIX path of (path to scripts folder from user domain)
my entireContentsOf:theFolder includeInvisibles:false includeFiles:true includeFolders:false resultType:"paths"
-- Returns the entire contents of a folder and its subfolders.
-- If includeFolders is false, only files (including packages) will be returned.
-- If includeFiles is false, only folders will be returned.
-- The resultType parameter takes a string:, "paths" to get POSIX paths, "names" to get just the names, "POSIX names" to get POSIX names (the / character appears as a :), "urls" to get an array of NSURLs, or "files" to get AppleScript file references. The latter is only available under macOS 10.11 and later.
on entireContentsOf:aFileOrPath includeInvisibles:incInvis includeFiles:incFiles includeFolders:incFolders resultType:resType
set theURL to my makeURLFromFileOrPath:aFileOrPath
set theOptions to 2 -- NSDirectoryEnumerationSkipsPackageDescendants
if not incInvis then set theOptions to 6 -- NSDirectoryEnumerationSkipsHiddenFiles
set theEnumerator to NSFileManager's |defaultManager|()'s enumeratorAtURL:theURL includingPropertiesForKeys:{} options:theOptions errorHandler:(missing value)
set theURLs to theEnumerator's allObjects()
if not incFolders then
set theURLs to my filesAndPackagesInURLArray:theURLs
else if not incFiles then
set theURLs to my foldersInURLArray:theURLs
end if
return my convertURLs:theURLs resultType:resType
end entireContentsOf:includeInvisibles:includeFiles:includeFolders:resultType:
-- This handler is called by other handlers
on makeURLFromFileOrPath:theFileOrPathInput
-- make it into a Cocoa object for easier comparison
set theFileOrPath to (NSArray's arrayWithObject:theFileOrPathInput)'s firstObject()
if (theFileOrPath's isKindOfClass:(NSString)) as boolean then
if (theFileOrPath's hasPrefix:"/") as boolean then -- full POSIX path
return |NSURL|'s fileURLWithPath:theFileOrPath
else if (theFileOrPath's hasPrefix:"~") as boolean then -- POSIX path needing ~ expansion
return |NSURL|'s fileURLWithPath:(theFileOrPath's |stringByExpandingTildeInPath|())
else -- must be HFS path
return |NSURL|'s fileURLWithPath:(POSIX path of theFileOrPathInput)
end if
else if (theFileOrPath's isKindOfClass:(|NSURL|)) as boolean then -- happens with files and aliases in 10.11
return theFileOrPath
else -- must be a file or alias
return |NSURL|'s fileURLWithPath:(POSIX path of theFileOrPathInput)
end if
end makeURLFromFileOrPath:
-- This handler is called by other handlers
on filesAndPackagesInURLArray:theURLs
set itemURLs to NSMutableArray's array()
repeat with aURL in theURLs -- is it a directory?
set {theResult, theValue, theError} to (aURL's getResourceValue:(reference) forKey:NSURLIsDirectoryKey |error|:(reference))
if theValue as boolean then -- is it a package?
set {theResult, theValue, theError} to (aURL's getResourceValue:(reference) forKey:NSURLIsPackageKey |error|:(reference))
if theValue as boolean then
(itemURLs's addObject:aURL)
end if
else
(itemURLs's addObject:aURL)
end if
end repeat
return itemURLs
end filesAndPackagesInURLArray:
-- This handler is called by other handlers
on convertURLs:theURLs resultType:resType
considering numeric strings
if resType = "names" then
return ((((theURLs's valueForKey:"lastPathComponent")'s componentsJoinedByString:(character id 0))'s stringByReplacingOccurrencesOfString:":" withString:"/")'s componentsSeparatedByString:(character id 0)) as list
else if resType = "POSIX names" then
return (theURLs's valueForKey:"lastPathComponent") as list
else if resType = "urls" then
return theURLs
else if resType = "files" and AppleScript's version > "2.4" then
return theURLs as list
else
return (theURLs's valueForKey:"path") as list
end if
end considering
end convertURLs:resultType:
Version with improvment: 3.57 sec
Original version: 4.73 sec
I have updated 2 scripts in the post #4, because the 2-nd script was posted with some error. And removed the properties, which is not in use. The timing results remain as was.
Using the AsObjC constant for True (instead of the coercion keyValue as boolean) gives me a 1.5x speed gain here:
-- Get Alias files of a folder and its subfolders (not inside Packages) (AsObjC)
use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions
-- get the folder URL
set posixPath to POSIX path of (path to scripts folder from user domain)
set theFolderURL to current application's NSURL's fileURLWithPath:posixPath
-- get URL of all items in folder, recursively
set isAliasKey to current application's NSURLIsAliasFileKey
set theFileManager to current application's NSFileManager's defaultManager()
set allURLs to (theFileManager's enumeratorAtURL:theFolderURL includingPropertiesForKeys:{isAliasKey} options:((current application's NSDirectoryEnumerationSkipsPackageDescendants) + ((current application's NSDirectoryEnumerationSkipsHiddenFiles) as integer)) errorHandler:(missing value))'s allObjects()
-- build an array with URLs to alias files
set filteredArray to current application's NSMutableArray's new()
set AsObjCTrue to current application's NSNumber's numberWithBool:true
repeat with oneURL in allURLs
set keyValue to end of (oneURL's getResourceValue:(reference) forKey:isAliasKey |error|:(missing value))
if keyValue is AsObjCTrue then (filteredArray's addObject:oneURL)
end repeat
-- convert URL array to file list
return filteredArray as list
Using the AsObjC constant for True (instead of the coercion isDirectory as boolean) gives me a 1.5x speed gain here:
-- Get Posix paths of Files of Folder and its Subfolders (AsObjC)
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
set theFolder to POSIX path of (path to scripts folder from user domain)
set theFiles to getFiles(theFolder)
on getFiles(theFolder)
set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
set fileManager to current application's NSFileManager's new()
set enumKey to current application's NSURLIsDirectoryKey
set enumOptions to (current application's NSDirectoryEnumerationSkipsPackageDescendants as integer) + (current application's NSDirectoryEnumerationSkipsHiddenFiles as integer)
set entireContents to (fileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{enumKey} options:enumOptions errorHandler:(missing value))'s allObjects()
set AsObjCTrue to current application's NSNumber's numberWithBool:true
repeat with anItem in entireContents
set {theResult, isDirectory} to (anItem's getResourceValue:(reference) forKey:(enumKey) |error|:(missing value))
if isDirectory is AsObjCTrue then set contents of anItem to {}
end repeat
return text of (entireContents's |path| as list)
end getFiles
Using the AsObjC constant for True (instead of the coercion as boolean) gives me a 1.4x speed gain here:
-- Get Posix paths of Executable Files of Folder and its Subfolders (AsObjC)
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
set theFolder to POSIX path of (path to scripts folder from user domain)
set theFiles to getExecutableFilesFromEntireContents(theFolder)
on getExecutableFilesFromEntireContents(theFolder)
set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
set fileManager to current application's NSFileManager's new()
set enumKey to current application's NSURLIsDirectoryKey
set enumOptions to (current application's NSDirectoryEnumerationSkipsPackageDescendants as integer) + (current application's NSDirectoryEnumerationSkipsHiddenFiles as integer)
set entireContents to (fileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{enumKey} options:enumOptions errorHandler:(missing value))'s allObjects()
set AsObjCTrue to current application's NSNumber's numberWithBool:true
repeat with anURL in entireContents
set {theResult, isDirectory} to (anURL's getResourceValue:(reference) forKey:(enumKey) |error|:(missing value))
if isDirectory is AsObjCTrue then
set contents of anURL to {}
else
set {theResult, isExecutable} to (anURL's getResourceValue:(reference) forKey:(current application's NSURLIsExecutableKey) |error|:(missing value))
if not (isExecutable is AsObjCTrue) then set contents of anURL to {}
end if
end repeat
return text of (entireContents's |path| as list)
end getExecutableFilesFromEntireContents
The speed improved about 1.32 times with 1981 items (files and folders) in entire contents of the folder:
-- Get the Files in Entire Contents of Folder by Extension, the Sort them (AsObjC)
use AppleScript version "2.4"
use framework "Foundation"
use scripting additions
property |⌘| : a reference to current application
property NSPredicate : a reference to NSPredicate of |⌘|
property NSFileManager : a reference to NSFileManager of |⌘|
property |NSURL| : a reference to |NSURL| of |⌘|
property NSMutableArray : a reference to NSMutableArray of |⌘|
property NSURLIsRegularFileKey : a reference to NSURLIsRegularFileKey of |⌘|
property NSSortDescriptor : a reference to NSSortDescriptor of |⌘|
property NSDirectoryEnumerationSkipsPackageDescendants : a reference to 2
property NSDirectoryEnumerationSkipsHiddenFiles : a reference to 4
set sourceFolder to POSIX path of (path to scripts folder from user domain)
set sourceURL to |NSURL|'s URLWithString:sourceFolder
set fileManager to NSFileManager's |defaultManager|()
set fileKey to NSURLIsRegularFileKey
set searchOptions to (NSDirectoryEnumerationSkipsPackageDescendants) + (NSDirectoryEnumerationSkipsHiddenFiles)
-- Get entire contents of folder, includung contents of subfolders, without packages and hidden files
set entireContents to (fileManager's enumeratorAtURL:(sourceURL) includingPropertiesForKeys:({fileKey}) options:(searchOptions) errorHandler:(missing value))'s allObjects()
-- Filter case-insensitively for items with "pdf" extensions.
set thePredicate to NSPredicate's predicateWithFormat:("pathExtension ==[c] 'scpt'")
set urlArray to entireContents's filteredArrayUsingPredicate:(thePredicate)
-- The result is probably just the files we want, but check for any folders among them while getting their paths.
set theFiles to NSMutableArray's new()
set AsObjCTrue to current application's NSNumber's numberWithBool:true
repeat with theURL in urlArray
if ((theURL's getResourceValue:(reference) forKey:(fileKey) |error|:(missing value))'s end) is AsObjCTrue then
tell theFiles to addObject:(theURL)
end if
end repeat
-- Sort the remaining URLs on their paths.
set sortDescriptor to NSSortDescriptor's sortDescriptorWithKey:("path") ascending:(true) selector:("localizedStandardCompare:")
theFiles's sortUsingDescriptors:({sortDescriptor})
set theFiles to theFiles as list
The speed improved about 1.24 times with 1981 items (files and folders) in entire contents of the folder:
-- Count files in Entire Contents of Folder by extension, then Report to text file (AsObjC)
use AppleScript version "2.3.1" -- macOS 10.9 (Mavericks) or later
use framework "Foundation"
use scripting additions
property |NSURL| : current application's class "NSURL"
property NSArray : current application's class "NSArray"
property NSFileManager : a reference to current application's NSFileManager
property NSURLIsRegularFileKey : a reference to current application's NSURLIsRegularFileKey
property NSURLIsPackageKey : a reference to current application's NSURLIsPackageKey
property NSDirectoryEnumerationSkipsPackageDescendants : a reference to current application's NSDirectoryEnumerationSkipsPackageDescendants
property NSDirectoryEnumerationSkipsHiddenFiles : a reference to current application's NSDirectoryEnumerationSkipsHiddenFiles
property NSPredicate : a reference to current application's NSPredicate
property NSCountedSet : a reference to current application's NSCountedSet
property NSNumber : a reference to current application's NSNumber
property NSDictionary : a reference to current application's NSDictionary
property NSMutableArray : a reference to current application's NSMutableArray
property NSSortDescriptor : a reference to current application's NSSortDescriptor
property NSUTF8StringEncoding : a reference to current application's NSUTF8StringEncoding
property NSRegularExpressionSearch : a reference to current application's NSRegularExpressionSearch
set theDestinationFile to (POSIX path of (path to desktop folder)) & "zCARPA.txt"
set destinationURL to |NSURL|'s fileURLWithPath:theDestinationFile
set theSourceFolder to POSIX path of (path to scripts folder from user domain)
-- get all files
set theSourceFolder to |NSURL|'s fileURLWithPath:theSourceFolder
set fileManager to NSFileManager's defaultManager()
set URLKeys to NSArray's arrayWithArray:{NSURLIsRegularFileKey, NSURLIsPackageKey}
set theOptions to ((NSDirectoryEnumerationSkipsPackageDescendants) as integer) + (NSDirectoryEnumerationSkipsHiddenFiles as integer)
set theFiles to (fileManager's enumeratorAtURL:theSourceFolder includingPropertiesForKeys:(URLKeys) options:theOptions errorHandler:(missing value))'s allObjects()
-- remove items with no extensions
set theFilter to NSPredicate's predicateWithFormat:"pathExtension != ''"
set theFiles to theFiles's filteredArrayUsingPredicate:theFilter
-- Build a counted set containing the extensions of those items which aren't folders.
set theSet to NSCountedSet's new()
set AsObjCFalse to NSNumber's numberWithBool:false
set theDict to NSDictionary's dictionaryWithDictionary:{NSURLIsPackageKey:AsObjCFalse, NSURLIsRegularFileKey:AsObjCFalse}
repeat with thisItem in theFiles
if not ((thisItem's resourceValuesForKeys:URLKeys |error|:(missing value)) is theDict) then
(theSet's addObject:(thisItem's pathExtension()))
end if
end repeat
-- build array of records so we can sort
set theResults to NSMutableArray's array()
set theSum to 0
tell (space & space) to tell (it & it) to set eightSpaces to (it & it) -- MacScripter /displays/ the literal string as a single space.
repeat with aValue in theSet's allObjects()
set theCount to (theSet's countForObject:aValue)
set theSum to theSum + theCount
-- The spaces at the beginning of theEntry are padding for an indent, whose size will be adjusted later.
(theResults's addObject:{theValue:aValue, theEntry:(eightSpaces & theCount) & (space & aValue)})
end repeat
-- sort on the dictionaries 'theValue' values.
set sortDesc to NSSortDescriptor's sortDescriptorWithKey:"theValue" ascending:true
theResults's sortUsingDescriptors:{sortDesc}
-- create the text with an entry for the total count at the end.
set theSum to theSum as text
theResults's addObject:({theEntry:linefeed & eightSpaces & theSum & " TOTAL"})
set theText to (theResults's valueForKey:"theEntry")'s componentsJoinedByString:(linefeed)
-- Adjust the width of the indent to the number of characters in the total.
set theText to theText's stringByReplacingOccurrencesOfString:("(?m)^ +(?=[ \\d]{" & (count theSum) & "} )") withString:("") options:NSRegularExpressionSearch range:({0, theText's |length|()})
-- Write the text to the specified text file as UTF-8.
theText's writeToURL:(destinationURL) atomically:(true) encoding:NSUTF8StringEncoding |error|:(missing value)
Do these numbers actually mean anything though ? How statistically significant (if at all) are they ? And can you be sure that the speed improvements are down to just the changes being made in the script, or are there other variables that aren’t being considered here ?
Largely rhetorical questions, but could be things to consider.
It would be more helpful if you improved the speeds of the scripts posted above in your own way. Instead of a useless comment about the themes of the universe.
I was working on a related script and decided to give KniazidisR’s speed-enhancement suggestion a try. My original script was:
use framework "Foundation"
use scripting additions
set theRegularFiles to getFiles("/Users/Robert/Records/")
on getFiles(theFolder)
set fileManager to current application's NSFileManager's defaultManager()
set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
set theKey to current application's NSURLIsRegularFileKey
set theOptions to 6 -- skip package descendants and hidden files
set theFiles to (fileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{theKey} options:theOptions errorHandler:(missing value))'s allObjects()'s mutableCopy()
repeat with i from theFiles's |count|() to 1 by -1
set anItem to (theFiles's objectAtIndex:(i - 1))
set {theResult, isFile} to (anItem's getResourceValue:(reference) forKey:theKey |error|:(missing value))
if isFile as boolean is false then (theFiles's removeObject:anItem)
end repeat
set fileCount to theFiles's |count|()
end getFiles
To employ the speed enhancement, I added the first of the following lines just above the beginning of the repeat loop and changed the line just above the end of the repeat loop to the second line below.
set AsObjCFalse to current application's NSNumber's numberWithBool:false
if isFile is AsObjCFalse then (theFiles's removeObject:anItem)
I used Script Geeek and my test folder contained 429 regular files and 114 folders. The script with KniazidisR’s speed enhancement was consistently 10 percent (about 7 milliseconds) faster, which seems a small but worthwhile enhancement.
Thie nature of any forum is to invite comments and opinion, and it surprises me that you’d view those considerations I put forward as “useless”, or that they would have upset you in any way. They weren’t criticisms nor any kind of judgment for or against, just questions that are actually pretty relevant to your priorities. Though you might not be unable to see any value in or use for someone’s comment, do try to remember that there are more people out there besides you, and it’s conceivable that one amongst them possibly would.