Library handler for querying folder

I’ve posted this elsewhere, but it probably belongs here too.

One of the banes of scripting is how slow the Finder can sometimes be, and this is particularly so when you want the contents of a folder. For example, this:

tell application "Finder" to set theFiles to (sort (every file of <some folder>) by modification date)

can be quite slow if there are a lot of files involved. And if you use “entire contents” with a folder containing a lot of files, there’s a good chance it will whir away for two minutes, then time out. Fortunately ASObjC- based libraries mean you can side-step the Finder.

This handler is a bit more complicated, but most of the effort is in copying, pasting and saving in a library. It could do with someone defining some terminology to make it a bit easier to use, but it’s not exactly rocket science to use as-is.

The handler requires a POSIX path for the folder, and boolean values for whether you want to include folders as well as files, whether to also search subfolders, and whether to sort in ascending order. And it takes a key to sort on, which is where it’s a little more complicated.

For instructions on how to save an ASObjC-based library, see macscripter.net/viewtopic.php?id=41638. Here’s the handler:

use framework "Foundation"

on sortFolder:folderPosixPath byKey:theKey includeFolders:includeFolders entireContents:entireContents ascendingOrder:ascendingOrder
	set keysToRequest to {current application's NSURLPathKey, current application's NSURLIsDirectoryKey, theKey} -- keys for values we want from each item
	set theInfo to current application's NSMutableArray's array() -- array to store new values in
	set theURL to current application's NSURL's fileURLWithPath:folderPosixPath -- make URL for folder because that's what's needed
	set fileManager to current application's NSFileManager's defaultManager() -- get file manager
	set theOptions to ((current application's NSDirectoryEnumerationSkipsPackageDescendants) as integer) + ((current application's NSDirectoryEnumerationSkipsHiddenFiles) as integer) -- options to use when enumerating directory
	if not entireContents then -- extra option to skip deep search
		set theOptions to theOptions + ((current application's NSDirectoryEnumerationSkipsSubdirectoryDescendants) as integer)
	end if
	-- enumerate the folder
	set theEnumerator to fileManager's enumeratorAtURL:theURL includingPropertiesForKeys:keysToRequest options:theOptions errorHandler:(missing value)
	repeat
		set aURL to theEnumerator's nextObject() -- nextObject() returns the next URL held by the enumerator
		if aURL is missing value then exit repeat -- missing value means there are no more, so exit
		theInfo's addObject:(aURL's resourceValuesForKeys:keysToRequest |error|:(missing value)) -- get values and add to array
	end repeat
	if not includeFolders then -- filter out folders
		set thePred to current application's NSPredicate's predicateWithFormat_("%K != YES", current application's NSURLIsDirectoryKey)
		theInfo's filterUsingPredicate:thePred
	end if
	set theDescriptor to current application's NSSortDescriptor's sortDescriptorWithKey:theKey ascending:ascendingOrder -- describes the sort
	theInfo's sortUsingDescriptors:{theDescriptor} -- do the sort
	return (theInfo's valueForKey:(current application's NSURLPathKey)) as list -- extract paths and convert to list
end sortFolder:byKey:includeFolders:entireContents:ascendingOrder:

And you’d call it like this:

use theLib : script "<lib name>" -- name of the library you saved it in
use scripting additions
set theFiles to theLib's sortFolder:(POSIX path of (choose folder)) byKey:(current application's NSURLContentModificationDateKey) includeFolders:false entireContents:false ascendingOrder:true

If you wanted files instead of POSIX paths, you could add:

repeat with i from 1 to count of theFiles
	set item i of theFiles to POSIX file (item i of theFiles)
end repeat

The tricky bit is knowing the key to use. The main ones are: NSURLAttributeModificationDateKey, NSURLContentAccessDateKey, NSURLContentModificationDateKey, NSURLCreationDateKey, NSURLLabelColorKey, NSURLLabelNumberKey, NSURLLocalizedLabelKey, NSURLLocalizedTypeDescriptionKey, NSURLNameKey, NSURLTypeIdentifierKey, NSURLFileSizeKey, NSURLFileAllocatedSizeKey, NSURLTotalFileAllocatedSizeKey, and NSURLTotalFileSizeKey.

Why bother? On my Mac, with about 400-odd items on my desktop, the difference in times is about 1.3 seconds vs 0.08 seconds. If I instead set entireContents to true and do the equivalent for the Finder, the ASObjC method blows out to about 0.2 seconds, while the Finder spends two minutes thinking and then times out.

For a hoot, I tried this:

set theFiles to theLib's sortFolder:(POSIX path of (path to home folder)) byKey:(current application's NSURLContentModificationDateKey) includeFolders:false entireContents:true ascendingOrder:true

It took a bit over a minute, although returning the list of more than 90,000 items took quite a bit longer. I wouldn’t want to try it with Finder…

Nice thank you for this.

In one of my own commands that checks for existence of a file/folder path I use this


set theClass to class of thePath
 
	if theClass is «class furl» or theClass is alias then
		
		set thePath to POSIX path of thePath
		
	else if theClass is text then
		set testString to current application's NSString's stringWithString:thePath
		set isRange to testString's rangeOfString:":"
		set tr to |length| of isRange as boolean
		if tr then
			set thePath to POSIX path of thePath
		end if
	end if
	

So the user can use either a POSIX path string, posix file string or an ‘alias’ or , POSIX file

This saves them the hassle of doing the coercing

Each type is coerced into a posix path where needed. This was a lazy cobble together and I am sure you can write it better if you want :slight_smile:

I don’t have adequate means for timing, but what’s the matter with this much simpler way to find files sorted by modified date?


set tFolder to POSIX path of (choose folder)
set tFiles to (do shell script "ls -Art " & tFolder)

Admittedly there’s some overhead for starting a shell, but still…

The script is for sorting the files – ls doesn’t distinguish between files and folders. You can use the -p option and grep to eliminate directories, but because ls can’t distinguish folders from packages, you then lose all packages.

The output from ls when used recursively also isn’t particularly useful to a script (and again, it will recurse through packages, which is not usually what you want).

Keep in mind that a POSIX path can contain a colon:

POSIX path of ("Mac HD:folder:file for 1/1/2013")
--> "/Mac HD/folder/file for 1:1:2013"

Hello Adam.

Hopefully you’ll find something that provides adequate timing at the end of this thread. :slight_smile:

I did. in the

set isRange to testString's rangeOfString:":"

part of the code.
Should have commented it up I guess

What Shane means is that when the file name contains a “/” in HFS, it will be presented as “:” in UFS (POSIX) paths. Checking if a path contains “:” is not enough. It’s better to check if the first character is an “/” because you (almost) never use relative paths and only absolute path.

set isPosixPath to character 1 of (thePath as string) is equal to "/"
--or
if character 1 of (thePath as string) is not equal to "/" then set thePath to POSIX path of thePath

Right. There’s no 100% foolproof way.

The other thing to watch out for is this sort of thing:

       set tr to |length| of isRange as boolean

There’s a bug in AS that makes “as boolean” fail to compile if you include a “use scripting additions” statement. Better to do something like:

       set tr to (|length| of isRange as integer > 0)

I did understand. I and I went back and check that bit of code would trip up on some thing like that.

I ran:

set thePath to POSIX path of ("Macintosh HD:Users:userName:Pictures:screenshots:Screen Shot 2013-11-25 at 21/41/57.png")
log thePath 
existence of file path thePath

This worked ok. the result was;
(/Users/userName/Pictures/screenshots/Screen Shot 2013-11-25 at 21:41:57.png)
true or false respectively.

I had a go at writing the code with terminology. Not sure if you guys will think it is great or not as my coding can be basic… but it’s a start.

Usage is ;

use theLib : script "sortLib" -- name of the library you saved it in

use scripting additions


sorted list from folder (choose folder) by Key "NSURLContentModificationDateKey"  including folders false  entire contents  false sort order ascending

Result:

{"/Users/UserName/Library/Script Libraries/a_folder/functions.php", "/Users/UserName/Library/Script Libraries/a_folder/b.png", "/Users/UserName/Library/Script Libraries/a_folder/c.png", "/Users/UserName/Library/Script Libraries/a_folder/a.png"}

The Applescript Library when opened looks some thing like this.

[code]
sorted list from folder‚v : Returns a sorted list of folder contents
sorted list from folder Specifier : The folder path to query. Can be a POSIX path string,‘alias’, POSIX file path -
by Key text : Name of the System’s Common File System Resource Key to use for sorting.
including folders boolean : Include sub folders in sort list
entire contents boolean : include the entire contents
sort order ascending/Œdescending : The Sort direction. ascending or descending

#example

use theLib : script “sortLib” – name of the library you saved it in

use scripting additions

sorted list from folder (choose folder) by Key “NSURLContentModificationDateKey” including folders false entire contents false sort order ascending

Result:

{“/Users/UserName/Library/Script Libraries/a_folder/functions.php”, “/Users/UserName/Library/Script Libraries/a_folder/b.png”, “/Users/UserName/Library/Script Libraries/a_folder/c.png”, “/Users/UserName/Library/Script Libraries/a_folder/a.png”}

To obtain a list of Common File System Resource Keys you can use

theLib’s RecourceKeys()

Or go to Apples documentation

(NSURL class Common File System Resource Keys) Documentation[/code]
The library code.

use framework "Foundation"

on sorted list from folder folderPath by Key theKey including folders includeFolders entire contents entireContents sort order ascendingOrder
    set theClass to class of folderPath
    
    
    -- check the class of the folder path var and coerce it into a posix path
    if theClass is «class furl» or theClass is alias then
        
        set folderPath to POSIX path of folderPath
        
    else if theClass is text then
        set testString to current application's NSString's stringWithString:folderPath
        
        -- take care of path strings with colons
        set isRange to testString's rangeOfString:":"
        set tr to (|length| of isRange as integer > 0)
        if tr then
            set folderPath to POSIX path of folderPath
        end if
    end if
    
    -- validate the sort order and set it if correct
    set validType to {ascending, descending}
    if ascendingOrder is in validType then
        
        if the ascendingOrder is ascending then
            set ascendingOrder to 1
        else if the ascendingOrder is descending then
            
            set ascendingOrder to 0
        end if
    else
        set errStr to ascendingOrder & " is not valid sort order" as text
        error errStr
        
    end if
    
    -- set the key from the string for the key
    set theKey to current application's NSSelectorFromString(theKey)
    
    
    set keysToRequest to {current application's NSURLPathKey, current application's NSURLIsDirectoryKey, theKey} -- keys for values we want from each item
    set theInfo to current application's NSMutableArray's array() -- array to store new values in
    set theURL to current application's NSURL's fileURLWithPath:folderPath -- make URL for folder because that's what's needed
    set fileManager to current application's NSFileManager's defaultManager() -- get file manager
    set theOptions to ((current application's NSDirectoryEnumerationSkipsPackageDescendants) as integer) + ((current application's NSDirectoryEnumerationSkipsHiddenFiles) as integer) -- options to use when enumerating directory
    if not entireContents then -- extra option to skip deep search
        set theOptions to theOptions + ((current application's NSDirectoryEnumerationSkipsSubdirectoryDescendants) as integer)
    end if
    -- enumerate the folder
    set theEnumerator to fileManager's enumeratorAtURL:theURL includingPropertiesForKeys:keysToRequest options:theOptions errorHandler:(missing value)
    repeat
        set aURL to theEnumerator's nextObject() -- nextObject() returns the next URL held by the enumerator
        if aURL is missing value then exit repeat -- missing value means there are no more, so exit
        theInfo's addObject:(aURL's resourceValuesForKeys:keysToRequest |error|:(missing value)) -- get values and add to array
    end repeat
    if not includeFolders then -- filter out folders
        set thePred to current application's NSPredicate's predicateWithFormat_("%K != YES", current application's NSURLIsDirectoryKey)
        theInfo's filterUsingPredicate:thePred
    end if
    set theDescriptor to current application's NSSortDescriptor's sortDescriptorWithKey:theKey |ascending|:ascendingOrder -- describes the sort
    theInfo's sortUsingDescriptors:{theDescriptor} -- do the sort
    return (theInfo's valueForKey:(current application's NSURLPathKey)) as list -- extract paths and convert to list
end sorted list from folder


-- call list of some of the System's Common File System Resource Keys
on RecourceKeys()
    set RecourceKeys to {"NSURLAttributeModificationDateKey", "NSURLContentAccessDateKey", "NSURLContentModificationDateKey", "NSURLCreationDateKey", "NSURLCustomIconKey", "NSURLEffectiveIconKey", "NSURLFileResourceIdentifierKey", "NSURLFileResourceTypeKey", "NSURLFileSecurityKey", "NSURLHasHiddenExtensionKey", "NSURLIsDirectoryKey", "NSURLIsExcludedFromBackupKey", "NSURLIsExecutableKey", "NSURLIsHiddenKey", "NSURLIsMountTriggerKey", "NSURLIsPackageKey", "NSURLIsReadableKey", "NSURLIsRegularFileKey", "NSURLIsSymbolicLinkKey", "NSURLIsSystemImmutableKey", "NSURLIsUserImmutableKey", "NSURLIsVolumeKey", "NSURLIsWritableKey", "NSURLLabelColorKey", "NSURLLabelNumberKey", "NSURLLinkCountKey", "NSURLLocalizedLabelKey", "NSURLLocalizedNameKey", "NSURLLocalizedTypeDescriptionKey", "NSURLNameKey", "NSURLParentDirectoryURLKey", "NSURLPathKey", "NSURLPreferredIOBlockSizeKey", "NSURLTagNamesKey", "NSURLTypeIdentifierKey", "NSURLVolumeIdentifierKey", "NSURLVolumeURLKey"}
    
    return RecourceKeys
end RecourceKeys

and the sdef code.

[code]<?xml version="1.0" encoding="UTF-8"?>

<suite name="Misc Utilities" code="MOXS" description="Miscellaneous commands">
    
    <command name="sorted list from folder" code="MOSXIDOL" description="Returns a sorted list of folder contents">
        <direct-parameter type="Specifier" description="The folder path to query. Can be a POSIX path string,'alias', POSIX file path - "/>
        <parameter name="by Key" code="ByKy" type="text" description="Name of the System's Common File System Resource Key to use for sorting."/>
        <parameter name="including folders" code="INcl" type="boolean"  description="Include sub folders in sort list"/>
        <parameter name="entire contents" code="Etcs" type="boolean"  description="include the entire contents"/>

        <parameter name="sort order" code="SrOr" type="sort direction" description="The Sort direction. ascending or  descending"/>
        
        
        <documentation>
            <html>
                <![CDATA[
                 
                  <style type="text/css">
                   p.p1 {margin: 0.0px 0.0px 0.0px 39.2px; text-indent: -13.8px; font: 12.0px Verdana}
                    p.p2 {margin: 0.0px 0.0px 6.0px 39.2px; text-indent: -39.3px; font: 12.0px Verdana; color: #042eee}
                  p.psort1 {margin: 0.0px 0.0px 0.0px 39.2px; text-indent: -39.3px; font: 12.0px Verdana; color: #5e6161}
                  p.psort2 {margin: 0.0px 0.0px 0.0px 39.2px; text-indent: -39.3px; font: 12.0px Verdana; min-height: 15.0px}
                  p.psort3 {margin: 0.0px 0.0px 0.0px 39.2px; text-indent: -39.3px; font: 12.0px Verdana; color: #0433ff}
                  p.psort4 {margin: 0.0px 0.0px 0.0px 41.6px; text-indent: -41.6px; font: 12.0px Verdana; color: #012fbe}
                  span.ssort1 {color: #000000}
                  span.ssort2 {color: #4f8f00}
                  span.ssort3 {color: #0433ff}
                  span.ssort4 {font: 12.0px Menlo; color: #942193}
                  </style>
                  </head>
                  <body>
                   <p class="psort2"><br></p>
                  <p class="psort1">#example</p>
                  <p class="psort2"><br></p>
                  <p class="psort1"><span class="ssort1"><b>use</b> </span><span class="ssort2">theLib</span><span class="ssort1"> : </span><span class="ssort3"><i>script</i></span><span class="ssort1"> "sortLib" </span>-- name of the library you saved it in</p>
                  <p class="psort3"><span class="ssort1"><b>use</b> </span><i>scripting additions</i></p>
                  <p class="psort2"><br></p>
                  <p class="psort4"><b>sorted list from folder</b><span class="ssort1"> (</span><b>choose folder</b><span class="ssort1">) </span>by Key<span class="ssort1"> "NSURLContentModificationDateKey"</span><span class="ssort4"><span class="Apple-converted-space"> </span></span><span class="ssort1"> </span>including folders<span class="ssort4"> false<span class="Apple-converted-space"> </span></span><span class="ssort1"> </span>entire contents<span class="ssort1"><span class="Apple-converted-space"> </span></span><span class="ssort4"> false </span>sort order<span class="ssort4"> ascending</span></p>
                  
                  
                  <p class="psort1"><b><span class="Apple-tab-span">	</span>Result:</b></p>
                  <p class="p1"><span class="Apple-tab-span">	</span> {"/Users/UserName/Library/Script Libraries/a_folder/functions.php", "/Users/UserName/Library/Script Libraries/a_folder/b.png", "/Users/UserName/Library/Script Libraries/a_folder/c.png", "/Users/UserName/Library/Script Libraries/a_folder/a.png"}</p>
                   <p class="psort2"><br></p>
                  <p>To obtain a list of Common File System Resource Keys you can use</p>
                   <p class="psort1"> </span><span class="ssort2">theLib's RecourceKeys</span>()</p>
                    <p>Or go to Apples documentation</p>
                     <p class="psort2"><br></p>
                     <p class="p2"><span class="s1"><a href="https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSURL_Class/Reference/Reference.html#//apple_ref/doc/uid/20000301-SW3">(NSURL class Common File System Resource Keys) <span class="Apple-converted-space">  </span>Documentation</a></span></p>
                  </body>
                ]]>
            </html>
        </documentation>
    </command>
    
    <enumeration name="sort direction" code="SRTD">
        
        <enumerator name="ascending" code="Acen" description=""/>
        <enumerator name="descending" code="Decn" description=""/>
    </enumeration>
    
    
</suite>

[/code]

Cool. If you felt really keen, you could use enumerators for the keys. You could work from Standard Addtions’ .sdef, using the same name/code combinations as the file information record.

:lol:

The only thing with that is It does not have things like label index/number or tag names.

But worth some thought as it would be nice to have the users not think too much about the key.
I was going to make the simple words that would be matched to the correct key in the script library. But thought better of it. As the way it works now it will always be up to date with apple if they add any new keys.

But I think a solution mixing both ways would possibly be better…

Hmm,

Actually the tag does not work in either of the code. I thought it did but then I only had one tag on one file. When i introduce more than one it fails on the sortUsingDescriptors…

There are one or two others that do not work. It may be worth just defining some of them.

Also if we did use the standard additions name/codes. How do we avoid the ambiguity of them being in both libraries with out forcing the user to write tell blocks or using terms from…

The result has to be compare-able, so nothing that returns an array, for example.

It’s not a problem with enumerators or properties, because the context is already clear. That’s why apps like the Finder and System Events use the same codes (as do lots of other apps, for that matter).

Think I must have done something wrong then?. I copied properties from the scripting additions and replaced their record/property tags with enumeration/enumerator . and removed the type and access.

But got a ambiguous error saying that the modified date was in both the script lib. and standard additions…

Also when I said more than one tag. I meant one tag name per image rather than multiple tag names per image. But because files can have more than one tag this would need some thought. I like the idea of being able to sort by tag

It’s very easy to do, I’m afraid. That’s one of the reasons I built an .sdef editor into ASObjC Explorer.

Hello.
I have been thinking a little about this, and I have made a handler that returns the posix path if the file exists, and missing value otherwise, it takes Filreferences, (Finder and System Events) as input, together with aliases, hfs paths, and posix paths, and a list of charactes/text items that may have been given as a parameter accidentally, it also removes any single quotes from the start/end of the filename.

It may be a little bit more expensive.

on UFS(whatever)
	local uxPth
	try
		if class of whatever is list then set whatever to whatever as text
		
		if class of whatever is not alias and class of whatever is not text then
			set uxPth to POSIX path of (whatever as alias)
			
		else if (class of whatever is alias) then
			set uxPth to POSIX path of whatever
		else
			if text 1 of whatever is "'" then ¬
				set whatever to text 2 thru -1 of whatever
			
			if text -1 of whatever is "'" then ¬
				set whatever to text 1 thru -2 of whatever
			
			if (offset of ":" in whatever) is not 0 then
				try
					set uxPth to POSIX path of (whatever as alias)
				on error e number n
					-- maybe it was ok as it was ..
					try
						(POSIX file whatever as alias)
						set uxPth to whatever
					on error e number n
						error e number n
					end try
				end try
			else
				(POSIX file whatever as alias)
				set uxPth to whatever
			end if
		end if
		return uxPth
	on error e number n
		return missing value
	end try
end UFS

:slight_smile: Sorry I don’t have ASObjC Explorer.

But I did find it easy to transfer the properties. But was getting that ambiguity error - clashing against standard additions. in the test.

All you have to do is follow my advice :frowning:

Turns out that in the brave new world of the use statement, using identical terminology is a bad idea if you’re going to use scripting additions. It’s like trying to compile this:

use scripting additions
use application "Finder"

modification date of (info for (choose file))