Loading file list from a folder

Hi technomorph.

Your first, second, and fourth fetch lines are correct.

The fifth should just be:

third list of (aList's second list) -- With or without the parentheses.

… or …

aList's second list's third list

… or …

list 3 of list 2 of aList

… or whatever mixture of styles you prefer.

You can’t get an indexed item of a record because the point of a record is that the values in it are labelled rather than being in a particular order. So your third fetch line would have to be something like this:

d of (aList's third record) ----> {dsub1:"tomato", dsub2:"potato"}

“List of list” simply means a list containing lists, the implication being that it either only contains lists or is empty. What any lists in it might contain isn’t specified.

item 1 = {[b]INFO:[/b]{[i]BITRATE:[/i]"128000", [i]GENRE:[/i]"Alternative", [i]COMMENT:[/i]"missing 20180401", [i]RATING:[/i]"Search In Playlists", [i]PLAYTIME:[/i]"353", [i]IMPORT_DATE:[/i]"2018/4/29", [i]FLAGS:[/i]"10", [i]FILESIZE:[/i]"5582", [i]|color|[/i]:"1"}, PRIMARYKEY:{|key|:"Tekno/:Users/:kerry/:Music/:iTunes/:iTunes Media/:Music/:LCD Soundsystem/:All My Friends - EP/:All My Friends (Franz Ferdinand Version).m4p"}, LOCATION:{DIR:"/:Users/:kerry/:Music/:iTunes/:iTunes Media/:Music/:LCD Soundsystem/:All My Friends - EP/:", |file|:"All My Friends (Franz Ferdinand Version).m4p", VOLUME:"Tekno", VOLUMEID:"Tekno"}, MAININFO:{MODIFIED_DATE:"2018/5/9", MODIFIED_TIME:"47417", AUDIO_ID:"AQoAABEBERERIhEQEiERERFERlVCIjVnQxRWZCIiNVZ1QhETZlUiRVIRNENoQUVVImdlQgJWZiRTEQAWd4UVZ2QQJnZhRUIRERIiEREiEREQABJmRTMhERVYUhFFVUERJERERCJHVFVSE0VCEUZkiDFFZRJmZVITVWUTQyIRFXd1E1Z2MRWGcyVEIiM0M0MyIUdlQQFGZSFGMzEAATITZldBJWVlM0dVd2dnQyIREyIzIiMiISERERA2ZUEmZlUgNWZhRUERAEd4ckZmYxFnZyJFQ0RmdUIRIiIhBHYlVVQzIhEjMjIjMzIiMzIhERERFDERElQxERNDMxACMyERAA==", title:"Test Title 02", ARTIST:"Test Artist 02"}, ENTRYNO:1}

which breaks down to
item 1 having these records:
ENTRYNO: record of 1 item
INFO: record of 9 items
-----BITRATE
-----GENRE
-----COMMENT
-----RATING
-----PLAYTIME
-----IMPORT_DATE
-----FLAGS
-----FILE_SIZE
-----COLOR
PRIMARYKEY : record of … etc
-----subrecord1
-----subrecord2
LOCATION : …etc
-----subrecord3
-----subrecord4
-----subrecord5
…etc

All of the sub records all have different user names.

  1. Am I able to access the subrecords directly IE:
    from previous example:
    1a)
    d of aList
    1b)
    dsub of aList
    1c)
    dsub of (aList’s third record)
    1d) or do I have to drill right down into the subrecords like:
    dsub of d of (aList’s third record)

  2. I’m guessing I’m probably not able to ask for a recordkey name that’s deep in a list.
    But in my ITEM 1 it just contains records. Can I

2a)
BITRATE of item 1

2b) or do I have to
BITRATE of INFO of item 1

  1. If I have to drill down would I be better to set up 3b rather that 3a
    3a)
    BITRATE of INFO of item 1
    GENRE of INFO of item 1
    COMMENT of INFO of item 1
    RATING of INFO of item 1
    FLAGS of INFO of item 1
    3b)
    set ITEMINFO to INFO of item 1
    BITRATE of ITEMINFO
    GENRE of ITEMINFO
    COMMENT ITEMINFO
    RATING of ITEMINFO
    FLAGS of ITEMINFO

  2. I’m guessing there is probably other functions/libraries out there that will “filter” an
    array for me by “keys”

I’m not wanting all of the INFO from the item 1 just certain sub records
providing key say {“BITRATE”, “GENRE”, “COMMENT”, “RATING”, “FLAGS”}
in the key case is it possible to use a list of lists?
{“INFO” {“BITRATE”, “GENRE”, “COMMENT”, “RATING”, “FLAGS”}, “LOCATION” {“VOLUME”, “DIR”, “|file|”}}

thanks

Hi technomorph.

We seem to have strayed a long way from the topic of this thread — and even from the items-of-a-class-from-a-list digression. :confused:

If ‘aList’ is a list, then 1a and 1b are both wrong because lists don’t have labelled properties. If ‘dsub’ is a property of a record which is the ‘d’ property of aList’s third record, then 1c is wrong and 1d is right.

You’d normally only use an expression like ‘aList’s third record’ where the list contained items of various classes and you specifically wanted the third record. If you know that the list only contains records, then ‘aList’s third item’ may be more convenient. It’s also theoretically more efficient, since it simply grabs the third item from the list without having to check the item’s class.

Yes. BITRATE is a property of the record which is item 1’s INFO value, not a direct property of item 1 itself.

Whichever you find more convenient at the time. Theoretically, if you need to extract several values from a subitem, it’s more efficient to use a variable, as in 3b, since it reduces the number of pointers which have to be followed with each individual value. But the difference is tiny.

It may be possible in ASObjC, but filtering is straying even further off-topic here.

If you know that a record heirarchy contains all the properties of interest, a convenient way to set variables to all the required values would be like this:

-- For this demo, aList is a list containing one record.
set aList to {{INFO:{BITRATE:"128000", GENRE:"Alternative", COMMENT:"missing 20180401", RATING:"Search In Playlists", PLAYTIME:"353", IMPORT_DATE:"2018/4/29", FLAGS:"10", FILESIZE:"5582", |color|:"1"}, PRIMARYKEY:{|key|:"Tekno/:Users/:kerry/:Music/:iTunes/:iTunes Media/:Music/:LCD Soundsystem/:All My Friends - EP/:All My Friends (Franz Ferdinand Version).m4p"}, LOCATION:{DIR:"/:Users/:kerry/:Music/:iTunes/:iTunes Media/:Music/:LCD Soundsystem/:All My Friends - EP/:", |file|:"All My Friends (Franz Ferdinand Version).m4p", VOLUME:"Tekno", VOLUMEID:"Tekno"}, MAININFO:{MODIFIED_DATE:"2018/5/9", MODIFIED_TIME:"47417", AUDIO_ID:"AQoAABEBERERIhEQEiERERFERlVCIjVnQxRWZCIiNVZ1QhETZlUiRVIRNENoQUVVImdlQgJWZiRTEQAWd4UVZ2QQJnZhRUIRERIiEREiEREQABJmRTMhERVYUhFFVUERJERERCJHVFVSE0VCEUZkiDFFZRJmZVITVWUTQyIRFXd1E1Z2MRWGcyVEIiM0M0MyIUdlQQFGZSFGMzEAATITZldBJWVlM0dVd2dnQyIREyIzIiMiISERERA2ZUEmZlUgNWZhRUERAEd4ckZmYxFnZyJFQ0RmdUIRIiIhBHYlVVQzIhEjMjIjMzIiMzIhERERFDERElQxERNDMxACMyERAA==", title:"Test Title 02", ARTIST:"Test Artist 02"}, ENTRYNO:1}}

-- Set variables to the required values from the first record in the list.
-- The record containing the variables must have the same structure as the record containing the values, but need only contain the required properties.
-- The order in which the records' properties are written in the source code is of course immaterial.
set {INFO:{BITRATE:theBitRate, GENRE:theGenre, COMMENT:theComment, RATING:theRating, FLAGS:theFlags}, LOCATION:{VOLUME:theVolume, DIR:theDirectory, |file|:theFile}} to item 1 of aList

return {theBitRate, theGenre, theComment, theRating, theFlags, theVolume, theDirectory, theFile}
--> {"128000", "Alternative", "missing 20180401", "Search In Playlists", "10", "Tekno", "/:Users/:kerry/:Music/:iTunes/:iTunes Media/:Music/:LCD Soundsystem/:All My Friends - EP/:", "All My Friends (Franz Ferdinand Version).m4p"}

While I’m not expecting to get hundreds of files returned, I’d like to speed up the following AppleScript version and add the ability to exclude a list of file extensions. The list of exclusions is the part about which I’m particularly concerned, as I’m not that familiar with some of the languages used it the preceding posts and can’t extend them. Any help would be appreciated.


		-- --------------------------------------------------------------------------------
		-- get all the files having a filename that is the same as the ".meta" file,
		-- but not the ".meta" file itself, into a list.
		-- --------------------------------------------------------------------------------
		try
			tell application "System Events"
				set theList to (name of files of alias (thePath) whose (name begins with theFilename & ".") and (name does not contain "." & theExtension))
			end tell
		on error error_message number error_number
			set this_error to "Error: " & error_number & ". " & error_message & return
			write_to_file(theStatus, this_file, true)
		end try

It currently looks for all files having the root name “theFilename” in “thePath”, but not with the extension “theExtension”.

I’m not too particular whether or not the substitute utilizes scripts or objective C. (But I’m hoping the suggestion is commented so I can learn from it). I just need to improve the speed, as the pure AppleScript way is quite slow.

I was playing around with bash and the terminal. I “think” the script needs to incorporate something like:


find . -type f -iname "GoodFile.*" -a -not \( -iname "*.meta" -o -iname "*.app" \)

then have its output fed to grep to use a regular expression to format it as a string for return to AppleScript.

To get the GoodFile and list of excluded extensions, I believe I’d have to create the bash script on the fly in a handler.

Am I on the right track?

Try this:

use AppleScript version "2.5" -- 10.11 or later
use framework "Foundation"
use scripting additions

on listFilesIn:sourceAliasOrFile fileStub:fileStub excludedExtensions:theExtensions
	set fileManager to current application's NSFileManager's defaultManager()
	set theURLs to fileManager's contentsOfDirectoryAtURL:sourceAliasOrFile includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
	
	set thePred to current application's NSPredicate's predicateWithFormat:"%K == %@ AND !(%K IN %@)" argumentArray:{"lastPathComponent.stringByDeletingPathExtension", fileStub, "pathExtension", theExtensions}
	set theURLs to theURLs's filteredArrayUsingPredicate:thePred
	return theURLs as list
end listFilesIn:fileStub:excludedExtensions:

Thank you, Shane.
I appreciate you putting together the code for the handler.

Could you let me know the intended format for each of the calling parameters? I tried to determine it from your code, but I’m getting some errors and I know it’s my fault.

I believe;

The sourceAliasOrFile is the “URL” for the directory to be examined, for example:

tell application "System Events" to set sourceAliasOrFile to URL of (the path to the desktop as alias)

The fileStub is the URL of the file for which one wants filter based on the name alone, for example:

set fileStub to quoted form of (sourceAliasOrFile & "Introduction to if.webloc")

The theExtensions is a list of quoted extensions, for example:

set theExtensions to {"jpg", "dmg", "app"}

The return is clear as to its format.

Here is a bit lengthy, pure AppleScript solution. The handler’s input arguments are as follows:

 folderHFSPath
      - Self-explanatory

 fileStubs
      - Text string: Files with this file name stub (i.e., file name without extension) will be considered
      - List of text strings: Files with these file name stubs will be considered
      - {}, "", missing value, or null: All files with the file name stub of files whose extension is in the excluded extensions list will be considered

 excludedExtensions
      - Text string: Files whose stub is in the file stubs list but without this extension will be returned
      - List of text strings: Files whose stub is in the file stubs list but without these extensions will be returned
on getFilesOfFolder:folderHFSPath withFileStubs:fileStubs excludedExtensions:excludedExtensions
	-- Embedded script containing a utility handler for the main handler
	script util
		on componentsForFileName:utilFileName
			-- Return the file stub and extension components of the input file name
			tell (get utilFileName's text items)
				if (its length = 1) or ((its length = 2) and (first item = "")) then
					-- Handle file names without an extension and those beginning with a period character but without a file extension
					set {utilFileStub, utilFileExtension} to {utilFileName, ""}
				else
					-- Handle all other file names
					set {utilFileStub, utilFileExtension} to {items 1 thru -2 as text, item -1}
				end if
			end tell
			return {utilFileStub, utilFileExtension}
		end componentsForFileName:
	end script
	-- Baseline return values
	set {fileNames, filePOSIXPaths, fileHFSPaths} to {{}, {}, {}}
	-- Convert an extension coded as a text string into a one-item list
	set excludedExtensions to excludedExtensions as list
	-- Get the names and POSIX and HFS paths of all files in the input folder
	tell application "System Events" to set {allNames, allPOSIXPaths, allHFSPaths} to {name, POSIX path, path} of files of folder folderHFSPath
	-- Set the text item delimiters to the period character "." here; it will be used by the utility script "util" to extract a file's stub and extension from its full name
	set tid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to "."
	try
		if {fileStubs} is in {{}, "", missing value, null} then
			-- If not provided in the input argument, get the file stubs of those input files whose file extension is one of the excluded file extensions
			set fileStubs to {}
			repeat with currName in allNames
				set {currStub, currExtension} to (util's componentsForFileName:(currName's contents))
				if ({currStub} is not in fileStubs) and ({currExtension} is in excludedExtensions) then set end of fileStubs to currStub
			end repeat
		else
			-- Convert a file stub coded as a text string into a one-item list
			set fileStubs to fileStubs as list
		end if
		-- Get the names and POSIX and HFS paths of those input files whose file extension is not one of the excluded file extensions
		repeat with i from 1 to allNames's length
			set {currName, currPOSIXPath, currHFSPath} to {allNames's item i, allPOSIXPaths's item i, allHFSPaths's item i}
			set {currStub, currExtension} to (util's componentsForFileName:currName)
			if ({currStub} is in fileStubs) and ({currExtension} is not in excludedExtensions) then set {end of fileNames, end of filePOSIXPaths, end of fileHFSPaths} to {currName, currPOSIXPath, currHFSPath}
		end repeat
		-- Restore the text item delimiters to their baseline value, then return the results
		set AppleScript's text item delimiters to tid
		return {fileNames, filePOSIXPaths, fileHFSPaths}
	on error m number n
		-- If an error is encountered, restore the text item delimiters to their baseline value, then optionally process the error
		set AppleScript's text item delimiters to tid
		### Error-processing code ###
	end try
end getFilesOfFolder:withFileStubs:excludedExtensions:

Examples:

my getFilesOfFolder:(path to desktop as text) withFileStubs:"GoodFileName" excludedExtensions:"meta"
--> Returns all files on the Desktop whose file stub = "GoodFileName" and whose extension ≠ "meta"

my getFilesOfFolder:(path to desktop as text) withFileStubs:{"GoodFileName", "AnotherGoodFileName"} excludedExtensions:{"meta", ".app"}
--> Returns all files on the Desktop whose file stub = "GoodFileName" or "AnotherGoodFileName" and whose extension ≠ "meta" or "app"

my getFilesOfFolder:(path to desktop as text) withFileStubs:null excludedExtensions:{"meta", ".app"}
--> Returns all files on the Desktop whose extension ≠ "meta" or "app", provided that another desktop file with the same file stub does have a "meta" or "app" extension

@GG

Maybe this would help:

(*

https://macscripter.net/viewtopic.php?id=46207&p=2

Loading file list from a folder

by Shane STANLEY
*)
use AppleScript version "2.5" -- 10.11 or later
use framework "Foundation"
use scripting additions

on listFilesIn:sourceAliasOrFile fileStub:fileStub excludedExtensions:theExtensions
	set fileManager to current application's NSFileManager's defaultManager()
	set theURLs to fileManager's contentsOfDirectoryAtURL:sourceAliasOrFile includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
	
	set thePred to current application's NSPredicate's predicateWithFormat:"%K CONTAINS %@ AND !(%K IN %@)" argumentArray:{"lastPathComponent.stringByDeletingPathExtension", fileStub, "pathExtension", theExtensions}
	set theURLs to theURLs's filteredArrayUsingPredicate:thePred
	return theURLs as list
end listFilesIn:fileStub:excludedExtensions:

set sourceAliasOrFile to current application's class "NSURL"'s fileURLWithPath:(POSIX path of (path to desktop))
--set sourceAliasOrFile to (path to desktop) # alternate
set fileStub to "First & Last Day"

set theExtensions to {"rtfd"}

my listFilesIn:sourceAliasOrFile fileStub:fileStub excludedExtensions:theExtensions

It returned correctly :
{file “SSD 500:Users::desktop:First & Last Day of Last Month by Nigel.applescript", file "SSD 500:Users::desktop:First & Last Day of Last Month by Shane.applescript”, file “SSD 500:Users:**********:desktop:First & Last Day of Last Month by StefanK.applescript”}

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) lundi 4 février 2019 18:57:52

Thank you bmose. Interesting way to do it. I’ll try it, because I’d like to see how fast it is.

Thank you, Yvan, for illustrating how to call the objective C routine from AppleScript. I’m amazed at how fast Shane’s code is. I like the AppleScript natural language paradigm, but I’m going to have to learn this language!

I should note that I am using Mojave and I Applescript solutions tend to make one bang into a barrage of requests to allow programs to communicate. It seems like the objective C may be able to do and end-run around this issue by avoiding the calls that trigger it. If I’m wrong about this, please let me know.

This modification to my original post is a little more efficient by extracting file stubs and extensions from the folder’s file names only once rather than twice. If you do try it, it would be interesting to know how it compares with Shane’s ASObjC solution in terms of execution speed. ASObjC is almost always the faster approach, but sometimes plain ol’ AppleScript is sufficiently fast. :slight_smile:

on getFilesOfFolder:folderHFSPath withFileStubs:fileStubs excludedExtensions:excludedExtensions
	script util
		on componentsForFileName:utilFileName
			tell (get utilFileName's text items)
				if (its length = 1) or ((its length = 2) and (first item = "")) then
					set {utilFileStub, utilFileExtension} to {utilFileName, ""}
				else
					set {utilFileStub, utilFileExtension} to {items 1 thru -2 as text, item -1}
				end if
			end tell
			return {utilFileStub, utilFileExtension}
		end componentsForFileName:
	end script
	set {fileNames, filePOSIXPaths, fileHFSPaths} to {{}, {}, {}}
	set excludedExtensions to excludedExtensions as list
	tell application "System Events" to set {allNames, allPOSIXPaths, allHFSPaths} to {name, POSIX path, path} of files of folder folderHFSPath
	set tid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to "."
	try
		set fileStubsWereComputed to ({fileStubs} is in {{}, "", missing value, null})
		if fileStubsWereComputed then
			set {allStubs, allExtensions, fileStubs} to {{}, {}, {}}
			repeat with currName in allNames
				set {currStub, currExtension} to (util's componentsForFileName:(currName's contents))
				set {end of allStubs, end of allExtensions} to {currStub, currExtension}
				if ({currStub} is not in fileStubs) and ({currExtension} is in excludedExtensions) then set end of fileStubs to currStub
			end repeat
		else
			set fileStubs to fileStubs as list
		end if
		repeat with i from 1 to allNames's length
			set {currName, currPOSIXPath, currHFSPath} to {allNames's item i, allPOSIXPaths's item i, allHFSPaths's item i}
			if fileStubsWereComputed then
				set {currStub, currExtension} to {allStubs's item i, allExtensions's item i}
			else
				set {currStub, currExtension} to (util's componentsForFileName:currName)
			end if
			if ({currStub} is in fileStubs) and ({currExtension} is not in excludedExtensions) then set {end of fileNames, end of filePOSIXPaths, end of fileHFSPaths} to {currName, currPOSIXPath, currHFSPath}
		end repeat
		set AppleScript's text item delimiters to tid
		return {fileNames, filePOSIXPaths, fileHFSPaths}
	on error m number n
		set AppleScript's text item delimiters to tid
		### Error-processing code ###
	end try
end getFilesOfFolder:withFileStubs:excludedExtensions:

Sorry, I should have given an example.

No, sourceAliasOrFile requires a file reference or alias. This will be bridged automatically to a Cocoa NSURL.

No, fileStub simply means the file name minus its extension.

Right.

Thank you Shane. I’ve got it working now. Once I understood what was happening, I was able to use other “Foundation” functions to speed up my scripts.

Hello Shane

I’m unable to retrieve threads in which you described the way to edit filters so that they are not case sensitive.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mardi 5 février 2019 16:13:13

Yvan,

You insert “[c]”, or “[cd]” if you also want to ignore diacriticals, after the argument. So “%K CONTAINS %@” could become: “%K CONTAINS[cd] %@”.

Thank you Shane.

I misunderstood what I read in Everyday AppleScriptObjC 3ed and tried: “%K[cd] CONTAINS %@[cd]”.

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 février 2019 10:05:31

Is it normal that using %K IN[c] %@ has no effect upon the test upon the extension?

With : set theExtensions to {“RTF”}

the script return:
{file “SSD 500:Users::desktop:Barbara studio.rtf", file "SSD 500:Users::desktop:BARBARA, Comme Un Soleil Noir.numbers”, file “SSD 500:Users:**********:desktop:Barbara.rtf”}

With : set theExtensions to {“rtf”} it return :
{file “SSD 500:Users:**********:desktop:BARBARA, Comme Un Soleil Noir.numbers”}

It’s not frequent to see the extension spelled “RTF” but the problem is common with “pdf” versus “PDF”.

Of course, if there is no need for a list of extensions we may use :
(%K ==[c] %@) and set theExtensions to “RTF” which return
{file “SSD 500:Users:**********:desktop:BARBARA, Comme Un Soleil Noir.numbers”}

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 février 2019 11:42:12

I think so. In fact, I suspect my example above doesn’t actually work – it probably only works with direct text comparisons, such as == or LIKE.

Thanks Shane.

I tested CONTAINS, LIKE and ==.
They apply to string comparison, not to comparison upon items of a list.

%K CONTAINS[c] %@ return file names containing “BARBARA” as well as “Barbara”

Yvan KOENIG running High Sierra 10.13.6 in French (VALLAURIS, France) mercredi 6 février 2019 12:19:58

Hi.

As a practical solution, this variation on Shane’s handler seems to work. I don’t know if it’s supposed to! :wink:

use AppleScript version "2.5" -- 10.11 or later
use framework "Foundation"
use scripting additions

on listFilesIn:sourceAliasOrFile fileStub:fileStub excludedExtensions:theExtensions
	set fileManager to current application's NSFileManager's defaultManager()
	set theURLs to fileManager's contentsOfDirectoryAtURL:sourceAliasOrFile includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
	
	set lowercaseExtensions to (current application's NSArray's arrayWithArray:theExtensions)'s valueForKey:"lowercaseString" -- List of lower-cased extensions.
	set thePred to current application's NSPredicate's predicateWithFormat:"%K ==[c] %@ AND !(%K.lowercaseString IN %@)" argumentArray:{"lastPathComponent.stringByDeletingPathExtension", fileStub, "pathExtension", lowercaseExtensions} -- Modified predicate.
	set theURLs to theURLs's filteredArrayUsingPredicate:thePred
	return theURLs as list
end listFilesIn:fileStub:excludedExtensions: