Sort files in folders

CWilcke · March 31, 2021, 1:35pm

Hi,

I it possible to sort n move files in folders by applescript?

I do have one Folder with 30k+ files and would like to split it and move about 300 files into a new folder.

Each folder should not have more than these 300 files.

Best,

KniazidisR · April 1, 2021, 10:14am

Hi,

One simple solution is this (Caution: test it on the copy of folder firstly):


set MAX to 300
set sourceFolder to (choose folder) as text

tell application "Finder"
	set theFiles to sort files of folder sourceFolder by name
	set subFolderIndex to 1
	repeat while (theFiles is not {})
		set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
		set filesMoved to 0
		repeat while ((filesMoved < MAX) and (theFiles is not {}))
			move contents of item 1 of theFiles to subFolder
			set filesMoved to filesMoved + 1
			set theFiles to rest of theFiles
		end repeat
		set subFolderIndex to subFolderIndex + 1
	end repeat
end tell

CWilcke · April 7, 2021, 6:13am

perfect! Thank You!

wch1zpink · April 8, 2021, 1:51am

KniazidisR:

Hi,

One simple solution is this (Caution: test it on the copy of folder firstly):


set MAX to 300
set sourceFolder to (choose folder) as text

tell application "Finder"
	set theFiles to sort files of folder sourceFolder by name
	set subFolderIndex to 1
	repeat while (theFiles is not {})
		set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
		set filesMoved to 0
		repeat while ((filesMoved < MAX) and (theFiles is not {}))
			move contents of item 1 of theFiles to subFolder
			set filesMoved to filesMoved + 1
			set theFiles to rest of theFiles
		end repeat
		set subFolderIndex to subFolderIndex + 1
	end repeat
end tell

I tested the above code in Script Editor on 5000 identical text files, named from 0001 to 5000. The results of timing the code was 490 seconds

While using Finder.app to move, copy, duplicate, etc., on only a few files or folders at a time, is generally OK, it becomes extremely inefficient when trying to process dozens, hundreds, or even thousands of items at a time.

It is also worth noting that creating a list of files or folders with Finder.app becomes much quicker by coercing it to an alias list.

set sourceFolder to (choose folder) as text
set MAX to 300
tell application "Finder" to set theFiles to files of folder sourceFolder as alias list

Once that alias list has been created, there is no need to use the sort command because the created alias list will automatically be sorted. (I do believe)

A more efficient solution would be to use Finder to generate the alias list first, then System Events.app to move the files instead of Finder.

set sourceFolder to (choose folder) as text
set MAX to 300

tell application "Finder" to set theFiles to files of folder sourceFolder as alias list

tell application "System Events"
	set subFolderIndex to 1
	repeat while (theFiles is not {})
		set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
		set filesMoved to 0
		repeat while ((filesMoved < MAX) and (theFiles is not {}))
			move contents of item 1 of theFiles to subFolder
			set filesMoved to filesMoved + 1
			set theFiles to rest of theFiles
		end repeat
		set subFolderIndex to subFolderIndex + 1
	end repeat
end tell

The results of timing this code was 47 seconds

KniazidisR · April 8, 2021, 9:13am

Hello, wch1zpink.

I ran detailed tests here with a folder containing 1690 files. Here are the test results:

My original script (227 seconds) works correctly, but it is slow. Your script runs 5-6 times faster (45 seconds), but it is wrong because it breaks sorting by name. In order for it to work correctly, you will have to abandon the alias list + coerce the Finder references into the System Events references inside the repeat loop. And this increases the execution time (158 seconds):

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set theFiles to (sort files of folder sourceFolder by name)

tell application "System Events"
	set subFolderIndex to 1
	repeat while (theFiles is not {})
		set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
		set filesMoved to 0
		repeat while ((filesMoved < MAX) and (theFiles is not {}))
			move file ((contents of item 1 of theFiles) as text) to subFolder
			set filesMoved to filesMoved + 1
			set theFiles to rest of theFiles
		end repeat
		set subFolderIndex to subFolderIndex + 1
	end repeat
end tell
-- 158 seconds

And here, I wrote other Finder version, which has same speed (165 seconds) with correct System Events version above:

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set theFiles to sort files of folder sourceFolder by name
set theCount to count theFiles

set N to theCount div MAX
if N > 0 then
	tell application "Finder"
		repeat with i from 1 to N
			set subFolder to make new folder at folder sourceFolder with properties {name:i}
			repeat with j from (i - 1) * MAX + 1 to i * MAX
				move contents of item j of theFiles to subFolder
			end repeat
		end repeat
	end tell
end if

set M to theCount mod MAX
if M > 0 then
	tell application "Finder"
		set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
		repeat with j from theCount - M + 1 to theCount
			move contents of item j of theFiles to subFolder
		end repeat
	end tell
end if
-- 165 seconds

Note: I think, adapting this last script for System Events instead of Finder will produce one fastest script:

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set theFiles to sort files of folder sourceFolder by name
set theCount to count theFiles

set N to theCount div MAX
if N > 0 then
	tell application "System Events"
		repeat with i from 1 to N
			set subFolder to make new folder at folder sourceFolder with properties {name:i}
			repeat with j from (i - 1) * MAX + 1 to i * MAX
				move file ((contents of item j of theFiles) as text) to subFolder
			end repeat
		end repeat
	end tell
end if

set M to theCount mod MAX
if M > 0 then
	tell application "System Events"
		set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
		repeat with j from theCount - M + 1 to theCount
			move file ((contents of item j of theFiles) as text) to subFolder
		end repeat
	end tell
end if
-- 130 seconds

KniazidisR · April 8, 2021, 11:51am

Here I wrote fastest script (31 seconds for 1690 files). It uses sorting names instead of sorting files, and speed of System Events:


use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

set sourceFolder to "HARD_DISK:Users:123:Desktop:sourceFolder:"
set MAX to 300

tell application "Finder" to set fileNames to name of files of folder sourceFolder
set sortedNames to sortListOfStrings(fileNames) -- you can use other sorting method as well
set theCount to count sortedNames

set N to theCount div MAX
if N > 0 then
	tell application "System Events"
		repeat with i from 1 to N
			set subFolder to make new folder at folder sourceFolder with properties {name:i}
			repeat with j from (i - 1) * MAX + 1 to i * MAX
				move file (sourceFolder & item j of sortedNames) to subFolder
			end repeat
		end repeat
	end tell
end if

set M to theCount mod MAX
if M > 0 then
	tell application "System Events"
		set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
		repeat with j from theCount - M + 1 to theCount
			move file (sourceFolder & item j of sortedNames) to subFolder
		end repeat
	end tell
end if


on sortListOfStrings(theList)
	-- convert list to Cocoa array
	set theArray to ¬
		current application's NSArray's arrayWithArray:theList
	-- sort the array using a specific function
	set theArray to ¬
		theArray's sortedArrayUsingSelector:"localizedStandardCompare:"
	-- return the sorted array as an AppleScript list
	return theArray as list
end sortListOfStrings

wch1zpink · April 8, 2021, 6:12pm

Actually, this following approach bypasses Finder completely (which proves to be the major bottleneck)

I tested this code in Script DeBugger on 5000 identical text files, named from 0001 to 5000. The results of timing the code was 46 seconds.

global startTime, timeTaken

set sourceFolder to (choose folder) as text
set MAX to 300
startTimer()

set theFiles to paragraphs of (do shell script "find " & quoted form of POSIX path of sourceFolder & " -type f -maxdepth 1 | sort -f")

tell application "System Events"
	set subFolderIndex to 1
	repeat while (theFiles is not {})
		set subFolder to make new folder at folder sourceFolder with properties {name:subFolderIndex}
		set filesMoved to 0
		repeat while ((filesMoved < MAX) and (theFiles is not {}))
			move file ((contents of item 1 of theFiles) as text) to subFolder
			set filesMoved to filesMoved + 1
			set theFiles to rest of theFiles
		end repeat
		set subFolderIndex to subFolderIndex + 1
	end repeat
end tell
log endTimer()

to startTimer()
	set startTime to current date
	return startTime
end startTimer

to endTimer()
	set timeTaken to ((current date) - startTime as string) & " Seconds"
	activate
	ignoring application responses
		display alert "Script Timer" message "Running This Code Took " & ¬
			timeTaken buttons {"OK"} giving up after 7
	end ignoring
	return timeTaken
end endTimer

RESULT 46 SECONDS

I tested this code in Script DeBugger on 5000 identical text files, named from 0001 to 5000. The results of timing the code was 73 seconds.

use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

global startTime, timeTaken

set sourceFolder to (choose folder) as text
set MAX to 300
startTimer()

tell application "Finder" to set fileNames to name of files of folder sourceFolder
set sortedNames to sortListOfStrings(fileNames) -- you can use other sorting method as well
set theCount to count sortedNames

set N to theCount div MAX
if N > 0 then
	tell application "System Events"
		repeat with i from 1 to N
			set subFolder to make new folder at folder sourceFolder with properties {name:i}
			repeat with j from (i - 1) * MAX + 1 to i * MAX
				move file (sourceFolder & item j of sortedNames) to subFolder
			end repeat
		end repeat
	end tell
end if

set M to theCount mod MAX
if M > 0 then
	tell application "System Events"
		set subFolder to make new folder at folder sourceFolder with properties {name:(i + 1)}
		repeat with j from theCount - M + 1 to theCount
			move file (sourceFolder & item j of sortedNames) to subFolder
		end repeat
	end tell
end if

log endTimer()

to startTimer()
	set startTime to current date
	return startTime
end startTimer

to endTimer()
	set timeTaken to ((current date) - startTime as string) & " Seconds"
	activate
	ignoring application responses
		display alert "Script Timer" message "Running This Code Took " & ¬
			timeTaken buttons {"OK"} giving up after 7
	end ignoring
	return timeTaken
end endTimer

on sortListOfStrings(theList)
	-- convert list to Cocoa array
	set theArray to ¬
		current application's NSArray's arrayWithArray:theList
	-- sort the array using a specific function
	set theArray to ¬
		theArray's sortedArrayUsingSelector:"localizedStandardCompare:"
	-- return the sorted array as an AppleScript list
	return theArray as list
end sortListOfStrings

RESULT 73 SECONDS

peavine · April 11, 2021, 2:17am

I decided to see if ASObjC might improve the timing results and wrote the script below. It took 18 seconds to move 5,000 files located on an external SSD, although better results might be achieved by rewriting the move handler. To set a base, I timed KniazidisR’s script from post 2 and it took 132 seconds.

A few comments:

The script assumes that the sourceFolder contains the files that will be moved and nothing else.
The script errors if a source file already exists in the target folder.
The files are sorted in lexicographic rather than numeric order.
The script needs some error notification.
This script should be used with test files until found to be reliable.

use framework "AppKit"
use framework "Foundation"
use scripting additions

on main()
	set sourceFolder to "/Volumes/Store/Source Folder/"
	set targetFolder to "/Volumes/Store/Target Folder/"
	set theFiles to getFiles(sourceFolder)
	set folderCounter to 0
	
	repeat with i from 1 to (count theFiles)
		if (i - 1) mod 300 = 0 then
			set folderCounter to folderCounter + 1
			set newFolder to targetFolder & "Folder " & folderCounter
			makeFolder(newFolder)
		end if
		set theFile to POSIX path of ((item i of theFiles) as text)
		moveFile(theFile, newFolder)
	end repeat
end main

on getFiles(theFolder)
	set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
	set theFiles to current application's NSFileManager's defaultManager()'s contentsOfDirectoryAtURL:theFolder includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
	set sortDescriptors to current application's NSSortDescriptor's sortDescriptorWithKey:"lastPathComponent" ascending:true
	set sortDescriptors to current application's NSArray's arrayWithObject:sortDescriptors
	set sortedFiles to (theFiles's sortedArrayUsingDescriptors:sortDescriptors)
	return sortedFiles
end getFiles

on makeFolder(theFolder)
	set fileManager to current application's NSFileManager's defaultManager()
	fileManager's createDirectoryAtPath:theFolder withIntermediateDirectories:false attributes:(missing value) |error|:(missing value)
end makeFolder

on moveFile(theFile, theFolder)
	set sourceFile to current application's NSString's stringWithString:theFile
	set sourceName to sourceFile's lastPathComponent()
	set targetFolder to current application's NSString's stringWithString:theFolder
	set targetFile to targetFolder's stringByAppendingPathComponent:sourceName
	set fileManager to current application's NSFileManager's defaultManager()
	set {theResult, theError} to fileManager's moveItemAtPath:sourceFile toPath:targetFile |error|:(reference)
end moveFile

main()

KniazidisR · April 11, 2021, 3:26am

Cool script, peavine. As I see, it skips hidden files like my scripts. I have question only with sorting. Your script sorts not exactly like Finder sorting by name:

“myFile1” – For me, it is preferable this Finder sorting by name order.
“myFile2”
“myFile10”

Your script sorts like this:

“myFile1”
“myFile10”
“myFile2”

You can sort exactly the Finder way using localizedStandardCompare selector:


set sortDescriptors to current application's NSSortDescriptor's sortDescriptorWithKey:"lastPathComponent" ascending:true selector:"localizedStandardCompare:"

TEST:


use framework "AppKit"
use framework "Foundation"
use scripting additions

on sortByNameFinderWay(theFolder)
	set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
	set theFiles to current application's NSFileManager's defaultManager()'s contentsOfDirectoryAtURL:theFolder includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
	set sortDescriptors to current application's NSSortDescriptor's sortDescriptorWithKey:"lastPathComponent" ascending:true selector:"localizedStandardCompare:"
	set sortDescriptors to current application's NSArray's arrayWithObject:sortDescriptors
	set sortedFiles to (theFiles's sortedArrayUsingDescriptors:sortDescriptors)
	return sortedFiles
end sortByNameFinderWay

my sortByNameFinderWay(POSIX path of (choose folder)) as list

Tip: you can get fileManager only once, out of repeat loop. Symply make fileManager global variable.

peavine · April 11, 2021, 5:45am

Thanks KniazidisR for looking at my script and for the suggestions. I revised my script to create file manager only once and I optimized the getFiles handler. These changes reduced the timing result with 5,000 files to 14.5 seconds. I also changed the sort selector, which works as you indicate. Using an AppleScript to move 30,000 files may be a risky thing to do, and so I used copy rather than move in the script.

use framework "Foundation"
use scripting additions

on main()
	set sourceFolder to "/Volumes/Store/Source Folder/"
	set targetFolder to "/Volumes/Store/Target Folder/"
	set filesPerFolder to 300
	set fileManager to current application's NSFileManager's defaultManager()
	set theFiles to getFiles(sourceFolder, fileManager)
	set folderCounter to 0
	
	repeat with i from 1 to (count theFiles)
		if (i - 1) mod filesPerFolder = 0 then
			set folderCounter to folderCounter + 1
			set newFolder to targetFolder & "Folder " & folderCounter
			makeFolder(newFolder, fileManager)
		end if
		copyFile((item i of theFiles), newFolder, fileManager)
	end repeat
end main

on getFiles(theFolder, fileManager)
	set theFolder to current application's |NSURL|'s fileURLWithPath:theFolder
	set enumOptions to (current application's NSDirectoryEnumerationSkipsPackageDescendants as integer) + (current application's NSDirectoryEnumerationSkipsHiddenFiles as integer)
	set theFiles to (((fileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{} options:enumOptions errorHandler:(missing value))'s allObjects())'s |path|)
	return (theFiles's sortedArrayUsingSelector:"localizedStandardCompare:") as list
end getFiles

on makeFolder(theFolder, fileManager)
	fileManager's createDirectoryAtPath:theFolder withIntermediateDirectories:false attributes:(missing value) |error|:(missing value)
end makeFolder

on copyFile(theFile, theFolder, fileManager)
	set sourceFile to current application's NSString's stringWithString:theFile
	set sourceName to sourceFile's lastPathComponent()
	set targetFolder to current application's NSString's stringWithString:theFolder
	set targetFile to targetFolder's stringByAppendingPathComponent:sourceName
	set theResult to fileManager's copyItemAtPath:sourceFile toPath:targetFile |error|:(missing value)
	if theResult is false then display dialog "An error was encountered while copyng the files" buttons "OK" cancel button 1 default button 1 with icon stop
end copyFile

main()