Repeat loop parsing a .plist is slow

[eakwarren]

Hi All,

Longtime lurker, first time poster. I wrote a script that powers a Logic Pro articulation switcher called LAS. The following repeat loop takes ~3 seconds to parse the articulations in the .plist file and build lists which are sent via OSC messages to populate the articulation buttons (usually around 20 buttons).

Repeat loop code from the getArtList handler is below. Entire script here. Are there obvious optimizations I’m missing?

					repeat with i from 1 to (number of items in p1)
						set a to ""
						set b to ""
						set c to ""
						try
							set varArticulationsID to the value of property list item "ID" of property list item i of property list item "Articulations"
							set varName to the value of property list item "Name" of property list item i of property list item "Articulations"
							set varName to ("\"" & varName & "\"") #wrap in double quotes to escape string
							set varSwitchesID to the value of property list item "ID" of property list item i of property list item "Switches"
							if (varSwitchesID as text) contains "." then #Studio Horns/Strings create 100n.0 IDs
								set t to (varSwitchesID as text)
								set varSwitchesID to (text 1 thru -3 of t) as number
							end if
							set varSwitchesID to ("\"" & varSwitchesID & "\"")
							set varType to the value of property list item "Status" of property list item i of property list item "Switches"
							if varType = "NoteOn" then #some art sets have different names made with different versions of Logic
								set varType to "Note On"
							else if varType = "Poly Pressure" then
								set varType to "Poly Aftertouch"
							end if
							set varType to ("\"" & varType & "\"")
							(*if exists property list item "MidiChannel" of property list item i of property list item "Articulations" then
								set varMidiChannel to the value of property list item "MidiChannel" of property list item i of property list item "Articulations"
							else
								set varMidiChannel to null
							end if*)
							(*if exists property list item "Symbol" of property list item i of property list item "Articulations" then
								set varSymbol to the value of property list item "Symbol" of property list item i of property list item "Articulations"
								set varSymbol to ("\"" & varType & "\"")
							else
								set varSymbol to ""
							end if*)
							if exists property list item "MB1" of property list item i of property list item "Switches" then
								set varSelector to the value of property list item "MB1" of property list item i of property list item "Switches"
							else
								set varSelector to null
							end if
							#set varSelector to ("\"" & varSelector & "\"")
							(*if exists property list item "Mode" of property list item i of property list item "Switches" then
								set varMode to the value of property list item "Mode" of property list item i of property list item "Switches"
								set varMode to ("\"" & varMode & "\"")
							else
								set varMode to ""
							end if*)
							if exists property list item "ValueLow" of property list item i of property list item "Switches" then
								set varValueStart to the value of property list item "ValueLow" of property list item i of property list item "Switches"
							else
								set varValueStart to null
							end if
							#set varValueStart to ("\"" & varValueStart & "\"")
							(*if exists property list item "ValueHigh" of property list item i of property list item "Switches" then
								set varValueEnd to the value of property list item "ValueHigh" of property list item i of property list item "Switches"
							else
								set varValueEnd to null
							end if*)
							my clearMsg({9})
						on error
							if g_artSetByTrack = 0 then
								my sendOSC("/message9 ", "s ", "Incompatible articulation set. ")
								return 0
							end if
						end try
						
						try
							set varOutputType to the value of property list item "Status" of property list item "Output" of property list item i of property list item "Articulations"
							if varOutputType = "NoteOn" then #some art sets have different names made with different versions of Logic
								set varOutputType to "Note On"
							else if varOutputType = "Poly Pressure" then
								set varOutputType to "Poly Aftertouch"
							end if
							set varOutputType to ("\"" & varOutputType & "\"")
							
							if exists property list item "MB1" of property list item "Output" of property list item i of property list item "Articulations" then
								set varOutputSelector to the value of property list item "MB1" of property list item "Output" of property list item i of property list item "Articulations"
							else
								set varOutputSelector to null
							end if
							
							if exists property list item "ValueLow" of property list item "Output" of property list item i of property list item "Articulations" then
								set varOutputValueStart to the value of property list item "ValueLow" of property list item "Output" of property list item i of property list item "Articulations"
							else
								set varOutputValueStart to null
							end if
							
							my clearMsg({9})
						on error
							if g_artSetByTrack = 0 then
								my sendOSC("/message9 ", "s ", "Incompatible articulation set. ")
								return 0
							end if
						end try
						
						
						set a to varName & ":" & varArticulationsID
						copy a to the end of myList
						set b to varSwitchesID & ":[" & varType & ", " & varSelector & ", " & varValueStart & ", " & varOutputType & ", " & varOutputSelector & ", " & varOutputValueStart & "]" as text
						copy b to the end of myList2
						(*
						set a to "varArticulationsID:" & varArticulationsID & ", " & ¬
							"varName:" & varName & ", " & ¬
							"varSwitchesID:" & varSwitchesID & ", " & ¬
							"varType:" & varType & ", " & ¬
							"varSelector:" & varSelector
						if varMode ≠ "" then set a to a & ", " & "varMode:" & varMode
						if varValueStart ≠ -1 then set a to a & ", " & "varValueStart:" & varValueStart
						if varValueEnd ≠ -1 then set a to a & ", " & "varValueEnd:" & varValueEnd
						set a to "{" & a & "}"
						copy a to the end of myList
						*)
					end repeat

Thanks for reviewing.

[robertfern]

Can you post the plist file so we can test?

[eakwarren]

That would be helpful, wouldn’t it.

plist file

[leo_r]

As far as I remember, vanilla AppleScript’s plist operations are indeed slow (although last time I used them was many years ago).

Using AppleScript-ObectiveC to do the same via NSDictionary (or/and other classes) should be much faster and easier (although if you’ve never done this then there’s a learning curve).

[alastor933]

Another alternative would be to read the entire plist into a variable, and manipulate that in plain AppleScript:

set path_ to (choose file with showing package contents) as text

tell application id "sevs" to set recPlistContent to value of property list file path_

[Nigel_Garvey]

Part of the slowness is probably because the script’s getting the values from System Events individually, sometimes more than once to test that they exist. Each fetch involves a communication exchange between the script and System Events and System Events interpreting each specifier and digging the data out of the plist file.

What may speed things up (as I see alastor933’s just mentioned) is to get all the data you want in one go before the repeat — as lists of records for the Articulations and the Switches:

tell application "System Events" to set {|Articulations|:theArticulations, |Switches|:theSwitches} to value of property list file g_pfile

Then instead of a command like this in the repeat to fetch a value from the file …

-- tell application "System Events"
	set varArticulationsID to the value of property list item "ID" of property list item i of property list item "Articulations" -- of property list file g_pfile
-- end tell

… you use this to retrieve the already fetched value from the appropriate list:

set varArticulationsID to |ID| of item i of theArticulations

Note that the labels in the records are the same as the names of the named property list items except that they’re between vertical bars instead of in quotes.

Where there’s a possibility that a property may not exist, you can use concatenation of records to supply the default value for your purposes. So …

-- tell application "System Events"
	set varOutputType to the value of property list item "Status" of property list item "Output" of property list item i of property list item "Articulations" -- of property list file g_pfile
	if varOutputType = "NoteOn" then #some art sets have different names made with different versions of Logic
		set varOutputType to "Note On"
	else if varOutputType = "Poly Pressure" then
		set varOutputType to "Poly Aftertouch"
	end if
	set varOutputType to ("\"" & varOutputType & "\"")
	
	if exists property list item "MB1" of property list item "Output" of property list item i of property list item "Articulations" then
		set varOutputSelector to the value of property list item "MB1" of property list item "Output" of property list item i of property list item "Articulations"
	else
		set varOutputSelector to null
	end if
	
	if exists property list item "ValueLow" of property list item "Output" of property list item i of property list item "Articulations" then
		set varOutputValueStart to the value of property list item "ValueLow" of property list item "Output" of property list item i of property list item "Articulations"
	else
		set varOutputValueStart to null
	end if
-- end tell

… becomes something like:

set outputValues to (|Output| of item i of theArticulations) & {|MB1|:null, |ValueLow|:null}
set varOutputType to |Status| of outputValues
if varOutputType = "NoteOn" then #some art sets have different names made with different versions of Logic
	set varOutputType to "Note On"
else if varOutputType = "Poly Pressure" then
	set varOutputType to "Poly Aftertouch"
end if
set varOutputType to ("\"" & varOutputType & "\"")

set varOutputSelector to |MB1| of outputValues
set varOutputValueStart to |ValueLow| of outputValues

Hopefully you get the idea.

[eakwarren]

Thank you all for the helpful info. I’ll refactor and test over the weekend.

[peavine]

I’ve included a partial ASObjC solution below. It creates one array each for Articulations and Switches. It might be a good idea to have a separate repeat loop for each array, but I didn’t work through the logic of eakrwarren’s script to see if that would be helpful. The timing result with eakwarren’s plist file was 25 milliseconds.

--this script  will error if an item is not found
--the repeat loop returns NSNumbers and NSStrings which are coerced to integers and text

use framework "Foundation"
use scripting additions

set thePlist to "1st Violins SSO" --set to desired value
set theDefaults to current application's NSUserDefaults's alloc()'s initWithSuiteName:thePlist
set theArticulations to (theDefaults's objectForKey:"Articulations")
set theSwitches to (theDefaults's objectForKey:"Switches")

repeat with i from 0 to ((theArticulations's |count|()) - 1)
	set articulationID to ((theArticulations's objectAtIndex:i)'s valueForKey:"ID") as integer
	set articulationValueLow to ((theArticulations's objectAtIndex:i)'s valueForKeyPath:"Output.ValueLow") as integer
	set articulationMB1 to ((theArticulations's objectAtIndex:i)'s valueForKeyPath:"Output.MB1") as integer
	set articulationMidiChannel to ((theArticulations's objectAtIndex:i)'s valueForKeyPath:"Output.MidiChannel") as integer
	set articulationStatus to ((theArticulations's objectAtIndex:i)'s valueForKeyPath:"Output.Status") as text
	set articulationName to ((theArticulations's objectAtIndex:i)'s valueForKey:"Name") as text
	set articulationArticulationID to ((theArticulations's objectAtIndex:i)'s valueForKey:"ArticulationID") as integer
	set articulationMidiChannel to ((theArticulations's objectAtIndex:i)'s valueForKey:"MidiChannel") as integer
	set switchesID to ((theSwitches's objectAtIndex:i)'s valueForKey:"ID") as integer
	set switchesMB1 to ((theSwitches's objectAtIndex:i)'s valueForKey:"MB1") as integer
	set switchesMode to ((theSwitches's objectAtIndex:i)'s valueForKey:"Mode") as text
	set switchesStatus to ((theSwitches's objectAtIndex:i)'s valueForKey:"Status") as text
end repeat

The above script errors if a key or value is not found, and one approach that deals with this is included below. This script segment can be expanded to do some of the other stuff that eakwarren wants done.

use framework "Foundation"
use scripting additions

set thePlist to "1st Violins SSO" --set to desired value
set theDefaults to current application's NSUserDefaults's alloc()'s initWithSuiteName:thePlist
set theSwitches to (theDefaults's objectForKey:"Switches")

set switchesStatus to ((theSwitches's objectAtIndex:0)'s valueForKey:"xStatusx") --key does not exist
if switchesStatus is (missing value) then
	set switchesStatus to "default value"
else if (switchesStatus as text) is "NoteOn" then
	set switchesStatus to "Note On"
else
	set switchesStatus to switchesStatus as text
end if

I suspect eakwarren is already working to edit his existing script and has no interest in an ASOBC suggestion. I’m posting this simply to demonstrate an alternative approach FWIW.

[robertfern]

@alastor933

Another alternative would be to read the entire plist into a variable, and manipulate that in plain AppleScript:

@alastor933 is correct. This was way faster.
When I run parsing of the plist data in Script Geek, it is over 28 times faster to get the data in one go, and then parse it in AppleScript.

I used the sample plist eakwarren posted above for testing.
Here are the two quick scripts I used for testing.

property pfile : (path to desktop folder as text) & "1st Violins SSO.plist"
property ArticulationsIDs : {}
property ArticulationsNames : {}
property SwitchesModes : {}

local theArticulations, theSwitches, i

tell application "System Events"
	set i to 1
	repeat
		try
			set end of ArticulationsIDs to value of property list item "ID" of property list item i of property list item "Articulations" of property list file pfile
			set end of ArticulationsNames to value of property list item "Name" of property list item i of property list item "Articulations" of property list file pfile
			set end of SwitchesModes to value of property list item "Mode" of property list item i of property list item "Switches" of property list file pfile
			set i to i + 1
		on error
			exit repeat
		end try
	end repeat
end tell

and the faster one…

property pfile : (path to desktop folder as text) & "1st Violins SSO.plist"
property ArticulationsIDs : {}
property ArticulationsNames : {}
property SwitchesModes : {}

local theArticulations, theSwitches, i

tell application "System Events" to set {|Articulations|:theArticulations, |Switches|:theSwitches} to value of property list file pfile
repeat with i from 1 to count theArticulations
	set end of ArticulationsIDs to |ID| of item i of theArticulations
	set end of ArticulationsNames to |Name| of item i of theArticulations
	set end of SwitchesModes to |Mode| of item i of theSwitches
end repeat

[eakwarren]

Thank you @peavine and @robertfern for additional insights. I am interested in learning multiple solutions, including ASOBC, so this is great! However, I’ll work through one approach at a time so my brain doesn’t explode.

There are other handlers where I can use the single .plist fetch, so I’m working on making theArticulations and theSwitches lists available globally. Eliminating all those fetches will make a big difference.

[peavine]

@eakwarren

However, I’ll work through one approach at a time so my brain doesn’t explode.

I know the feeling.

[eakwarren]

Here’s a pass at refactoring for @Nigel_Garvey’s single fetch. The getArtList handler is executing an avg. of 1.36 seconds which may still be slow.

LAS-singleFetchDev.scpt.zip

Here’s a video showing the lag between clicking the button and having the active articulation highlight to match.

[eakwarren]

Reporting back on refactor using @peavine’s ASOBC. Average execution of getArtList is still 1.36 seconds. (Hey, at least they’re consistent and it’s less than the original ~3 seconds! )

Perhaps I’ll build the artList only when there’s a change.

							if g_tw_artSetNew ≠ g_tw_artSetOld then #only update on art set change
							    my sendOSC("/artList ", "s ", my getArtList()) #additional sendOSCs in handler for supporting art set elements
							    my sendOSC("/artColors ", "s ", my getArtColors())
							end if

[robertfern]

I wish I had Logic pro to test, but me thinks your slowness lies elsewhere in the script, not in the plist reading/parsing.

[eakwarren]

Right, the plist parse is just one aspect of getArtList().

on getArtList() #returns list of arts from artSet.plist type:Controller|Note On (|Note Off|Poly Aftertouch|Program|Aftertouch|Pitchbend|Velocity not implemented yet)
	if (g_artSetByTrack = 0) and (g_tw_artSetNew is "None") then
		my sendOSC("/message4 ", "s ", "Articulation Set is None. ")
		my clearMsg({7, 8})
		return "{}"
	else if (g_artSetByTrack = 1) and (g_tw_artSetNew is "None") then
		#my sendOSC("/message4 ", "s ", "Articulation Set is None. ")
		my clearMsg({7})
	end if
	if g_tw_artSetNew starts with space then return "{}" #depending on timing script grabs Staff Style with leading space instead of Articulation Set
	
	tell application "System Events"
		set g_pfile to my getPlist()
		if g_pfile = 0 then
			my sendOSC("/message8 ", "s ", "Articulation Set not found in " & g_userName & "/Music/Audio Music Apps/Articulation Settings (or sub-folder). ")
			my clearMsg({4, 6, 7}) #6 select a region
			return "{}"
		else if g_pfile = 2 then
			my sendOSC("/message7 ", "s ", "Multiple Articulation Sets with same name, please make unique.</br><div class=\"marquee\">" & g_pfile2 & "</div></br>")
			my clearMsg({4, 6, 8})
			return "{}"
		else if g_pfile ≠ "" then
			my clearMsg({4, 7, 8})
			
			tell application "System Events" to set {|Articulations|:g_plistArticulations, |Switches|:g_plistSwitches} to value of property list file g_pfile
			
			try
				my clearMsg({8})
				
				#myList = {"Legato":0,"Long":1} etc.  example nested {color: "red", wheels: 4, engine: { cylinders: 4, size: 2.2 }}
				#{varArticulationsID:1001, varName:"Legato", varSwitchesID:1001, varType:"Controller", varSelector:4, varMode:"Permanent (Trigger) ", varValueStart:0, varValueEnd:127, varInputMidiChannel:0 - 15}
				
				set myList to {}
				set myList2 to {}
				
				
				repeat with i from 1 to count of g_plistArticulations
					set a to ""
					set b to ""
					set c to ""
					
					try
						set varArticulationsID to |ID| of item i of g_plistArticulations
						set varName to ("\"" & |Name| of item i of g_plistArticulations & "\"")
						set outputValues to (|Output| of item i of g_plistArticulations) & {|MB1|:null, |ValueLow|:null}
						
						set varOutputType to |Status| of outputValues
						if varOutputType = "NoteOn" then #some art sets have different names made with different versions of Logic
							set varOutputType to "Note On"
						else if varOutputType = "Poly Pressure" then
							set varOutputType to "Poly Aftertouch"
						end if
						set varOutputType to ("\"" & varOutputType & "\"")
						
						set varOutputSelector to |MB1| of outputValues
						set varOutputValueStart to |ValueLow| of outputValues
						
						my clearMsg({9})
					on error
						if g_artSetByTrack = 0 then
							my sendOSC("/message9 ", "s ", "Incompatible articulation set. ")
							return 0
						end if
					end try
					
					
					set varSwitchesID to |ID| of item i of g_plistSwitches
					if varSwitchesID contains "." then #Studio Horns/Strings create 100n.0 IDs
						set t to (varSwitchesID as text)
						set varSwitchesID to (text 1 thru -3 of t) as number
					end if
					set varSwitchesID to ("\"" & varSwitchesID & "\"")
					
					set varType to |Status| of item i of g_plistSwitches
					if varType = "NoteOn" then #some art sets have different names made with different versions of Logic
						set varType to "Note On"
					else if varType = "Poly Pressure" then
						set varType to "Poly Aftertouch"
					end if
					set varType to ("\"" & varType & "\"")
					
					set outputValues to (item i of g_plistSwitches) & {|MB1|:null, |ValueLow|:null, |ValueHigh|:null} --may also have Mode, Status
					
					set varSelector to |MB1| of outputValues
					set varValueStart to |ValueLow| of outputValues
					
					--commented code below WIP
					(*if exists property list item "MidiChannel" of property list item i of property list item "Articulations" then
								set varMidiChannel to the value of property list item "MidiChannel" of property list item i of property list item "Articulations"
							else
								set varMidiChannel to null
							end if*)
					(*if exists property list item "Symbol" of property list item i of property list item "Articulations" then
								set varSymbol to the value of property list item "Symbol" of property list item i of property list item "Articulations"
								set varSymbol to ("\"" & varType & "\"")
							else
								set varSymbol to ""
							end if*)
					
					
					
					
					#set varSelector to ("\"" & varSelector & "\"")
					#set varValueStart to ("\"" & varValueStart & "\"")
					(*if exists property list item "Mode" of property list item i of property list item "Switches" then
								set varMode to the value of property list item "Mode" of property list item i of property list item "Switches"
								set varMode to ("\"" & varMode & "\"")
							else
								set varMode to ""
							end if
					
					if exists property list item "ValueHigh" of property list item i of property list item "Switches" then
								set varValueEnd to the value of property list item "ValueHigh" of property list item i of property list item "Switches"
							else
								set varValueEnd to null
							end if*)
					
					
					set a to varName & ":" & varArticulationsID
					set end of myList to a
					set b to varSwitchesID & ":[" & varType & ", " & varSelector & ", " & varValueStart & ", " & varOutputType & ", " & varOutputSelector & ", " & varOutputValueStart & "]" as text
					set end of myList2 to b
					(*
						set a to "varArticulationsID:" & varArticulationsID & ", " & ¬
							"varName:" & varName & ", " & ¬
							"varSwitchesID:" & varSwitchesID & ", " & ¬
							"varType:" & varType & ", " & ¬
							"varSelector:" & varSelector
						if varMode ≠ "" then set a to a & ", " & "varMode:" & varMode
						if varValueStart ≠ -1 then set a to a & ", " & "varValueStart:" & varValueStart
						if varValueEnd ≠ -1 then set a to a & ", " & "varValueEnd:" & varValueEnd
						set a to "{" & a & "}"
						copy a to the end of myList
						*)
				end repeat
				
				#these only exists once in .plist, process outside repeat
				(*if exists property list item "InputMidiChannel" then
						set varInputMidiChannel to (the (value of property list item "InputMidiChannel") + 1) #valid values 0-15, All is -1, so +1
					else
						set varInputMidiChannel to 1
					end if
					if varInputMidiChannel = 0 then set varInputMidiChannel to 1 #adjust if ch is All
					my sendOSC("/g_midiCh ", "i ", varInputMidiChannel)
					if exists property list item "OctaveOffset" then
						set varOctaveOffset to the value of property list item "OctaveOffset" #valid values -10 to -1 & 1 to 10
					else
						set varOctaveOffset to 0
					end if
					my sendOSC("/g_octaveOffset ", "i ", varOctaveOffset)  *)
			on error
				return "{}"
			end try
			
		end if
	end tell
	
	#format list as string with , separators
	set {TID, text item delimiters} to {text item delimiters, ","}
	set myList to myList as text
	set text item delimiters to TID
	set myList to "{" & myList & "}" #wrap in {} for OSC /artList message as object
	
	set {TID, text item delimiters} to {text item delimiters, ","}
	set myList2 to myList2 as text
	set text item delimiters to TID
	set myList2 to "{" & myList2 & "}" #wrap in {} for OSC /typeArray message as array
	my sendOSC("/switchesObjectArray ", "s ", myList2)
	
	set g_tw_artSetOld to g_tw_artSetNew
	
	return myList
end getArtList

[robertfern]

From what I’m seeing, most of the code doesn’t seem to be needed inside a ‘tell applications’ block.
Also you have a ‘tell application “System Events” ‘ inside of a ‘tell application “System Events” ‘. Why?

[eakwarren]

Oversight on my part when refactoring. I’ve now removed the outer tell/end tell and just have the single line

tell application "System Events" to set {|Articulations|:g_plistArticulations, |Switches|:g_plistSwitches} to value of property list file g_pfile

[robertfern]

By removing the outer tell block, you can remove the ‘my’ from infront of each method call.

[eakwarren]

Thanks @robertfern. I learned something new today!