Performance question

Scripting Forums AppleScriptObjC and Xcode
#1

The short version of the question:
How much does it cost to call an Objective-C method from AppleScript as compared to the cost of calling it in “pure” Objective-C?

The somewhat longer version:
I am well aware of that coercing data back and forth between Objective-C and AppleScript comes at a rather significant cost if the amount of data is somewhat large, such as list to NSArray, record to NSDictionary, etcetera. For single calls this is usually not noticed, but in long loops this may be noticeable.

Assuming the data can stay in Cocoa/Objective-C form, such as an NSArray, and I pass it between 20 methods, and the whole thing is run in a loop 5 000 times, there are 100 000 calls, but no coercing. If I do this in ASOC (using an AppleScript loop), how much extra does that cost compared to doing the same thing in “pure” Objective-C?

An alternative is to stay within ASOC and its AppleScript loop, but do the same thing with AppleScript methods. Of course, no one can say how much slower or faster that could be (in the general case) but I know there are cases where it is significantly faster to do things with an “ugly” TID based solution rather than coerce to NSString and call componentsSeparatedByString_ on it. I am only saying this as a possible alternative for simple cases, but I would expect that if I can keep the data as an Objective-C class and calling methods on it, it would be faster in most cases, but perhaps not necessarily so?

#2

It is hard to say how much faster it would be. The AppleScript operators and controls are self a lot lot slower than objective-C. So without talking about objects and coercing anything you need to keep in mind that your repeat of 5000 times is slower specially when there are a bunch of if statements in as well without calling any instance methods.

I would say, keeping it into Objective-C objects (so your variable is only a pointer to and cocoa object) is the fastest way in general/most cases. Remember that AppleScript is event based. So there is an event for every command you have in Applescript and that is slower than sending an message (Objective-C is message based and don’t have calls) to cocoa through AppleScriptObjC. I haven’t noticed it yet but there could be situations where it is otherwise of course.

But to ask your question about comparing it with purely Objective-C I would say it’s much slower and noticeable too. That’s why most programming I’ll do in Objective-C.

#3

Because I found this topic very interesting I started to do some little research.

What I wanted to do is to show you the difference between pure applescript, applescriptObjC and Objective-C in calling/messaging and speed of controls and operators. The fastest way of programming is without coercing (which I called the hybrid version later). What I do is next:

create a empty list/array
loop 125,000 times with a loop counter
add to the end of list the loop counter
when we’ve created a list from 1, 2, 3, 4, 5, … 124,999, 125,00 we loop through this list
every time we sum the value with the previous sum

IN purely applescriptL:

script speedUp
		property listRef : {}
	end script
	repeat with x from 1 to 125000
		set end of speedUp's listRef to x
	end repeat
	set total to 0
	repeat with aValue in speedUp's listRef
		set total to total + aValue
	end repeat

it takes 8,5 seconds on my machine to complete the task. As you can see in the code I don’t use anything from cocoa and optimized the speed as you would do in script editor with a list reference.

in Hybrid:

set theList to NSMutableArray's alloc()'s init()
		repeat with x from 1 to 125000
			theList's addObject_(x)
		end repeat
		set total to 0
		repeat with x from 0 to 124999
			set total to total + (theList's objectAtIndex_(x)'s doubleValue())
		end repeat

it takes 18,5 seconds on my machine to complete the task. As you can see I use an NSMutableArray and add objects to it. This way for such a simple task we need 250,000 coercions. We place integer values into a NSMutableArray and they will be coerced into an NSNumber (because NSArray’s only contains objects, integer is not an object). When summing all values in the Array we need a coercion as well.

in Objective-C:

It takes 0,08 seconds and it is much faster Objective-C is than both applescript methods.

conclusion
It’s shows us how many faster Objective-C is compared with AppleScriptObjC. However this was scripting/programming in it’s most simplest form so don’t let this result mislead you to write pure applescript as much as possible. It’s is not cocoa that is expensive for your application, but the coerion betweenAppleScript and Cocoa done by the appleScriptObjC framework.

Also don’t forget that many methods contains many lines of code which not only takes a lot of effort away but gains also a lot of speed. You’ll only need one coercion for multiple applescript commands. Here is when pure AppleScript is too expensive.

This test showed is there is no rule what you should do and what is best. In this situation it is better to use pure applescript but in a lot of cases it is definitely better to use cocoa objects than applescript Objects.

#4

To continue my post and to complete your answer:

We take a string and append 125000 times the string “x” to it.

In pure Applescript:

set theString to ""
repeat 125000 times
	set theString to theString & "x"
end repeat

It takes about 30 seconds to complete and is very expensive.

in hybrid:

set theString to NSMutableString's alloc()'s init()
repeat 125000 times
	theString's appendString_("x")
end repeat

It takes about 5.5 seconds to complete and is almost 6 times faster than in pure applescript. This is simply because we use only 1 coercion (this time 125,0000 times a coercion) of a small string (“x”) and leave the rest to cocoa.

in Objective-C:

Only this takes about 0.01 seconds.

This shows that using cocoa objects with instance methods is sometimes better than using pure appleScript.

#5

Hi!
I am glad you found the topic interesting!
And I found your experiments interesting.
(But your computer is obviously much faster than mine.grrr.)

I played around with the samples and did variations to see what it would do.
As for the hybrid version of adding numbers in a list, it is pretty extreme in the number of coercions required. To see what would happen if there are no coercions, I did:

set theList to current application's NSMutableArray's alloc()'s init()
set y to current application's NSNumber's numberWithInteger_(9)
repeat 125000 times
	theList's addObject_(y)
end repeat
theList's |count|()

It is still pretty slow, but 1.5 times faster than the pure appleScript version:

script speedUp
	property listRef : {}
end script
set y to 9
repeat 125000 times
	set end of speedUp's listRef to y
end repeat
length of speedUp's listRef

So, although the coercions are very costly, the actual calls to the method still costs a lot.
In the very extreme case, let’s run:

set theList to current application's NSMutableArray's alloc()'s init()
repeat 125000 times
	theList's |count|()
end repeat

Despite this does almost nothing, it takes almost as long time as if the method had to create a list of 125000 elements!!
Compare that with pure appleScript:

script speedUp
	property listRef : {}
end script
repeat 125000 times
	length of speedUp's listRef
end repeat

This computes in virtually no time! (About 50 times faster).

All these cases are extreme of course, but it certainly shows that both coercions and method calls can be very costly. On the other hand, they also show that the cost of a method call can win over pure appleScript even in the simple case of just making a long list of equal elements. In more complicated tasks, the advantage can of course be dramatically much greater. (But. the speed of pure Objective-C is really tempting. hmm…)

Thanks for listening.

#6

Well it seems a bit extreme but sometimes you need to get extreme to notice the difference. Your examples has proven proven again that sometimes it is faster to use pure AppleScript instead of using cocoa. I’m not disapproving your tests but length and count are powerful commands because the count and length has already been set when creating the object. When you add or change the list it is immediately recounted and the properties count and length are updated.

Yes the speed of pure Objective-C is tempting. Even knowing that AppleScript is an interpreted language (like php, javascript etc) it is still an slow programming language. So if AppleScript is your reference every programming language is fast:D. AppleScript handles strings with a lot of overhead with allocating and re-allocating memory. A string, the word speaks for itself, is an character array and when programming C you’ll be amazed how fast it can handle character and wide character arrays (wide character is for multibyte strings like UTF-8). It is even faster than NSString objects in Objective-C, but only if you write it right. The problem with C is that you have to write efficient, avoid memory leaks (for stay open applications), avoid buffer overflows (which is now protected by the kernel since Leopard) and efficient memory allocations.

I would recommend anyone to look into Objective-C, even if some of my applications are an AppleScriptObjC project I still write data sources (classes with a lot of data handling and messages) for instance in Objective-C. I use AppleScriptObjC classes for things that can’t be done with Objective-C or takes too much time (money). I think programming AppleScriptObjC gives scripters the perfect opportunity to learn Objective-C by rewriting classes from AppleScriptObjC to Objective-C and starting with smallest and simplest classes first.

#7

AppleScript compiles scripts before running them

#8

You can call it parsing, compiling or whatsoever but it still is an interpreted language and not a compiler language. They all need an interpreter who create a virtual run time environment to run the code. They all have some sort of ‘compiling’ but PHP and Javascript does it right before running the code while AppleScript can do it earlier and store it for later use.

#9

Yea, I fully agree with everything you said; if I was still with AppleScript Studio, I would probably still know almost nothing about Objective-C, whereas now I can at least read the documentation without looking like a dumb question mark;-). At some stage, I will probably convert at least some of my stuff to Objective-C, but I am pretty certain that, as things are now, it’s faster for me to get things done with ASOC, since I am pretty fluent in AppleScript. From what you said, it appears that also you are faster doing things in ASOC than pure Objective-C, despite you know them both pretty well.

As for heavy string processing (for which AppleScript is next to useless) I have used a lot of Perl and regular expressions in the past, and it can really be darn fast, despite it’s interpretive! It is well worth the cost of a do shell script, even for relatively simple stuff. And for chewing hundreds of strings in a whole bunch of files, it simply shines!

In the tests, my choice of “length of” was intentional – it’s pretty close to a noop, so the time mostly consisted of overhead for the loop. From this, one can conclude that the time for the ASOC version mainly consisted of the cost of doing the call (since the method itself consumes almost zero execution time). These experiments, and some other I have done, have led me to the following rough estimates:

¢ Each call to an Objective-C method costs roughly 0.1 ms on an oldish laptop (if run inside AppleScriptObjC Explorer, which doesn’t consume all cpu-power – the load of the two CPUs is about 60%).

¢ When stacking several methods after one another, each one costs roughly the same.

¢ Coerce of a smallish NSArray to list costs about the same (or slightly less) as a method call. But if done many times, AppleScript appears to do it slower and slower! It just keeps growing until you restart the thing that runs the script! Coercing large lists can be very time consuming.

¢ String processing that easily can be done with TIDs are about 10-20 times faster within AppleScript than calling a few methods to do the same thing, provided the string is relatively small – but at about 40 000 characters, they are about equal, and beyond that TID looses.

¢ The cost of calling “do shell script” is about 100 times longer than a call to an Objective-C method!

¢ Doing things in native C or Objective-C is cheating. (Ok, just joking.)

I am sure these conclusions can be criticized, since the timing methods can be debated (doing things in a tight loop may not always show real-life conditions). My hope is to at least to get a reasonable feeling for what kind of solutions lend themselves best for different tasks.

Best Regards,

#10 
my choice of "length of" was intentional -- it's pretty close to a noop

Not really. I was doing some timing the other day for using NSRegularExpression to change some text, to compare it with some examples someone posted using a scripting addition and using do shell script and sed. And I was puzzled at why it was taking four or five times longer. It turned out that the NSRegularExpression method includes a range argument, so I was getting that using “length of x”.

A more thorough check showed that line was, in fact, taking 90% of the time. So by making an NSString and getting |length|(), the time taken went from about 0.7 seconds to about 0.075 seconds.

This is probably an extreme case – the string was more than 5MB.

#11

Yes and no. Yes, getting the length of a string can take a lot of time for long strings, but in my case, I took the length of a zero-length list, and in this case, the cpu consumption for that is pretty close to a noop, at least in comparison to the time it took to do a call to a method (was about 1.5 microsecond per iteration for getting the length and about 80 microseconds for doing the same thing by calling an Objective-C method). So, I still think my conclusion is pretty ok, namely that a method call costs roughly 0.1 ms per call on an oldish laptop. Further tests support that.

But I fully agree that taking the length of a long string can almost kill AppleScript. I just did some extensive tests that strongly supports using Objective-C calls whenever the data (string or list) is large:

LISTS (on an oldish laptop, and run from within AppleScriptObjC Explorer):

For a list of single characters, taking the length of that in AppleScript costs:
about 0,001 ms/iteration for zero items, about 0.002 ms/iteration for 10 items, 0.07 ms/iteration for 1000 items, 0.65 ms for 10 000 items, 7 ms/iteration for 100 000 items!

Doing the same thing with an Objective-C method, assuming the NSArray is already in place:
about 0.075 ms/iteration for 0 to 1000 items, about 0.085 ms/iteration for 10 000 items, about 0.1 ms for 100 000 items.

So, the two methods break-even at around 1000 items.

But to create the NSArray from the list costs 5 ms for a 1000-item list, but only about 0.17 ms for 10 items. For 10 000 items it costs 55 ms, and for 100 000 items it costs a wopping 550 ms! I.e extremely dependent on size! For long arrays that means it is a LOT better if it can be created (or extended) on the Objecive-C side rather than coerce from AppleScript.

STRINGS (on an oldish laptop, and run from within AppleScriptObjC Explorer):

Taking the length of strings in AppleScript costs:
about 0.0013 ms for length 0, about 0.004 ms for length 10, about 0.16 for length 1000, 1.6 ms for length 10 000, 16 ms for length 100 000!

Doing the same thing with an Objective-C method, assuming the NSString is already in place:
0.08 ms/iteration for a zero to 100 000-string, i.e constant speed!

The two methods break-even at string lengths about 450 characters. Surprisingly short!

But to create the NSString from the text costs about 0.1 ms for a 10-length string, 0.12 ms for a 1000-length string, 0.3 ms for a 10 000-length string, and 2 ms for length 100 000. So, the cost of making NSStrings does not increase as much as making NSArrays did.

Regards,

#12

Since the do shell script feature AppleScript itself became more powerful by this. I use awk, sed and grep a lot depending on the goal. Note that these tools has been optimized and written in C and they outperform Objective-C’s equivalent, no matter how well it’s written. The costs of do shell is for many repetitive task huge and therefore it’s advisable to create complete shell scripts and run them once in AppleScript to get rid of the overhead.

But you’re able create a pipe in AppleScriptObjC and then you don’t need do shell script but open your own shell. When using multiple shell commands you’re application becomes more efficient.