Parsing a log file to segment a video clip.

[BenJi]

Hey every one. I’m working on a project at work, and as a newbie to apple script ( i know some other languages) this is bothering me.

I have a server who plays video clips during the recording of a tv show. Afterwards, i play the whole playlist of previously played clips and record it on tape for archiving/backup. The playlist app generates a file wih clip names and time code on the tape . ( vtr and computer are sync).

My goal is to do that automatically and digitally ( we’re runing out of tapes :p)

I started by scheduling a recording on a mac pro ( sdi input).
I play my playlist on the video server and it records on the mac.
It stops automaticaly, and send the file into a folder.

Now that i have the whole playlist in one file, i need to segment it in multiples files, with proper durations and names.

For the parsing, i generate with the video server a .log file is like this :
12:32:45, clip1, 00:00:04:21, 12:32:49, clip2, 00:02:23:12

In this exemple, the clip1 was recorded on 12h32,45 and have a duration of 4 sec and 21frames.

First, i need to parse the datas in list or array i think, then process the duration in a way that applescript understand ( i dont know if i can specify frames)

Then trim the movie according to the duration specified in my log file.

I want the clips to be exported with proper names.

What i have done yet is a folder action that opens any new file in the folder, trims it to a specific durationand export it.

on adding folder items to this_folder after receiving this_file

Set file_name to name of (info for name of this_file)
     

tell application "QuickTime Player"
Activate
Open this_folder & file_name
	set the movie_length to the duration of document 1
	
	set start_time to 0
        set end_time to 60 
	
		
		set current time of document 1 to start_time
		trim document 1 from start_time to end_time
		set the target_file to ((path to movies folder as string) & base_name & "-" & (i as string) & ".mxf")
		export document 1 in file target_file using settings preset "Computer"
		delay 1 
		my reset_movie()
		set start_time to end_time
	
	set current time of document 1 to 0
end tell

end adding folder items to

on reset_movie()
	tell application "System Events"
		tell process "QuickTime Player"
			keystroke "z" using command down
                        keystroke "q" using command down
		end tell
	end tell
	delay 2
end reset_movie

It works well exept for my reseting part. When a first trimm is done, the auto “cmd z” doesnt work because the export windows is in front, therefore, quicktile doesnt cancel, and retrim on the previously trimmed portion…
It doesnt close either…

My two questions are :

• how can i parse the log file properly
• how can i loop the triming and exporting action using those datas
• and close quicktime afterwards.

Thank you for your time, andthanks for your help.

[McUsrII]

Hello.

Can you please put in a little bit more of your logfile?

Two full lines so that we know how the lines ends, and such unless there is only one long lines, then I’d like to see how the log for two clips looks like.

[BenJi]

Hey. The log file is actually a big long line. I can’t show you the exact log right know, i’m not working on the week end, but its very similar to what i’ve posted. Maybe there is a tab beetwzen clips, but i’m not sure, how can i know? I’ll tell you that on monday!

Basiccaly, the datas are separated by comas.

Thanks for your help

[McUsrII]

I’ll parse the log for you into a list of lists, where each sublist contains the hour, clipname and duration, is that what you want?

Then you can access each clip like it was in an array, and get the clip, the hour and the duration easily out of the sublist.

[BenJi]

Exactly! That would be great.

I think i might need a while loop afterwards, or somthing like
x= count of theList
for( i=1; i=x){
my actions
i++
}

[McUsrII]

Here you are, while I figure out why I can’t make the sed script to work from within an AppleScript.

local logfilename, parsedLog, tids, rawloglist, numFields, recCount, i, recList
set logfilename to quoted form of "/Users/you/your/videologfile.log"

set parsedLog to do shell script "cat " & logfilename & " ; echo "
set {tids, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ","}
set rawloglist to text items of parsedLog
set AppleScript's text item delimiters to tids

set numFields to length of rawloglist
if numFields mod 3 is not 0 then error "the video clip log is malformed"

set recCount to numFields div 3

set {recList, i} to {{}, 1}
repeat recCount times
	set recOffset to ((i - 1) * 3)
	set end of recList to {item (2 + recOffset) of rawloglist, item (1 + recOffset) of rawloglist, item (3 + recOffset) of rawloglist}
	set i to i + 1
end repeat

@BenJi

think i might need a while loop afterwards, or somthing like
x= count of theList
for( i=1; i=x){
my actions
i++
}

You can use AppleScript’s simili of a for each loop It is important to understand that the variable you use here, is like a normal iterator, but it only stores a reference to the active item in the list, so you should really dereference it by using the contents operator within your list, or you may end up with no values, and just references to the items in the list you iterate over.

set endTimes to {}
repeat with aRec in theList	
	set end of endTimes to calcEndtime(contents of item 2 of aRec, contents of item 3 of aRec)
end repeat

But you may also do like this:

set recCount to numFields div 3

set endTimes to {}
repeat with i from 1 to recCount
	set end of endTimes to calcEndtime(contents of item 2 of item i of theList, contents of item 3 of  item i of theList)
end repeat

[McUsrII]

This is another take on the same problem: shorter, and in gratitude to Nigel Garvey for the trick with the literal linefeeds! It looks ugly when transformed into an AppleScript, and the comments had to go, but it works! (gnu sh started interpreting parenthesis in in them.)

set reclist to paragraphs of (do shell script "{ cat " & theLog & " ; echo ; } |sed -n -e '
/^[[:alnum:]]\\{1,\\}/ {" & linefeed & "
:start" & linefeed & "
h" & linefeed & "
s/^\\([[:space:]]*\\)\\([^,]\\{1,\\},[^,]\\{1,\\},[^,]\\{1,\\}\\)\\(,\\)\\(.*\\)/\\2/p" & linefeed & "
t prune" & linefeed & "
n" & linefeed & "
b start" & linefeed & "
:prune" & linefeed & "
g" & linefeed & "
s/^\\([[:space:]]*\\)\\([^,]\\{1,\\},[^,]\\{1,\\},[^,]\\{1,\\}\\)\\(,\\)\\(.*\\)/\\4/" & linefeed & "
b start" & linefeed & "
}" & linefeed & "' -e 's/\\([^,]\\{1,\\}\\),[[:space:]]*\\([^,]\\{1,\\}\\),[[:space:]]*\\([^,]\\{1,\\}\\)/\\2,\\1,\\3/p' ")

[BenJi]

thanks a lot for your help.

If i understand well, i will end up with a list like this {{12:23:22,clip1,00:02:34},{12:36:22,clip2,00:05:22}}
that would be awsome.

Then i will be able to do somthing like you suggested.

It will be more like this :

Set thebiglist to parsedlist
Set num_of_clips to the count of thebiglist
Local previous_end_time


Repeat with i from 1 to num_of_clips

        Set thesmalllist to item i of thebiglist as a list

       Set start_rec_time to item 1 of thesmalllist
       Set clip_name to item 2 of thesmalllist
      Set clip_duration to item 3 of thesmalllist

Set start_time to previous_end_time
Set end_time to start_time + clip_duration

My trimOperation(start_time,clip_name,clip_duration,start_time,end_time)

Set previous_end_time to start_time + clip_duration
Set i to i+1

End repeat 

On trimOperation

--I will trim every thing here and export



End trimoperation

Do this looks good to you?

Anyway i’ll test everything on monday

Thanks again for yout help

[McUsrII]

It looks basically good, but you can’t add up times like that!

And the property for the count of the records are named length.

That is not interesting to program, but I thought I’d make a couple of handlers for adding up time anyways, you’ll see them before midnight −1 CET. I guess you are using 24 hour format for your clock? That is, if you use the start time, as the time the first recording started, and don’t set that time to zero?

[BenJi]

Hey, that’s very kinf of you.
Yes i’m using 24h clock.

For adding times, i guess i could convert everything into seconds, since quictime accepts seconds as start and end time?

Anyway, the time the clip was recorded doesn’t matter. I wont need it anymore. It’s a legacy from the time we recorded the clips on Beta sx tape, and needed the time code to be the same on the log and on the tape ( vtr was set on hour time-code) so we could print it and have like a list of clips. Easier to find the clips on the tape afterward.

The only things that matter are clip durarions and names, so i can segment the file and name the files.

[McUsrII]

Hello!

This should work.

### last version
# not sure what to do with the timeframes, so I'll remember them, and add the extra second, when 60 is reached.
property frameRate : 25
on run
	local reclist, theLog
	set AppleScript's text item delimiters to ""
	set theLog to quoted form of "/Users/you/your/logfile"
	
	set reclist to paragraphs of (do shell script "{ cat " & theLog & " ; echo ; } |sed -n -e '
/^[[:alnum:]]\\{1,\\}/ {" & linefeed & "
:start" & linefeed & "
h" & linefeed & "
s/^\\([[:space:]]*\\)\\([^,]\\{1,\\},[^,]\\{1,\\},[^,]\\{1,\\}\\)\\(,\\)\\(.*\\)/\\2/p" & linefeed & "
t prune" & linefeed & "
n" & linefeed & "
b start" & linefeed & "
:prune" & linefeed & "
g" & linefeed & "
s/^\\([[:space:]]*\\)\\([^,]\\{1,\\},[^,]\\{1,\\},[^,]\\{1,\\}\\)\\(,\\)\\(.*\\)/\\4/" & linefeed & "
b start" & linefeed & "
}" & linefeed & "' -e 's/\\([^,]\\{1,\\}\\),[[:space:]]*\\([^,]\\{1,\\}\\),[[:space:]]*\\([^,]\\{1,\\}\\)/\\2,\\1,\\3/p' ")
	
	global frameCount
	local endTimes, tids
	set {endTimes, frameCount} to {{}, 0}
	
	
	repeat with aRec in reclist
		local valueList
		# we split the sublist into its parts 
		set {tids, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ","}
		set valueList to text items of contents of aRec
		set AppleScript's text item delimiters to tids
		set end of endTimes to calcEndtime(item 1 of valueList, item 3 of valueList)
	end repeat
	endTimes
end run

to calcEndtime(startTime, duration)
	global frameCount, startTimeParts
	local tids
	set {tids, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ":"}
	set durationParts to text items of duration
	set startTimeParts to text items of startTime
	set AppleScript's text item delimiters to tids
	set frameCount to frameCount + (item 4 of durationParts as number)
	local tellerSec
	if frameCount > frameRate then
		set tellerSec to 1
		set frameCount to frameCount - 1
	else
		set tellerSec to 0
	end if
	local secs, tellerMin
	# adding secs 
	set secs to (item 3 of startTimeParts as number) + (item 3 of durationParts as number) + tellerSec
	if secs > 59 then
		set tellerMin to 1
		set secs to secs - 60
	else
		set tellerMin to 0
	end if
	local _mins, tellerHour
	# adding mins
	set _mins to (item 2 of startTimeParts as number) + (item 2 of durationParts as number) + tellerMin
	if _mins > 59 then
		set tellerHour to 1
		set _mins to _mins - 60
	else
		set tellerHour to 0
	end if
	local _hours
	# adding hours	
	set _hours to (item 1 of startTimeParts as number) + (item 1 of durationParts as number) + tellerHour
	if _hours > 23 then
		set _hours to _hours - 24
	end if
	# we build back the string
	local retH, retM, retS, Endtime
	set retH to text -2 thru -1 of ("00" & _hours)
	set retM to text -2 thru -1 of ("00" & _mins)
	set retS to text -2 thru -1 of ("00" & secs)
	set Endtime to (retH & ":" & retM & ":" & retS)
	return Endtime
	
end calcEndtime
# 12:32:45, clip1, 00:00:04:21, 12:32:49, clip2, 00:02:23:12

[BenJi]

Actually, the last number in the duration like : 00:12:23:21
12 is minutes, 23 seconds and 21 is the 21st frame of this 23rd seconds.
As i am in europe we use a pal systeme so there is 25 frame per seconds. I think your exemple works the string as if the last digits were seconds… Am i right? I should add a second when these last digits reach 25, or even 24

I dont need the time of the start of the recording to be processed i think. Adding clips duration could do the work, that way we’ll be processind similar datas.
The in point of thhe triming part will start to 0 ( the start of the recorded file) and then increase.

Anyway thanks again for your time. You’ve been very helpfull.

Last question : what doesn tids mean?
And i dont quite understand the part that look like a regex with the shellscript. Could you briefly explain what you did? I’d apriciate it!

[McUsrII]

Hello.

I already had incorporated the frames, and added a second whenever the number of frames reaches the equivalent of a second.

The sed script firstly parses the long string of columns into records, consisting of three, then it rearranges it, so that the start times comes first, then the clipname, then the duration. Then it is returned as paragraphs of text, which effectively leaves us with a list of strings, that are parsed further down the road.

I thought you woul’d like the hours while you are trimming, having the endtimes, but then later on, setting the start time for the collated clip to 0?

Anyways, tomorrow I’ll toss in a total time handler.

[BenJi]

Whenever you trim with quicktime, you have to have some sort of a cmd+z action, to get back to the whole long clip before doing the nex trim.

So basically, the first exported part will be from 0 sec of the whole long clip to “the duration of the first clip”. Export the trimmed video clip then cancel the action to proceed the next clip.
This second clip will be a trim of the whole movie again, but from “duration of the first clip” second to “duration of clip1 + duration of clip2”. The duration of clip1 in this case will also be the end point of the previously trimmed clip. Thats why y created a variable previous end time, to,avoid multiple adition each time.

[McUsrII]

Ok, so you will use this as the end log, and not during clipping, so that the time should start at zero?

[McUsrII]

Hello. Now the endtimes starts at time 00:00:00. The last time in endtime holds the total time.

property frameRate : 60
on run
	set AppleScript's text item delimiters to ""
	set theLog to quoted form of "/Users/you/your/videologfile.log"
	
	set recList to paragraphs of (do shell script "{ cat " & theLog & " ; echo ; } |sed -n -e '
/^[[:alnum:]]\\{1,\\}/ {" & linefeed & "
:start" & linefeed & "
h" & linefeed & "
s/^\\([[:space:]]*\\)\\([^,]\\{1,\\},[^,]\\{1,\\},[^,]\\{1,\\}\\)\\(,\\)\\(.*\\)/\\2/p" & linefeed & "
t prune" & linefeed & "
n" & linefeed & "
b start" & linefeed & "
:prune" & linefeed & "
g" & linefeed & "
s/^\\([[:space:]]*\\)\\([^,]\\{1,\\},[^,]\\{1,\\},[^,]\\{1,\\}\\)\\(,\\)\\(.*\\)/\\4/" & linefeed & "
b start" & linefeed & "
}" & linefeed & "' -e 's/\\([^,]\\{1,\\}\\),[[:space:]]*\\([^,]\\{1,\\}\\),[[:space:]]*\\([^,]\\{1,\\}\\)/\\2,\\1,\\3/p' ")
	
	global frameCount, prevHour, prevMin, prevSec
	local endTimes, tids
	set {endTimes, frameCount, prevHour, prevMin, prevSec} to {{}, 0, 0, 0, 0}
	# initializes the previous times when we start to sum up the end time.
	# The latest time is the total time. that is: >>>item -1 of endTimes<<<
	
	repeat with aRec in recList
		set {tids, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ","}
		set valueList to text items of contents of aRec
		set AppleScript's text item delimiters to tids
		set end of endTimes to calcEndtime(item 3 of valueList)
	end repeat
	endTimes
end run

to calcEndtime(duration)
	global frameCount, prevHour, prevMin, prevSec
	local tids
	set {tids, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ":"}
	set durationParts to text items of duration
	set AppleScript's text item delimiters to tids
	set frameCount to frameCount + (item 4 of durationParts as number)
	local tellerSec
	if frameCount > frameRate then
		set tellerSec to 1
		set frameCount to frameCount - 1
	else
		set tellerSec to 0
	end if
	local secs, tellerMin
	# adding secs 
	
	set secs to prevSec + (item 3 of durationParts as number) + tellerSec
	
	if secs > 59 then
		set tellerMin to 1
		set secs to secs - 60
	else
		set tellerMin to 0
	end if
	set prevSec to secs # adds up for next run 
	local _mins, tellerHour
	# adding mins
	set _mins to prevMin + (item 2 of durationParts as number) + tellerMin
	
	if _mins > 59 then
		set tellerHour to 1
		set _mins to _mins - 60
	else
		set tellerHour to 0
	end if
	set prevMin to _mins # adds up for next run 
	local _hours
	# adding hours	
	set _hours to prevHour + (item 1 of durationParts as number) + tellerHour
	if _hours > 23 then
		set _hours to _hours - 24
	end if
	set prevHour to _hours # adds up for next run 
	
	# we build back the string
	local retH, retM, retS, Endtime
	set retH to text -2 thru -1 of ("00" & _hours)
	set retM to text -2 thru -1 of ("00" & _mins)
	set retS to text -2 thru -1 of ("00" & secs)
	set Endtime to (retH & ":" & retM & ":" & retS)
	return Endtime
	
end calcEndtime
# 12:32:45, clip1, 00:00:04:21, 12:32:49, clip2, 00:02:23:12

[BenJi]

Very nice! Thanks a lot.
It works well within my script

[McUsrII]

Glad to hear it, I really suspected you wanted somehting else. It is a bit too much inside that script, but with the pieces of both posts with scripts, that is left as an exercise for you!

[BenJi]

Exactly i’ll work my way with both scripts.