[Help] Audio Syncing

[deleted user]

Here's an idea, but it would require some coding and tinkering: Record the 2 channels of audio output and simultaneously hook up some kind of electrical trigger to a third audio input and record that with it, so that for each frame there is a spike in the audio curve of the third channel. Then you could write a software that finds all those spikes and normalizes the speed (tho it would require some clever interpolation methods probably), and then it would just be a matter of syncing. Not a simple method, but it might be possible.

After some tinkering with C++, I would probably be able to do something like this now. Turns out there's a library that can do this kind of resampling with variable speed. Just thought I'd mention it in case it ever becomes an issue for anyone ever again.

[zoidberg]

Tom,

I think something similar to your concept is used in the retrosync sound module, except it uses the projected image shone onto a photosensor to create the sync pulses, which can then be used to conform the captured audio to the desired fps.

In reverse there is software called film-o-sync which can lock a digital soundtrack to film playback by using the equivalent of a timecode signal generated by the projector (kind of like how DTS works)

[deleted user]

Tom,

I think something similar to your concept is used in the retrosync sound module, except it uses the projected image shone onto a photosensor to create the sync pulses, which can then be used to conform the captured audio to the desired fps.

In reverse there is software called film-o-sync which can lock a digital soundtrack to film playback by using the equivalent of a timecode signal generated by the projector (kind of like how DTS works)

I could do it a bit cheaper than that retrosync sound module, heh. Wonder what algorithm they use for resampling.

Yeah the film-o-sync does the reverse, not very useful to us here I think.

[zoidberg]

It's probably some kind of time stretch, the distance between 2 pulses should be x amount of ms, the actual amount as captured is y so whatever stretch/compression is needed to get y to match x. The retrosound module refers to eliminating wow so it's obviously capable of adapting the amount of adjustment at a given point, how it does this and to what quality who knows.

Yes the film-o-sync is definitely not of use for film scans but I mentioned it as it's the exact reverse of what we're after (ie taking a audio file of known speed and conforming it to the often variable playback speed of a projector). It could possibly be reverse engineered

[deleted user]

Well yes, time stretch=resampling, heh. Question is what method they use, bc they range from very good to pretty lousy (in terms of artifacts etc). I haven't found a perfect one either but a pretty decent one.

Ah I see what ur saying. Well reverse engineering it would be more work than doing one from scratch, heh.

[Turisu]

So what's generally considered to be the best method of stretching/resampling?

[zoidberg]

Yeah I see your point about time stretching = resampling, unless the software is merely raising/lowering the sample rate then there will be resampling taking place (and even then there is probably some internal resampling taking place within the computer during playback).
I guess the alternative to all this is to know the projector's actual running speed and to adjust the captures based on that known speed. A well maintained projector should hold its fps fairly consistently once it's warmed up, there are inexpensive tachometers which can do the job.

[deleted user]

So what's generally considered to be the best method of stretching/resampling?

I don't know if there is a known "THE" best method, but here you can compare various resamplers: https://src.infinitewave.ca/

Yeah I see your point about time stretching = resampling, unless the software is merely raising/lowering the sample rate then there will be resampling taking place (and even then there is probably some internal resampling taking place within the computer during playback).
I guess the alternative to all this is to know the projector's actual running speed and to adjust the captures based on that known speed. A well maintained projector should hold its fps fairly consistently once it's warmed up, there are inexpensive tachometers which can do the job.

Well it's admittedly a bit pedantic but technically resampling simply means to sample again. The audio was originally sampled at certain points. When you "re-" sample, you sample it again at different points. That requires a good interpolation algorithm to know what values are supposed to be at those new points. So technically it doesn't matter whether the speed changes, you can play back the number of samples you have at any speed you like after all.

About using the projector running speed... well for the projectors that can hold their speed well enough for this to be feasible, of course your proposed solution is the way to go, but at least as far as my own motivation here went, it was explicitly for the cases where that's not the case, where the speed varies from start to end. Requiring a non-constant resampling and thus making the problem that much more difficult.

And yeah as said in an earlier post in this thread, ideally you'd capture an additional second/third channel that picks up a signal representing the framerate. I suppose that's what a tachometer is? So basically there would be a little spike in the signal every time one frame (or a third-frame or whatever the projector uses) is played back. This signal would be trivial to analyze and from that you could then build a map of exactly which point in time should be where, then run the resampler over it, and voila.

[zoidberg]

Or you could use the magnet/hall effect sensor approach, especially if it's already in place as a trigger for image capture, one pulse per frame

[williarob]

My experience with the Retrosync Sound Module has been very hit and miss. When it works it works perfectly and when it doesn't it's often way off the mark. Roger sent me a replacement unit, but both were manufactured mid pandemic so I think he had to source parts from different suppliers, which may not have been so good.

For educational purposes, you can easily take a peek at the source code of the Retrosync software to see how they do the resampling - just download it from the moviestuff site and open it with something like dotnetpeek from Jetbrains. The code has not been obfuscated at all, so it's all right there.