[Proposal] Halloween 2020 pipefan413 preservation: THE EXORCIST (1973)

[wilcof]

Hi. I apologize for asking about something a bit off-topic (I can't seem to send PMs), but I'm curious how that "Exorcist" video was created. I know it was done with Virtualdub. I ask because I'm planning to start making such videos soon-ish, as I've finally procured some video capture equipment. Some pointers would be nice. If it helps, if things go well, I might be able to handle some of the other jobs in the future. (I've got most of the LDs from the poll.)

Thanks.

[pipefan413]

Hi. I apologize for asking about something a bit off-topic (I can't seem to send PMs), but

You can't send PMs because turning up on a forum as a new user and bothering someone in a PM as your very first move is not only incredibly bad form, but literally against the rules. Incidentally, asking them the same question you were going to ask in PM in a thread about something that isn't what you're asking about isn't much better. The fact that you don't know this suggests that you've neglected to read the forum rules and just went straight to trying to PM me before posting here, which does not exactly serve you well as a first impression. There is even a dedicated "presentation" subforum which new users are enocouraged to post in before doing anything else, but no, you jumped straight over that as well. You don't *need* to post in Presentation, but you do need to read the rules, which you evidently didn't, despite apparently being here without posting for almost a year (going by the account creation date).

I'm curious how that "Exorcist" video was created. I know it was done with Virtualdub. I ask because I'm planning to start making such videos soon-ish, as I've finally procured some video capture equipment. Some pointers would be nice.

It wasn't, it was done with AviSynth. There's not really a simple way for me to take you through the entire process from start to finish; you really need to research what AviSynth is, how it relates to VirtualDub, and how to get it to do what you want. The general gist is that you write a script in AviSynth (potentially using an editor with some realtime feedback like AvsPmod, though you don't have to), then potentially feed that script through VirtualDub to preview it with sound, then render the output if you want to make an actual stored video/audio file once done. AviSynth is essentially streaming an "avi" (or if you prefer, serving frames, hence "frameserver") to whatever output you use (e.g. VirtualDub, which is itself a frameserver, or something else like ffmpeg) with whatever processing applied that you've put in your script. So in this case, I was doing things like adding and removing frames of video (and cutting the audio to match), resizing and aligning the videos beside each other, and adding timestamps ("ShowSMPTE") and frame numbers for reference.

The good news is there's a crapload of info online (much of it on this forum, written by people who are far more knowledgeable and experienced) for you to read. I'm far from being an expert myself; if I can do it, so can you. I can give more specific help if you ask more specific questions, but please keep it elsewhere in either a more relevant existing thread or in its own new one.

If it helps, if things go well, I might be able to handle some of the other jobs in the future. (I've got most of the LDs from the poll.)

Thanks.

I'll be doing all of these anyway, just not all before Halloween. THE EXORCIST is kind of a s**tshow because it's missing massive chunks of the audio so it's going to need a lot more attention than some of the other resyncs I've looked at, and I want to do it justice. I may or may not try to do a slightly "easier" one (maybe HALLOWEEN) in the meantime, but we'll see. I have some Actual Life Priorities that also need attention at the moment so I need to make sure I'm not just sitting in a room fiddling with audio all month.

[pipefan413]

THE EXORCIST update: Now capturing 1993 US fullscreen LaserDisc 1007. Apart from possibly using this as a syncing reference, I may as well sync it to the Blu-ray and the now out of print fullscreen DVD, which I previous said I suspected was sourced from the exact same master (and seeing it now in front of me, I'm pretty sure I was right about that). But I'm very curious to see what this 1996 JP one PILF-2196 looks and sounds like, given that it seemingly makes no reference to Dolby Surround at all (just says it's stereo).

For patching purposes I'm most likely going to use the 48 kHz analogue stereo recordings even though - and indeed, specifically because - they will sound worse than the bit-perfect 44.1 kHz digital captures. I'm hoping this'll help a little with blending to the analogue mono recording, though the 1985 mono one will still sound worse because it's from an older and significantly more beaten-up print source than the much later 1993 disc. As expected, the side changes are in different places, which is of course good news because it means audio will not be missing from the same places.

Summary of THE EXORCIST LaserDiscs I have captured or will capture:

1985, JP [10JL-1007]
First fullscreen release with analogue mono that wasn't time-compressed (unlike 1983 US release 1007-LV)

1989, JP [NJEL-01007]
"Ever Green" reissue of 10JL-1007

1993, US [1007]
Digitally remastered fullscreen re-release, this time with digital Dolby Surround instead of analogue mono, I believe most likely sourced from the 1979 theatrical re-release's 35 mm Dolby Stereo track. I believe the video is functionally identical to or at least sourced from the same master as the fullscreen transfer on one side of the now out of print double-sided 1997 US DVD.

1996, JP [PILF-2196]
Digitally remastered widescreen re-release, which curiously only lists the audio as "stereo" and makes no mention whatsoever of Dolby Surround / Pro Logic. I'm wonder if it's a unique mix, strange as that would be, and if it is plain old stereo rather than Dolby matrixed it's most likely a better patching source for the analogue mono than the Dolby Surround track.

EDIT: One more...

1983, US [1007 LV]
Original time-compressed fullscreen transfer with analogue mono. I anticipate this to be useful for side-changes at best, nothing more (since it's almost certainly from the same source as the two JP analogue mono LDs, and will presumably have the same missing frames in addition to the extra pulldown from time compression)

[spaceboy710]

I'm beyond excited for that EXORCIST project. If there is any other formats or media you need captured, let me know.

I had considered seeking out the first VHS release in the green clamshell. It is mono but I can only imagine how degraded the VHS sounds after 40+ years. Still, if it would be helpful, let me know.

[pipefan413]

I'm beyond excited for that EXORCIST project. If there is any other formats or media you need captured, let me know.

I had considered seeking out the first VHS release in the green clamshell. It is mono but I can only imagine how degraded the VHS sounds after 40+ years. Still, if it would be helpful, let me know.

I thought about that but I'd guess probably not: it's most likely from the same print as both the early JP LD transfers I already captured (1985 and 1989) so at best might give like less than 1 second of patching material at side changes, but probably wouldn't help with the far bigger problem of missing frames at reel changes.

I'm keen to hear this weirdo late JP LD to see if it's just basically the same as the Dolby Surround but with the branding weirdly forgotten, or if it's actually a different stereo track. I'm not even sure how I'd really know if it's plain stereo or matrixed, honestly, so if anybody's got a reasonably reliable method for figuring that out, I'm all ears. In the meantime I'm capturing some Genesis stuff and grumbling about missing the end of an auction for a PAL disc I really wanted...

[pipefan413]

Still waiting on that last THE EXORCIST source (PILF-2196) so looking less and less likely that'll be done in time for the end of the month but in the meantime my Criterion ERASERHEAD is here so I now have two HD video options for that and the analogue mono is captured (NDH-101) and I suspect the more commonly held one will be this one from Criterion rather than the previous UK release which has since gone out of print. I might look into that today to see if the analogue mono for this is as bad as the analogue mono for THE EXORCIST in terms of how many big chunks may or may not be missing from it. Could be that it too needs a major patch job, but I hope not!

Tempted to check out HALLOWEEN (AML-0040) as well but seeing as there's already a mono sync done for it, I should probably shunt it a little bit lower in the priority list and maybe focus on something like NEAR DARK, particularly if indeed ERASERHEAD is another tricky one. We'll see.

[pipefan413]

I'm capturing PILF-2196 now. One would assume this is Dolby Stereo/Surround except that there is not a single hint of Dolby branding of any kind anywhere on the packaging. Just says "stereo". Keen to compare to my captures of 1007, which *does* carry Dolby Surround branding. I would have guessed that both were basically the 1979 Dolby Stereo 35 mm mix, but who knows? Why is there a difference on the packaging? I suppose it could be a simple oversight / misprint. EDIT: Well that's interesting... the first side change is in the exact same moment as the side change for the old fullscreen Japanese transfer with the mono soundtrack. Which may mean it's useless for patching, because it'll probably have audio missing in the same place, or if it *doesn't* have audio missing (and indeed it sounds like it has a little less missing than the old transfer does) it may end up being the easiest thing to match to the mono if it's derived from similar elements. Very keen to see what happens here.

Either way, I'll most likely be using one or both as patching material (after creating some sort of dual mono downmix) for the mono track from 10JL-1007 and/or NJEL-01007. I also need to work out whether the older or newer copy of the mono seem to be in better shape, which may not be a quick or straightforward process. I could go through and attach bits of both into one track, depending on which bits of each are in better shape, which is probably what I'll do to some extent but I don't want to get too carried away with that in case there's any difference beyond noise/dirt/rot levels (also it'd take absolutely forever).

[pipefan413]

"What an excellent day for an exorcism..."

Jeeeeeeesus... finished figuring out the structure of THE EXORCIST resync and I kinda want to dig a big hole and jump into it because, to quote my own AviSynth notes...

# 34 + 35 = 69 frames missing at 1st reel change
# 33 + 28 = 61 frames missing at 2nd reel change
# 1 + 8 = 9 frames missing at 1st LD side change
# 12 + 9 = 21 frames missing at 3rd reel change
# 25 + 19 = 44 frames missing at 4th reel change
# 31 + 11 = 42 frames missing at 5th reel change
# 10 frames missing at 2nd (final) LD side change
# 10 + 9 = 19 frames missing at 6th (final) reel change

# Total missing frames:
# 69 + 61 + 9 + 21 + 44 + 42 + 10 + 19 = 275
# 275 / (24000/1001) = 11.470 seconds (!)
# x 48000 = 550,550 samples at 48 kHz

No wonder @The Aluminum Falcon was struggling to make it work well! I dunno how the hell he managed what he did, looking at the sizes of the holes in the LD source. Bloody hell.

We're really talking about gaps that are way too large to bridge with extension crossfades, I think, and there is no alternative source that I know of apart from my other LD copy (which is the first pressing of the same disc, so almost certainly has identical frames). If I were to attempt to patch this, I'd have to create a custom mono downmix from something else, like one of the 5.1 mixes or perhaps preferably a 2.0 track from something else like one of the newer LDs. But even that is kind of a potential s**tshow; it may not be a transparent transition regardless of what I do with it.

The final alternative would of course be to resync in the opposite direction: remove the frames from the Blu-ray that do not appear on the LaserDisc(s) in order to avoid editing the soundtrack. Normally I would be completely against that, but it may make slightly more sense in this case given that my primary motivation for doing this is "theatrical" reconstruction (I'd be using it with @The Aluminum Falcon's custom regrade/edit of the Blu-ray rather than the Blu-ray itself, since the Blu-ray as it is remains a fairly abhorrent thing from my perspective). I'm honestly tempted to use the information I've gleaned from this analysis to just do the video edit instead...

Rather than do this with the Blu-ray, I had what I think is a better, or at least slightly more interesting idea: I used the old 1997 US DVD, the one with the fullscreen transfer on one side of the disc and a widescreen transfer on the other.


VIDEO

The video source was a 480i NTSC DVD encoded in a 3.63 GB MPEG-2 stream at 60/1.001 interlaced fields per second. I up-then-downscaled it to 720p and encoded with x264 at 24/1.001 fps, resulting in a final video stream size of 7.13 GB.

I could have used something a little sharper as the main video source here, definitely. The obvious candidate would be the Blu-ray Disc release. The trouble is that it's admittedly beautiful for the most part but also rather revisionist in a few ways. In terms of the picture (as opposed to the sound, which is obviously not the original mono), it has blanket blue tinting over the climactic bedroom scenes (no, not like that) and a somewhat artificial warm grading on the opening sequence in Iraq. It also (accidentally or otherwise) leaves in one digitally replaced effects shot right at the climax of the entire film, and is missing the old 70s Saul Bass Warner logo at the beginning, which has been replaced by a distractingly modern computer generated version. These are all things that @The Aluminum Falcon made an impressive job of correcting for with the available resources for his theatrical reconstruction but seeing as he's already done that and I didn't want to just lift his work wholesale, I figured I'd use this DVD instead, which has none of those problems to begin with.

Method:

1. Synced DVD to LD without running IVTC on either to start with to check sync looked OK. It did. (I also exported the chopped down DVD audio so I had a waveform target as well as a video one; this was a stereo 16-bit mixdown of the 5.1 off the DVD, with the right offset for correct sync according to PgcDemux. I just did that in eac3to.)

2. Ran IVTC on both then synced DVD to LD again, with the understanding that sync with the audio would now be somewhat less precise. Did a cursory check and found it to still seem OK so decided to render out an initial "test" version of the video, with the following process...

3. Upscaled 4x from 717 x 480 to 2868 x 1920; I did this as an intermediate step because I thought it might look very compression artefacty if I left it SD and re-encoded the already low bitrate video, or did a straight upscale to 480-720 upscale; doing it like this means I can then downsample it and hopefully get a slightly less horrible final picture. If shooting for square PAR, I'd have to rescale it even if I kept it SD or did straight upscale anyway, so I figured this made more sense. I tried many different methods for this and found that although nned3_rpow2 is widely recommended, it didn't give me quite as nice results as a simple BicubicResize (it looked a bit too sharp otherwise, with the video noise from the low-bitrate DVD looking worse). The bicubic kernel made it look a little softer which looks better to my eyes, for this particular source.

4. Cropped off horizontal blanking. In the source this would have been 1 pixel from left and 3 from right, but I couldn't crop to individual pixel accuracy because staying in YUV (in this case YV12) colour space meant that adjustments had to conform to modulus 2. Instead, since I'd upscaled by a factor of 4, I cropped by 1 x 4 (4) and 4 x 4 (12) after the upscaling step. Incidentally, I noticed that I had to adjust this slightly to account for horizontal shift if I used nnedi3_rpow2, instead taking off 2 and 14 respectively (the filter has a correction function for this but it involves yet another resize, which was unnecessary since I was cropping it slightly already anyway).

5. Used ResampleHQ to perform a bilinear downscale to 1280 x 720, fixing aspect ratio for square pixels, anti-aliasing, and Since I decided to do a conservative upscale to 720p, I converting the colour from YV12 with Rec.601 coefficients to YV12 with Rec.709 coefficients purely to improve compatibility, in case players ignore the Rec.601 instruction in the metadata and use Rec.709 instead, which would mess up the colours on playback.

6. Cropped off the top and bottom 14 pixels then added 14 px black borders to matte to the theatrically appropriate 1.85:1 aspect ratio instead of the DVD's 16:9. Not a big deal, but in addition to being more theatrically correct, also helps a teensy bit with cramming as much bitrate into the image as possible without inflating the filesize, since the black bars eat less bitrate than the strips of actual image they replaced. (I ideally wanted this to fit on a DVD9 as AVCHD, which only gives me 8.5 GB to play with.)

AUDIO

The audio is one of my recordings of the 1989 reissue of the original Japanese LaserDisc transfer (NJEL-01007), simply because my copy of that is in slightly better condition that my copy of the much older original 1985 release (10JL-1007). They contain otherwise exactly the same audio and video with the side changes in the same places. It's noisy, it's crackly, but most importantly, it's original.

The audio for this quick project is 1.0 DTS-HD Master Audio with a 318 kbps DTS core because the 2.0 PCM is almost 2 GB, whereas this way it's only 758 MB but no quality is lost. As with the 1.85:1 matte, this helps a bit with keeping the total file size down while allowing me to optimise the bitrate in the image. It was recorded off the LaserDisc at a lower level then boosted and dithered back to 24-bit, to avoid the risk of clipping during recording. Apart from that a tiny fade in and fade out at the start and end (mostly to encourage it to decode bit-perfectly from a DTS-HD container). The audio samples you'll get here are otherwise almost entirely untouched for this initial release; no cleanup, noise reduction, or editing/resynchronisation of any kind has been used, so what you hear is what the LaserDisc sounds like, subject to the recording capabilities of the Blackmagic Design Decklink Studio 2.

Since the audio itself has not been significantly edited, with the video being cut down to match it instead, sync theoretically may not be quite as precise as if I'd done a full resync of the audio (which is not really possible without significant patching, as I mentioned). Feedback welcome, though any adjustments for this specific project will likely only be done in terms of video framed (at 24/1.001 fps) so bear that in mind. In other words, I can knock out an extra frame here and reinsert an extra one there, but I don't think I'll cut the audio at all for now. That's for another day and another project, which I am still planning to do! That all being said though, it should be pretty damn accurate as long as the LaserDisc was in the first place.


If we know each other already to some extent, or if we don't but you're an active member of the forum and have been for a wee while, throw me a PM if you're interested. If we haven't spoken before, you've just signed up and have yet to get involved, please do not PM me or anybody else (or post elsewhere) asking for this project. It's not something to be shared widely, it's just a quick one I wanted to put together mostly for my own sake since I love this film and hate that the official releases have changed so much about it.

Happy Halloween!

[pipefan413]

The main thing I'm realising after watching this on the projector is that I definitely misjudged the gain adjustment, hahah...



It isn't clipping or anything, it's just quite loud compared to other soundtracks according to my usual volume setting on my AVR. I'm fiddling with it now, in any case.

One thing that's bothering me, though, is that the audio appears to only be using 20 of the available 24 bits according to eac3to: if I do a stream copy or something, it reports "the original audio track has a constant bit depth of 20 bits", presumably meaning that there are basically 4 bits worth of zeroes packing it out to 24. EDIT: I think it's actually just because the Decklink Studio 2 is an SDI capture card and supposedly SDI audio is generally 20-bit: http://www.ffmpeg-archive.org/DeckLink-O...77754.html

Original comments on what might be causing the 20-bit-inside-24-bit PCM captures:

I don't know whether that's because...

1. the recording level just didn't go into 24-bit range, so only 20 bits of dynamic range were used (which might make sense, because it did sound quite quiet but that's the maximum level the capture card will record at: maxing out the levels manually or setting it to "use hi-fi audio levels" appear to give the same results)

or

2. the Blackmagic Decklink card records at 20-bit depth, but this is a weird bit depth so the Media Express packs it out with zeroes to 24-bit for storage

or

3. the Blackmagic Media Express software records at 20-bit depth, but this is a weird bit depth so it then packs it out with zeroes to 24-bit for storage

or

4. something about how I extract the audio from the raw .avi capture is iffy

I don't think it's #4 because it'd be kinda strange if a direct stream copy (regardless of if done in ffmpeg or VirtualDub) randomly discarded 4 bits' worth of data and rewrote it with zeroes (the files are not actually 20-bit, they're 24-bit but only using 20 bits of it for meaningful non-zero data according to eac3to). And both VirtualDub and ffmpeg give the exact same results, bit-perfectly, when I've tested them against each other (eac3to just doesn't read the raw video .avi files so can't extract the audio at all, so I have to use something else to pull it out). But none of the first three possibilities would particularly surprise me.

Incidentally, both ERASERHEAD and THE EXORCIST exhibit the same properties: the audio stream is stored as 24-bit, but eac3to reports that it's actually 20 bits of real data with 4 bits worth of zeroes.

Anybody have any ideas? The documentation for the card is pretty crap and info online is sparse.

In the meantime, I've just encoded fresh .dtshd files with the gain boosted much less, dithered back to 24-bit (so that, like the original version I included, they actually do use the full 24 bits of depth). I've done a 2.0 one with "surround" encoding so that the AVR routes it to C unless surround decoding is disabled, and another one as 1.0 like the original version. Will see which one sounds subjectively more appropriate on my system. The only meaningful difference from the included audio is the gain change, really, so you can just turn the volume down a bit during playback and it'll be otherwise much the same as what I'm fiddling with just now.

Current test: boosted by +3 db, +6 dB, and +9 dB, without discarding a channel (so still 2.0). Will try to gauge which sounds most correct on my AVR. Will then either just use as-is or discard R and compare the chosen one with the one 3 dB below to see how my AVR handles it. As in, I don't know if sending 2.0 "Lt+Rt" to C will sound 3 dB louder/quieter than sending only one of the two channels from the same track to C, and if it does, I'll boost the 1.0 version 3 dB less/more to compensate. Curious to see how that goes.

[pipefan413]

Aye OK so I think +9 dB is probably about right to my ears in 2.0 mode but +6 dB isn't terrible. I had previously boosted it +12 dB (!) and had to turn down my AVR from my usual preferred volume so that makes sense to a degree, though the +12 dB was 1.0 going to C, but this is +9 dB going to sides instead, because... well...

I found something interesting, which I was curious about: I wondered if the AVR would actually decode it as dual mono (routing everything to C) given that the 2 channels are non-identical. Turns out it doesn't quite! It did something quite amusing, in fact: it basically put all the analogue noise in L+R and the majority of the actual audio in C! So perhaps the "dual mono" approach is not the best way to go after all, and I should stick to my original plan: 1.0 all to C by default unless intentionally putting AVR in "stereo" (which appears to apply a -3 dB cut and duplicate, which I think would be correct to avoid it seeming too loud due to the channel coming out if 2 speakers instead of 1). Alternatively, I imagine I could discard 1 channel, duplicate it, and encode as 2.0 as I just did and it theoretically should be more successful in routing the whole thing to C in surround mode or to the sides in stereo mode, but that seems inefficient storage-wise (I'd reckon that's only worthwhile at all if doing it to preserve both channels, but I'll be doing that regardless for archival purposes). Since the video source is a DVD and I'm trying to be efficient with file size for this particular project, I think I'll stick to 1.0 DTS-HD MA but I'm not sure if it'll sound most appropriate at +9 dB or +6 dB (+12 dB was definitely too loud but maybe not by so much that 6 dB less would be correct). Will need to test that next and report back. Of course, part of this is reliant on how my particular AVR behaves, so YMMV!