[Help Request...]Various frame rates in single project

[jerryshadoe]

Ok, so I'm working on a multi-source project at the moment that contains video streams with three different frame rates: 23.976 - 29.97 - 59.94 - ALL progressive video(!!)

I need footage from all sources (I have 27 videos that are split between these three frame rates) and am wondering how to go about making everything play as smooth as possible.

My biggest problem here is the video that is at 23.976fps, because for the other two; I can either double the 29.97fps video stream or drop/blend (although don't want to blend) half the frames of the 59.94fps video stream BUT how do I "conform" the footage from the 23.976fps video??

Any suggestions are welcome because this is so broad of a topic that google yielded me nothing useful

[dvdboy]

Hi Jerry,

You can 'add' pulldown back to your 23.976fps to create 29.97fps interlaced, and then frame double this to 59.94 progressive.

HTH

[jerryshadoe]

First off, thanx for responding

Ok, so I see no reason to interlace progressive footage, just to make it progressive again by way of losing detail and this also makes the video slightly jerky. Unfortunately, after numerous tests, it looks like the only way for me to make things smooth is to use frame blending. In this case, that's ok as it just creates a slight motion blur which is barely visible due to the nature of the footage I'm working with. To make things "worse"/different, I realized that I actually have five different frame rates; in addition to the ones I mentioned in OP, I also have 24fps and 25fps footage.

I could see some people wondering why I'm not just using one damn frame rate, but the footage for this comes from so many different sources that I don't have a choice but to work with it and the footage is unique so there is no possibility of re-recording it all with one frame rate

[Chewtobacca]

Ok, so I'm working on a multi-source project at the moment that contains video streams with three different frame rates: 23.976 - 29.97 - 59.94 - ALL progressive video(!!)

True 29.97fps progressive material is rare, and true 59.94fps progressive material is even rarer. Are you absolutely sure that is what you have? How are you determining the frame-rates? Samples would be useful.

If I had such material, I would make everything 23.976fps.

[dvdboy]

Like Chewtobacca says, it would be good to know what the project is, as it's very rare to have content which is all 'native' to all those framerates unless it was shot by lots of members of the public!!

Frame 'doubling' or introducing a frame repeat pattern is fairly standard practise - when integrating film (23.976) footage with video (29.97) interviews, editors will often add pulldown to the film to take it back to 29.97 - 4 frames becomes 10 fields if I remember correctly.  When you have 'mixed' footage like this, often one way to de-interlace it, is to take it to 59.94 fps (59.94 fields, separated then doubled in height using something like nnedi or gtqmc).

It sounds like you might have some film, some video which has been de-interlaced one way, and some film being de-interlaced another way.

Repeat frames will introduce a slight 'jerkiness', but it comes down to personal preference in the end - 'good' frames repeated or frame blending to reach the new frame rate.

24fps and 25fps could each be slowed down to 23.976fps so you're back to having only three framerates, but again, it would be good to get samples.

[jerryshadoe]

Ok, so I could see some of the confusion with how vague I was about the source material that I am using. I won't post samples because that would ruin the "surprise" element of this project. Slowing down the 24fps and 25fps video is NOT an option either.

Let me clear this up...

- ALL of the source material is native, progressive at the various frame rates I specified (including true 59.94fps[frames-per-...])
- ALL of the source material is shot with various cameras (also ranging from 1080p to 4k)
- ALL of the source material is of the same event and slowing down the framerate would cause audio sync issues, as well as the fact that the video would actually be playing in "slow-motion," LOL

I am wondering why Chewtobacca would make everything 23.976fps, as I would be loosing a lot of info, especially from the 59.94fps material. But this might just be because I didn't explain much earlier and I'm trying to mention only what is necessary without spoiling the project

[Chewtobacca]

- ALL of the source material is of the same event and slowing down the framerate would cause audio sync issues, as well as the fact that the video would actually be playing in "slow-motion," LOL

Of course, adjusting the audio is implied.  Whether or not the video would play in slow motion depends on the material.  We're shooting in the dark here.  Slowing down the 24fps footage to 23.97fps would not be noticeable in terms of motion or resampling of the audio.  Even if your 25fps material is progressive, it could still be sped-up film-material, so slowing it back down would restore the original run-time.  Again, we don't know.

I am wondering why Chewtobacca would make everything 23.976fps, as I would be loosing a lot of info, especially from the 59.94fps material.

I would make everything 23.976fps because there is no perfect way to do what you're doing.  It is true that you would lose temporal resolution, especially from the 59.94fps material, but the way I see it is that that material is filmed at a far higher frame-rate than film anyway.  What would the same content look like if it were to have been shot on film in the first place?  Well, you'd have far fewer frames.  What would it look like if you threw some of the existing ones away then?  More or less the same, I guess, but it wouldn't look odd (in terms of motion) when combined with the film content.

The alternative is to blend, duplicate or interpolate frames in the other clips until they match.  I can't stand blending, try to avoid duplicating, and am wary of frame interpolation, unless under certain circumstances, none of which apply here.  Consequently, I'd convert the 29.97fps and 59.94fps content to 23.97fps.

Code:

WhateverSource()
ChangeFPS(24000,1001)

I'd slow down the 24fps content and would have to handle the 25fps content based on my analysis of it.  But many knowledgeable people who know what they're doing wouldn't agree with my approach, for the reasons that you have mentioned.  It's a case of picking your poison, and I know that I've used the solution outlined above successfully in the past.

[jerryshadoe]

Picture this:

There is a live, musical show event (with lots of movement and is at night) that is shot from numerous angles and perspectives by many different people, that are all just shooting for their own benefit. Then someone comes along (in this case, me) and takes all this footage to edit it together into one seamless presentation. This is the reason for all the different frame rates, (and why the video/audio can not be altered in speed) because the original intention of the people capturing it was personal usage and they never knew it would need to "conform" to some "standard," because they were unaware that someone would be trying to use it for purpose of combing into one presentation.

Now, as far as my tests are concerned, for the nature of this project, I think that 29.97fps is the best option because the majority of the footage is either 29.97fps or 59.94fps and only some of it will need to be sourced from the other frame rates where frame-blending produces the smoothest result and kinda works well as a "motion-blur" for some of the footage.

Audio is a whole different issue, as I have just received a LINE recording of the footage directly from the mics on stage which is great for the "main" part of audio and I have to figure out how to use the miscellaneous audio for the "live ambiance" and I want to see if there is any way that I can try to configure it into a 5.1 surround sound mix since I have so many different sources.

Sorry that I'm not being specific about the content of the footage, but that would just completely spoil the surprise.

[Chewtobacca]

Meeting in the middle is also feasible, especially given the sort of material you have described.  Let me know if you need scripts to perform specific frame-rate changes in AviSynth.  From your posts, I get the impression that you are comfortable performing the changes in an NLE and have the situation in hand.

Sorry that I'm not being specific about the content of the footage, but that would just completely spoil the surprise.

No apology is needed.  That's understandable; it just makes it harder for those who are trying to help.  And I'm sorry if you felt that people doubted you about the progressive nature and high frame-rates of your footage.  It's just that people often think they have that sort of content but don't, so I was simply trying to rule out the most common explanation.

By the way, I can and would keep your project a secret.  And the chance of my having any interest in, or even recognizing, the sort of thing you have described is remote.  But I get it if you want to keep it under wraps.  Good luck!  

[jerryshadoe]

I appreciate the offer of help and can verify your impression of me being comfortable with an NLE. I have some experience with AviSynth and only use it when absolutely necessary. When using an AviSynth script, are you basically blending frames to achieve the "desired" frame-rate or are you using some interpolation (which can cause jerky motion depending on type of footage) when dealing with progressive footage to be rendered progressive as well? This is assuming, of course (and in this case, this is the assumption) that the duration must remain the same.

I wasn't even sure about posting this thread because I acknowledge the fact that I didn't exactly make the question clear and, therefore, made it harder to answer. I completely understand why the doubt about the progressive nature and high frame rate, as that would have been my first thought if I was reading my OP from someone else. So, no worries

I'm pretty sure there is a few of you here that would keep the project secret (including you, Chew) but I want this one to be a surprise to all. Also, it's kind of a test for me in a different kind of video/audio editing, (versus restoration or film fanediting work) where I don't want my result to be influenced by other opinions as I am curious about my abilities.

Either way, as far as the technical work is concerned... that part I was curious of other people's opinions, but can see how without samples this may be hard. I was just under the impression that a "formula" may exist for dealing with this type of footage (i.e. true, progressive, various frame-rates that need to be all exported into one-single frame rate)

Thanx to all that have tried to contribute to this mess of a thread and thanx for the patience