AviSynth TSMC (Temporal Soften Motion Compensated)(Denoiser)

[Chewtobacca]

^ I'm not keen on interpolating frames in this situation.

[althor1138]

I think that would help. If each frame has 1 duplicate, tsmc could be tweaked to only use the duplicates for denoising. Then you could crank up the mthresh as high as you want and simply blend the frames to remove noise. I will do a tsmc version aimed at anime denoising and see if it is viable or not.

Such a version would be a great asset. What might complicate matters is that occasionally animation has sections (usually those involving a high degree of motion) that are animated at 24fps, as opposed to the usual 12fps with every frame duplicated.

Not knowing much about animation I was unaware of this complication but it makes sense. If there is a filter that can detect duplicates and give feedback, it might be possible to automatically apply the right type of denoising for different scenarios at the runtime level using scriptclip or conditionalfilter. I will have to dig into the list of filters and see if this can be approached.

[Chewtobacca]

Scrubbing through one of the aforementioned 24fps sections, I discovered that many of the unique frames are in fact extractions of slightly different portions of the same image. It's almost as if someone applied the Ken Burns effect to a cell but limited it to two frames to create the appearance of motion. It's a cheap of way of handling the high-motion sections without animating at more than 12fps, I suppose. It's quite clever, but I don't think it looks as smooth as true 24fps.

[Chewtobacca]

I have developed another version that allows you to manually control the blur on the prefiltered clip which helps prevent MVTOOLS from locking onto noise as "motion" which might help things further. I will update the first post with that when I get near my pc again this weekend.

Did you ever post an updated version, althor?  No worries if you didn't!  I've only just gotten round to using this filter again today...

[althor1138]

Here's what I've got so far! I've kind of decided to give the user enough rope to hang themself with this version. I haven't yet come up with a solution to the animation "problem" but I think this version should handle it pretty well regardless.  I should say that everything I'm doing now is geared towards avs+ x64 because that's what I'm using now.  This should work with 2.6 x32 however I think. I've included a lot of new parameters from mvtools for more tweakability. This new version will probably be a bit slower. I've added notations to help with the parameters. They all default to pretty safe settings.

Code:

function TSMC(clip input, int "Bradius", int "BLthresh", int "BCthresh", int "tradius", int "mthresh", int "mthresh2", int "lumathresh", int "chromathresh", int "search", int "searchparam", int "pelsearch", int "blocksize",int "overlap", int "dct", bool "MT")
{
    #TSMC Version 0.2
    #Supports Yuy2,Yv12,Yv16,Yv24 colorspaces
    #Requirements:
    #AVS 2.6 or latest AVS+
    #MVTOOLS2 2.6.0.5 or higher
    
    #Blurring the prefiltered clip helps mvtools to "see through" grain, etc. without locking onto it as motion.
    #Too much blur can prevent it from locking onto good data though so you'll have to tinker with it.
    #The defaults should be pretty safe.
    
    #The motion vectors from the prefiltered clip are passed on to the motion compensation stage.
    #The prefiltered clip is discarded after generating vectors and the original clip is used during motion compensation.
    
    Bradius=default(Bradius,3)
    #Spatial Blur Radius. Used only for prefiltered clip.
    
    BLthresh=default(BLthresh,16)
    #Spatial blur luma threshold. Used only for prefiltered clip.
    
    BCthresh=default(BCthresh,16)
    #Spatial blur chroma threshold. Used only for prefiltered clip.

    tradius=default(tradius,3)
    # temporal radius-number of frames analyzed before/after current frame.
    #1-10 are valid numbers.
    
    mthresh=default(mthresh,70)
    # motion threshold-higher numbers denoise areas with higher motion.
    #Anything above this number does not get denoised.
    
    mthresh2=default(mthresh2,mthresh)
    #Define the SAD soft threshold for the furthest frames.
    #The actual SAD threshold for each reference frame is a smooth interpolation between the original Mthresh (close to the current frame) and Mthresh2 (farthest from current frame).
    #Setting Mthresh2 lower than Mthresh allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the risk of
    #blurring when the result of MCompensate is passed to a temporal denoising filter.
    #Defaults to Mthresh value unless you change it.
    
    lumathresh=default(lumathresh,255)
    # luma threshold- Denoise pixels that match in surrounding frames.
    #255 is the maximum and default. 0-255 are valid numbers.
    
    chromathresh=default(chromathresh,255)
    # chroma threshold- Denoise pixels that match in surrounding frames.
    #255 is the maximum and default. 0-255 are valid numbers.
    
    search=default(search,4)
    searchparam=default(searchparam,2)
    pelsearch=default(pelsearch,2)
    
    #search decides the type of search at every level, searchparam is an additional parameter (step, radius) for this search, and pelsearch is the radius parameter at finest (pel) level. Below are the possible values for the search type:

    #0    'OneTimeSearch'. searchparam is the step between each vectors tried (if searchparam is superior to 1, step will be progressively refined).
    #1    'NStepSearch'. N is set by searchparam. It's the most well known of the MV search algorithm.
    #2    Logarithmic search, also named Diamond Search. searchparam is the initial step search, there again, it is refined progressively.
    #3    Exhaustive search, searchparam is the radius (square side is 2*radius+1). It is slow, but it gives the best results, SAD-wise.
    #4    Hexagon search, searchparam is the range. (similar to x264).
    #5    Uneven Multi Hexagon (UMH) search, searchparam is the range. (similar to x264).
    #6    pure Horizontal exhaustive search, searchparam is the radius (width is 2*radius+1).
    #7    pure Vertical exhaustive search, searchparam is the radius (height is 2*radius+1).
    
    blocksize=default(blocksize,4)
    #larger numbers = faster processing times. Must be 4,8,16,32 or 64.
    
    overlap=default(overlap,2)
    
    dct=default(dct,0)
    #Using of block DCT (frequency spectrum) for blocks difference (SAD) calculation. In particular it can improve motion vector estimation at luma flicker and fades.
    
    #0    Usual spatial blocks, do not use DCT.
    #1    Use block DCT instead of spatial data (slow for block size 8x8 and very slow for other sizes).
    #2    Mixed spatial and DCT data; weight is dependent on mean frame luma difference.
    #3    Adaptive per-block switching from spatial to equal-weighted mixed mode (experimental, a little faster).
    #4    Adaptive per-block switching from spatial to mixed mode with more weight of DCT (experimental, a little faster).
    #5    SATD instead of SAD for luma.
    #6    Same as 2 only use SATD.
    #7    Same as 3 only use SATD.
    #8    Same as 4 only use SATD.
    #9    Similar to 2, use SATD and weight ranges from SAD only to equal SAD & SATD.
    #10    Similar to 3/4,use SATD weight is on SAD, only on strong luma changes.
    
    #I don't recommend using anything greater than 0 unless you see a need to.
    
    MT=default(MT,true)
    #True enables multithreading. This is the default.
    #If you are using in multithreaded script with setmtmode or setfiltermtmode this should be false.
        
     prefilt2=input.isyuy2() ? input.spatialsoften(Bradius,BLthresh,BCthresh) : input.converttoyuy2().spatialsoften(Bradius,BLthresh,BCthresh)
    
     prefilt=input.isyv12() ? prefilt2.converttoyv12()
     \ : input.isyv16() ? prefilt2.converttoyv16()
     \ : input.isyv24() ? prefilt2.converttoyv24()
     \ : prefilt2
    
     super=MSuper(input, pel=2,mt=MT,hpad=blocksize,vpad=blocksize)
     superfilt=MSuper(prefilt,pel=2,mt=MT,hpad=blocksize,vpad=blocksize)
    
     multivectors=Manalyse(superfilt,multi=true,delta=tradius,mt=MT,blksize=blocksize,search=search,searchparam=searchparam,pelsearch=pelsearch,badsad=1000,badrange=32,dct=dct)
     multivectors2=Mrecalculate(superfilt,multivectors,thsad=mthresh,tr=tradius,mt=MT,blksize=blocksize,overlap=overlap,search=search,searchparam=searchparam,dct=dct)
     mc=Mcompensate(input,super,multivectors2,thsad=mthresh,mt=MT,tr=tradius,center=true,thsad2=mthresh2)
     mc.temporalsoften(tradius,lumathresh,chromathresh,15,2)
     selectevery(tradius * 2 + 1,tradius)
}

An example of using this in a multithreaded environment (avs+):

Code:

setmemorymax(8192)
setfiltermtmode("default_mt_mode",2)
setfiltermtmode("ffvideosource",3)
ffvideoSource("E:\STARWARS\Return.of.the.Jedi.Grindhouse.35mm.LPP.mkv")
tsmc(5,16,16,5,200,120,255,255,4,2,2,4,2,0,false)
prefetch(8)

EDIT: Turn the blur radius down to 2 or 3 for SD content.

[Chewtobacca]

Thanks for the update!

Wow! That's a hefty set of parameters. I tried the settings that you posted (changing the blur-radius to 2), and they seem to work fine in non-MT AviSynth. Presumably, the last setting should set to "true" for MT.

[althor1138]

Thanks for the update!

Wow! That's a hefty set of parameters. I tried the settings that you posted (changing the blur-radius to 2), and they seem to work fine in non-MT AviSynth. Presumably, the last setting should set to "true" for MT.

Great to hear it works. Please let me know if you find a bug or have any suggestions. Setting MT to true in your case is the correct thing to do. If I understand right, it should be false when using setmtmode or similar because it's like multithreading it twice which wastes ram and CPU cycles.

[Chewtobacca]

Setting MT to true in your case is the correct thing to do. If I understand right, it should be false when using setmtmode or similar because it's like multithreading it twice which wastes ram and CPU cycles.

Do I really it to "true" when using non-MT? That's certainly counter-intuitive. To be clear, the only line that I used from your settings is:

Code:

tsmc(5,16,16,5,200,120,255,255,4,2,2,4,2,0,false)

I didn't call setfiltermtmode() or prefetch() because I assumed that they related only to MT.

[althor1138]

Mt=true makes it run multithreaded internally without the need for setmtmode or setfiltermtmode.

If you aren't using setmtmode or setfiltermtmode in your script you can have it set to true for mvtools to run multithreaded internally. False is fine too but it will be slower.

If you are using setmtmode or setfiltermtmode in your script set Mt=false to disable mvtools internal multithreading and rely on setmtmode or setfiltermtmode instead to handle multithreading. It will work if you set Mt= true but it will be running double multi-threaded, so to speak, and waste ram and CPU.

It is counterintuitive for sure lol.