Hello guest, if you like this forum, why don't you register? https://fanrestore.com/member.php?action=register (December 14, 2021) x


Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
AviSynth AiUpscale
#1
This could sure come in useful:

https://github.com/Alexkral/AviSynthAiUpscale

Some cuts of films might never make it to (official) UHD, or even HD, so decent upscaling is always useful.

Doom9 thread:

https://forum.doom9.org/showthread.php?t=181665

Just a heads-up!
Reply
Thanks given by:
#2
Thanks for the heads up Chew. Looks like it is semi-based on MysteryX's AVIsynthshader which is what I still use for SR upscaling. Gigapixel is better for fine details (when it works) but seems to be producing odd artifacts for me. Especially in the dark details. So I've been looking for something in-between. Better scaling via AI but with the faster speed of SR. I'm talking 1 or 2fps versus .5 to .8fps.

I'm intrigued and will give it a try.
Reply
Thanks given by:
#3
Great! Let us know your thoughts. You've done way more upscaling than I have, and the results have been impressive.
Reply
Thanks given by:
#4
I have to say I am interested in AI upscaling but literally know nothing about it. The dream for me would be to be able to pair a HD and SD master, using the existing HD footage to 'train' the AI in order to create more accurate upscaling of the SD footage.

I'm mainly thinking of 1.66:1 Robocop (aren't we all) but I think it could also be really helpful when it comes to regrading new masters with older/better colour timed sources
Reply
Thanks given by:
#5
(2020-07-29, 05:40 PM)zoidberg Wrote: I have to say I am interested in AI upscaling but literally know nothing about it. The dream for me would be to be able to pair a HD and SD master, using the existing HD footage to 'train' the AI in order to create more accurate upscaling of the SD footage.

I'm mainly thinking of 1.66:1 Robocop (aren't we all) but I think it could also be really helpful when it comes to regrading new masters with older/better colour timed sources


TomArrow was doing a bit of experimentation with training AI in a way you are talking about. Perhaps he will chime in.
Reply
Thanks given by:
#6
Curious to see some comparison between this and Gigapixel... any volunteer? Wink
Reply
Thanks given by:
#7
(2020-07-29, 05:40 PM)zoidberg Wrote: I have to say I am interested in AI upscaling but literally know nothing about it. The dream for me would be to be able to pair a HD and SD master, using the existing HD footage to 'train' the AI in order to create more accurate upscaling of the SD footage.

I'm mainly thinking of 1.66:1 Robocop (aren't we all) but I think it could also be really helpful when it comes to regrading new masters with older/better colour timed sources

That's possible. There's two codebases I know of that can do this. One is for single images and one is for sequences. I havent tried the sequence one yet. Believe it needs a really beefy GPU setup, so it might not even work for me. But I did the one for the images and got pretty nice results sometimes.

The single image one is/was called ESRGAN (upscaling) and BasicSR (training).

The sequence one is/was called EDVR.

I believe they are now kinda all bundled together in something called "MMSR" or something similar. You can find all that info on Github.

You need a CUDA GPU and Python and a few packages, but the readme tells you how to install it basically.

Edit: For a sample, check out that Dark Knight thread of PDBs somewhere on this forum, I think I posted about it there. I also made a sample in motion, maybe PDB still has the link floating around.
Reply
Thanks given by: zoidberg , spoRv
#8
Here are the experiments Tom was referencing:

https://fanrestore.com/thread-2807-post-...l#pid56276

I haven't tested this upscaling yet, having trouble getting it working but I will eventually. Just need time to work it out. Regardless, I used the THX Halloween DVD as an upscale test since I had it handy. I used this particular pic because fine hair is a great test for upscaling. Please don't take this as truth from on high, its just my little rough test. A lot of experience users here will find these results trivial. So here for baseline reference:

Original DVD (IVTC'd in AVIsynth, Converted to 709)
[Image: FiXyODr.png]

From here out everything is IVTC'd and converted from 601 to 709 in AVIsynth and then upscaled. No additional filters unless noted.

Bicubic/x200
[Image: e8ffhzf.png]

Spline36/x200
[Image: sJNY24M.png]

Nnedi3_rpow2/x200
[Image: y57mhyy.png]

SuperRes x2 with Nnnedi3 upscale (AVIshader Version)/x200
[Image: wXTxGTG.png]

SuperResXBR (AVIshader Version)/x200
[Image: ojQtiU8.png]

SuperRes x2 with Nnnedi3 upscale (AVIshader Version)/x200 (this is with filters for deblocking, noise, halo removal, shaprening, blurring and re-graining)
[Image: ryBOMEO.png]

Gigapixel Photo/x200 (50% sharpen, 0% noise reduction, upscale to 4K and down to 1080p)
[Image: gHikDzy.png]

Gigapixel Photo/x200 (50% sharpen, 0% noise reduction, upscale to 4K and down to 1080p plus a little blur and grain)
[Image: eoIgvlw.png]

So the summary, if it is difficult to see, is that Superres is better than most AVIsynth filters. SRXBR is about the same level. Giga still wins hands down. With Giga you get a lot more detail in the hair.

The MPEG-2 of the DVD naturally has macroblocking. Running a deblock filter in AVI takes care of the problem whereas Giga naturally removes it. Adding grain covers a lot of the problems.

Macroblocking (Superes, SR processed with deblock/grain, Gigapixel, Gigal processed with grain)
[Image: yeHAB1a.png]

So the next steps is that I need to add examples of AIupscale, SR from info and Gigapixel video.
Reply
Thanks given by:
#9
(2020-07-29, 07:48 PM)TomArrow Wrote: You need a CUDA GPU and Python and a few packages, but the readme tells you how to install it basically.

What a python... ehr, pity, I have an AMD GPU! Happy

(2020-07-29, 11:26 PM)PDB Wrote: Here are the experiments Tom was referencing:

https://fanrestore.com/thread-2807-post-...l#pid56276

Thanks to you for the link, and Tom to have produced it; even if with too much noise, I like very much the third image! I wonder how many time it takes to make a single frame, though...

(2020-07-29, 11:26 PM)PDB Wrote: So the next steps is that I need to add examples of AIupscale, SR from info and Gigapixel video.

Thanks for the comparison! Ok
Eagerly waiting for the next steps!
Reply
Thanks given by:
#10
As a gamer I've been watching a lot about DLSS (Deep Learning Super Sampling) recently and how it can upscale low res source. I wonder if this could ever be used on videos instead.



Reply
Thanks given by:


Possibly Related Threads…
Thread Author Replies Views Last Post
  AVISynth AutoOverlay frame delay Mediahead 6 3,077 2020-05-29, 12:45 AM
Last Post: Mediahead
  Guide: Upgrade to AviSynth+ for high bit depth support deleted user 3 4,330 2018-12-20, 07:33 AM
Last Post: deleted user
  Dummies guide to AviSynth? Jetrell Fo 11 12,060 2016-08-01, 09:11 AM
Last Post: jerryshadoe

Forum Jump:


Users browsing this thread: 1 Guest(s)