Other [FIJI] Film Restoration macros.

[althor1138]

I've been working a bit on using ImageJ/FIJI as a method for performing various film restoration methods. It is a work in progress and there are quite a few new ideas that I've yet to implement and the code is probably atrocious.

Dual Image Sequence Registration:
https://github.com/althor1138/DISR-FIJI-MACRO

It basically takes two identical film clips from separate sources and registers them spatially.

If you don't want to mess with installing the plugin just download the fiji.app folder here and open up FIJI. The plugin is in the plugins tab under FILM TOOLS.

https://drive.google.com/drive/folders/1...sp=sharing

[deleted user]

I'm probably making some really dumb mistake but here's what I get when I try to run Image Registration on two PNG files that sit in the same folder (they have random names from screenshotting):

ImageJ 1.52a; Java 1.8.0_202 [64-bit]; Windows 7 6.1; 40MB of 6067MB (<1%)

java.lang.IllegalArgumentException: First frame is out of range 0:-1
at ffmpeg_video_import.FFmpeg_FrameReader.makeStack(FFmpeg_FrameReader.java:221)
at ffmpeg_video_import.FFmpeg_FrameReader.run(FFmpeg_FrameReader.java:83)
at ij.IJ.runUserPlugIn(IJ.java:221)
at ij.IJ.runPlugIn(IJ.java:185)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at ij.IJ.run(IJ.java:301)
at ij.macro.Functions.doRun(Functions.java:613)
at ij.macro.Functions.doFunction(Functions.java:96)
at ij.macro.Interpreter.doStatement(Interpreter.java:249)
at ij.macro.Interpreter.doStatements(Interpreter.java:235)
at ij.macro.Interpreter.run(Interpreter.java:118)
at ij.macro.Interpreter.run(Interpreter.java:89)
at ij.macro.Interpreter.run(Interpreter.java:100)
at ij.plugin.Macro_Runner.runMacro(Macro_Runner.java:161)
at ij.plugin.Macro_Runner.runMacroFile(Macro_Runner.java:145)
at ij.plugin.Macro_Runner.run(Macro_Runner.java:60)
at ij.IJ.runPlugIn(IJ.java:191)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at java.lang.Thread.run(Unknown Source)

Edit: I was using the 02 Registration Macro

[althor1138]

The macro requires the images to be in separate folders as it's meant to be importing image sequences. If you are simply trying to register 2 images the macro is not needed at all.

You would simply open the two images in imagej, then go to Plugins->Feature Extraction and run "extract sift correspondences". When the settings window pops up, change the expected transformation to affine and hit ok.

After the correspondences are found don't click on the images or you might lose them. Now, go to Plugins->Transform and click on Landmark correspondences. Make sure you have the source image and template image in the correct place, change the transformation class to affine and hit ok.

That's it. If you need something more robust or non-rigid you can use bunwarpj but this method gets it right most of the time and the bunwarpj method would require a lot more typing.

[deleted user]

Awesome, thank you. That worked.

Now, I noticed that the image I chose to transform was resized down to the resolution of the template. Is there a way to keep the original resolution or will I have to upscale the other (DVD) source? Will your macro accept an .avs input file? And will your macro just apply the same transformation on every single frame or will it adapt frame-by-frame?

I also noticed that the resulting frames weren't perfectly aligned out of the box and I had to run a second-pass alignment in Photoshop to get them to match up perfectly. I don't think it's a big deal, but is there anything I can do to make them match even better out of the box?

[althor1138]

Great!

I will try to answer your questions in order here:

1) Is there a way to keep the original resolution or will I have to upscale the other (DVD) source?

A: It's probably better to upscale the other source or switch which source you are using as a template.

2) Will your macro accept an .avs input file?

A: No. I have not found a way to connect avisynth and imagej this way. Hopefully somebody will.
Currently you have to import with ffmpeg, an image sequence, or a raw avi that works with imagej.

3) Will your macro just apply the same transformation on every single frame or will it adapt frame-by-frame?

A: It will adapt frame-by-frame where it can. Currently, if there are none or not enough correspondences between the 2 images it will call upon the last known good set of correspondences to perform registration.
Williarob has some ideas that I'd like to implement that will account for mismatched frames but I've yet to implement those in a way that would be helpful so I haven't updated the macro with it.

4) Is there anything I can do to make registration match even better out of the box?

A: The macro goes through a more extensive registration process and will likely give better results than just doing an affine transformation on the landmark correspondences.

If I remember right, the macro performs:

1) The steps I told you about in the previous post.
2) Then it goes on to do it again once they are matched in resolution which give many more correspondences usually.
3) It then does a bunwarpj step with the correspondences from step 2 which is the first non-rigid step in the process that allows(hopefully) for it to wriggle everything into it's proper place.

The color matching macro only performs the first 2 steps as this is close enough for color matching and saves a bit of time.

[deleted user]

Thanks for writing all that down. I'll try these tips out when I can.

[Serums]

Has anyone tested FIJI to see whether it can be utilised to use part of an image to fill in another source that has embedded subtitles?