Ghost in the Shell

[der richter]

Hi, and welcome to FanRes! And thanks for Judgment, it's info, and the work you put into it!   

I never could get straight the pedigree of GITS with all the various abbreviations and resolutions and iterations (for restoration projects).

Is this the correct pathology?

1) HDTV MPEG2 HD (1080i?) .TS capture (sometime in 2000-2005)
2) WMV3 HD file (1080p?) was a re-encode of the MPEG2 (when?)
3) "PSNR" (short for ??) WMV3 HD file (720p) was a re-encode of the WMV3 HD (1080p?) (when?)
4) "WOWOW" MPEG2 HD (1080i?) .TS capture is an independent capture (when?)

1) yes, i would just add that it is telecine, though that's probably obvious.
2) yes, the earliest available mention i can still find is 2007. my mind might play tricks on me but i think the first time i was looking for exactly that source the upload date was either 2003 or 2005.
3) yes, beginning of 2007. sry, no idea what the PSNR stands for. i always thought their group name was just a reference to the peak signal to noise ratio.
4) yes, same format as 1. captured on 8th of february 2018.

BTW, do you know anything about a digital-download (?) extra from a DVD release, mentioned earlier in this thread?

sry, i know nothing about the digital download extra.

SO ... Judgment (version 1) was the file/samples used in my earliest comparisons (for visible area, and for color testing from published backgrounds posted as samples at various websites)?
AND ... Judgment-1's source was #2 (above) and Judgment-2's source was also #2?
AND ... your release-name Judgment means your "my own judgment"?  

Yes, yes and yes. on a side note, i only re-encoded from the WMV3 source because i didn't have a better source of that specific transfer at that point of time.

i guess the name Judgement has various reasons. for a start, i use my nick "der richter" since nearly 20 years now and it's just german for "the judge". so Judgment felt natural, i guess. around 10 years now i am rather active in the anime scene i compared thousands of releases of hundreds of anime (see my comparison site i linked before) and was a moderator on a curated anime torrent tracker. one can say i passed judgment on releases.

If  you don't mind, would you mention the software (and add-on filters/plug-ins) you used, and your opinion of each's effectiveness for it's purpose? Were there particular procedures that worked miracles for particular fixes? Aside from the AUDIO, did you use any commercial releases as fill-ins or corrections/enhancements? -- if so, where were they inserted?

I hope you don't mind all the questions -- we Titans have really big noses (i.e. "nosy").  

i don't mind, but writing that might take me a whole day. i wrote scripts to automate quite a lot of the tasks to make my work easier and reusable for other projects. just from the top of my head (not in that order), i might forget something, a short list of the various procedures (via Avisynth).
1. Spot, Dirt removal
- one script to detect spots and to output the temporal and spatial position into a text file (modified version of DeSpot). an ASS file was used here so the detection can be previewed and edited easily via Aegisubs. the output is basically a binary mask for every frame where the spots are marked. it's also easy to add new spots via Aegisubs that weren't detected.
- one script to remove spots via temporal motion compensation (MVTools, RemoveDirt) up to 3 previous or next frames used as reference, mostly none persistent spots
- one script to remove spots via spatial interpolation (ExPaint), mostly for persistent spots
- one script to automatically output every processed frame. one before processing, one after processing, and two where the sports are marked via two different methods. this was used to check the accuracy of the sport removal.
2. Lost cause frames, severely damaged, cleaning up
- a script to insert hand edited frames (via PS) into the clip. this probably answers your question commercial available sources i used. one frame uses the first Japanese BD release to reconstruct it (the headshot/exploding head in the beginning). otherwise around 40 other frames were reconstructed via PS.
- some heavy temporal reconstruction via up to 7 previous and/or next frames.
- masking of low contrast details to prevent blurring and unintentional detail loss (MaskTools)
- some light spatial denoising (NLMeans)
- some medium temporal denoising (fslg-denoise)
- everything was masked to only filter affected parts of frames
3. Frame border fixes (black borders/gradients)
- one script to detect broken/black borders and output frame ranges and the position (left, right, top, bottom) to a text file.
- two scripts to fix the detected borders, one uses BalanceBordersGT, the other one is a bit more complex. depending on the result and scene either one was used to fix the border.
4. Scene detection
- one script to detect scene changes and output frame ranges to a file. this was used to create scene ranges or scene group ranges for the scene based filtering.
- one script to output at least of one frame of every scene (based on scene length) to a file (one screenshot before filtering, one after) to check the final or partial filtering result.
5. Color, Level correction
- via white balance and SmoothAdjust. level corrections mostly only upper and lower ranges to not alter the mid ranges when not necessary.
- one script to detect big level differences between BD and HDTV, frame ranges are output to a file.
6. Risidual combing fix
- via Vinverse and some interpolation.
7. Haloing and ringing fix
- simple Dehaloing for bright and dark halos via fine tuned masking to not destroy the line art. (DeHaloAlpha)
- light deringing to prevent some artificial sharpening (HQDeRing)
8. Credit fixes
- some simple grayscale filtering to get the original sharpness back.
9. Chroma bleeding
- fix of chroma bleeding via colour extreme masking and warping. (aWArpSharp2)
10. Frame Stablisation (DePan)
11. Debanding
- some light debanding per scene to fix some of the heavy filtering. (f3kdb)
12. Risidual logo removal
- selective denosing and interpolation for the affected areas at the top right.

for a more exact descriptions and explanation of every script with examples i would need to take a bit more of time.

BTW, when you begin your remake, would you consider starting a new thread here to document your procedures and progress? That would be cool!

i can try to. documentations aren't really my strong point though. it might take quite a while till i start.

From what I gather it is:
4) "PSNR" (an anime release group: http://anidb.net/perl-bin/animedb.pl?sho...p&gid=2258 ) - 720p re-encoding of #2. (Probably early 2007)
(Although I disagree about the source.  PSNR has different duplicate, missing frames & interlace artifacts than #3, so likely sourced from #1.)

i am pretty sure they were completely frame exact when i checked. but maybe you or me were hit by a bug (or even PSNR) i encountered with ffmpeg/ffms2 where the WMV3 stream was output differently depending on the version. i did some frame checking with one version and later switched to another (newer), resulting into my already filtered frame ranges to be different. i was quite annoyed till i found the source of that problem.
do you have an example at hand, for missing frames and different artifacts? just looking at the logo removal of both, the very same artifacts were introduced. it isn't completely unlikely that they used the exact same setting and delogo file, but it also isn't very likely. especially that one horizontal white line that is left and the artifacts introduced around the removal from encoding. though it's probably not very important since PSNR's release is quite bad anyway and completely negligible.

i meant to ask anyway about the interlacing artifacts that were mentioned before. which ones do you mean exactly? asking in the case i missed something that should have been fixed.

@CSchmidlapp
sure if you want you can send me the file. though no promises when a new release will happen.

[BronzeTitan]

Wow ... thanks for going into the details! Those frame-by-frame fixes must've been tedious, to say the least.

What's your opinion on using the published artwork (see the books I came across, some posts back) as the best (available) guide for color correction?

And there is that eBay auction for an original cel with background -- where we get to see what an absolutely clean source (just before being put onto film). That, too, can be a guide for general fixes -- like processing for line thickness (pardon eBay's .JPG compression - I cleaned it a little in the close-up)!



Or is that too insane?    (I thought that might help for really good looking 1080p HD, or even UHD.)

[der richter]

yeah the frame by frame fixes were very tedious, at least the part that had to be done by hand.

i am not opposed to using printed reference material, since that's probably as close as we can get. the problem is mostly to acquire them. cels are a bit of a mixed bag maybe and expensive. they don't always represent what the director, the colour engineer or whoever was responsible had in mind as an end result. photos of cels or art should probably be taken with caution. we don't know under what conditions they were taken and should only be taken as a lose general indicator.

just in my opinion, i didn't like that teal colour tint(?) on the BD. the UHD BD is a bit better though. i general if i correct colours i try to look for reference points. usually for anime at the white of the eyes as a 'reference' white, or at the bright moon/sun also for whites. this helps at least with the white point. usually the sky for mid-day scenes is blueish to white, depending on the angle. just to name a few examples, but yeah this is quite a liberty i take.

another problem is, when taking reference material into consideration, is that one needs references for various scenes since "one change fixes all" is not possible.

[CSchmidlapp]

Here is a quick comparison between the Judgment Release and the JP UHD.
(Both screenshots we're not created by me.)

Screen shot taken from Judgment release


UHD Converted to HD SDR


Ive since grabbed the V2 Judgment release for a full viewing

[Lio]

A bit astonishing how many fine details seem to be missing on the UHD. The middle alleyway in particular is a blurry mess.

[CSchmidlapp]

A bit astonishing how many fine details seem to be missing on the UHD. The middle alleyway in particular is a blurry mess.

I agree.
It's had a HDR pass and I imagine some of what we're seeing (or not in this matter) is due to the down conversion from the UHD. But not to the degree we're seeing Im sure.
Does anybody know if, or how much a 'HDR pass' on the master would affect detail like this?

[zoidberg]

The HDR pass in and of itself shouldn't cause any loss of detail as it is essentially a colour grading process. Most likely the texture of the grain/noise from the scan was exacerbated by the HDR and the 4K resolution so they filtered it. Or perhaps it was a 'creative decision'.

So is it only the 'original' GITS that has been released on UHD? Perhaps 2.0 has been sent to the trash heap of history.

[Doctor M]

Here is a quick comparison between the Judgment Release and the JP UHD.
(Both screenshots we're not created by me.)

Screen shot taken from Judgment release


UHD Converted to HD SDR


Ive since grabbed the V2 Judgment release for a full viewing

Raw WOWOW screenshot:


My knee jerk response was to suggest that some edge enhancement was used in the original HDTV print... but looking closely, that is absolutely not what I'm seeing.

Having my hot little hands on the raw WOWOW release now, I'm in love with it.  Sure I could do some light clean-up (deblock/dehalo) and IVTC, etc.  But I have half a mind to leave it alone, just sync some tracks, and remux as a BD (since mpeg2 doesn't require recoding for compatibility).

Sure some of the grain is messed up from compression, but some grain is actually still there. Some of the worst macroblocking the HDTV releases are known for is gone (like the exploding head at the beginning).  My TVs media player deinterlaced the file on the fly just fine.  Honestly, it's a tough call whether it is better to be OCD and fix the small issues or to not re-encode it again.

[CSchmidlapp]

The HDR pass in and of itself shouldn't cause any loss of detail as it is essentially a colour grading process. Most likely the texture of the grain/noise from the scan was exacerbated by the HDR and the 4K resolution so they filtered it. Or perhaps it was a 'creative decision'.

So is it only the 'original' GITS that has been released on UHD? Perhaps 2.0 has been sent to the trash heap of history.

I doubt there has been a new scan of the original elements, due to its creation history.
The master used is most probably a 'variation' of the same source as previous releases.
Doing a HDR pass, (and im talking the 'metadata' information that would be read to control the grade on various monitoring technology's) may have 'stretched' or upsampled the image into areas that when down-sampled again, are missing due to the adaption.

It would be fantastic if the 2.0 version was considered a mishap, and maybe kept as an extra feature as part of the history of the movie only.

To be honest England has just won 2.0 in the world cup and Im drunk. Off to celebrate that 'football maybe coming home'

[BronzeTitan]

I wonder if wiping clean the "cel elements" (including line "edge thinning/sharpening" -- I once read of an Avisynth plug-in that was designed to do that) and then regraining them back into the frame, would be useful for creating a fake "true HD/UHD"?

BTW, while looking around, I came across a couple of "archived"  reddit threads about Judgment (1) & Judgment v2. They're mostly "chatty" but still good for information digging.

reddit - [ANIME] Ghost in the Shell (1995)[HDTV][x264][1080p][4.59GB]

reddit - [ANIME] Ghost in the Shell (1995)(Judgment v2)[HDTV][x264][1080p][10.69GB]