2024-01-30, 06:02 PM
this is an idea ive had for a good while, but haven't really gotten that far: Blu-rau PiP (Picture-in-Picture) ripping/archival.
PiP is a kind of bonus feature where a secondary video stream is presented simultaniously alongside the main feature presentation. it allows interviews/commentary, BTS footage, etc. to be shown at relevant parts of a film, and imo its pretty neat.
![[Image: 371cgD1.png]](https://i.imgur.com/371cgD1.png)
while the vast majoirty of Blu-ray extras are just in the form of a video file, which is pretty easy to rip (and therefore archive outside of the Blu-ray format). however, PiP tracks are not. PiP video is a seperate video track which is composited onto the main video stream at playback.
it is possible to rip the PiP video and audio themselves; while MakeMKV seems to ignore the video, MKVToolnix will see and mux it just fine from a decrypted backup. the video seems to all be 720x480 and audio is 2.0 E-AC-3 or DTS-Express. the streams themselves offer little issue.
however, its not quite that simple. for starters, i have not encountered any player that will playback some kind of PiP image outside of Blu-ray player software (which only do it when playing back the Blu-ray itself). it doesnt seem too hard to overlay the video/audio in post, but there is some missing information unfortunately.
the PiP video/audio arent a constant. the video turns on and off, and the audio fades in and out (shifting the volume of the main audio alongside it). the video and audio come in and out at different points. the PiP video doesn't have a constant position, it can be shown in different corners. and finally, ive seen some titles show additional text onscreen.
and so, i'd like to look into the archival of these titles, so they can be as true to the disc as possible, while not being bound to the Blu-ray format. which basically means burning everything into a single video file. imo there are genuine reasons to do this. personally, i like to include extras on my media server and these are extras which i have and enjoy, but cant have on my media server, because they are more than just a video stream. but also just general archival stuff as a video file is so much more versatile than a Blu-ray backup.
ideally, it may be possible to generate a file from the PiP data on the Blu-ray itself; it could theoretically be done automatically for an arbitrary number of titles with no intermediate and custom quality settings. that being said, haven't made much progress in that direction. diving into the files of decrypted backups, ive found nothing. using BDedit, ive managed to extract some "PiP metadata" supposedly, but of the ones ive tried, some look to be tiny (<100 bytes) so not sure wut kind of data is here or how useful it may be, not to mention its in a binary format i have not been able to decypher.
alternate routes:
playback capture. ive managed to capture The Matrix's PiP presentation, its a bit shoddy if we're going for perfection but not bad in terms of viewing quality. capture of course requires full length playback and large intermediate files to try and maintain quality. could potentially get better if i can figure out how to capture a 24Hz image at 24fps rather than the 60fps capture i have. however if that doesnt shake out theres always:
manual reconstruction. capture can be used to simply obtain reference to be used to manually reconstruct the presentation in an editor. main downside is it takes a lot of time and may not be perfect. could be augmented with some automation like perhaps analysing the video/audio for black frames/silence to determine rough timing.
regardless, i.. dont really know how to proceed with this idea, research has dug up absolutely nothing helpful beyond some top level technical details, but maybe im not looking in the right places, and i know just enough about the tech stuff to know roughly how it works, but not enough to actually reverse engineer anything useful. any ideas/input/help would be appreciated
PiP is a kind of bonus feature where a secondary video stream is presented simultaniously alongside the main feature presentation. it allows interviews/commentary, BTS footage, etc. to be shown at relevant parts of a film, and imo its pretty neat.
while the vast majoirty of Blu-ray extras are just in the form of a video file, which is pretty easy to rip (and therefore archive outside of the Blu-ray format). however, PiP tracks are not. PiP video is a seperate video track which is composited onto the main video stream at playback.
it is possible to rip the PiP video and audio themselves; while MakeMKV seems to ignore the video, MKVToolnix will see and mux it just fine from a decrypted backup. the video seems to all be 720x480 and audio is 2.0 E-AC-3 or DTS-Express. the streams themselves offer little issue.
however, its not quite that simple. for starters, i have not encountered any player that will playback some kind of PiP image outside of Blu-ray player software (which only do it when playing back the Blu-ray itself). it doesnt seem too hard to overlay the video/audio in post, but there is some missing information unfortunately.
the PiP video/audio arent a constant. the video turns on and off, and the audio fades in and out (shifting the volume of the main audio alongside it). the video and audio come in and out at different points. the PiP video doesn't have a constant position, it can be shown in different corners. and finally, ive seen some titles show additional text onscreen.
and so, i'd like to look into the archival of these titles, so they can be as true to the disc as possible, while not being bound to the Blu-ray format. which basically means burning everything into a single video file. imo there are genuine reasons to do this. personally, i like to include extras on my media server and these are extras which i have and enjoy, but cant have on my media server, because they are more than just a video stream. but also just general archival stuff as a video file is so much more versatile than a Blu-ray backup.
ideally, it may be possible to generate a file from the PiP data on the Blu-ray itself; it could theoretically be done automatically for an arbitrary number of titles with no intermediate and custom quality settings. that being said, haven't made much progress in that direction. diving into the files of decrypted backups, ive found nothing. using BDedit, ive managed to extract some "PiP metadata" supposedly, but of the ones ive tried, some look to be tiny (<100 bytes) so not sure wut kind of data is here or how useful it may be, not to mention its in a binary format i have not been able to decypher.
alternate routes:
playback capture. ive managed to capture The Matrix's PiP presentation, its a bit shoddy if we're going for perfection but not bad in terms of viewing quality. capture of course requires full length playback and large intermediate files to try and maintain quality. could potentially get better if i can figure out how to capture a 24Hz image at 24fps rather than the 60fps capture i have. however if that doesnt shake out theres always:
manual reconstruction. capture can be used to simply obtain reference to be used to manually reconstruct the presentation in an editor. main downside is it takes a lot of time and may not be perfect. could be augmented with some automation like perhaps analysing the video/audio for black frames/silence to determine rough timing.
regardless, i.. dont really know how to proceed with this idea, research has dug up absolutely nothing helpful beyond some top level technical details, but maybe im not looking in the right places, and i know just enough about the tech stuff to know roughly how it works, but not enough to actually reverse engineer anything useful. any ideas/input/help would be appreciated
