I've been asked a lot of times to outline how to make a high quality film scanner.
There are multiple approaches to this, and I keep putting it off, so this is a Work In Progress thread, where I will bash out the concepts, and with some feedback from everyone as to which approach they would prefer, we will put together a guide.

Again, this is going to be a bit of a mess in this thread, and at the end of it, I'll put together a proper guide. In the meantime, I will keep editing this first post as I get time.

So, let's get started.

OVERVIEW.
We need to take film, move it through a film gate, and take an image of each frame and store it for editing later.

Basically, there are three main approaches.

1) Frame by Frame Scanning
2) Realtime Continuous Movement Scanning
3) Realtime Intermittent Scanning

We will get back into these later, but let's take a look at the hardware options.
Again, there are a few ways to go here.

FILM TRANSPORT:
We need to be able to move the film through the scanner so we can take an image of each frame. This needs to be fairly high precision and robust, we will be scanning 175000 frames or so for each 2hr movie.
Some of the options are:

1) Build from scratch, using 3D printing and traditional tools.
2) Use an existing projector for the film transport (i.e. to move the film)
3) Use another piece of existing cinema equipment such as a camera or film editor
4) Use a flatbed scanner or similar.

SENSOR:
This is the 'Camera' that takes the image of each frame. It really can be anything that can capture an image, but the main requirements are that it has a really wide dynamic range (to be able to capture detail in the shadows and the highlights), be consistent with exposure (this usually means global shutter) and can be controlled or synchronised in some way with the film movement.
The sensor can be mono or colour.
With mono you will need to do three exposures, one each for Red, Green and Blue. This can be with separate R,G,B light sources, or white light and RGB filters.
With Colour you will need a full spectrum light source, such as Halogen or a very high CRI white LED source, or with an array of Red, Green and Blue LEDs.
You may also want to get a sensor that is sensitive to Infra-red so that you can capture a damage matte. Colour film is invisible as far as Infrared light is concerned, so the film appears perfectly clear in that light. This means the only thing captured in infrared is any dirt or scratches or damage. This makes it easy later to use the infrared image as a mask for digital cleanup later.

1) USB3 Machine vision cameras with external trigger option e.g. Point Grey Grasshopper/Blackfly etc.
2) Realtime Cine cameras e.g. Black Magic Production Camera 4K
3) Stills cameras e.g. Nikon/Canon/Fuji/Sony/Pentax with electronic shutter
4) Video Cameras

LIGHT SOURCE

This is really super important. There are a few ways you can go as mentioned above in the sensor section.
One important consideration is that the spectral response curves of your light source attach the spectral response of the sensor/camera you choose.
Otherwise you will not be capturing the full range of colour that is on the film.
Another consideration is heat, you don't want to damage the film, so either use LED light sources, or halogen with a 'hot mirror' (e.g.) https://www.edmundoptics.com/f/hot-mirrors/11665/
Also, you want the light source as diffuse as possible, this minimises the scratches showing up on the sensor. A truly diffuse light source can be nearly as good as a wet-gate in removing scratches on the base of the film.
The most diffuse light comes from an integrating sphere, you can build your own via 3D printing or using holy styrofoam spheres that you buy from a hobby shop.
You can also use a diffusion filter, holographic ones are the best (e.g. https://www.edmundoptics.com/p/25mm-diam...-kit/8212/) but ground glass or 'opal glass' filters also work well.

Intermittent vs Constant Light

INTERMITTENT LIGHT:
If you are going with LED, you can either keep the LED on all the time, or only turn it on when you want to capture an image. This is known as 'Flash-Scanning'.
The advantage of flashing the light source is that you don't need big heatsinks for the LEDs as they are only on for a fraction of a second at a time. You can also use the duration of the flash to control the exposure, negating the need for a shutter. You can also flash the Red, Green and Blue LEDs separately if using a mono sensor.

CONSTANT LIGHT:
You want to drive the LEDs with a constant current, so that the light does not flicker. The best white LEDs for film work come from Yuji https://store.yujiintl.com/collections/frontpage

I will add more to this section later.

LENS
This is just as important as the sensor. The lens needs to be flat corner to corner and free of chromatic aberration. Working distance (how close the lens needs to be to the film) is also a consideration.
The best lens for this work is a Nikon 105mm printing lens. These are hard to find, and often sell for around $2000.
Next best is the Schneider/Componon-S 50mm enlarger lens, it can be found for under $100 sometimes on ebay, but requires the lens to get very close to the film, which can be a problem if using a projector as your film transport. The 80mm or longer lens lets you get a bigger distance from the film, but they aren't quite as sharp.
I will put some other lens recommendations here.

STORAGE.

This is a real issue. Storing a 2hr feature at uncompressed 4K,  in deep colour, with infrared damage matte take around 21TB of hard disk space.
Storing in something lossy like ProRes 4444 takes up considerably less space, you can fit a feature onto a single 4TB HDD.

If capturing realtime via USB3, you will need very, very fast storage, either a SATA SSD RAID or a fast NVME SSD, and a PC that is fast enough to keep up with processing.

Capturing to a BMD camera or other camera a frame at a time, you can use the internal camera storage, and then transfer to a PC once it is full.

FILM GATE
Dry vs Wetgate.
Keeping the film flat

FRAME SENSING:
Sprocket detection
Hall Effect Sensor on shutter mechanism


TRANSPORT
Film Tension
Rewinding
Sprocket vs Capstan

CLEANING
Physically cleaning the film makes a huge difference. Isopropyl alcohol (98% pure or better) is safe and works reasonably well. Perc works better but is dangerous (can be sourced from dry cleaners) , there are newer cleaners available from 3M that are safe.
Filmguard is not good for cleaning, but can be applied during scanning using a Kelmar style film cleaner. This can work effectively as an alternative to wet-gate as the film-guard fills in the scratches and has a similar refractive index to film.



That is all for this morning, I just wanted to get this started so I don't keep putting it off, consider this post a skeleton that I will continue to fill in when I get time.
Feel free to post questions and suggestions in the thread.

INTERMITTENT SCANNER BUILD

This is in some ways the easiest one to implement, *if* you decide to use a projector head or projector as your film transport, it is very easy to do.

The first step is to find a projector. You only need the film transport or the 'head', you don't need the lamp house or lenses or amplifiers or anything else.
For 35mm a portable projector is a good idea, as they are more compact, or something like a Simplex 35.
For 16mm or Super8 or other gauges, getting one that has an 'open' film transport is a good idea, as you may want to be able to unload the film mid-reel if required.

OVERVIEW.
Using an existing projector, regardless of the type, follows the same basic steps.

1) Remove the shutter
2) Fit a hall effect sensor (so that you know when each frame of film is in the gate)
3) Replace the light source with an LED light source
4) If necessary, replace the motor with a motor with a speed control. (Not needed if your camera is fast enough to do 24fps)
5) Get a camera/sensor and lens to suit.
6) Mount the Camera and focus
7) Capture the footage.

If you are using a machine vision camera, with a trigger input that can capture at 24fps or higher, then you can just run the projector at normal speed and capture directly to your computer.

If you are using a slower camera, then you will want to replace the motor with something that can run much slower, perhaps as low as 1fps. This will require removing and replacing the existing motor, and creating a mount for the new one, rigging up belts etc. I will cover that in a separate post, as it is common to the other builds as well.

1) REMOVING THE SHUTTER
In most projectors the shutter is simply a rotating disc with pieces cut out of it to allow the light through when the film is in the gate, and to block out the light sheen the frame is being 'pulled down' to the next frame. i.e. the shutter blocks off the light when the film is moving, and lets light through when each frame is sitting stationary in the film gate.

[Image: shutters.png]

More modern shutters have 3 or more openings in the disc, to make the film look less flickery, by effectively bumping up the flicker rate.
What you need to do is remove the shutter completely, or cut off the two 'increase flicker' blades, and leave the 'advance film' blade intact, or just cut off all three blades.
Most projectors will have a manual on how to remove the shutter, and if you want to restore the projector for normal usage, you should do this rather than cutting off the blades.
The blade material is very thin, and easy to bend or cut. 
*WARNING* If cutting the blades, it is very easy to accidentally bend the shutter so that the remaining part will hit the other parts of the projector as it spins.

It is not possible to cover all different projector types here, feel free to post me photos of your specific projector for advice if required.

2) FITTING THE HALL EFFECT SENSOR
Basically, we need to know when the film is in the gate, so that we can take a 'photo' of it with our camera.
A hall effect sensor is basically a switch that is triggered by a magnet. Kind of a solid-state version of a reed-switch. Basically if you put a magnet near the sensor, it closes a switch, and when you move the magnet away, it opens the switch (or vice versa).
I recommend the SS443A sensor. It is tiny and very reliable and inexpensive ( a few dollars)
e.g.


You will also need a powerful and small magnet, you can either pilfer one out of an old hard disk, or purchase a 5mm or smaller neodymium magnet.
Something like these. https://www.ebay.com.au/itm/25-50-100pcs...2337546782

A cylindrical magnet is easiest to use, get something long and thin enough to suit your projector.

(Note, other switches could be used, like a reed switch, mechanical or optical switch, but with extensive testing I have found this hall effect sensor to be *by far* the most reliable, especially at realtime or faster usage, 24fps - 40fps)

Before you mount the magnet in the projector, test it with the Hall Effect Sensor to make sure it works. The sensor is unipolar, i.e. it will work better with either the South or the North pole of the magnet, so you will need to try both ends of the magnet and see which end works best for you. Just wave it past the sensor at around 5mm (1/5th of an inch) from the sensor and see which end triggers it best.

Now we have to find a place for the magnet in the projector.

MOUNTING THE MAGNET IN THE PROJECTOR
Again, this will be very projector specific.
Take a look at your projector, hand crank it slowly and look at all of the moving parts. There should be some parts that are in a particular location every time the film is in the gate.
Some projectors have a 'manual frame advance' knob that protrudes out the back of the projector, and one 360 degree rotation of that knob equates to one film frame being advanced. In that case you could drill a hole in the knob and mount the magnet in the hole, and mount the sensor near the knob.
In other projectors a part will move or rotate once per frame, you will need to look at your paritcular mechanism and find a good spot to mount the magnet, and the sensor.
Often the main drive shaft rotates once per frame.

Here is an example of the magnet and sensor mounted (Thanks to Frank V)



This part may seem complex, but it is really very easy to do, again feel free to send me pictures for advice on mount points.

If the part you are mounting the magnet to is made of iron, then it will pretty much stay in place by itself. A drop of cyanoacrylate (Super Glue, Cray Glue, Kragle) will keep it in place and stop it flying off at speed.

Mount your Hall Effect Sensor on a piece of veroboard or similar and attach it so the the sensor moves very close to, but doesn't hit it.

Now you can hand crank the projector and check that each time the film is in the gate, that the sensor is triggered. Check this manually then run the projector at full speed if required, and check that it all still works correctly and nothing flies off!

More to come!

CONTINUOUS SCANNER BUILD

A continuous scanner moves the film at a constant speed, rather than stopping and starting like a film projector does.
The advantage is a much more simple film transport, you don't need an intermittent system with all of the complexity that brings.
Another advantage is your LED array does not need a heatsink, as it is only flashing once every 24th of a second or so, for a few milliseconds, so it doesn't get hot, as they are 'on' for less than 10% of the time. This also allows you to use a lower spec power supply.

The downside is that you need a way to sense when each frame is in front of the camera, and you can only take one exposure per frame.

With modern cameras, this isn't much of an issue, their dynamic range is typically much wider than the range on a print, and $100,000 scanners like the ScanStation, the Spinner S (https://www.youtube.com/watch?v=NugH6zdXVmw) or even the $30,000 scanner like the BMD Cintel use this method, so very high-end results can be achieved. You can see by looking at those scanners that the film path is quite basic.

The basic way it works, is the film is moves past the camera at a constant speed. Laser sensors shine through the sprocket holes (perforations) at the side of the film to detect when the fame is in front of the camera.
An LED array is 'flashed' briefly to 'freeze' the frame and take the exposure.
The amount of exposure is typically set by the duration of the LED flash.

To do this reliably, you need a fast camera, with a global shutter - the USB3 cameras from Point Grey/FLIR are ideal for this application.
You need a global shutter otherwise the 'flash' will give exposure issues on rolling shutter cameras.

You also need a reliable LED control circuit with big capacitors to ensure the flash is consistent at high speed.

It is advisable to use an array of Red, Green and Blue LEDs.
This gives a big advantage of being able to control the exposure of each colour individually, to allow correct exposure of different film stocks, with different colour temperatures. It also allows you to get better captures of faded films.

It also gives you the option to upgrade later to individual exposures for each channel, either by doing multiple passes or slowing the film down.

*more to come*

RESERVED FOR SCANNER BUILD

RESERVED FOR OTHER BUILDS

Thanks poita for making this thread and getting the conversation started. Looking forward to your wisdom and guidance.

My vote would be frame by frame scanning, yes it takes a hell of alot longer but i think there would be alot less problems that would arise with this method. This would then give you the benefit of RGB triple flashing each frame and possibly an IR matte for dirt using an integrating sphere for diffusing.

I kind of like the idea of building from scratch just because you can now fix something easier since your the one who built it from plans. At the same time can be more difficult because it's built from scratch. I suppose if detailed plans were drafted this would make things go smoother, could be fun too.

Could a film gate be 3D printed or would this be something better made with polished metal, or are you just better off using an already manufactured gate?

I didn't know you could use Isopropyl alcohol to clean film, how would you do a full featured film safely without dragging dirt along the film and scratching it? Could this be done on a Kelmar cleaner or is that strictly for FilmGuard? Though i suppose if you put Isopropyl alcohol on the cleaning pads it would evaporate quickly.

Awesome, thanks for this. Personally I'd be most curious about the cheapest methods overall, whatever they are.

Yes, you can use isopropyl with a Kelmar cleaner, you just soak the pads.

Cheapest methods, it completely depends on your situation, or what you might already own.

If you already have a projector, or a video camera or whatever, then it makes a big difference. It also depends where you live. You can pick up a projector head very, very cheaply in the US, in Australia, not so much.

Also, it depends on your skillset, or what you are willing to learn. For example, if you can handle PID control and motor control, then you can make a realtime scanner, controlling the motor with a VFD synchronised to the video pulse of the camera for perfect transfers.

Absolute cheapest, a white LED light, hand crank and a camera with electronic shutter and remote control. Manually advance a frame, and press the button on the camera.

(2018-09-09, 01:48 AM)DoomBot Wrote: My vote would be frame by frame scanning, yes it takes a hell of alot longer but i think there would be alot less problems that would arise with this method. This would then give you the benefit of RGB triple flashing each frame and possibly an IR matte for dirt using an integrating sphere for diffusing.

Could a film gate be 3D printed or would this be something better made with polished metal, or are you just better off using an already manufactured gate?

Sure, frame by frame scanning has those advantages, but it takes a big time and processing hit. You are doing four exposures per frame, and your frame needs to be absolutely rock solid to stop alignment issues, and you will need to write custom software.

So, say you are taking 10 seconds per frame, including advance time, then it is 486 hours to scan a 2 hour movie, which equates to 12 weeks/3 months if scanning 8 hours a day, 5 days a week. If you are taking 30 seconds per frame, then its 1458 hours, or the better part of a year doing it 8hrs a day, 5 days a week.

If you can get it down to 2 seconds per frame, then you are down to 97 hours for the scan, which is more manageable, but still two weeks at 10 hours a day, 5 days a week, and this is all assuming no issues during scanning, which is rarely the case

But anyway, I will be showing solutions for all of the various methods, and at various price points. You could make a really good 2.5K scanner for under $2000 depending on what equipment you can get, that could scan up to 24fps.

So we will be exploring all of the different ways of scanning, and people can run with it from there.

I can only imagine who prompted the idea of posting this work-in-progress guide
I'll follow this thread eagerly