Even when I learnt about the concept of calibrating the data in it's raw form before processing from some of the more expert blogs, the importance didn't immediately sink in. In theory you'd think that any defect captured in a dark frame for example would exist in the same form in the light frames regardless of whether either were de-bayered or not. This is reasonably correct if you use a 'superpixel' approach to de-bayering where each raw pixel only maps to a single target pixel. However, in the more normal approaches like VNG de-bayering (which preserves the resolution) each raw pixel determines the value of numerous target pixels to varying degrees.
To illustrate this I engineered a raw DSLR dark frame that represents a single hot pixel on an otherwise perfect sensor.
As can be seen this single pixel stands out as an obvious outlier. The normal calibration and cosmetic correction processing can easily remove its effect from the light frames. If the above was subtracted from a light frame it would only be modifying a single pixel - which is ideal as the rest are in this case 100% perfect.
If, however, we first de-bayer the above raw frame using the superpixel approach we get the following.
The hot pixel was obviously a red pixel as the de-bayered pixel has become pink. This is because the process has combined four raw pixels (RGGB) with average green and blue values and 'hot' red value to produce a single pink pixel. The image is obviously four time smaller as well but that is of little consequence. As the hot pixel has only affected a single pixel in the de-bayered image the impact is fairly small. At the end of the day the calibration processing is only going to be messing with a single pixel in the light frames (which also now need de-bayering first). What is probably not so apparent is that our single hot sensor pixel has now in reality affected three other pixels (the two green and one blue pixel that along with the dodgy red pixel form the de-bayered pixel). We have in effect multiplied out single hot pixel into four in terms of the raw sensor data.
If we use the standard VNG de-bayering things get much worse.
The VNG process has caused the effect of the hot pixel to expand into the surrounding pixels, In this case 16 pixels in the de-bayered image which should really be grey have been affected by the colour pixel. In real terms this equates to 67 of the original raw sensor pixels having been affected by the one hot pixel.
It can obviously be said that although the impact of the single hot pixel has expanded out significantly in the dark frames it will have caused the same effect in the light frames (assuming lights and darks are de-bayered the same), In theory this means that the effect cancels out, However, the above is a highly contrived perfect world example with just the one faulty pixel in an otherwise perfect sensor, In reality there will be lots of hot (or at least warm-ish) pixels and potentially cold/cool pixels as well. Put several in proximity and although they may be discrete in the raw data they start to merge in the de-bayered images.
However, the effect should still be similar in the light frames so it should still in theory cancel out. The first problem with that theory is that in real life we will be dealing with more than one dark frame and these will be stacked together and these frames although similar will not be identical. If for example we were stacking 200 of the above dark frames into a master frame and each had one random noise pixel the effects would be far less pronounced stacking raw (left) data than de-bayered (right) data. Also, with the raw data the random noise pixels would tend to stand out far more as outliers allowing them to be excluded from the resulting master frame.
What it boils down to is that in the calibration frames (dark and bias anyway) we are attempting to identify the fixed sensor noise only and do not want any random noise included. This is far simpler with raw sensor pixel data than with the stirred up data in a de-bayered image.
With software such as DeepSkyStacker this is done for you automatically so long as you feed it with raw data in the first place. Clearly if you feed it de-bayered data the calibration process is going to be hindered. And jpeg is obviously a non-starter as it's de-bayered, stretched and compressed so it would be a miracle if a pixel survived the process intact.
With PixInsight it's straight forward to calibrate using the raw data. All you have to do is ensure that the DSLR format settings are set to load the raw data and not de-bayer it or apply white balance etc. It's then just a case of treating the images as grey-scale up until the point where you come to register/align the light frames. You obviously have the de-bayer the lights before aligning as the process only knows the colour of each raw pixel based on its position in the image - once you alter the geometry by aligning you can't de-bayer any more.
One technique that I haven't tried (mainly because my raw frames tend to be more noise than signal) is to separate the raw image data from the DSLR frames into separate RGB channels. This then allows you to process the data in the same way as you would with a monochrome CCD camera and colour wheel, This means that you have to split every single frame (lights, darks, bias, flats) into three separate frames (red, green, blue) before you start. You then calibrate and process the red, green and blue data separately before combining the channels at the end. I've not come across a 'proper' way of splitting out the three channels (four if you count the two greens in DSLR images). The closest I've found is to de-bayer the frames as superpixel and then split the result into three images (one per channel). The only slight flaw in this is that a DSLR image has in reality two green channels which will end up merged (not that there is any advantage in them being separate).
Finally, a highly unscientific comparison. Both images are master darks created using the exact same process in PixInsight. The one on the left is formed from de-bayered data and the one on the right from raw data. (The de-bayered one has been converted to grey-scale to aid comparison). The same part of the image is shown in both windows. The raw data (to my eye anyway) looks far crisper and hot pixels are clearly delineated. The deb-bayered data by comparison looks smoothed and many of the offending pixels appear lost. Obviously, the same smoothing effect will apply to the lights so it should cancel out; the danger is that the smoothed out calibration data also smooths out signal data in the light frames (not that I tend to have too much of that :) ).
Anyway, that's my thoughts on the subject of working with the raw DSLR data rather than de-bayering from the outset...
