Randy
Jul 11th, 2007, 1:28 pm
In an earlier post I suggested that the best way to store digital image captures was in the camera’s native raw format. Here is a brief explanation of what a raw file is and why it is a good idea.
Raw file capture is available on almost every digital SLR and on many high end point and shoot digital cameras. You would need to consult your owner’s manual to see if your camera supports the raw format.
What is the raw format?
All digital cameras use an image sensor which creates a digital conversion of the image you capture. This digital conversion contains 100% of the data your image sensor records. In a camera that supports the raw format this digital data is stored, without alteration or compression, as a raw image file. This data is not processed for color balance, luminance or color saturation – it is exactly representative of the light that the sensor captures. Most of the manufactures employ a lossless compression scheme on the raw files, but these files are still close to one megabyte per megapixel.
Why JPEG?
JPEG (http://en.wikipedia.org/wiki/JPEG) is an acronym for Joint Photographic Experts Group. This group came up with a file format as well as the codec to compress and decompress the digital image. JPEG employs ‘lossy’ compression which means that some of the picture data is lost in the conversion. JPEG can be adjusted to minimize the data loss or to minimize the stored file size. JPEG only applies to still images. In my large 16 megapixel digital SLR a full sized image with only minimal compression will be stored in a file of approximately 1.5 megabytes. The raw file for that same image will be over ten times that size. Since JPEG uses compression, the final size of the stored file will be affected by the amount of noise in the image, the actual image and the amount of compression used in creating the JPEG. A picture of just the bright blue sky will compress more than an image of a forest, because there is little detail in the sky.
Because JPEG is lossy by nature, there will be some data that is lost forever in the compression and encoding of the image. When you take a picture in your digital camera and save it as a JPEG a number of things take place: The image sensor captures the light (image) based upon the mechanical exposure controls of aperture and exposure time. The ISO setting of the camera will affect this capture as well, as ISO settings usually control the actual gain of the sensor, affecting the digital values for a given amount of light captured. The output of the sensor is raw data and a raw file would store this data.
This captured data is output from the sensor and then processed by the camera for white balance, luminance and color levels. These adjustments are imposed by the cameras white balance settings and the auto leveling parameters built into the camera. The auto leveling sets arbitrary values for black and white, then discards all data above or below those arbitrary values. Auto leveling also sets mid tone levels, which will affect the contrast, shadow detail and other elements of the image which fall between black and white. Most cameras add a little color saturation to give the image a little more dynamic impact.
The image is sharpened. All digital images need to be artificially sharpened or they will look a little soft. This sharpening is usually done at an arbitrary level and cannot be changed in the cameras JPEG conversion. Sharpening is a process where light level changes from one pixel to the next are altered to give the impression of sharper edges on objects within the image.
After all of the processing the image is compressed and stored on the cameras memory media as a JPEG imageAs you can tell from the description above, a lot of what your camera ‘sees’ is simply discarded in the JPEG conversion process. This is usually not a problem on a properly exposed image without any technical difficulties involved.
I will attempt to illustrate, by example, why raw image capture can be really beneficial. I am going to use an image from a recent trip of mine that illustrates what can happen when you do not get a proper exposure of a technically difficult capture. This image was strongly back lit on a sunny day and I was careless and neglected to let my camera meter on the proper item in the image. Focus was good, but the exposure was awful. This is the opening gate to Yellowstone at the north entrance. I will first show the JPEG that came out of the camera with normal processing parameters.
You can click on any picture in this post, except for the graphs, for a larger view.
http://www.prades.net/images/jpeg1sm.JPG (http://www.prades.net/images/jpeg1.JPG)
As you can see, the sun and a bright blue sky were behind the gate and my camera’s metering used the very dark stone of the gate averaged against the little bit of blue sky in the center of the archway to determine the metering value. This made the whole exposure dark, as you can see in this graph of the raw image. The left of the graph represents the low levels or black in the image and the right side represents highlights or whites. It is underexposed by more than one full stop.
http://www.prades.net/images/rawbefore.JPG
There is the possibility to alter the levels in a JPEG image, but because of the lost data there are severe limits as to what you can recover. The following image will show a quick and dirty attempt at regaining the shadow detail in the gate. More could be done with some time in PhotoShop, but this still gives a good representation of the challenge.
http://www.prades.net/images/jpegprocsm.JPG (http://www.prades.net/images/jpegproc.JPG)
As you can see there is some shadow detail to be recovered, but there is also a lot of noise. The highlights also lose all of their detail. Again, a better job could be done with the recovery, but the amount of shadow detail that is lost in the JPEG conversion is considerable.
With raw conversion the first thing I can do is to impose some exposure compensation. This will recover a tremendous amount of shadow detail that is buried in the raw file, which was discarded in the JPEG conversion. Here is a graphic of the image luminance data after exposure compensation.
http://www.prades.net/images/rawafter.JPG
This is a screen capture from BreezeBrowser. As you can see there are many other items I can adjust in this raw conversion. I can set the final image size, adjust color saturation, set black point, white point and even mid tones. I can choose sharpening methods and amounts, image gamma and even correct for certain types of lens distortion. Below is the image, processed from raw in PhotoShop CS3. Most of the image adjustments were done in the raw conversion adjustments. There was little to do other than sharpening and resizing before saving it as a JPEG.
http://www.prades.net/images/jpegfromrawsm.jpg (http://www.prades.net/images/jpegfromraw.jpg)
As you can see, the raw file had enough data to help me recover from this disaster of an image. In most cases I would have been more careful in the first place, reviewed the histogram of the image on the spot and adjusted the exposure properly when I captured the image. I was driving through and only captured it once. Without the raw file, there would have been little to do. I realize this is not a great image, I only chose it to illustrate the challenges that a raw file can help you overcome.
A raw file will not help you if there is motion blur in the image, either by motion in the camera or the subject. It cannot help you if the image is not focused or composed properly. A raw file can help you recover from exposure problems or white balance problems.
Almost every camera that is capable of raw image storage comes with software to process that raw file. I use BreezeBrowser for the vast majority of my images, because I like and am accustomed to the workflow. Adobe PhotoShop also includes ACR for raw conversion. Photoshop is robust and complicated, but the learning curve to get started is not too steep. There are also programs that come with the camera, Canon provides DPP with theirs. There are also dozens of other raw conversion applications from the professional Capture One to the popular Bibble. I highly recommend that you look into raw image storage and processing as it will appreciably improve your results. It will also make a better photographer out of you as you learn the pitfalls of complicated exposure situations. You will begin to pay much more attention to light, which is the absolute key element in any quality photograph.
BreezeBrowser (http://www.breezesys.com/BreezeBrowser/index.htm)
Capture One (http://www.phaseone.com/Content/Software/ProSoftware/ProductOverview.aspx)
Bibble (http://www.bibblelabs.com/)
PhotoShop (http://www.adobe.com/products/photoshop/photoshop/)
Hope this helps!
If you want an even more technical treatise of the raw conversion, Northlight Images has a good one here (http://www.northlight-images.co.uk/article_pages/why_use_raw.html).
Raw file capture is available on almost every digital SLR and on many high end point and shoot digital cameras. You would need to consult your owner’s manual to see if your camera supports the raw format.
What is the raw format?
All digital cameras use an image sensor which creates a digital conversion of the image you capture. This digital conversion contains 100% of the data your image sensor records. In a camera that supports the raw format this digital data is stored, without alteration or compression, as a raw image file. This data is not processed for color balance, luminance or color saturation – it is exactly representative of the light that the sensor captures. Most of the manufactures employ a lossless compression scheme on the raw files, but these files are still close to one megabyte per megapixel.
Why JPEG?
JPEG (http://en.wikipedia.org/wiki/JPEG) is an acronym for Joint Photographic Experts Group. This group came up with a file format as well as the codec to compress and decompress the digital image. JPEG employs ‘lossy’ compression which means that some of the picture data is lost in the conversion. JPEG can be adjusted to minimize the data loss or to minimize the stored file size. JPEG only applies to still images. In my large 16 megapixel digital SLR a full sized image with only minimal compression will be stored in a file of approximately 1.5 megabytes. The raw file for that same image will be over ten times that size. Since JPEG uses compression, the final size of the stored file will be affected by the amount of noise in the image, the actual image and the amount of compression used in creating the JPEG. A picture of just the bright blue sky will compress more than an image of a forest, because there is little detail in the sky.
Because JPEG is lossy by nature, there will be some data that is lost forever in the compression and encoding of the image. When you take a picture in your digital camera and save it as a JPEG a number of things take place: The image sensor captures the light (image) based upon the mechanical exposure controls of aperture and exposure time. The ISO setting of the camera will affect this capture as well, as ISO settings usually control the actual gain of the sensor, affecting the digital values for a given amount of light captured. The output of the sensor is raw data and a raw file would store this data.
This captured data is output from the sensor and then processed by the camera for white balance, luminance and color levels. These adjustments are imposed by the cameras white balance settings and the auto leveling parameters built into the camera. The auto leveling sets arbitrary values for black and white, then discards all data above or below those arbitrary values. Auto leveling also sets mid tone levels, which will affect the contrast, shadow detail and other elements of the image which fall between black and white. Most cameras add a little color saturation to give the image a little more dynamic impact.
The image is sharpened. All digital images need to be artificially sharpened or they will look a little soft. This sharpening is usually done at an arbitrary level and cannot be changed in the cameras JPEG conversion. Sharpening is a process where light level changes from one pixel to the next are altered to give the impression of sharper edges on objects within the image.
After all of the processing the image is compressed and stored on the cameras memory media as a JPEG imageAs you can tell from the description above, a lot of what your camera ‘sees’ is simply discarded in the JPEG conversion process. This is usually not a problem on a properly exposed image without any technical difficulties involved.
I will attempt to illustrate, by example, why raw image capture can be really beneficial. I am going to use an image from a recent trip of mine that illustrates what can happen when you do not get a proper exposure of a technically difficult capture. This image was strongly back lit on a sunny day and I was careless and neglected to let my camera meter on the proper item in the image. Focus was good, but the exposure was awful. This is the opening gate to Yellowstone at the north entrance. I will first show the JPEG that came out of the camera with normal processing parameters.
You can click on any picture in this post, except for the graphs, for a larger view.
http://www.prades.net/images/jpeg1sm.JPG (http://www.prades.net/images/jpeg1.JPG)
As you can see, the sun and a bright blue sky were behind the gate and my camera’s metering used the very dark stone of the gate averaged against the little bit of blue sky in the center of the archway to determine the metering value. This made the whole exposure dark, as you can see in this graph of the raw image. The left of the graph represents the low levels or black in the image and the right side represents highlights or whites. It is underexposed by more than one full stop.
http://www.prades.net/images/rawbefore.JPG
There is the possibility to alter the levels in a JPEG image, but because of the lost data there are severe limits as to what you can recover. The following image will show a quick and dirty attempt at regaining the shadow detail in the gate. More could be done with some time in PhotoShop, but this still gives a good representation of the challenge.
http://www.prades.net/images/jpegprocsm.JPG (http://www.prades.net/images/jpegproc.JPG)
As you can see there is some shadow detail to be recovered, but there is also a lot of noise. The highlights also lose all of their detail. Again, a better job could be done with the recovery, but the amount of shadow detail that is lost in the JPEG conversion is considerable.
With raw conversion the first thing I can do is to impose some exposure compensation. This will recover a tremendous amount of shadow detail that is buried in the raw file, which was discarded in the JPEG conversion. Here is a graphic of the image luminance data after exposure compensation.
http://www.prades.net/images/rawafter.JPG
This is a screen capture from BreezeBrowser. As you can see there are many other items I can adjust in this raw conversion. I can set the final image size, adjust color saturation, set black point, white point and even mid tones. I can choose sharpening methods and amounts, image gamma and even correct for certain types of lens distortion. Below is the image, processed from raw in PhotoShop CS3. Most of the image adjustments were done in the raw conversion adjustments. There was little to do other than sharpening and resizing before saving it as a JPEG.
http://www.prades.net/images/jpegfromrawsm.jpg (http://www.prades.net/images/jpegfromraw.jpg)
As you can see, the raw file had enough data to help me recover from this disaster of an image. In most cases I would have been more careful in the first place, reviewed the histogram of the image on the spot and adjusted the exposure properly when I captured the image. I was driving through and only captured it once. Without the raw file, there would have been little to do. I realize this is not a great image, I only chose it to illustrate the challenges that a raw file can help you overcome.
A raw file will not help you if there is motion blur in the image, either by motion in the camera or the subject. It cannot help you if the image is not focused or composed properly. A raw file can help you recover from exposure problems or white balance problems.
Almost every camera that is capable of raw image storage comes with software to process that raw file. I use BreezeBrowser for the vast majority of my images, because I like and am accustomed to the workflow. Adobe PhotoShop also includes ACR for raw conversion. Photoshop is robust and complicated, but the learning curve to get started is not too steep. There are also programs that come with the camera, Canon provides DPP with theirs. There are also dozens of other raw conversion applications from the professional Capture One to the popular Bibble. I highly recommend that you look into raw image storage and processing as it will appreciably improve your results. It will also make a better photographer out of you as you learn the pitfalls of complicated exposure situations. You will begin to pay much more attention to light, which is the absolute key element in any quality photograph.
BreezeBrowser (http://www.breezesys.com/BreezeBrowser/index.htm)
Capture One (http://www.phaseone.com/Content/Software/ProSoftware/ProductOverview.aspx)
Bibble (http://www.bibblelabs.com/)
PhotoShop (http://www.adobe.com/products/photoshop/photoshop/)
Hope this helps!
If you want an even more technical treatise of the raw conversion, Northlight Images has a good one here (http://www.northlight-images.co.uk/article_pages/why_use_raw.html).