Dynamic Range (DR) is the range of exposure, i.e., scene brightness, over which a camera responds with good contrast and/or a good Signal-to-Noise Ratio (SNR).
Sensor manufacturers are developing High Dynamic Range (HDR) image sensors that claim exceptional dynamic ranges: 120dB to as much as 150dB. Measured system (camera) dynamic range are typically much less than the specified sensor dynamic range.
The primary cause is flare light in the lens— stray light reflected between lens elements and off the barrel (on the interior of the lens) that fogs the image and sometimes causes “ghost” images. High quality lens coatings help— but only so much. Different test chart designs have different density distributions, hence have different amounts of lens flare on the chart image— leading to differences in measured dynamic range. No camera system we know of (where light has to pass through optics) reaches 120 dB. (100dB is about the best we’ve heard of (summer 2016).)
Sensor dynamic range (the range of brightness between sensor saturation and SNR = 1 (0 dB)) can be measured from absolutely raw images (no demosaicing or processing of any kind) with Multicharts or Multitest using a technique described in Multicharts/Multitest/eSFR ISO Noise. It can also be measured with a sequence of flat field exposures (often involving calibrated neutral density filters). These measurements should not be confused with a camera system measurements (especially for HDR systems) because each image used for the measurement has essentially zero dynamic range, and hence don’t correspond to real scenes in real cameras.
ISO 15739: The dynamic range corresponding Scene-referenced Signal-to-Noise Ratio SNRscene = 1 (0dB) corresponds to the intent of the ISO 15739 Dynamic range definition in section 6.3 of the ISO noise measurement standard: ISO 15739: Photography — Electronic still-picture imaging — Noise measurements. The Imatest DR measurement differs in several details from ISO 15739 (though we do measure ISO 15739 Dynamic Range); hence the results are not identical. Imatest may well produce more accurate results because it measures DR directly from a transmission chart, rather than extrapolating results for a reflective chart with maximum density = 2.0.
|Dynamic range from a single transmissive chart image.||Stepchart, Multicharts, Mutitest||A transmissive chart is such as the Imatest 36-patch Dynamic Range or HDR chart is required because reflective charts do not have sufficient tonal range.|
|Dynamic range from multiple (differently exposed) images||Dynamic Range module||Uses CSV output of Multitest or Stepchart for several differently exposed images. Usually used with reflective charts, but transmissive charts may also be used.|
|ISO 15739 Dynamic Range from patch with density ≈ 2||Multicharts, Multitest, eSFR ISO||Extrapolates Dynamic Range from a single patch with density ≈ 2.|
|Raw sensor Dynamic Range||Multicharts, Multitest||Fits raw data to an equation from the EMVA 1288 standard, then extrapolates to find DR. The test chart does not have to have as large a tonal range as the DR, but transmissive charts with tonal range ≥ 3 are recommended.|
|Contrast Resolution||Multicharts, Multitest||Measures the visibility of low contrast features over a wide tonal range. A better indication of true usable DR than traditional measurements. New in Imatest 5.0. Not yet an industry standard (as of Sept. 2017), but we’re working on it.|
The most straightforward way to measure a camera’s (or scanner’s) dynamic range is with a transmission step chart illuminated from behind by a lightbox (taking care to minimize light reaching the front of the chart). Reflection step charts such as the Kodak/Tiffen Q-13 or Q-14 are inadequate because their density range of around 1.9 is equivalent to 1.9 * 3.32 = 6.3 f-stops (38 dB), well below that of digital cameras (although they can be used to measure Dynamic Range from several different exposures combined in the Dynamic Range postprocessor for Multitest and Stepchart).
The table below lists several transmission step charts, all of which have a density range of at least 3 (10 f-stops). All except the Imatest and TE 241 charts are linear (one row of patches), and may suffer from vignetting (light falloff) as a result. All except the Imatest charts have dark backgrounds, which makes is difficult to achieve correct exposure in autoexposure systems.
|Imatest 36-patch ITDR36 Dynamic Range test chart||36||~0.10 (1/3 f-stop);
Reference file included
|Imatest 36-patch ITWDR36 Wide Dynamic Range test chart (recommended)||36||~0.10 – 0.30 (1/3 – 1 f-stop); Reference file required||>5||8×10″|
|Imatest 36-patch ITUHDR36 Ultra High Dynamic Range test chart||36||”||>7.5||8×10″|
|Imatest Contrast Resolution chart||20||0.25 (with 2:1 inner patches)||4.75||8×10″|
|Kodak Photographic Step Tablet No. 2 or 3. Calibrated or uncalibrated (which is usually sufficient).
||21||0.15 (1/2 f-stop) Linear pattern*||3.05||1x5.5″ (#2)
|Stouffer Transmission Step Wedge T4110||41||0.10 (1/3 f-stop) Linear pattern*||4.05||1x9″|
|Danes-Picta TS30D (Digital Imaging page)||28||0.15 (1/2 f-stop) Linear pattern*||4.2||(0.4x9″)|
|DSC Labs Xyla 15, 21, and 26 step HDR Greyscale||13||0.30 ( 1 f-stop) Linear pattern*||3.7||(large)|
|Image Engineering TE241, TE264,
|from table; 20-patch charts have large steps in dark regions. Circular pattern of squares. Most are half toned.
|*Region selection is less robust with linear (single-row) charts because it’s difficult to locate features in very dark areas. Also, linear charts are more subject to vignetting than charts with a circular arrangement of patches.|
The Imatest 36-patch Wide Dynamic Range (HDR) test chart, with density steps from 0.1 to 0.3 and a maximum density of at least base+5 (~17 f-stops), is recommended for traditional Dynamic Range measurements. A nearly circular patch arrangement ensures that vignetting has minimal effect on results. A CSV or CGATS reference file with actual densities (required with the HDR chart) is supplied. the chart has an active area of 7.75×9.25 inches on 8×10 inch film.
It also contains slanted edges in the center and corners with 4:1 contrast for measuring MTF. Registration marks make the regions easy to select, and fully automated region detection is available in Imatest 4.0+. A neutral gray background helps ensure that the chart will be well-exposed in auto exposure cameras (compared to charts with black backgrounds, which are sometimes strongly overexposed).
This chart is produced with a high-precision LVT film recording process for the best possible density range, low noise, and fine detail.
The standard 36-patch Dynamic Range chart can be used with the Dynamic Range postprocessor to measure dynamic ranges larger than 11 f-stops if several manual exposures are available. Dmax = 3.4 is sufficient for camera phones and digital cameras with small pixel sizes, but high-end DSLRs and HDR security and automotive cameras generally have higher dynamic ranges, which makes them well suited for measurements with either Dynamic Range or the HDR chart (which never requires the Dynamic Range postprocessor).
You’ll need a lightbox that can evenly illuminate the transmission step chart. 9×12 inches is large enough in most cases. Avoid thin or “mini” models, which may not have even enough illumination. Fluorescent light boxes should have high frequency ballasts to eliminate flicker.
Our recommended lightbox is the ITI LED Lightbox (shown on the right with the 36-patch HDR chart), available from the Imatest store. It has several advantages over the lightboxes mentioned in the previous paragraph.
Several lightboxes are available from the Imatest Store. There are compared in the Lightbox Comparison Guide.The GLE-10 and GLE-16E have well controlled color temperatures. The Artograph LED light boxes model) are nice because they’re inexpensive, flicker-free (DC power supplies) and easy to dim (if you have a current-limited DC power supply), but they are not reliable for color measurements.
Imatest Dynamic range is defined in two ways (both displayed in Multicharts, Multitest, and Stepchart).
the range of exposure where
SNR tends to be worst in the darkest regions. Recommended in most cases.
|SNRscene = 10 = 20dB; high quality
SNRscene = 4 = 12dB medium-high quality
SNRscene = 2 = 6dB; medium quality
SNRscene = 1 = 0dB; low quality
(introduced in Imatest 4.4)
The range of exposure where slope of the density curve is greater than 7.5% of the maximum slope (on the dark side) and less than 98% of the saturation level (on the light side).
This has to be done with care since noise and irregular density steps in the original data affect patch levels, and hence slope measurement. Test chart density is interpolated to have 101 evenly-spaced points, then the log pixel level for these points is found using the Matlab interp1 function. The curve is smoothed using a rectangular kernel 9 points wide (approximately 1/11 of the total range).
This algorithm gives better results than earlier (pre-4.4) Patch range and detected DR measurements, but it should be interpreted with care because SNR can be extremely low (often well under 1 = 0 dB) in the the dark portion of the range. This can result in measurement inconsistencies as well as overly optimistic measurements, since image features are indecipherable when SNR < 1 (all you can see is noise). Slope-based DR can be falsely increased by tone-mapping and by medium-range flare light (bleeding from light chart patches, visible in the Image Statistics images below). For this reason we don’t recommend its use. The Contrast Resolution chart and analysis provides a much better indication of true camera (system) DR.
Dynamic Range can be measured by three Imatest modules: Multicharts, Multitest, and Stepchart. (We omit Dynamic Range, which is a postprocessor for Multitest and Stepchart.) We recommend Multicharts (interactive) and Multitest (fixed; batch-capable), which are newer and more versatile than Stepchart (an older module that we will continue to support).
|Module||Brief description and recommendations|
|Multicharts||Interactive module for measuring color and grayscale charts. Best when you need to closely examine the results.||Supports several measurements not in Stepchart, including ISO noise, sensor dynamic range, and Contrast Resolution. Share most INI file settings.|
|Multitest||Fixed (batch-capable) module for measuring color and grayscale charts. Same functions and INI file settings as Multicharts. Largely replaces Colorcheck and Stepchart (both older modules with limited capabilities).|
|Stepchart||Older module for measuring grayscale step charts (at first it only supported the Kodak Q13/Q14 charts).||Supports temporal noise (will eventually be added to Multicharts/Multitest).|
|f-stops||log2; Factor of two||also called zones or EV (Exposure Value);||Photographers|
|decibels (dB)||20 log10||1 density unit = 20dB; one f-stop = 6.02dB||electrical engineers|
|density units||log10; Factor of ten||1 density unit = 3.322 f-stops||optical scientists|
The lightbox and chart are shown inside a black box that minimizes stray light reflecting back to the chart.
|Tip: to facilitate Dynamic Range plots in Multitest we’ve added a button to the settings window. Pressing it enters the correct settings for a DR display (checks the Noise/SNR plot, and selects 15. F-stop…SNR and Density response above noise). the X-axis & DR units are unaffected.|
Here are Multitest and Stepchart results for the Panasonic G3 (a Micro Four-Thirds camera with 3.75 micron pixel pitch) at ISO 160, converted from raw using dcraw with the following settings: Demosaicing: Normal RAW conversion (demosaiced), Output gamma: 2.2, White Balance: Camera, Output color space: 48-bit, Quality: Default.
The slope-based dynamic range is greater than 11 f-stops. Slope-based DR is not very meaningful because it often includes patches with SNR below 0 dB, where the noise overwhelms visible features.The DR at low quality level (scene-referenced SNR = 1 = 0dB) is 8.7 f-stops (52.4 dB), decreasing to 5.17 f-stops at high quality level (SNR = 10 = 20dB). These results are similar for 24-bit raw conversion.
The shape of the response curve is a strong function of the conversion software settings. The plot below is for the same exposure, saved as a JPEG file inside the camera. Note that the transfer curve is quite different: it has a “shoulder” in the highlights, which improves pictorial quality by reducing the tendency of highlights to saturate (“burn out”). Dynamic Range is increased due to software noise reduction (absent in the dcraw conversion).
In Stepchart, units for displaying Dynamic Range can be set in the X-axis scale (Figs 2-4) and Dynamic Range units (Fig. 2) dropdown menu. To convert dynamic range from f-stops into decibels (dB), the measurement normally given on sensor data sheets, multiply the dynamic range in f-stops by 6.02 (20 log10(2)). The dynamic range for low quality (f-stop noise = 1; SNR = 1) corresponds most closely to the number on the data sheets. (A valid sensor Dynamic Range measurement requires completely raw images.)
The Contrast Resolution chart is a new chart design that was developed to overcome critical limitations of standard DR measurements derived from grayscale charts. It is not yet standardized, but we’re working on it, especially in the IEEE P2020 standard for automotive system image quality.
The key to understanding the Contrast Resolution chart is that Dynamic Range can be defined more meaningfully when it predicts the visibility of low contrast features over a range of tones, i.e., answers the question, “Will a low contrast feature be visible in a dark (or light) area of the image?” Standard DR measurements don’t provide a clear, unambiguous answer to this question.
The Contrast resolution chart consists of 20 large patches that cover a 95 dB tonal range. Each large patch contains four smaller patches. The light and dark gray patches have a 2:1 (6 dB) contrast ratio and the same mean density as the surrounding large patch. They are used to define a signal for the newly-developed Contrast Resolution Signal-to-Noise Ratio (CRSNR) measurement, where the noise is measured in the larger gray patch (which has better noise statistics). The red and blue patches are for visual analysis.
Before we say more, we have to answer the question,
“Why only 95 dB when sensors have up to 150 dB dynamic range, and there is getting to be quite an intense DR competition? (This competition is somewhat akin to the automotive horsepower race of the 1950s, which culminated when TV comedians adopted the line, “My car can pass anything on the road except a gas station.”)”
The answer is we have yet to see a real practical system (which will be unavoidably limited by flare light) that reaches 95 dB, much less surpasses it. (Also, it’s difficult to manufacture practical charts of this type with a higher tonal range). We also have to note that customers who expect higher dynamic ranges for sensors specified at up to 150 dB DR may not be pleased by the harsh reality of a lens between the sensor and the test chart or scene. Please contact us at email@example.com if your camera’s DR exceeds 95 dB.
Here is a brief explanation of Contrast Resolution measurements. See the Contrast Resolution page for full details.
The example on the right, run on Multicharts, uses an image of the Contrast Resolution chart taken with the Canon EOS-6D and 100mm f/2.8 macro lens at f/4— a very high quality DSLR camera and lens, but not specifically marketed as HDR (High Dynamic Range).
Log pixel levels for the large gray patches, which are used as the signals in standard DR analyses, are shown in the upper plot as thin black lines. The Contrast Resolution signal (light-dark levels) is shown as thick magenta lines.
The lower plot shows standard SNR (thin black line) and Contrast Resolution SNR (CRSNR), where the signal is the light-dark patch level and the noise is measured in the larger adjacent gray area.
The image on the right displays all patches at a constant mean pixel level, making it easy to see the effects of noise. Even at the small size it’s apparent that the 2:1 features are only clear in patches 1-13, are fading badly in patches 14 and 15, and are pretty much gone beyond that. Patch 13 has about 8 dB (20*log10(2.42)) Contrast Resolution SNR.
We can define a Contrast Resolution Dynamic Range based on the range of tones where CRSNR is greater than a comparable amount (6-14 dB). Note that CRSNR = 1 (used to define DR in conventional measurements) corresponds to really horrible image quality, where the feature is only marginally visible if you know exactly what to look for.
Click on the images to display them full-sized Image with patches normalized
to have the same mean pixel level.
A key advantage of the Contrast Resolution chart is that it provides good results in the presence of tone mapping— a type of nonuniform tonal compression that maintains local contrast. Tone mapping is widely used for HDR images intended for human viewing (on displays with limited dynamic range). Standard grayscale charts do not provide reliable results for tone mapped images.
Once again, we have to state that although we believe that the Contrast Resolution chart provides a superior indication of system performance, it is not yet an industry standard. We’re working on it. For more details, see the Contrast Resolution page.
There are a number of things to consider when measuring dynamic range. Most importantly, dynamic range measurements are strongly affected by flare light from the scene (i.e., the chart), image processing (especially bit depth, black level offset, noise reduction, and tone mapping). The experimental setup is critical.
Light from the environment reflecting back to the front of the test chart should be minimized. It can seriously degrade DR measurements, especially for HDR (DR > 100dB) systems. The test environment should be totally darkened. Where possible, walls should be dark. Black cloth should be used to cover lights and light surfaces might reflect light back to the chart. Stray light is the major cause of DR measurement error.
For Imatest Dynamic Range or Contrast Resolution charts (shown above) the chart size in the image should be at least 600×600 pixels if possible. For high resolution cameras it doesn’t need to fill the image. A 2000×2000 chart image is more than sufficient for high resolution cameras.
Flare light (stray light that reaches the image sensor after bouncing between lens elements and the lens barrel) is the most significant practical limiting factor for DR. It can take the form of fog in shadows (veiling glare) or ghost images (challenging to measure). Flare light can arise from light areas or light sources inside the chart or (in real-world situations) outside the chart. Because of this, the choice of test chart can have a significant effect on DR measurements. Measurement errors caused by flare light are described below.
When possible, DR should be measured in images converted from raw format (the sensor output) with minimal processing and if possible with bit depth of at least 16. This can be done in Imatest using dcraw or Readraw, which do not apply a tonal response curve, sharpening, or noise reduction, hence have relatively little effect on DR.
Dynamic Range can be measured by the Imatest Multicharts, Multitest, and Stepchart modules. Multicharts (which is highly interactive) and Multitest (a batch-capable module that includes virtually all Stepchart functions) are recommended because they are newer and have more noise analysis detail than Stepchart.
Image processing can be divided into two categories: processing routinely done during raw conversion and post-processing that is applied afterwards. Both can affect DR measurements.
Tone mapping is fairly easy to identify. Measured gamma is much lower than values typical for standard color space images (around 0.5), and patch levels may not decrease monotonically. The Image Statistics example below shows evidence of tone mapping. The effects of tone mapping on DR measurements are not easy to predict: it should be regarded as a complete wildcard. Dynamic Range measurements of tone-mapped images are not reliable.
The slope-based Dynamic Range (which is sometimes considered total DR because it tends to be larger than SNR-based DR measurements) is defined as the range of exposure where slope of the density curve is greater than 7.5% of the maximum slope (on the dark side) and less than 98% of the saturation level (on the light side). Since tone mapping reduces the maximum slope, it increases the slope-based DR. The slope-based DR often has an extremely low SNR in the dark part of its range, −10 dB or worse, which makes it quite inconsistent, even without tone mapping.
The Image Statistics module can be used to examine the image for tone mapping, pixel level offsets, and other surprises. It provides information that can be useful when testing DR. The example below shows a XYLA chart, which is designed to minimize flare light (the lighter patches are relatively small). Density steps = 0.3. The image below is displayed lightened (this does not affect the results below the image). The lower plot is a 35 pixel-wide cross-section of from the chart (averaged to reduce noise). There appears to be a pixel offset of 8 in the black areas outside the chart patches, though it could be long-range flare light (that affects the image uniformly). Looking at the cross=sections near the borders of the images, we see a fairly consistent mean of 8 for this image, which is strongly indicative of a pixel offset (rather than flare, which tends to be largest near the bright regions). Some short-range flare light is visible in about 60 pixels to the left of the brightest patch. It’s actually quite well controlled; we’ve seen worse.
The flattening of the response in patches 4-6 seems to indicate that tone mapping has been applied. If it were not caused by tone mapping, this would mean that there would be no image contrast at these densities— a quite unacceptable result. This is a case where the Contrast Resolution chart provides information that is simply not available from a straight grayscale chart.
This decrease can cause measurement errors because it can resemble response to the changes in chart density, when it is caused entirely by flare light.
The Image Statistics horizontal cross-section on the right shows the effects of significant flare light on an image of the XYLA (linear) chart. Very strong flare light is visible to the left of the chart, and the pixel level continues to drop in areas on the right where chart patches are no longer visible (even when the image is extremely lightened).
The lower image— a horizontal cross section of the same image taken below the the chart (displayed with extreme HSL lightening)— shows that this decrease is caused by flare light— not from the camera’s response to the chart.
X-cross-section taken at the center of the XYLA chart
X-cross section taken below the XYLA chart,
showing significant effects of flare light.
Note that the decrease in pixel level on the right of the chart (around x ≥ 1500) erroneously increases the measured Dynamic Range from slope. Currently Imatest is unable to distinguish a pixel level decrease due to flare light from a decrease due to patch densities.
Noise is a random stochastic process. When noise is high in relation to signal level (low SNR), is can have a strong effect on measurement consistency. In particular, the slope-based dynamic range has been discussed in several places above. A key problem with this measurement is that SNR is frequently very low (−10 dB or worse) at its darker limit. (It can also be affected by tone mapping and flare light.) For this reason we don’t recommend its use.
Region selection in linear charts such as the XYLA charts (which have 21 or more f-stops of dynamic range) can be extremely difficult because (a) It is difficult to see dark patches in the denser parts of the causes, (b) it is affected by optical distortion, which can be corrected with difficulty, and (c) it can be affected by keystone distortion (which can also be corrected, but with considerable difficulty). Unfortunately small errors in region selection can significantly affect measurements. This issue was the primary reason we designed the 36-patch Dynamic Range charts (as well as the Contrast resolution chart), both of which have patches arranged in a circular pattern with registration marks for automatic region selection (or easy manual selection if automatic doesn’t work).
What is scene-referenced noise and Signal-to-Noise Ratio (SNR)?
Scene-referenced noise and SNR (measured relative to the scene rather than to the image pixel levels) are used by imatest for measuring dynamic range because they are independent of the camera’s gamma-encoding (contrast), which is applied as a part of the image processing pipeline.
In calculating scene-referenced noise and SNR we need consider the behavior of the human eye, which responds to relative changes of illumination. For example, doubling or halving the illumination (multiplication by 2 or 1/2) results in a similar perceptual change. That’s why we think of exposure in terms of zones, f-stops, or EV (exposure value), all of which correspond to factors of two change in exposure. The eye’s relative sensitivity is expressed by the Weber-Fechner law, ΔL ≈ 0.01 L –or– ΔL/L ≈ 0.01 where ΔL is the smallest luminance difference the eye can distinguish. (This equation is approximate; effective ΔL tends to be larger in dark areas of scenes and prints due to visual interference from bright areas.)
Expressing noise in relative exposure units, such as f-stops, where f-stop = k log2(exposure), corresponds more closely to the eye’s response than standard pixel or voltage units. Noise in f-stops is obtained by dividing the noise in pixels by the number of pixels per f-stop. (I use “f-stop” rather than “zone” or “EV” out of habit; all are equally valid.)
f-stop noise = (Noise in pixels) / (d(pixel level)/d(f-stop))
where d(pixel)/d(f-stop) is the derivative of the pixel level with respect to exposure measured in f-stops = log2(exposure). Note that d(f-stop) = d(k log2(exposure)) = 1.44274k d(loge(exposure)) = 1.4427k d(exposure)/exposure. Dropping the 1.4427k factor and noting that exposure is the signal level,
f-stop noise = (Noise in pixels) / (d(pixel level)/d(exposure) * exposure) where exposure is the signal level. Hence,
Scene-referenced Signal-to-Noise Ratio = SNRscene = exposure / ((Noise in pixels) / d(pixel level)/d(exposure))
The above image illustrates how the pixel spacing between f-stops (and hence d(pixel)/d(f-stop)) decreases with decreasing brightness. This causes f-stop noise to increase with decreasing brightness.
Since scene-referenced noise and SNR are defined relative to the scene exposure and are independent of image processing or pixel levels, they are universal measurements that can be used to compare digital sensor quality when sensor RAW data is unavailable. They play a key role in Imatest’s diverse dynamic range calculations.
Incident light has to be measured to obtain the camera’s Exposure Index (ISO Speed).
Reflective charts: Incident light measurement is relatively straightforward. Place an incident illumination meter with a flat diffuser (such as the inexpensive BK615) just in front of the chart, pointed towards the camera, making sure not to block any of the illumination. Use the lux measurement directly.
Transmissive charts: These are not so straightforward: the lux measurement cannot be used directly. You will need to measure a clear (white) area of the chart (not the lightbox itself). For meters like the BK 625, where you can switch the orientation of the sensor, set it to point towards the light source (on the opposite side from the display). Measure the illumination in a white area of the chart. For the 36-patch Dynamic Range chart, use the lightest grayscale patch. You need to make an adjustment because the input to Imatest is designed for reflective charts, where white areas reflect about 90% of the incident light. To compensate for this, use
Lux (input to Imatest) = Lux (measured) * 1.11.
Useful equation: If your meter reads in EV (Exposure Value): Lux = 2.5 * 2EV @ ISO 100
Note: EV @ ISO 100 is also known as Light Value (LV).