Imatest Resolution Webinar

Estimated read time: 1:20

Summary

In the Imatest Resolution Webinar, the creator dives deep into the intricate world of measuring sharpness and resolution in imaging systems. The discussion covers the fundamentals of resolution, challenges involving different measuring methods, and how signal processing can affect image quality. Advances in measurement standards and tools for assessing image sharpness and resolution are also highlighted, offering valuable insights for improving image assessment across various applications. The session concludes with a Q&A segment addressing specific audience queries about imaging techniques and tools.

Highlights

Introduction to the concept of resolution and its definition in imaging. 📏
Discussion on the imaging system workflow from scene capture to image processing. 📸
Different factors affecting image resolution and challenges in measuring it. 🧩
Understanding the difference between sharpness and resolution: the detail of the details. 🔬
In-depth exploration of the methods and tools used for measuring resolution through historical and modern lenses. 🕰️
The impact and mitigation of signal processing on image resolution. 🎛️
Analysis of different measurement standards, including the new ISO updates. 🌐
Best practices in setting up a test environment for effective resolution measurement. 🛠️
Future improvements in resolution and sharpness measuring techniques. 🚀
Q&A session addressing common issues and advanced queries. 🤝

Key Takeaways

Resolution isn't just about pixel count; understanding the detail nuances is crucial! 🖼️
Sharpness versus resolution: not interchangeable but complementary concepts. 🔍
Watch out for saturation and aliasing when assessing image quality. ⚠️
Know your measurement tools and use them correctly to avoid misinterpretation. 🛠️
Signal processing effects on image quality need careful control. 🎛️

Overview

The Imatest Resolution Webinar thoroughly explores the fundamentals of resolution and sharpness in imaging systems. It sets the stage with definitions and dives into the nuanced differences between resolution and sharpness — crucial for professionals in fields relying on imaging accuracy.

Throughout the session, various methods and historical advancements in measuring resolution are discussed. These include ISO standards updates and the impact of signal processing, highlighting the importance of understanding these techniques' capabilities and limitations.

The webinar concludes with practical insights into setting up test environments for resolution measurement and previews upcoming advancements in the field. Additionally, a robust Q&A session tackles challenges faced by professionals, providing expert advice on tailored imaging solutions.

Chapters

00:00 - 01:00: Introduction to Webinar The chapter 'Introduction to Webinar' introduces the topic of measuring sharpness and resolution. It outlines the plan to cover definitions of resolution, different methods to measure resolution, and the challenges and solutions associated with these methods. Additionally, the chapter hints at discussing future developments in this area, followed by a question and answer session.
01:00 - 03:00: Understanding Resolution In the chapter titled 'Understanding Resolution,' the concept of resolution is explored primarily within the context of imaging. Resolution is defined in terms of the smallest interval that can be measured by a scientific or optical instrument, which correlates to the resolving power or the degree of detail visible in an image. This is typically referenced in everyday equipment like photographic devices or high-resolution monitors, emphasizing the importance of resolution in determining image quality.
03:00 - 04:00: Imaging System Overview In the chapter titled 'Imaging System Overview,' the concept of sharpness versus resolution is discussed. Sharpness is defined as a more comprehensive measure of how much spatial information is reproduced across various frequencies, in contrast to resolution, which is often related to the limit of detail that can be discerned. This highlights the complexity of translating these descriptors into a single objective measurement, as they each play distinct roles in defining the clarity and detail of imaging systems.
04:00 - 06:00: Challenges in Measuring Resolution The chapter 'Challenges in Measuring Resolution' discusses the factors affecting the resolution of images. It highlights that sensor resolution, which refers to the number of pixels in an image, is not the sole determinant of image quality. Other factors include the sensitivity of the pixels, noise levels, isolation between different pixels, and the ability of the lens to resolve fine details. This indicates that merely having a higher number of pixels does not guarantee a high-quality image without considering these additional aspects.
06:00 - 09:00: Spatial Frequency Response (SFR) and MTF The chapter starts with an introduction to imaging systems, highlighting the process of capturing a scene through a lens and focusing it on a focal plane array, typically an image sensor. It explains how raw data from the image sensor is processed by the signal processor.
09:00 - 16:00: Methods for Measuring Resolution The chapter discusses different methods for measuring resolution in imaging systems. It highlights the distinction between human visual interpretation and machine vision processes, such as neural networks, in analyzing images. The chapter also notes how resolution can become ambiguous depending on the method of interpretation used.
16:00 - 21:00: Challenges and Accuracy in Measurement The chapter discusses the challenges in accurately identifying the customer, particularly in determining whether the customer is one entity or another, or possibly both. The difficulties in the scene are also addressed, including determining the distance to an object, its placement within an image, the angle of the object, its spectral properties, and the background on which it appears. Additionally, atmospheric properties are mentioned as factors that could affect the resolution of the scene.
21:00 - 25:00: Saturation and Signal Processing Impacts This chapter delves into the concept of 'visibility' in the realm of signal processing, using a visual representation to illustrate varying levels of saturation and contrast. The discussion centers around analyzing different levels of visibility within a group of objects, highlighting how each subsequent level loses its contrast and visibility as one moves down the sequence. This gradual loss of visibilty is pivotal in understanding how saturation impacts signal processing and visibility.
25:00 - 28:00: Wedge MTF Measurement Issues This chapter discusses the 'Wedge MTF Measurement Issues', focusing on the frequency at which 50% contrast loss occurs, commonly referred to as MTF50. It explains that MTF50 is a standard measure used for assessing image sharpness. Additionally, for resolution concerns, higher frequency MTFs are considered more relevant.
28:00 - 33:00: Slanted Edge MTF Measurement Issues The chapter discusses the common use of mtf10 as a standard for visibility in slanted edge MTF measurements. However, it acknowledges that this standard isn't universally accepted, as losing 90% of contrast doesn’t always mean an object can’t be resolved. The chapter notes the role of human vision and display systems in MTF measurements, suggesting that these factors can influence the results and interpretations.
33:00 - 40:00: Measurement System Overview This chapter focuses on the measurement systems and their limitations, specifically in terms of visibility thresholds. The discussion centers around the Modulation Transfer Function (MTF) and its standard threshold, MTF 10, which is commonly used as a benchmark for visibility. It highlights that human visibility is limited by factors such as display quality, image size, and viewing distance or angle. The chapter initially references a 99% contrast loss as a point where visibility is no longer effective.
40:00 - 43:00: Wide Field and Long Range Testing This chapter discusses the differences between human visual systems and machine algorithms. It highlights that human vision is limited by spatial, temporal, and contextual contrast sensitivity. In contrast, machine vision is constrained by the design of the algorithm, computational power, and energy consumption.
43:00 - 46:00: ISO Standards and Future Developments The chapter focuses on the main sharpness and resolution metric known as spatial frequency response (SFR), which is used interchangeably with modulation transfer function (MTF). It explains that MTF is the contrast of a sine pattern at a spatial frequency relative to contrast at low spatial frequencies, providing a definition and example of such a pattern that has not been degraded.
46:00 - 65:00: Q&A Session In this chapter, the focus is on analyzing sine patterns in terms of frequency resolution and modulation transfer function (MTF). It is noted that low frequency sine patterns are resolved well, whereas at higher frequencies the response starts to degrade, as evidenced by dropping contrast and responsive metrics. The discussion highlights the consistent performance at low frequencies with a noticeable decline as the frequency increases.

Imatest Resolution Webinar Transcription

00:00 - 00:30 all right we'll get started then thank you all for joining uh today i'm going to talk about measuring sharpness and resolution i'm going to go through some definitions what is resolution some of the different methods of measuring it challenges associated with those methods some solutions we have for measuring resolution and future developments that are in progress right now we'll do a question and answer at the
00:30 - 01:00 end as well if you have questions you can put them in the chat or raise your hand uh so let's get into definitions what is resolution uh if you look it up in the dictionary uh at least the definition that's closest to what we think of in the imaging term is the smallest interval measurable by a scientific or optical instrument the resolving power the degree of detail visible in a photographic or television image such as a a high resolution monitor that's the
01:00 - 01:30 example they give so this is difficult to translate into a single objective measurement and i will explain why first off uh sharpness is uh is is a more complete uh definition of what how much uh spatial information is reproduced across all frequencies where resolution is is usually uh associated with that the kind of limit of of the
01:30 - 02:00 finest detail that you can see in that image so sensor resolution we don't usually use that as the as the the key definition of resolution because it's necessary uh for for a good uh high resolution image but just because you have a lot of pixels doesn't mean that they're sensitive that they're free from noise or that they're isolated from the other pixel pixels or that your lens really can resolve
02:00 - 02:30 uh details on those pixels uh so let's talk about what is an imaging system so uh we start with the scene uh we we take that scene uh and and image it through a lens uh and and try to focus that that scene on a focal plane array uh usually a sensor image sensor and then the the data raw data comes out of the image sensor it goes through the signal processor and at that point it goes to
02:30 - 03:00 your customer which could be a couple different things it could be displayed on some sort of display and then presented to a human and then uh interpreted by the human visual system or it could be that there is some sort of machine vision a neural network that's that's taking that image and uh interpreting it so this is this is where the resolution starts to become a little bit foggy because uh
03:00 - 03:30 we don't know who our customer is it could be one the other or both of these uh folks here so and another problem is uh in the scene what what exactly we're trying to resolve what's the distance to that object where does that object appear in the image what's the angle of that object you know what are the spectral properties of that object in the background that it's appearing on you know atmospheric properties those can also limit the resolution of the scene
03:30 - 04:00 then there's the definition of what is visible so uh we have uh here uh clearly the one on top is definitely visible um the question is how many different uh is this visibles can you see here and there's actually nine of them and they have lost increasingly lost contrast as you go down down the row here so
04:00 - 04:30 uh the 50 contrast loss is is kind of uh similar to mtf50 or it's it's mdf50 is the frequency where you've lost 50 of the contrast uh so uh these these other types of this is this is typically the mtf we use for sharpness um or at least one of the defaults but as far as resolution it's going to be one of these much higher frequency mtfs uh maybe
04:30 - 05:00 mtf10 is fairly commonly used as the standard for visibility but it's not necessarily universally agreed upon that if you lose 90 of your your contrast that you can't resolve that object anymore and also you know i mean it depends somewhat on i mean if it's a it's a human vision then it may be that that the human visual system and the display also have some factors in the
05:00 - 05:30 ability to resolve so i think we can all agree that 99 contrast law 99 contrast loss is definitely uh not visible anymore and so uh we're going to focus mostly on mtf 10 because i think that's the the fairly standard threshold for visibility so in terms of uh what different systems have limits and what their limit is uh dependent on the humans are limited by the display the image size and the viewing distance or angle to that image
05:30 - 06:00 that they're viewing and they're also limited by the human visual system and it's spatial temporal and contextual contrast sensitivity machines are very different and they're limited by the algorithm that you've designed the computation power and the energy consumption that you can afford so the
06:00 - 06:30 main sharpness and resolution metric that we use is known as spatial frequency response sfr which we kind of use interchangeably with mtf or the modulation transfer function this is the contrast of a sine pattern at a spatial frequency relative to contrast at low spatial frequencies so we have a definition of contrast here and then we also have an example of a sine pattern uh which has not been degraded
06:30 - 07:00 and you can see the low frequency sine pattern is resolving just as well as this high frequency one and then below here we have a degraded sine pattern where at higher frequencies is actually you can see the contrast dropping off and so if we take the mtf of this sine pattern we can see that low frequencies are all good and then at higher frequencies we start to drop our response drops monotonically
07:00 - 07:30 as we go to higher frequencies so uh it does usually drop as we the frequency increases it does not always uh especially in in signal process systems and we'll get into discussing that here uh so this is just an example of a mtf plot uh this is something obtained from a slanted edge and the uh spatial domain is shown on top where we have the
07:30 - 08:00 function of the average edge going from light to dark and then we go through the spatial the slanted edge calculation to you know take the derivative of that edge perform a fourier transform and then that gets us to the frequency domain where and then we also normalize to one that low frequency response and then we have our uh spatial frequency response uh mtf as a function of frequency
08:00 - 08:30 and so you can see going uh from this low frequency up to the nyquist frequency 0.5 cycles per pixel this is usually uh the area that that we care about um the performance of the most uh so this is a um example of kind of okay mtf and and we have another example of mtf that's that's not not quite as good so you can see that uh our mtf50
08:30 - 09:00 uh it falls it drops from about a quarter of a cycle per pixel down to about 0.15 cycles per pixel here so that's that's the mtf50 that's the frequency where we've lost half of our contrast uh but i think we're more interested in this discussion about resolution or where we've lost 90 percent of our contrast or most of our contrast so uh in in this sharp case uh it's
09:00 - 09:30 actually beyond the nyquist frequency uh at 0.67 cycles per pixel that's our mtf 10. uh in this not so sharp case the the resolution is about half it's about three five um cycles per pixel so somewhere around here so you can see that that big change in resolution power and actually this is all from the same uh same image from the same camera and it's just a different part of the image so
09:30 - 10:00 uh this is close to the center and this is close to the corner so that that kind of shows that you know don't just measure resolution in one place and think that you're done you may have a much different resolution somewhere else on the image talk about some methods for measuring resolution now this is kind of a timeline of uh historical test targets actually uh and this is going back 70 years
10:00 - 10:30 or yeah 70 71 years so the us air force 1951 target is uh still very commonly used actually especially with optical designers so this is a target which we don't support in imates we probably never will just because of the limited frequencies and the kind of subject subjective nature of you know looking at these these tri bars and saying where where does do these stop resolving
10:30 - 11:00 so it's very very uh a target which i wish people would stop using there are much more modern standards out there which introduced these linear wedges in the eia 1956 and and then those turned into these hyperbolic wedges and the iso one two two three three 2000 target which is still a very popular resolution target i will explain some reasons why
11:00 - 11:30 you should not use it the more modern targets here are the iso12233 2014 targets which introduced a a dedicated wedge measurement target high contrast wedge measurement target sinusoidal siemens star and then the slanted edge a lower contrast slanted edge target and we'll talk about some of the evolution of this standard which is happening this year in the 2222 edition of iso one
11:30 - 12:00 two two three three these are a collection of targets that are supported uh for sharpness and resolution measurements in imatest software and we'll go over uh most of these and describe some strengths and weaknesses a lot of them are slandered edge at least the ones on the top are primarily slanted edge sfr targets and you know they may add wedges to them uh or
12:00 - 12:30 um you know you may see slanted edges appear on these other targets as well these other targets down below are the the cyclical targets so this is a a sine modulated uh polar sine modulated siemens star and then this is the log frequency target and then these are these random we call them spill coins targets are also known as dead leaves so first off i'll talk about the slanted edge sfr and resolution and this is
12:30 - 13:00 introduced in iso12233 in 2000 22 years ago it is a method that is it works best you get the most accurate results when you disable sharpening or you use raw images to test uh i mean especially for resolution the sharpening will make the resolution appear much much higher from these slanted edges but i think that is a
13:00 - 13:30 that is an anomaly of signal processing that doesn't actually mean that your system will resolve uh those those higher spatial frequencies well it's more of an aliasing um translating lower spatial frequencies to appear as if they're sharper higher spatial frequencies so i guess um the real advantage of this is the precision that we can measure so many different uh points across the lens it's also a fast calculation the disadvantage uh there's only one angle of mtf per
13:30 - 14:00 edge and it is very impacted by sharpening and we'll get into some other little quirks with this too but this is our kind of main method the the wedge resolution is also part of um uh iso one two two three three two thousand and also in the the subsequent versions of iso one two three three and this is something that can be used for a tv lines measurement which is uh similar to
14:00 - 14:30 line widths per picture height that's the mtf units that are related to tv lines so there's a if you look at mtf 10 in the units of line widths per picture height that is roughly equivalent to tv lines so the advantage of this target is is it's less impacted by spatial signal processing than the slanted edge so if you have a high frequency wench pattern and uh it
14:30 - 15:00 gets blurred out by your lens which is maybe not in focus or not not a high quality lens uh the signal processing uh at least current signal processing can't just go and reconstruct that that wedge and um restore it whereas the slanted edge you get this big boost in mtf it's less in the wedge the disadvantage is that there's only one angle it's this this big long target which means that there's low spatial precision
15:00 - 15:30 you know you're measuring a different uh point on on your image on low frequencies than you are in the high frequencies uh often this target gets used for subjective analysis and we don't recommend that because that's that's something that can introduce error there's also a poor precision for very sharp systems at fractions of nyquist frequency and we'll get into some details about what that actually means uh practically
15:30 - 16:00 uh next message method is the sinusoidal siemens star um sometimes it's known as ssr this came with the iso 2014 revision of the iso standard the big advantage for this is that it has all angles of mtf not just vertical or horizontal it's it gets a 360 degree mtf so you can see not you can see sagittal and tangential mtf um no matter
16:00 - 16:30 where this target appears in your image so the uh it's also much less impacted than signal processing than the slanted edges the disadvantage is this target is very large and so again the same problems with the wedge you're measuring different parts of your lens at different frequencies and that means low spatial detail and uh it's a lot of a lot of pixels per target and that can impact the calculation speed
16:30 - 17:00 so the random pattern is the target known as spilled coins or dead leaves this is a part of a incorporated at least the color version of this is incorporating the iso technical specification five six seven dash two um it's also part of the cpiq the ieee cpiq standard for image quality the advantage of this target is it it
17:00 - 17:30 measures how well your system preserves fine detail and uh especially for things like foliage or skin uh the signal processing has a way of reducing uh removing the texture from from those kind of areas that get confused with noise and so the other other disadvantage of this is that in low light there can be some accuracy issues whether that resolves um
17:30 - 18:00 that involves confusing noise as texture or in the case of the iso standard version of the calculation misregistering the position of the reference target compared to the actual target that you're measuring and that that's much easier to do in low light or or blurred conditions so this is more of a test of signal processing i probably wouldn't use this for uh a a pure resolution
18:00 - 18:30 measurement if you're wanting that the log frequency contrast again this is a something that is mostly useful for seeing the effects of signal processing this is a sinusoidal pattern that has many different contrasts and frequencies and it is um you can see what what the signal processing does to these different frequencies so uh this and contrast so usually the um the non
18:30 - 19:00 nonlinear processing that's done in most image signal processors isps um is is highly dependent on contrast so you may see that the higher contrast portions of this target have their are boosted by this processing uh the sharpening and then the lower contrast portions of this and especially the higher frequencies may have a reduction in contrast as those frequencies
19:00 - 19:30 get mistaken for noise in the isp it is also only one angle so you know this isn't good for measuring kind of lens uh properties sagittal tangential directions uh and uh because of its size uh it does have low spatial precision and also speed uh concerns so very very good for um seeing the effects of signal processing though so we'll go over some challenges
19:30 - 20:00 so the goal of any measurement system should be to achieve both precision and accuracy uh so this is a example of we could have something that's precise but not accurate we could have something accurate but not precise um or maybe we'd have problems with both and this is what we're going for where we have a repeatable measurement that is um the right the right answer or close to
20:00 - 20:30 it so there's uh the main factor on the precision is the uh the signal-to-noise ratio in the image uh you want more signal and less noise and also depending on the algorithm you're using to analyze uh sharpness and resolution different algorithms have different performance with regards to precision for accuracy there's there's some things that will cause systematic
20:30 - 21:00 problems with accuracy and those may go in either direction they may cause improvements or you know they may be misinterpreted as improvements if if the resolution is increased by that problem or they may um they may cause also a decrease in in the metric you're you're measuring so uh there's many different things that that can impact accuracy such as saturation signal processing non-uniformity and chart quality and
21:00 - 21:30 we'll go over most of these here so saturation this is a a big thing that degrades accuracy and this causes the results to actually increase uh the the metrics the resolution metrics you might get a much higher resolution and sharpness than you actually deserve if um if you have saturation in your sensor so you can see saturation here in this function of the average edge
21:30 - 22:00 where it's basically clipped the signal here and for all you know there may be blur that's concealed in this kind of saturated zone uh but because of the way it's exposed or perhaps because of the high contrast of the target uh you're not seeing the the full um response here so it's uh this is something that is especially a problem with um high contrast
22:00 - 22:30 targets uh imaged by a low dynamic range system or if you have an improv improper exposure um you'll get this wrongful increase in sharpness and resolution so this is something where uh it's actually a method that uh some manufacturers i won't name any names here i've used to kind of cheat the system and try to pass their their cameras off that aren't so good um because they can
22:30 - 23:00 um you know either use this high contrast target or overexpose intentionally and um they get this better score and they're like okay yeah we pass more devices well that is not a good way uh to get good production yield and um yeah i get a little worked up over that when when i see that happening especially for devices where um you know they're safety related things medical and automotive things uh you know don't don't try to use our
23:00 - 23:30 tools to cheat and i mean this is a big reason why we'll never support this 2000 or we'll never fully support this 2000 iso one two two three three target because uh i mean we could write an algorithm to do automatic detection on that but it's uh it's it's gonna give bad results and so we're going to try to encourage people to use more modern standards that are uh that have accurate results so um yeah that there is a clipping warning
23:30 - 24:00 you'll see when when this happens there's also some saturation detection that we have in our software that can kind of um warn you about this um along with reporting back the results but yeah the main indicator is this kind of cusp in this function of the average edge and you can see if you ever have that sharp cusp that's that's a good sign that um there's saturation and um you should be aware of that so uh as far as signal processing
24:00 - 24:30 if you care only about your lens and sensor then ideally you want to try to bypass that signal processing and use a raw or a minimally processed image if you're unable to obtain the raw image then you need to be aware of the impacts of signal processing and maybe use some of those methods that are less affected by this wheel processing so if you care more about your full system performance then go ahead use the processed image you know use slanted mtf and um that's going to give you a good description of
24:30 - 25:00 actually what is perceived in the image as you know how sharp does this image look uh so uh let's talk about some issues with wedge mtf measurement so uh for for very sharp lenses uh if the bars of this this wedge target are in phase with your pixel array uh then you're gonna get a fantastic response uh from them now if they move to be um
25:00 - 25:30 out of phase of the pixel array uh then then those bars which did resolve very well before will disappear so this is something that is problematic particularly at nyquist frequencies and fractions of nyquist frequency so this is an example of a image we we got from a customer actually very very sharp system that is resolving
25:30 - 26:00 um pretty good contrast it it's got like you know 40 contrast at the nyquist nyquist frequency so this is a you know very high quality lens or maybe it's over spec for the sensor that it's on um anyways you see you see this big kind of this sort of oscillation in the mtf well that's always a sign that something is going weird and you should be aware of that and so you see at nyquist frequency uh there you have this great response and you can see close to this red line
26:00 - 26:30 is where the onset of aliasing is or we we lose count of um our ability to count all the bars and then it goes um and then it has this very quick drop off and so what happens there is it because of the distortion these bars are going from out of phase or in phase of the pixels to outer phase of the pixels and they you can see here they completely disappear and our contrast goes to close to zero so uh if you were to take this chart and
26:30 - 27:00 move it up a half a pixel uh this response would drop a bunch and and you would get a very different or significantly different resolution measurement here so um this onset of alie scene is something that we get it's also known as the limiting resolution this is something that comes just from the uh the wedge uh mtf calculation uh so it's it's kind of an alternative to using the mtf 10 and so one of the things uh that you can
27:00 - 27:30 do is you can take the minimum of um like we don't really consider uh frequencies past nyquist as being very well resolved because they're actually you know those could be lower frequencies so um one thing you can do is you can take the the minimum of mtf10 the onset of aliasing and the nyquist frequency and that is a a reasonable metric to use for wedge mtf resolution in this case it's about 1050 line widths per picture
27:30 - 28:00 height in this hd camera so uh this is uh there's a post here that that describes some of this uh these issues here and so i'll move on to the uh actually let me look at the chat and just make sure i haven't missed any good comments here um uh yeah lou says you can also get spurious resolution where instead of eight bars you see seven sharp ones
28:00 - 28:30 and actually i think that's kind of what's going on uh in this one here well it's not eight bars but um it actually aliased right before nyquist and i think um this red line is actually the count of bars and so um we've lost some of these bars [Music] it's actually nine bars here and so uh that uh is i think we lost it because but it's you know i mean if
28:30 - 29:00 you take a raw image from the system you'll see it has a very excellent performance even even beyond nyquist frequency um thanks for the comment uh so this is uh another thing you'll see in wedge measurement sometimes in in systems where you have uh a lot of noise or um you know contrast boosting or i think this is a common combination of noise and saturation and high contrast here sometimes the
29:00 - 29:30 uh you may see this kind of plateau as you get to these higher frequencies and and this is another problem in mtf curves if you see they flatten out like this then then something is is wrong like they should be monotonically decreasing ideally so uh i think uh this is this is something to keep an eye out in this case um the the the plateau here is actually right below the 10 mark so um it could be that
29:30 - 30:00 a small change in the properties of this system caused that plateau to increase a little bit in which case you would have this giant jump in in mtf10 and that's something to be aware of that mtf10 is not always the most stable of measurements especially in in conditions where you have these sort of plateaus in your mtf uh so uh i had talked some about micro positioning changes this is just kind of a example of what it looks like um this is uh ten different frames
30:00 - 30:30 um that are where the the the wedge is offset by a tenth of a pixel between frames and you can see that our mtf is jumping around some because of that kind of um problem with resolving there and so these um the stability of um there's there's stability issues across the frequencies here but as you see as you go up to higher frequencies
30:30 - 31:00 things become increasingly unstable uh from wedge uh measurements uh so that's enough about wedges i'm gonna talk about some issues with the slanted edges and the uh slanted edge non-uniformity is uh is a big problem so if you have a i mean ideally you take a uh you measure mtf from a system that has gone through the um the lens shading correction or the flat fielding of the
31:00 - 31:30 isp so that's that's that's a really good thing to do in in your signal processing unlike the sharpening which can cause accuracy problems so this flat fielding will increase accuracy and uh non-uniformity introduces this this error where and it depends whether the the non-uniformity is um so you've got your edge here depending on with whether the non-uniformity um goes um goes with the edge or against the edge you you may see that
31:30 - 32:00 you can either get a boost or a decrease in your uh edge sfr measurements and and so this is an example of a sort of uniform control image here and then a non-uniform uh image where or a collection of non-uniform images where that non-uniformity is is kind of rotating around 180 degrees and so you can see that the mtf uh drops a lot and it's all
32:00 - 32:30 based on this low frequency reference so what's happening is that we're we're referencing it to the low frequency but the non-uniformity is impacting that low frequency and so you can see that it actually looks almost like well the mdf is going uh above one it almost kind of masquerades as as a form of sharpening and so i mean if you if you have a raw image and you're seeing mtf going over one that's a big sign that non-deformed non-uniformity is impacting um your
32:30 - 33:00 results another thing another thing you can do is you can you can take a slanted edge and you can you can flip it so that instead of going from light to dark and dark it goes from dark to light and if you get a different result that's another sign that non-uniformity is affecting your measurement so we uh we realized this um roughly seven years ago six years ago and introduced the correction in imatest 4.5
33:00 - 33:30 and the correction works by kind of doing a fit to the the slope of the of the function of that edge away from the transition and then it kind of flattens that out so the corrected version here you can see most of that error in the non-corrected version is gone so it's not perfect it's better to flat field uh or even flat field and do this um so this this correction is going to be
33:30 - 34:00 part of the upcoming iso one two three three 2022 standard which is being in the process of being going through final revisions right now so i mean i think measurement repeatability it is something that affects all types of measurements it's something you should be aware of and uh you know take your measurement repeat it several
34:00 - 34:30 times and see if you're getting back the same results so and depending on the metric that you use you may see that you have much less repeatability so generally these higher frequency things like mtf10 do jump around a lot more especially in the presence of noise so there's ways of mitigating this you can maximize your samples by um making having adequate sized regions of interest you can take multiple exposures and and
34:30 - 35:00 signal averaging them to kind of average out the noise assuming it's shot noise not fixed pattern noise you won't average that out you can ensure that you get adequate signal level by you know that you have enough contrast on your target and um you have a proper exposure uh so uh actually you wanna you wanna the more the more photons you have the less shot noise you're gonna have so um having a bright a brighter light box
35:00 - 35:30 is going to mean that you can not have to boost the sensitivity of your system to get more contrast and that sensitivity boost could boost noise as well so yeah you want to get a fairly bright light box and get it properly exposed without being saturated to get a accurate and precise measurement one another thing you can do is is you can use lower frequency metrics which are more stable
35:30 - 36:00 than these higher frequency ones now they may not exactly be resolution they may be more sharpness but they also they that stability may uh improve things especially if you're doing manufacturing so the um you can see here mtf 10 is actually off the charts in this study we did where we had simulated noise and we had some sharpening this mtf-10 kind of went to a very very high level in some of these expo exposures
36:00 - 36:30 and these other metrics uh mtf 30 and tf50 were much more stable actually mtf area was by far the most stable metric um in this study and that's something that integrates across all frequencies up to nyquist and that's not very commonly used mtf metric but i think it's an extraordinarily stable one uh and it's fairly it's kind of similar to the accutanes measurement which is weighted to the contrast sensitivity
36:30 - 37:00 function of the human visual system so but this one is not so mtf area is a good sharpness metric doesn't get used much surprisingly uh so now i'm going to go through some of the measurement systems uh that we have you know the the goal of a test setup is to get a target at a working distance you know where you're going to actually be using your camera so this is going to be somewhere uh hopefully
37:00 - 37:30 beyond the minimum focus distance of your camera and up to the hyper focal distance of your camera which uh depending on the focal length that could be it could be uh you know you know it could be a meter or it could be 10 meters so that hyperfocal distance is the distance where it's effectively the same your depth of field goes out to uh effectively the same as infinity so you know like for example for a typical
37:30 - 38:00 sort of camera phone we might measure it at 10 centimeters to see you know autofocus performance up close we'd use a high precision target to ensure that we had a a target that is more precise than the system we're trying to measure and then we may measure at something like two meters to the hyper focal distance uh so um and then the other goal is to fill the field of view out to the corners of the image and also have uniform illumination of
38:00 - 38:30 the target as best you can uh so uh for close range targets uh there's there's these high precision targets highest precision target we have is this chrome on glass which has some uh it's costly it's unitone uh so and it has to be backlit so there's some disadvantages to that type of target uh for sure um there's other like film targets or high precision film we have it's not as sharp as the chrome and glass but let's
38:30 - 39:00 see have color lets you have different tones it's also backlit and the highest precision resolution target that is reflectively lit we have is is called this res checker target which has a lot of tonal targets and color targets on it but also some very high precision reflective targets so you know at close range you know one of the problems is if you have specular reflection on your target and this is especially a
39:00 - 39:30 problem if you have a built-in light source which is turned on well if you have a specular reflection on your target that you're measuring and you're especially the region you're measuring that'll destroy the measurement and so um the that's that makes it very hard to measure endoscopes that have built-in light sources this reflective target is is one way of doing that otherwise you kind of have to disable the built-in light source and use the backlit backlit target to measure at close range
39:30 - 40:00 uh at medium range i the we have a product called the imates modular test stand and this is a target and camera alignment system also light source alignment so this is something that uh works with these rails and we can um adjust the distance between the the camera and the chart up to 3.5 meters we have a chart holder here and we have light stands where you can get you can
40:00 - 40:30 kind of get the lights in a certain position lock them down and there is actually a motorized version of this where you can automate the the distance changes and also the tilt tilt and pitch and yaw so this is a good good solution for measuring up to 3.5 meters now if you have to go to beyond 3.5 meters there's a longer range test that we have available to you but let me talk about wide field
40:30 - 41:00 of view so yeah the challenge for these very wide field of view cameras is to get a measurement regions in the corners of the image so uh up to around 160 degrees you can use a pre-distorted target and that can help you to get closer to the corners but as you get to 180 degrees or beyond 180 degrees these uh these system uh you you can't fill the field of view with the plane or
41:00 - 41:30 target it's not possible it'll be infinitely large so the uh we use a uh system that attaches the modular test stand that can kind of have a polar polar arrangement of these um slanted edge targets we call um sfr reg and these are can be a position in the corners um up to 200 degree field of view uh so uh and then this is mount uh lit reflectively from from behind
41:30 - 42:00 instead of uh instead of kind of from the sides as is done with the um kind of normal reflective targets so um yeah you can only use mat targets with this uh because of the the problems with the specular reflection which which are exacerbated on these wide field of view cameras uh so if you have to test beyond 3.5 meters uh well it gets difficult to fill a
42:00 - 42:30 field of view especially if you have a wide field of view at a significant distance so uh relay lenses or collimator lenses can create a virtual image at a longer working distance up to infinity and so this uh target projection system involves uh a test test chart on a light box a collimator lens and then mechanical stages that can align the unit under test
42:30 - 43:00 and also line align it to the optical axis of the collimator and also set the distance between the collimator and the light box which determines the virtual target distance which could be up to infinity so depending on the lens you use with this you can cover between a 5 degree or up to 120 degree field of view if you need to test long range beyond 120 degrees field of view then things
43:00 - 43:30 get complicated you may have to add some additional projectors uh in the corners or use multiple exposures and that that is challenging uh so next i'll talk about some of the uh resolution measurements and sharpness measurements that are in development right now right so i mentioned iso iso12233 2022 uh actually just submitted a comment for
43:30 - 44:00 that this morning and uh we're trying to finish up the standard along with the members of the working group um the iso tc42 working group 18. uh so there is a slanted edge uh uh the slanted square was used in the the older version of this iso target uh we're moving to a slanted star target for which will enable sagittal and tangential measurements closer to the corners of the image and so there's also a
44:00 - 44:30 fifth order polynomial curve fitting a two key window that's applied to the um the edges to improve the stability some and then and then the non-uniformity correction which we added you know six years ago is finally making it into the standard we actually just missed the revision of the last standard before um which we would have tried to have that improvement in and so we're getting at this go around so
44:30 - 45:00 uh in actually our next release of imatest which um we're we're going to have an alpha release available shortly uh like typically the the radial plots that imitate us have produced have just been vertical and horizontal but we're we're changing that to have a sagittal and tangential uh plots so in this case the these sagittal regions on this we're using an sfr plus
45:00 - 45:30 target here which has these slanted squares on it and so uh there isn't um there isn't really uh really great tangential and sagittal and tangential mdf in the corners but you can use you know vertical regions on the sides or horizontal regions on the top to get sagittal mtf and you can get use vertical regions on the top and bottoms or
45:30 - 46:00 horizontal regions on the sides to get tangential mtf so this um i mean the whole benefit of using using sagittal and tangential mtf is that it's going to convey what the minimum and maximum performance of your optical system are whereas vertical and horizontal are more related to the sensor so the uh i think especially for very wide field of view cameras we see the sagittal and tangential mtf
46:00 - 46:30 differ from each other more and um also the lens lens diagrams that you get are all going to be in sagittal intangible mtf so we think this is a big improvement i guess the you know something to note is that you know we want to get closer to the corner than um we are right now like you see the sagittal mtf um it's only going to about 70 of the field um the tangential mtf is going only about to um 85 percent of the field
46:30 - 47:00 so when we move to the slanted star target and we have some you know actually modifications of that slanted star that are immature specific uh we'll get those sagittal and tangential mdfs closer to the corner and um all four corners not just the um you know the sides or the top and bottom uh so uh yeah if you're interested in joining our pilot program you can go to imatest.compilot and that release will be out
47:00 - 47:30 uh probably later this week so that is the uh the end of this webinar and i am happy to take any questions uh from you and i appreciate i hope it was a good use of your time so you can follow up by contacting our support or sales or if you want to reach out to me individually there's my email address we also have a weekly office hour that uh is at different times this is the time
47:30 - 48:00 for the i guess the u.s and europe office hour we also have a kind of asia timed office hour that we have on alternating weeks uh so uh if you would like you can raise your hand ask questions actually had a comment actually i think fabrizio had submitted quite a few questions in advance maybe i can work through those um so uh yeah fabrizio i don't know if you want to commute and say this but fabrizio said many times we notice uh very different values between vertical
48:00 - 48:30 resolution horizontal resolution of the image uh what are the parameters of the image signal processing of the image to obtain the same resolution values both horizontally and vertically okay so i mean i think that there's there's two potential sources for that one would be i think lens related um differences between sagittal intentional mtf which just happened to be a lining up with the vertical and
48:30 - 49:00 horizontal which you might see in the sides uh so another thing is that you know the the non-uniform processing the sharpening it may be treating vertical different than horizontal so that is uh that's that's probably the i mean to get the same uh resolution i think you would first need to look at the the lens design and say okay uh you know is this
49:00 - 49:30 you know are we uh did we have a lower cost design that has worse mtf uh or bigger differences between sagittal and tangential um you know should we try a different lens design and uh so if you if you can get sagittal and tangential to align with each other on the lens side then you can go and um if you can't get that then you could go on the processing side and you could try to adjust them to try to equalize them
49:30 - 50:00 in signal processing that was not not treating all directions of uh information the same so maybe it would apply more sharpening to the outer portions of the field and it would apply to the center maybe it would uh apply more sharpening to the the different angles uh more than the others that's a tricky tricky question so i think you probably have quite a few more of those here so um if we have time i'll go look at the rest of your questions uh
50:00 - 50:30 and demetrio said could you please send the link of the recorder some recording summary to our emails we will send that out after the event here and uh let's see uh unless anybody else has any other questions and make sure nobody's raised their hand yeah you can raise your hand or by i think at the bottom of the display there is a was it a reaction thing you can say raise hand um
50:30 - 51:00 so [Music] i think i'm just going to find fabizio's questions and uh continue to answer them uh so give me one minute to pick uh open these up here we go uh so oh yeah so there was a questions of differences between 4k and 1080p i mean essentially it's just more pixels and then as far as different encoding there's this different kinds of
51:00 - 51:30 4k encoding as far as there's uh four four four where the luminance and chrominance channels all have the same sort of sampling and then there's other samplings where um such as 4 2 2 or 4 2 0 where there's some sort of decimation of the chroma channel so you have less less resolution for chroma
51:30 - 52:00 and then you do for luminance so i mean i think with most of our kind of monochrome grayscale sort of resolution targets you're probably not going to see much difference between these different types of 4k whether it's 444 or 420 uh you may you may have to actually have a something that has uh a transition in chroma instead of in
52:00 - 52:30 luminance so that would be something where it's like going from blue to yellow or or red to green and and you could use that to to see how those different color channels uh perform um in terms of their sharpness so yeah i didn't really get into chromatic aberration or axial axial or lateral chromatic aberration but those are all factors that can influence the um the color and you know either shift those colors
52:30 - 53:00 uh to different positions change the magnification or have have it so um you have much much better performance at some wavelengths of light than you have at other wavelengths of light so that is that is something i mean if i mean most of the mtfs we were looking at here we use the luminance channel which is a weighted combination of mostly green and some red and some blue and so that kind of green heavy channel is
53:00 - 53:30 is meant to kind of correlate with the human visual perception of luminance which very heavily favors green and also the the silicon sensors uh sensitivity which kind of peaks out in in the green region as well uh so [Music] um not sure where i'm going with this but uh yeah just just realize that uh illumi the luminance mtf that we've
53:30 - 54:00 been looking at is uh kind of a human-oriented uh um simplification of this multi-spectral mtf where not all colors are gonna perform the same uh okay uh let's see uh so uh black and white color and chess blocks um so i'm not sure what you mean by color chest blocks maybe i mean i think i think of checkerboards uh the
54:00 - 54:30 checkerboard pattern is usually what we call it uh it could be chester checkers i guess uh so uh the um i mean the great thing about the the checkerboard pattern uh is that it has a it is it has the most slanted edges of any any pattern uh let me get back to here where it was when i passed it so the um yeah this is the checkerboard pattern uh so
54:30 - 55:00 it's got tons of slanted edges it's great for distortion measurement it's probably more slanted edges than you need to be honest but it's it's something that has a it's very resistant to different um or it's very tolerant to different framing so you can go close to it you can go further away and you'll still get still get good automatic detection and um i would say as far as a target that is not good at different distances of framing sfr plus target is not great at that because you kind of
55:00 - 55:30 need the bars to be present in the target excuse me esfr iso is somewhere in between because you need to have these registration markets in the target um to to do the automatic detection um let's see here uh okay i got a question from max is there a different method to submit information for the pilot pop-up form doesn't appear to be submitting correctly when i click the join button
55:30 - 56:00 i am sorry for that uh max we will follow up we'll make sure that you're part of our pilot program i'm not sure what's going on there with our form uh so um but thank you for your interest and we'll um we'll be contacting you when the either the alpha or the beta release are ready to try out uh so fabrizio says uh uh there exists a benchmark for 4k and 1080p resolution measures to understand
56:00 - 56:30 how far we are from the best possible professional broadcast 4k and 1080p image i mean i guess that is that is a i think that's it's a hard thing to say because uh usually the best possible metric that you get out from immitus is the result of um kind of extreme amounts of signal
56:30 - 57:00 processing and sharpening so i mean this is something where certain industries uh the sharpening is you know it may produce some some halos and fringing artifacts on the edges but it you know not everybody's trying to take a beautiful picture um there's there's things like security and automotive where you don't care about these sort of artifacts you care about being able to
57:00 - 57:30 identify somebody in your in your scene you want to find you know not back over your your your kid and with your car or you want to see the bad guy in the dark so in these cases the extreme sharpening is actually a reasonable processing step to perform and so uh it's it's hard i mean because those that that processing can make the metrics go kind of off the charts then it's hard to say that oh yeah these are really good because they're
57:30 - 58:00 sharpened to heck uh and really um you want to have a good performance like for for a beautiful particular image you don't want those artifacts so maybe you don't you don't need the the resolution to be that high so it's it's something where uh we really can't say this one system is the best and and it's because i mean you may uh i mean there's some things that are done like sensors uh
58:00 - 58:30 are they have antioxidant filters on them or optical low-pass filters which are meant to kind of um spread the the information between the different pixels so that you don't have this color more color murray patterns i think we saw the color murray happen in uh this image here you could see the color marae happening um this is actually a simulated image so there was no there was no low pass filtering so you see this really ugly color marae uh so
58:30 - 59:00 uh color marae is um made worse by not having that filter uh it's made better by these really advanced forms of signal processing that take multi-image exposures and handshake and and kind of mitigate the color uh moire with that and actually actually can super resolve some so uh it's yeah i mean it's hard to say that a image is uh you know that there's a there's no
59:00 - 59:30 there's no reference imaging system there's always going to be some sort of disadvantages to any particular system and um that's why we we kind of don't get in the game of of saying hey this is what your mtf should be uh we say okay yeah maybe your mtf is really high maybe you have very high resolution sensor with tiny pixels but maybe that has introduced a big problem with noise in your system or texture result resolution so there's trade-offs across the board with any any form of
59:30 - 60:00 imaging and um or all the image quality metrics and and and so the hope where you just have a good you know what's the best uh it's it's it's a hard thing to answer so okay um let's see uh which codex salvador asks which codex can manage to get information from the file sometimes in transcoding we get wrong information um so i guess i i'm wondering what do
60:00 - 60:30 you mean by wrong information is that are those artifacts are they um i guess i'd need more information to answer that question um certainly the the transcoding uh works much better with kind of this sinusoidally modulated features than it does with uh edges sharp edges so uh yeah i would say the the problem of
60:30 - 61:00 the effects of um going over a transport stream those could definitely impact resolution uh it's something where you'd you'd have to look at many different frames not just a single frame but you know you want to look at keyframes you want to look at in between frames and and really having a still a still target is uh is is the best easiest case to resolve
61:00 - 61:30 properly and so you know you may want to have i mean if you want to challenge a codec you need to have a lot of things moving in the scene and i mean i think that's um you know a lot of times we we produce these kind of yeah sharp targets and uniformly lit but in the real world you know there's there and you know everything's gray in the background in the real world the systems are gonna be the scenes are going to be much more challenging and um so that that's where you know there may be things like a moving target
61:30 - 62:00 um that are beneficial to challenging codex um we have a slider that is part of the mts system you can find this here that where you can put a resolution target on a on the mts and um and then have uh have this kind of linear stage that moves that target back and forth um so
62:00 - 62:30 uh that's for motion blur uh that's that's a something that's valuable uh for measuring motion blur and you need to you need a moving target and you could put i think for autofocus uh the texture texture patterns are gonna be part of the next autofocus stat standard and um you'll need to have them on some sort of moving stage to um to test the autofocus um okay uh yeah uh salvador if you have more details on that i'd be happy to see
62:30 - 63:00 them um maybe this is the la we're at the hour here so maybe this is the last question here um what happens when you use a monochrome monocolor backlight like red or blue to measure mtf uh would that identify resolution of that wavelength uh uh short answer yes yes it would so uh i think you would um you would probably um i mean in in the mtf measurement in and i guess i can show you
63:00 - 63:30 quickly here i'm going to this is our new interface for immatus i will select a image here and [Music] do a quick run of the i'm going to do the esfr iso module so so here's an esf iso image and i'm going to select the esfri so analysis actually one of our features in our next
63:30 - 64:00 release is that it'll automatically recognize what kind of target your you just loaded in uh so um yeah when you run this analysis i mean the default is that it's going to use that luminance channel for mtf and so there's settings where you can i mean in the output file it will uh it will output all the different colors at least red green blue and luminance and um what we're uh what you want to do would
64:00 - 64:30 want to do in the settings is uh if you have you can you can select the channel that you're using here so see this is a little small here let's see if i can zoom in on this sorry so yeah there's a channel here where you can pick um you can pick the channel you're working with red green or blue so yeah you don't want to use luminance if you're if you have a monochrome light source um
64:30 - 65:00 or alternatively you could just i guess if you have a monochrome light source i would think that you would probably also have a monochrome sensor in which case you wouldn't have to worry about the call the channel selection because it it wouldn't uh that wouldn't be a applicable to a monochrome image uh hope that answers the question uh okay so uh the uh i guess we're at the official end of
65:00 - 65:30 the webinar uh i'm happy to answer any more questions um after uh um offline i appreciate you for joining and [Music] thank you