Paradigm shift for data acquisition: Realtime up-sampling noise filter

Session

DHA.3 - Machine assisted acquisition and analysis of microscopy data

Authors

Mr. Akimitsu Ishizuka (1), Dr. Koji Kimoto (2), Dr. Kazuo Ishizuka (1)

Affiliations

1. HREM Research Inc.
2. NIMS

Keywords

Low dose image 

Noise Filter

Realtime  filter

Up-sampling

Abstract text

In microscopy, the images should be smooth and high contrast, since image quality is usually judged by visual inspection. Therefore, data acquisition is usually performed at a high-magnification to get the images that are smooth and look beautiful. However, such image data is usually oversampled. This becomes apparent when the image data is subjected to Fourier transform. Here, the signal intensity appears only in the central region of the Fourier transform. According to the sampling theorem, the image information is retained even if the signal intensity has spread to the full range of the Fourier transform. Nevertheless, the data acquired with such optimal sampling is not attractive for a human operator/ observer.However, a smooth image can be simply obtained by up-sampling with Fourier transform, when the data is obtained by satisfying the sampling theorem.

We may note that up-sampling and noise reduction can be performed simultaneously through Fourier transform. Furthermore, if the image size is moderate, for example, 512 x 512, the new up-sampling noise filter works in live. Using this live filter, the operator can align the microscope and set up experimental conditions while looking at the noise-reduced up-sampled image. Moreover, the live up-sampling noise filter is effective for low dose observation, since it will make possible for an operator to inspect a sample by eyes. Alternatively, if this up-sampling technique is used, a wider field of view can be observed using the same camera. For example, you can obtain an image that is equivalent to 4kx4k pixels using a 2kx2k camera. 

Figure 1 illustrates noise filtering of low dose TEM images of MOF taken at 300kV [1]. (a) shows an area of 2kx2k pixels acquired with a Gatan K2 camera working at the electron counting mode, while (b) an enlarged part of the ROI (512x512), where consecutive ten frames are accumulated to reveal the sample feature. The histogram in (a) indicates the low dose rate of 0.25 el/pix/frame.  An image of a single frame within the same ROI is also shown in (a), which is almost black, and does not reveal any feature, becuase most of the pixels have no intensity. The noise filtered image of this single frame is shown in (c). It may be noteworthy that the noise filtered image of the single frame (c) is better than that of the accumulated image over 10 frames (b).

Figure 2 sows an example of the up-sampling noise filter. A series of low dose STEM images (256x256 pixel) of single-layer graphene was taken at 80kV, where an average signal corresponds to only 1.6 el/pix/frame [2]. A single frame of the common area after an off-line drift correction is shown in (a), where it is difficult to observe a sample structure. However, its Fourier transform gives distinct spots due to a periodic sample structure, and indicates that this data is over-sampled more than by 4. (b) shows the image that is down-sampled by 4, and its Fourier transform. (c) is an image obtained by accumulating 300 frames, which clearly shows the sample structure. (d) is a noise filtered image of (a), while (e) is a noise filtered image of (b) with up-sampling by 4. We may note that the noise-filtered up-sampled image is almost identical to the noise-filtered oversampled image. It may be stressed that both the noise filtered images, (d and e), of single frame are comparable with the accumulated image (c). 

A recent trend of data acquisition is a live drift correction to accumulate the frames obtained with a short exposure. However, it is not a good idea to accumulate many frames especially from a beam sensitive sample. In this report, we have demonstrated that the real-time up-sampling noise filter can reveal the sample structure even from a single frame and help to perform experiments. Then, the real-time noise filter and off-line non-rigid alignment [3] is a better strategy of data acquisition than the live drift correction. The real-time up-sampling noise filter will require a drastic reassessment of existing data collection scheme. The real-time up-sampling noise filter has been implemented as a new functionality of HREM-Filters Pro [4], a plug-in for DigitalMicrograph.

Figure 1. Noise filtering for low-dose images of MOF [1]. (a) an image (2kx2k image) accumulated over ten frames to reveal the sample. Insets show a histogram corresponds to a single frame and a part of the single frame image. (b) an enlarged part of the accumulated image. (c) a noise filtered image of the part of the single frame. 

Figure 2.  Up-sampling noise filter of single-layer graphene. (a) a single frame and its Fourier transform. (b) an image binned by 4 from (a) and its Fourier transform. (c) an accumulated image over 300 frames. (d) a noise filtered image of (a). (e) a 4-time up-sampled noise filtered image of (b). 

References

[1] Data courtesy of Xiaoxiao Cao, Gatan China.

[2] S. Yamashita et al., Microscopy 64 (2015) 409. 

[3] SmartAlign: https://www.hremresearch.com/Eng/plugin/SmartAlignEng.html; L. Jones et al., Advanced Structural and Chemical Imaging (2015) 1:8. 

[4] HREM-Filters Pro: https://www.hremresearch.com/Eng/plugin/FiltersEng.html