How RGB-IR Cameras can enhance medical devices

Diagnostic and therapeutic accuracy has significantly increased as a result of the usage of cameras in the medical device industry. Camera-enabled medical devices, such as retina scanners, automated blood analyzers, digital microscopes, and skin scanners, have greatly decreased medical personnel’s workload and improved the overall patient experience.

Advances in embedded vision and camera technology have largely contributed to this transformation, with new cameras supporting many current medical applications. RGB-IR cameras have been a game changer due to their ability to record IR and visible light data with the same camera without color corruption.

This article discusses how RGB-IR cameras, which can capture images that adhere to the stringent requirements of current medical procedures, improve the performance of medical devices.

RGB-IR cameras and their growing popularity in the medical industry

Before examining how RGB-IR cameras benefit medical and life science applications, it is important to comprehend what they are and how they operate.

Sensors that collect visible and infrared light in the same frame are used in camera modules. This frequently resulted in a deterioration in image quality due to a process known as color corruption. This occurs when the IR data interpolates with the color (or RGB) data and affects the color fidelity of the resulting image.

Two approaches have commonly been employed to address this challenge:

1. Capture RGB and IR data with two cameras
2. Using an IR cut filter during the day to prevent IR light from falling on the sensor

In the first case, the system gets bulkier and more difficult to manage. However, the IR-cut filter solves this problem.

The IR cut filter is attached to the lens and is controlled by a mechanical switch. However, this causes wear and tear, dramatically reducing the camera system’s lifetime.

RGB-IR cameras help in this situation.

Technology

Sensors with distinct RGB and IR pixels are used in RGB-IR cameras. Unlike regular camera sensors with the Bayer BGGR pattern, RGB-IR cameras use a new form of CFA (Color Filter Array) with an extra set of pixels that permits only IR light to pass through.

The CFA configuration of an RGB-IR sensor is as follows:

Figure 1. Bayer pattern. Image Credit: e-con Systems

Figure 2. RGB-IR filter pattern. Image Credit: e-con Systems

This innovative technique allows for the separation of RGB and IR data from the camera output. Numerous medical applications benefit from this, including ophthalmology, fluorescence-guided surgery, and remote patient monitoring.

The following are some of the major advantages RGB-IR cameras provide:

• Increased imaging capability in day and night conditions for continuous 24/7 image and video capture.
• Smaller form factor due to eliminating the need for two separate cameras to gather RGB and IR data.
• Improved camera lifetime due to non-mechanical switching between visible and IR imaging. This would also mean that the medical equipment linked with the camera would have a longer lifespan.
• A distinct IR channel that helps measure the amount of IR light in the RGB component for intelligent color adjustment.

Designing an embedded camera for RGB-IR imaging

As RGB IR imaging necessitates a distinct CFA, the camera system must be precisely developed. This implies that different camera components, such as the sensor, optics, and ISP, must be capable of receiving and processing images suitable for RGB-IR imaging.

The essential parameters to consider when assessing a camera for RGB-IR imaging in terms of its component features are discussed in this section.

Sensor

The camera sensor must include a pixel array that includes both RGB and IR pixels. Various sensor manufacturers, including onsemi and OmniVision, provide RGB IR sensors with specialized RGB and IR pixels.

The pixel pattern of a sensor with RGB-IR capabilities is shown below:

Figure 3. Pixel array of an RGB-IR sensor. Image Credit: e-con Systems

Optics

Most off-the-shelf lenses on the market include an IR-cut filter to restrict IR light from entering during the day. This is because most cameras still employ mechanical switches to shift between day and night settings.

However, RGB-IR imaging requires a lens with a dual bandpass filter. This ensures that light from the visible and NIR spectrums falls on the sensor.

The transmission diagram for a dual bandpass filter is provided below:

Figure 4. Transmission diagram of a dual bandpass filter. Image Credit: e-con Systems

ISP

The Image Signal Processor (ISP) is one of the most significant components in an RGB-IR camera since extracting the RGB and IR frames from the data given by the sensor requires considerable ISP-side acumen.

e-con Systems, for example, does this through its proprietary algorithm. The company’s engineers also used their ISP expertise to devise an algorithm to remove IR interference from the RGB channels to achieve the correct colors.

One of its other distinguishing features is the ability to tune the ISP to produce processed RGB and IR frames based on the needs of the host platform.

Figure 5. RGB window imaging processing. Image Credit: e-con Systems

Figure 6. IR window imaging processing. Image Credit: e-con Systems

Key medical applications where RGB-IR cameras are making a wave of change

Some of the benefits of RGB-IR cameras have already been discussed. The following section details how they impact some of the most popular camera-enabled medical applications.

Remote patient monitoring

Remote Patient Monitoring (RPM) devices have benefited the medical industry by minimizing the need for in-person patient monitoring. Using artificial intelligence, RPM systems are now powerful enough to detect patient falls and movements autonomously. These life-changing devices require cameras to collect images and videos to make all of this feasible.

One of the major features of a remote patient monitoring system is that it must be operational around the clock to enable ongoing patient monitoring. This necessitates that the camera functions flawlessly during the day and night. This is where RGB-IR cameras come in.

They allow the RPM device to take images utilizing IR imaging even at night without requiring two separate cameras or a mechanical switch to activate an IR cut filter.

Figure 7. (Left) RGB image during daytime and (Right) IR image during night-time. Image Credit: e-con Systems

Image-guided surgery

In surgeries such as laparoscopic cholecystectomy, colorectal surgeries to evaluate bowel viability in hernia cases, mapping of sentinel lymph nodes, etc., fluorescence-guided surgical techniques such as Indocyanine Green (ICG) surgery are used (source: sciencedirect.com).

The dye can become fluorescent when subjected to a wavelength of 750 to 800 nm, which is how ICG surgery works. As a result, the camera utilized must be capable of capturing images in the Near-Infrared (NIR) spectrum.

But why not observe the organs with an RGB camera and an IR light source simultaneously?

The difficulty here is the low spectral resolution. To better grasp this, consider cancer diagnosis as an example application.

In this case, capturing the fluorescent image with a standard RGB camera may not provide enough detail. On the other hand, the IR capabilities of an RGB-IR camera can be utilized to efficiently enhance the cancer diagnosis procedure. At the same time, the RGB data can be used to evaluate any anomalies discovered using the IR output. All of this may be accomplished with a single-camera system.

Figure 8. (Left) RGB image during ICG Surgery and (Right) IR image during ICG Surgery. Image Credit: e-con Systems

Ophthalmology

The quality of images provided by camera modules inside ophthalmic devices such as fundus cameras and retina scanners substantially influences diagnosis accuracy.

IR imaging is required in these devices to observe sub-retinal characteristics because the deposits appear bright and thicker compared to the optic nerve heads, retinal vessels, and choroidal vessels found in darker regions. Visible light imaging, on the other hand, is essential to ensure that the camera gets the required details of the eye.

This necessitates the use of a single camera that is capable of both visible and IR imaging. As a result, RGB-IR cameras are ideal for such applications. Medical professionals can now efficiently detect and treat eye problems using these new-generation embedded cameras, which provide visible and IR images from the same camera.

Final words

As detailed above, RGB-IR cameras have numerous advantages and are well-suited for various medical applications requiring concurrent RGB and IR imaging. At the same time, specialized NIR cameras are favored for applications that require exceptionally high NIR sensitivity and performance. Before proceeding with camera integration into the device, it is advised that users consult with an imaging professional such as e-con Systems.

About e-con Systems

e-con Systems™ is a leading OEM camera manufacturer with 18+ years of experience and expertise in embedded vision. It focuses on delivering vision and camera solutions to industries such as medical, retail, and industrial. The company’s wide portfolio of products includes MIPI camera modules, USB 3.1 Gen 1 cameras, GMSL cameras, stereo cameras, etc. It has built over 250+ product-based solutions and shipped millions of cameras around the globe. What sets the company apart is its deep expertise in building customized product designs while ensuring rapid prototyping and custom modifications in hardware and software. e-con Systems™ has close partnerships with Sony, ON Semiconductor®, Omnivision, NVIDIA, Xilinx, Socionext™, Cypress, Connect Tech, Toradex, Variscite, Toshiba, Diamond Systems, Avermedia and Texas Instruments.

Giving sight to medical applications:

e-con Systems™ offers end-to-end camera solutions to meet the needs of the medical and healthcare industry. It has a strong foothold in the medical device industry – having empowered clients to integrate unique camera solutions for medical and life science applications in ophthalmology, dentistry, dermatology, laboratory equipment, microscopy, assistive technology, point-of-care technology etc. e-con Systems also has a solid track record of working with the medical device industry leaders such as Thermofisher, Idexx, Perkin Elmer, Amwell, Welch Allyn, Seegene and others.”

Sponsored Content Policy: News-Medical.net publishes articles and related content that may be derived from sources where we have existing commercial relationships, provided such content adds value to the core editorial ethos of News-Medical.Net which is to educate and inform site visitors interested in medical research, science, medical devices and treatments.