A crab-inspired artificial vision system for both terrestrial and aquatic environments

To efficiently navigate real-world environments, robots typically analyze images collected by imaging devices that are integrated within their body. To enhance the performance of robots, engineers have thus been trying to develop different types of highly performing cameras, sensors and artificial vision systems.

Many artificial vision systems developed so far draw inspiration from the eyes of humans, animals, insects and fish. These systems have different features and characteristics, depending on the environment in which they are designed to operate in.

Most existing sensors and cameras are designed to work either in on the ground (i.e., in terrestrial environments) or in water (i.e., in aquatic environments). Bio-inspired artificial vision systems that can operate in both terrestrial and aquatic environments, on the other hand, remain scarce.

Researchers at the Institute for Basic Science (IBS), Seoul National University, Gwangju Institute of Science, Massachussets Institute of Technology (MIT) and Technology and the University of Texas at Austin have recently created a new vision system inspired by crabs, which can work both on the ground and in water. This amphibious system, presented in a paper published in Nature Electronics, allows robots to gain a panoramic 360° view of their surroundings, so that they can detect obstacles and navigate environments more effectively.

“Previous works (including our group’s research) on wide field of view (FoV) cameras were always at fewer than 180°, which is not enough for the ‘full’ panoramic vision, and they were not suitable for changing external environments,” Young Min Song, one of the researchers who carried out the study, told Tech Xplore. “We wanted to develop a 360° FoV camera that can image both in air and water.”

The artificial vision system developed by this team of researchers draws inspiration from the eyes of fiddler crabs. This unique species, also known as calling crabs, can gain a full panoramic view of their surroundings without having to move their eyes and body. To artificially reproduce the fiddler crab’s eyes, Min and his colleagues used a flat camera lens.

“If you use a conventional lens with curvature for imaging, its focal point changes when you dip the lens into the water,” Song explained. “On the other hand, if you use a lens with a flat surface, you can see a clear image regardless of ambient conditions. The fiddler crab living in the intertidal region has this kind of flat surface of its lens and we just imitated this crab-eye-lens.”

To create their sophisticated vision system, the researchers integrated an array of flat microlenses with a graded refractive index and an array of flexible comb-shaped silicon photodiodes on a spherical structure. The microlenses they used can retain their focal length irrespective of changes in the external refractive index between air and water.

“To the best of our knowledge, it is the first time to demonstrate amphibious and panoramic vision systems worldwide,” Song said. “Our vision system could pave the way for 360° omnidirectional cameras with applications in virtual or augmented reality or an all-weather vision for autonomous vehicles.”

Song and his colleagues tested their system in a series of optical simulations and imaging demonstrations, considering the characteristics of both terrestrial and water environments. So far, they found that it achieved highly promising results, thus it could soon be tested and implemented on several different hybrid and amphibious robots.

“In our next studies we will conduct further engineering to achieve higher resolution and superior imaging performance,” Song added. “In addition, we are still interested in developing a new type of camera with unique imaging features inspired by other animal eyes.”

© 2022 Science X Network