Google explains how it created ultrasonic features for Nest smart displays

It'll also soon work in even more ways.

What you need to know

• Google rolled out ultrasonic sensing features last month to Nest displays, something that was inspired by a Googler's older father walking closer to the display to read information.
• The feature uses inaudible sound waves to detect when a person is near or far away from the display and changes the size of the text on the screen.
• It currently works with timers, commute times, and weather. Support for reminders and appointments are coming soon.

Back in November, we reported on a new feature in Nest smart displays such as the Nest Hub and Nest Hub Max which reacts based on your proximity to the device. For example, when setting a timer, the numbers on the display become larger when you're further away from the screen.

Now, Google is detailing on its blog how this feature came to be and new ways it will be used in the near future. It turns out the feature originated after a Googler noticed how their father would walk closer to the display to read information.

Last year, I gave my 74-year-old father a Nest Hub for Christmas. Over the following months, I noticed he would often walk up to the device to read the information on the screen, because he couldn't see it easily from across the room. I wondered if other people were having the same issue.

After noticing this, the Nest team at Google began researching if it was a common occurrence among older people who use the devices. In the process, they discovered one in three people over the age of 65 have a vision-reducing eye disease, as well as millions of other people who are affected with some sort of vision impairment.

The next step was to figure out how to improve the experience for these millions of users and how to enable it. The Nest Hub doesn't feature a camera like the larger Nest Hub Max and there are also privacy concerns when it comes to a camera monitoring your home. To get around these obstacles, the engineers came up with the ingenious idea to use ultrasonic sensing similar to the echolocation used by animals with vision impairments.

Animals with low vision—like bats and dolphins—use echolocation to understand and navigate their environments. Bats emit ultrasonic "chirps" and listen to how those chirps bounce off of objects in their environments and travel back to them. In the same way, Nest Hub and Nest Hub Max emit inaudible sound waves to gauge your proximity to the device.

By using this technology, your Nest display can detect large scale movements to determine if someone is near or far away from the display, and with this information, it can then change what is shown on the screen. Not only did this improve the experience for those with low vision, but the team also received positive feedback from all testers, hearing comments such as "it just feels easier to read" regardless of their age or level of vision.

Ultrasonic sensing first rolled out for timers, commute times and weather. In the coming week, it will also begin working with reminders, appointments, and more alerts.

Big screen smart display

Nest Hub Max

$229 at B&H Photo

A Google smart display with a camera

Google's latest smart display is anything but ordinary. It stands out from the crowd with an ambient EQ display, facial recognition, and hands-free gestures to control media playback.