Artificial Intelligence Powered Video Search for Consumer Security Cameras: The Discovery of Missing Ear and Lost Things by Typing on Ring Doorbell Footage
In an ideal world, the Ring app wouldn’t tell me when I need to know if my cat is outside in the backyard or at the chicken house, so I can cut down on notification fatigue. There is a lot of use for the search function. Beyond the obvious one of looking for specific things after an emergency or incident, I could see myself using it to check whether my son had his tennis bag with him when he left for school, for example.
Ring is the first consumer security camera company to publicly offer an artificial intelligence-powered video search to all of its users. Wyze is launching a pilot program for its video search feature today, while Smart Video Search is going public. The search for its camera history was announced in August, but it hasn’t been rolled out yet.
Ring blocks searches that it considers offensive, inappropriate, or harmful, says Hamren, who oversees three other Amazon home technology units. She declined to elaborate beyond saying that names of weapons and “qualitative terms about people or situations” are among those barred.
The search uses what the company calls Ring IQ, which Eric Kuhn, GM of Ring experiences and subscriptions, explains is a combination of Ring AI technology and “Visual Language Modeling to match text to images and quickly deliver results.”
Typing “red sweater front door” brought up several images on the Ring Doorbell showing various visitors wearing red. There’s no person recognition here, though, which is a feature offered by competitors. It just sees generic people. Adding the name of the red sweater woman narrowed the results down. It does this when it runs out of relevant matches.
Security cameras are also useful for finding lost things. Yes, I have scrolled back through doorbell footage to see if I was carrying my purse when I left the house or when I couldn’t find my red sweater. This feature will save me a lot of scrolling time.
The feature can search your entire history, but there are some limitations I get into. I wanted to look for the animals in my yard as I have many late-night visitors. It was less successful than my search for a cat. It mostly surfaced videos of my cat and dog and a lot of squirrels. But a few blurry images of a possum did show up (from a Ring floodlight camera mounted up high — hence the blur).
The feature is going to be available to users in the US with the Ring Protect Pro plan. Ring says that it can look for queries relating to animals, vehicles, packages, and people by location, time, and weather, as well as by the question, “raccoons in the backyard last night.” There are various actions that it can identify.
I’ve been playing with Smart Video Search for a few days, and it’s largely worked well. It showed me where my cat Smokey had been so I knew where to look for him, showed when the UPS truck rather than the FedEx truck had been at my door, and identified a possum in my backyard. But despite receiving several Amazon deliveries (which my Ring video doorbell alerted me to), the search pulled up my rabbit hutch when I typed in “Amazon truck.” I was able to find the delivery events by searching for a person named “Amazon delivery person,” who looked like they wore a blue vest.
I have an indoor cat, and I always wonder where he is in the morning. (I’ve written about how I use security cameras as a digital cat flap.) Several videos were pulled up by typing in the name of the cat. By default, they are sorted by the most relevant, but I can choose tosort by date, which puts the most recent activity first. This shows me that there was Smokey in the backyard about 10 minutes ago.
Do I Need a Babysitter? Investigations in Hamren’s Charged Retailer and Social Media Accounts at a Cross-Centric Location
Hamren’s reign at Ring also has brought a significant shift in the company’s pitch to potential customers. ads showing suspicious characters were removed from the internet. She pulled back on partnerships with police, which had seen Ring develop tools for users to easily share videos with law enforcement agencies without them having to obtain a warrant.
Despite the fact that there was footage matching those descriptions, our searches for terms that might be sensitive yielded no results. People in wheelchairs are being pushed by mothers with strollers. Ring notes that it’s going to block searches for disabled due to the risk of misuse.
Historically, the latter option required her to tediously swipe through a video timeline in Ring’s app to identify if and when the box was picked up and by whom. But in recent weeks, Hamren has been able to let AI do the searching. She types “package today” into the app and right away can see the clip of her husband completing his duty, assuming he has.
Liz, the CEO of Amazon’s Ring camera business, often wonders if her husband remembered to grab the frozen goods that are regularly delivered to their home. She could nag him, or she could check the footage from their Ring.