Sometimes it’s hard to explain what you’re searching for. That’s why AT&T Labs researchers have developed a mobile app that allows use of visuals or audio clips as search terms, instead of text. Visual Search is a powerful app that allows users to find products, consumer photos and audio tracks, using a locally captured audio or photo sample. It satisfies the “in-the-moment” need for information and content with high accuracy and low latency. For example, if you see a painting that you’re curious about, you can simply take a photo of it – the app will search for it in its database and tell you the name and artist. And it’s not limited to web searches, researchers are working to include this technology in cloud offerings so you can search your own multimedia.
As an innovator in video search, researchers at AT&T Labs began to think about different ways to not just search for what you’re looking for on the web, but also in your own personal files. “For a long time all searches were based on text – you had to have a textual descriptor of something and type a text query to find what you were looking for,” explains Labs researcher Behzad Shahraray. Recognizing the void in search capabilities, Behzad and his team set out to find a search method using multimedia, such as photos and audio. The resulting app, Visual Search, demonstrates advances in near-duplicate multimedia copy detection work of AT&T Labs.
Visual Search is currently in prototype form as researchers continue to expand its search database and functionalities. Labs researchers envision many applications for this technology, including:
- The Cloud. Organizing hundreds of family photos that are stored in the cloud is no easy task. Visual Search will allow users to search by faces and places to find that snapshot they thought was lost.
- Guides. Lost in a museum? Easy, Visual Search can provide a simple and customized guide for museum-goers as they weave their way through a series of portraits.
- Shopping. Further simplifying the search for the best deal, this technology will allow shoppers to search by snapping a photo of the packaging, rather than scanning a barcode.
Behzad Shahraray is the Executive Director of Video and Multimedia Technologies Research at AT&T Labs. In this role, he leads an effort aimed at creating advanced media processing technologies and novel multimedia communications service concepts. Behzad is the recipient of the AT&T Medal of Science and Technology and holds sixteen US patents in the areas of image, video and multimedia processing. He is a Senior Member of IEEE, a member of the Association for Computing Machinery (ACM), and is on the editorial board of the International Journal of Multimedia Tools and Applications.
Eric Zavesky joined AT&T Labs in 2009 as a subject matter expert. At AT&T, he has collaborated on several projects to bring alternative query and retrieval representations to multimedia indexing systems including object-based query, biometric representations for personal authentication and work to incorporate spatio-temporal information into near-duplicate copy detection.
Zhu Liu is a subject matter expert in multimedia content processing, multimedia databases, video search, pattern recognition, machine learning, data mining and natural language understanding. He is on the editorial board of the IEEE Transaction on Multimedia and the Peer-to-peer Networking and Applications Journal. Liu is also a senior member of IEEE.