Search Engines Get a Machine Learning Boost

Online retailer eBay is attempting to extend its machine language capabilities beyond automatic language translation to e-commerce uses designed to make product searches more relevant. As automation improves, the company said one goal eliminating the search box.

Meanwhile, development cycles have been reduced as more machine learning libraries are released to the open source community. That has stimulated innovation. “As machines get better at decoding natural language, commerce should become increasingly conversational — eventually rendering the search box redundant,” eBay CEO Devin Wenig noted recently.

Wenig added that the pace of machine intelligence development has quickened over the last year. Hence, “We already use machine learning algorithms to recognize objects in listings, find similar products and rank recommendations,” Wenig added. Along with machine translation and analyzing structured data, eBay also is applying artificial intelligence in areas such as “risk and fraud management.”

The primary focus is leveraging machine learning to help simulate human cognitive abilities for commerce applications including item-to-product matching, price prediction and item categorization to help improve the relevance of customers’ search and navigation.

Translating search queries was a key initial step as eBay sought to “localize” online shopping. The next step is leveraging machine learning to optimize searches while taking advantage of traditionally human cognitive attributes such as visual processing. “Machine learning is especially helpful at problems that require human cognitive capabilities like perception, language processing and visual processing,” eBay research scientist Selcuk Kopru noted in a company blog post. Kopru is working to integrate machine learning with eBay’s current search platform.

Specific e-commerce applications for machine learning include price prediction and item categorization tasks. The company also stressed that search has moved well beyond keyword matching. Machine learning is used to extract semantics from item titles and descriptions, for example, to boost the relevance of search results.

The company has begun scaling its machine learning efforts with an application called “Best Match,” an algorithm used to boost search relevance to connect buyers and sellers. The algorithm analyzes factors like item popularity, potential value to buyers along with service terms such as sellers’ return policies. The company said it also is using statistical learning techniques to come up with the best deals, whether it’s from an individual seller or a large retailer. The bottom line is buyers are looking for the best deal, noted eBay technical fellow David Goldberg.

Kopru added that emerging open source machine learning libraries have allowed the auction giant to speed search application development without having to implement platforms from scratch. “We can also run very complex and deep models with today’s computing clusters,” Kopru added. “These enabling technologies let us model new search experiences from many angles faster than we ever could before.”

The retailer’s approach to automating search differs from the requirements of enterprise search, where many types of data are stored in Hadoop and other database systems. Tools like machine learning are being applied in these settings to mine data for business insights.

Still, high-volume applications of machine learning such as those being deployed by eBay promise to extend the technology from search engines to higher-level business intelligence uses.

Recent items:

Rethinking Enterprise Search in the Big Data Age

Here’s Another Option For Hadoop Enterprise Search