As Yahoo! struggled to lift itself out of the dot-com slump a few years ago, it faced a big decision. Google was dazzling the world of Internet...

Share story


SAN JOSE, Calif. — As Yahoo! struggled to lift itself out of the dot-com slump a few years ago, it faced a big decision.


Google was dazzling the world of Internet search engines with its building of a massive index of Web documents.


And a Pasadena, Calif., company called Overture Services was beginning to convince the world that search-engine advertising could be wildly lucrative.


Yahoo! had neither its own search engine nor a way to fully reap the profits from Internet searching. Yahoo! executives needed to decide whether to get into the massively expensive search-engine game — and how.


So after Overture sent over a proposal about a partnership in mid-2001, the light bulbs began going on for Yahoo!’s top brass.


“When people started looking at the numbers, there was almost disbelief,” recalled Jeff Weiner, Yahoo!’s senior vice president of search and marketplace. “It’s like ‘What?! Search is capable of generating this much value?’ ”


Today, Internet searching has become a cornerstone of Yahoo!’s resurgence from the dot-com slump.


The company doesn’t break out numbers. But American Technology Research analyst Mark Mahaney estimates Yahoo!’s search-engine advertising revenue of $1.2 billion last year accounted for 58 percent of all ad revenue and 45 percent of the company’s total net revenue.


Weiner credits Chief Executive Terry Semel with much of the success. Semel took the helm of a struggling Yahoo! in May 2001 and immediately set out to refocus the company’s strategy. When the management team identified search technology as a priority, the company moved swiftly to put the right pieces in place — enabling Yahoo! to ride a new wave of online advertising.


While Google’s name is still synonymous with search, many consider Yahoo!’s search technology to be at least as good as anybody’s. And the Sunnyvale, Calif., company has pulled close to its Mountain View, Calif.-based rival in search-market share.


U.S. Internet users went to Yahoo! for 31 percent of their search queries in February, according to comScore Networks. Google captured 36 percent.


“I think they’re extremely strongly positioned,” said Chris Sherman, associate editor of Search Engine Watch. “They’re definitely on par with Google when it comes to search products. They’re creating buzz again.”


That seemed inconceivable just a few years ago. Yahoo! was struggling to emerge from a post-bubble funk. And an upstart named Google had taken the Internet search world by storm with a concept called Page Rank — which returned search results based on a Web site’s popularity — and a computing infrastructure that was building a massive index of billions of pages on the World Wide Web.


The technology was so good and popular that Yahoo! signed a contract with Google to supply Yahoo! users with search results.


Meanwhile, Overture’s system of matching ads to search-engine results was taking off as advertisers found it effective and cheap to target text ads to specific search queries.


For Yahoo!’s top executives, two areas emerged as key, Weiner said: e-mail and search. “They’re two of the most popular services on the Web, and they create tremendous value, both directly and indirectly,” he said.


Yahoo!’s management concluded that it needed to own both the search engine itself and the technology to match ads to search results.


“We said, ‘How do we make this happen in the span of a year?’ ” Weiner said. “We didn’t even have a search business unit.”



Inktomi

Yahoo! executives found a jewel in the rubble of the dot-com bust. Money-losing Inktomi of Foster City, Calif., the once high-flying search-technology company, was laying off employees and shedding business units. Yahoo! knew Inktomi well. It had once used Inktomi to provide search results.


Just days before Christmas in 2002, Yahoo! announced a $235 million deal to buy the company.


But Yahoo! was just getting started. In January 2003, it hired Overture’s vice president of search, Tim Cadogan.


Then, Semel struck a deal in July 2003 that would be the capstone of his search strategy: the $1.63 billion purchase of Overture itself.


With Inktomi, Yahoo! had a search-engine technology it could build upon. And with Overture, it had the ability to match and place ads next to search results.


Yahoo! also got two other well-regarded search engines that Overture had acquired — AltaVista and FAST, which owned the AllTheWeb search engine.



Varied strengths

Each of the companies that Yahoo! acquired had different strengths.


Inktomi knew how to build a large-scale search engine and it understood real-world search problems. AllTheWeb was good at building indexes of the Web and keeping them fresh. And AltaVista, considered the best search engine in pre-Google days, had some of the best minds in the search world.


“We got a lot of very skilled, experienced people in search technology,” Cadogan said. “We brought them together in one place with one goal, which was launching the world’s best search engine in a short amount of time.”


Former Yahoo! senior search manager Jon Glick remembers vividly a weeklong brainstorming session in fall 2003 among employees of all the search companies Yahoo! had acquired.


Meeting at the former Inktomi headquarters in Foster City, engineers presented white papers on various search technologies and debated what to include in Yahoo!’s new engine.


“There was a tremendous amount of learning, innovation and brainstorming that went on,” said Glick, who joined the new shopping search engine Become.com in February.


Many observers consider Yahoo!’s search technology to be little more than the Inktomi search engine with a pretty interface. But Glick and others say major portions of the algorithm — the core of any search engine — were rewritten from the ground up.


“It wasn’t a case of Lego-ing these things together,” Glick said. “By adding those companies together, you were adding a critical mass of brain power that was competing with Google on even terms.”



Death-march mode

Yahoo!’s search team had now expanded from fewer than 20 people to more than 200 in less than a year. It took over most of Building B on the Yahoo! campus.


Sixteen-hour work days were common. In late fall 2003, Yahoo! set a deadline for itself of February 2004 for launching its own search technology to the world.


“Everyone was in death-march mode,” Glick said. “People gave up their vacations. Everyone worked.”


Motivation came in the form of Yahoo!’s partnership with Google. Every time engineers and product managers looked at the Yahoo! search page, they were reminded that Yahoo! was relying on another company for its search results.


“It was a lot of, ‘How quickly can we displace the current partner and get our product out there,’ ” said Ken Norton, senior director of product management for Yahoo! search and a former Inktomi manager.


On Feb. 16, 2004, at 9:30 p.m. Pacific time — less than a year after it completed the Inktomi acquisition — Yahoo! flipped the switch on Yahoo! Search Technology. The early reviews were generally good.


“The impressive thing is that the atmosphere it happened in was one in which Google had convinced everyone that nobody could deliver search like they did, that they had this unique smarts and ability to do this stuff,” said Doug Cutting, a search-technology consultant. “Search is not easy; there are these little finicky things to getting search results to look right.”



Evolving industry

Yahoo! can hardly afford to rest on its laurels. The search industry is evolving rapidly beyond just ordinary Web search. Desktop search, video and image search, and the ability to find local businesses are all must-haves for a search engine today.


In fact, building its own search engine was just the first step in a far broader strategy for Yahoo! Taking a pointed jab at Google’s mission to “organize all the world’s information,” Weiner said Yahoo!’s mission is to spur the creation of more Web content.


Weiner envisions Yahoo!’s 165 million registered users — with the help of Yahoo! tools — vastly expanding the amount of information they share with each other through the Web.


When that happens, he said, the 8 billion Web documents now in search-engine indexes will look trivial.


“We have only scratched the surface,” he said. “You get into the potential of hundreds of trillions of documents. That’s the vision — to enable people to find, use, share and expand all human knowledge. Not simply organize what already exists.”