New Media? The Political Economy of Internet Search Engines

New Media? The Political Economy
of Internet Search Engines
A paper presented to
The Communication Technology Policy section
2004 Conference of the International Association of Media &
Communications Researchers (IAMCR)
Porto Alegre, Brazil, July 25-30
Elizabeth Van Couvering
London School of Economics and Political Science
Department of Media & Communications
Houghton Street
London WC2A 2AE
e.j.van-couvering@lse.ac.uk
Key words: Internet, search engines, advertising, spam, mass media, political
economy

---

Abstract
This paper presents a structural analysis of the Internet search engine market. Search engines are
a core element of the Internet media system, topping the Nielsen NetRatings Top 10 list of
online properties in every market measured and generating advertising revenues of billions of
dollars each year. Using a search engine is the most popular online activity after reading email. A
political economy approach is used to identify the relevant parties in the system, their ownership,
their exchanges, and the constraints to which they are subjected. The market is shown to be
similar to other media markets in that it is both highly concentrated and global in scope, and
funded primarily by advertising. However, there are several important differences. First,
although traditional media conglomerates operate online, they do not control the most powerful
actors such as Google and Yahoo!. Second, the advertising market is structured differently.
Advertising is normally linked to words entered as search terms by users, and this keyword-based
advertising is syndicated to other search engines, large and small, primarily by Google and
Yahoo!-owned Overture, increasing their influence. Third, the situation is complicated by new
commercial actors called search engine optimisation companies who operate within a grey market
often in opposition to search engine companies, to strengthen the rank of their clients in search
engine results. Recent studies within the computer science discipline have shown search engine
results to be systematically biased in favour of commercial sites, popular sites, and US-based sites.
With no public service mandate and little regulation, the study raises the question of whether the
current system of search provision online serves the ‘public good’.
Introduction
The political economy of communication focuses critically on what structural issues in mass
media – ownership, labour practices, professional ethics, and so on – mean for products of those
mass media and thus for society more generally. Within the new media field, analysis from a
political economy perspective has been relatively lacking, with the exception of studies of the
Internet infrastructure1. So is it the case that these factors no longer matter within the area of
Internet content? Can we take at face value the idea that the Internet makes it easy for anyone
with access to basic technology to have a voice in the new Information Society?
It can be argued that the Internet is not a mass medium in the classical sense; that the thousands,
or even millions of sites visible on the Web are not the result of an industrial production process,
and that nor do they represent a common substrate of everyday life, in the way that Silverstone,
for example, characterises television:
“Watching television and discussing television and reading about television
takes place on an hourly basis: the result of focused or unfocused, conscious
or unconscious attention. Television accompanies us as we wake up, as we
breakfast, as we have our tea and as we drink in bars. It comforts us when we
are alone. It helps us sleep…We take television for granted in a similar way to
how we take everyday life for granted” (Silverstone, 1994: 3).
1 For example, the distribution of access and more recently skills and competences, also known as the “digital divide” has been one
area that has received considerable attention (see e.g., Lazarus & Mora, 2000; Mansell, 2001; Norris, 2001)
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 2

---

It can even be argued that the Internet does not conform to a basic definition of mass
communication, such as that offered by Thompson, who says that mass communication is: “the
institutionalized production and generalized diffusion of symbolic goods via the transmission and
storage of information/communication.” (Thompson, 1990: 219). While information and
communication are certainly transmitted and stored, can we really take seriously the argument
that my friend Joe’s blog about his daily life represents either institutionalized production or
generalised diffusion? Perhaps for this reason, research that approaches the study of Internet
content from a political economy of communications perspective is rare.
In this paper I suggest that in accepting the argument that some online content is small-scale
craft production, scholars are neglecting the study of an important new medium of mass
communication as an industry. It would be similarly easy to argue that desktop printing makes
everyone a publisher, and thus that there is no need to focus on the economics of the newspaper
industry; or that as video editing and handheld cameras come within the reach of many,
Hollywood and television networks need receive no further attention. Such a line of argument
would be rightly dismissed in those contexts, and a discussion of the Internet along those terms
deserves a similar dismissal.
In considering the Internet as a medium of mass communication, there is at least one clear set of
large industrial players akin to the television networks or the Hollywood studios, and those are
the search engines. Four large players dominate the search engine market: AOL, Yahoo!,
Google, and Microsoft/MSN. Not only do these companies dominate the search-engine market,
it can be argued that they dominate the Internet advertising market as a whole. In 2003, the
Internet Advertising Bureau reported that the top 10 ad-selling companies accounted for 70% of
US online advertising revenue of $7.3 billion (PricewaterhouseCoopers, 2004). The big four
alone contributed $3.4 billion (see Table 1) although as this is total revenue United States revenue
will be somewhat less2. If 75% of their advertising revenue is earned in the United States, a
conservative estimate, then between them these four earn 32% of all advertising dollars spent
online, not just those spent on search engines or ‘portals’3.
It can even be suggested that the big four search engines dominate Internet content around the
world. They come in the top 10 Nielsen//NetRatings listings in all of the markets that Nielsen
surveys, apart from Japan (lacking Google) and Hong Kong (lacking Google and AOL). The
figure below, derived from the Nielsen//Netratings data for the month of March 2004, shows
the enormous reach of these sites globally:
2 Advertising revenues are not broken out by country in the financial reports. However, Google received 26% of its revenue overall
from international sources and Yahoo! 17%. MSN international revenues are not reported separately from Microsoft overall, nor are
AOL’s international revenues.
3 Although long-term figures are not available, in the week of May 3 to May 9, 2004, Portals and search engines were responsible for
39% of all pages with advertisement viewed by the online public (Nielsen//NetRatings, 2004).
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 3

---

Microsoft
Google
Yahoo!
Time Warner
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
en
France
Spain*
Australia
Germany
Japan*/**
Swed
Kingdom
Hong Kong*
Netherlands*
Switzerland*
United States*
United
Figure 1: The big four search engines, global reach %, March 2004
Data source: Home/Work Panels from Nielsen//NetRatings (*Home panel only, **MSN only,
not all Microsoft)
It is easy to see from the graph above how this industry, similarly to the Hollywood studios and
the television networks, is concentrated in the hands of a few large players. As further sections
of this paper will show, it has other similarities, for example a heavy reliance on advertising
revenue and a preponderance of American actors.
This paper presents a preliminary investigation of the search engine market, in terms of its
ownership, its revenues, its structure, the products it sells, its geographic spread, and the policies
and regulations which govern it. A related theme is a critical evaluation of the ways in which the
search engine market distinguishes itself from previous mass media. The paper is structured as
follows. First, it considers the general economic structure of the search engine market in terms
of business models and sources of revenue; second, it describes and compares the advertising
products sold to traditional media products; third, it discusses the phenomena of search engine
optimisation and search engine spamming; and fourth, it considers the infrastructure currently in
place to regulate search engine content.
The Public Sphere and the Public Good
In a 1997 article, Roland Bettig talks of an “enclosure” of the “electronic commons.” He
explicitly links historical processes, which deprived the common worker of the land that they
cultivated and changed them from peasants to wage labourers, to current processes of
development of the Internet. He cites three “consistent structural tendencies” which shape the
development of communication technologies. These are first, marketplace concentration;
second, expanding intellectual property rights; and third, the growing commercialisation of
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 4

---

information and cultural output, including the introduction and increase of online advertising
(Bettig, 1997: 139-140). These, he argues, are combining to “consistently undermine the
liberatory potential that may be inherent in the technology” (Bettig, 1997: 54).
By taking this line of argument, Bettig is working within the tradition of political economy in
rejecting the notion that the market alone should be the arbiter of the structure of the media
industry, as might be appropriate for other types of products. Instead, he is operating with an
assumption that the mass media are sites of public interest and public discussion, or, in short, of
the public sphere which Habermas (1992) details as an essential element of a rational and
democratic government.4
Many other less radical treatments have shown how the Internet is affecting the processes that
govern normal news media – increasing the need for speed, altering the balance of funding,
changing the way journalists report news, adding alternative viewpoints (refs). Again, the
Internet is seen within the paradigm of the media as public sphere, rather than the media as
market. The political economy perspective, however, is that the media’s effectiveness in fulfilling
its role in the public sphere is intimately connected to how media industries operate.
The political economy of communication has focused on the study of the historical, political,
economic, and material constituents of the communications industry within capitalist societies,
whilst viewing them at the same time as “integral to fundamental economic, political, social, and
cultural processes in society” (Mosco, 1996: 71). The study of the mutual constitution of
communication and late capitalist society has become more and more important as we enter an
age where speed appears to bring places closer together, drawing us into globalization and a
‘space of flows’ (Castells, 1996). Processes of communication, particularly electronic
communication, are fundamental to the current transformation of capitalism; and this is reflected
in names such as “The Information Society” or “The Knowledge Society.” A political economy
of communication, therefore, challenges the pure economic logic of this transformation and
enables us to enquire, critically, about the relation of that transformation to social class, gender,
race, and place; and more fundamentally, as Golding and Murdock stress, about “justice, equity
and the public good” (2000: 73).
Two common metaphors for search engines are the library catalogue or the Yellow Pages index
of phone numbers. In other words, search engines, viewed from a naïve perspective, bring us
objective results from the whole Web based on our query. The reality is that they are shaped at
every stage by commercial processes. The significance of this paper derives from a concern with
4 This is not to say that the ‘media as market’ perspective is opposite to the ‘media as public sphere’ perspective. Rather, it is argued
that by virtue of their function in the public sphere, the media should not be treated solely as a market.
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 5

---

what that might mean for those of us concerned with constructing an information society that is
equally accessible to all.
Search Engines as Media of Mass Communication
Taking to Thompson’s definition of mass communication as “the institutionalized production
and generalized diffusion of symbolic goods via the transmission and storage of
information/communication” (Thompson, 1990: 219), we have on the one hand, ‘symbolic
goods’ – texts, images, sounds, etc. – and on the other hand commercial or institutional
production, transmission, and storage. This section focuses specifically on the institutional
production and diffusion of search engine results, describing and contrasting them with similar
processes in other forms of mass media.
Thus, in this section and more generally within this paper, we are comparing search engines to
other forms of mass communication. As noted in the introduction, it is possible that not
everyone will agree with this characterisation. Because media markets are often characterised by
their technology of distribution – radio, television, the press, cable television – it is difficult to say
whether or not a new technology is also a new medium. Some have argued that ‘the Internet’ in
general is a medium by implicitly including it in general studies of mass media (for example,
McQuail, 2000). However, many media scholars, while acknowledging this possibility, have
specifically excluded it from the body of their recent theorizing (for example, Couldry, 2003).
Others have noted that a major feature is the commercial exchange between businesses (Mansell
& Steinmuller, 2000) and a major strand of research has argued for its function as a means of
self-expression and a new form of community. Therefore, this study takes the view, strongly
suggested by Miller and Slater (2000), among others, that ‘the Internet’ is too wide an analytical
category to be of practical use for in-depth study. Rather, the Internet is a technological
infrastructure that affords a range of uses. This paper therefore makes no argument for ‘the
Internet’ as a whole being a mass medium. Instead, it concentrates primarily on search engines,
and argues for their place alongside other mass media, while noting that core differences do exist.
Business models
Economically speaking, it is widely recognized that mass media in general operate in what is
known as a ‘dual product’ market. They produce, on the one hand, cultural commodities for
individuals. These products are normally quite tangible – books, newspapers, films, magazines.
They also produce a second product for their advertising customers. This product is the audience
(see Smythe, 1995). The percentage of revenue from each product class is different for different
types of media vehicle (Croteau & Hoynes, 2001). Books, for example, derive almost all their
revenue from individual consumers, while advertising makes up an average of 80% of a
newspaper’s total income, a substantial fraction of which comes from classified advertising
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 6

---

(Sparks, 2000), although this differs from country to country as newspaper subscription is more
common in some areas than others. Commercial television derives all of its revenue indirectly
through advertising and sponsorship while public television, on the other hand, has the state as
its primary support.
How do search engines fare when compared to these other more well-known media enterprises?
The answer is complex. Table 1 shows the percentage of revenue from advertising for Google,
Yahoo!, AOL, MSN, AskJeeves, Lycos, Lycos Europe, and LookSmart5 in 2003, derived from
their annual financial reports6.
2003 Total Revenue
2003 Advertising
% from Advertising
(Terra Lycos 2002)
Revenue
Yahoo!
$1,625,097,000
$1,326,905,0007
82%
Google
$961,874,000
$916,603,000
95%
AOL/Netscape
$8,600,000,000
$767,000,000
9%
MSN
$1,953,000,000
$563,000,000
29%
Terra Lycos
€621,971,000
€267,448,000
43%
LookSmart
$156,229,000
$140,886,000
90%
AskJeeves
$107,292,000
$102,767,000
96%
Table 1: Percentage of 2003 revenues due to advertising
This table shows a wide variation, between 95% for Google at number two in terms of
advertising revenue and 9% for AOL/Netscape at number three.
The explanation for this variation, I suggest, is as follows. The search engine business, as it has
developed, has split into two somewhat overlapping camps. First, we have those who operate
search services as their primary business (which includes Google, AskJeeves, and LookSmart).
This is what I refer to as the search model. Second are those who also provide people access to the
Internet via dial-up or broadband(which includes AOL/Netscape, MSN and Terra Lycos). This I
refer to as the access model. Finally, Yahoo!, the largest company in terms of advertising revenues,
operates on a middle ground of diversified services which include partnerships with access
providers and direct-to-subscriber services such as premium email and web-hosting services8.
Whether this points to the wider development of a mixed model is still unclear.
This division has important implications for the structure of the market. Companies in the first
category operating with the search model are heavily dependent upon the search services to bring
5 Revenue figures are unfortunately not available for Excite (privately held until March 2003, when it was acquired by AskJeeves).
6 Google figures are derived from their S-1 filing for Initial Public Offering (IPO).
7 Includes $1,199,733 in display ads, sponsorships, text-links, paid-performance, paid-includsion, search syndication and transaction
revenue sharing. Also includes £127,172 in listings fees from jobs, autos, real estate and other small-ads services.
8 Lycos Europe, a purely European provider which operates lycos.de and lycos.co.uk among other sites, shows a similar pattern,
making 52%, or €43,360,000 of its€83,674,000 revenues in 2003 from advertising. Lycos Europe depends heavily on its email and
web-hosting businesses.
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 7

---

traffic9 and, with it, advertising revenue to their websites. Control of search technology is thus
core to their businesses, and in recent years has meant that these companies have acquired a
range of companies providing these technologies. To refresh their search capabilities against the
threat of Google, AskJeeves purchased the well-regarded search engine Teoma, in 2002. Yahoo!
purchased search provider Inktomi in 2003. It also purchased Overture (formerly GoTo) in
2002, an advertising provider which already owned AltaVista and FAST, two big names in search
technology. Google and now Microsoft appear to be betting on their own internal technical
capacity, although this may change.
Businesses operating in the second category, with the access model, have found search less crucial
and have been content, in recent years, to license their technology from providers operating with
the search-based model10. This practice is known as ‘white-labelling’. Access-based models sell
services to customers – the connection via dial-up or broadband, email addresses, space for their
own website and domain names, among other things – w ith advertising as a secondary income
stream. The archetypal company in this area is AOL, which developed as an online service
provider in the early 1990s with a closed network, before the Internet was widely available. It still
remains by far the largest earner.
This dual-strategy industry has a parallel in cable television, in which cable channels such as
Discovery or TNT book and earn money through advertising, while cable operators such as
Comcast or Adelphia earn money through direct subscription (Carroll & Howard, 1998) to the
network.
Figure 2 shows the ‘white-labelling’ of search results between the major search providers and
distributors. Each circle in the figure represents a search-related website. Larger diameter circles
are owners, smaller circles touching them are subsidiaries. AOL/Netscape and MSN are publicly
traded on the New York Stock Exchange since they are owned by Time Warner and Microsoft,
respectively. Yahoo!, AskJeeves, and LookSmart are traded on the NASDAQ exchange. Terra
Lycos is traded on the Madrid exchange. At the time of writing, Google was in the process of its
initial public offering, and will also be traded on NASDAQ.
The thickness of the line around the circles indicates the volume of search traffic on the site. In
2003, Yahoo! terminated its contract for search results provision from Google, and MSN did the
same from LookSmart, and is widely expected to drop Yahoo!-owned Inktomi when its MSN
search is prepared.
9 “Traffic” is a somewhat technical term that refers to both page views and user clicks and is roughly equivalent to a term like
‘audience’ in other mass communication industries.
10 In 2003 however MSN terminated its relationship with LookSmart and Microsoft has begun to develop its own web search
technology, possibly with the intention to integrate web-search capability into its next operating system (Hu, 2003).
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 8

---

Open
Netscape AOL Directory
Look
Lycos
Smart
HotBot
Google
MSN
Teoma
Ask
Alta
Jeeves
Vista
Overture
Excite
All The
Inktomi Yahoo! Web
Figure 2: Search site ownership and white-labelling of search results, April 2004
Figure 2 shows us that there are, at present, two major providers of search results (Google and
Yahoo!) and possibly another in the wings (with Microsoft’s MSN Search). The minor providers
(AskJeeves and LookSmart being the most prominent examples) are probably targets for
acquisition.
If these acquisitions were to occur, it would be continuing a trend of consolidation that began in
the late 1990s and accelerated after the dot-com crash of 2000. In 1998, for example, in a highly-
cited article, Lawrence and Giles listed six “major search engines” upon which they based their
calculation of the coverage of search engines on the Web. These were: AltaVista, Excite,
HotBot, Northern Light, Infoseek, and Lycos. Historically, each of these engines was a separate
company, operating separate technology. Today, none of these search engines is still an
independent company (with the exception of Lycos), and none of them operates their own
search technology.
One factor encouraging this consolidation is certainly the scale that search engine providers need
to attain to be competitive. Search provision is a capital-intensive industry. The sheer size and
scale of the Internet have meant that companies that want to compete in searching the Web in
real-time must invest hugely in hardware, software, and connection capacity. One senior
program manager at a major search provider who has had a long career in the industry reflected
on this:
“The scale problem is still there, although a bunch of people say it’s not a big
deal any more. They’re kind of right, and they’re kind of wrong. It’s not a big
deal if you’ve got a bunch of money. People have figured out how to solve
the scale problem, which is to throw a bunch of machines at it and partition
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 9

---

your index…You can still scale effectively. But now, it costs you a ton of
money. This is why ever since 2000, 2001, most of the search research done
at the universities is what I call Metacrawler-esque, which is people not
building a search engine but doing something on top of a search engine,
because they just can’t afford to build their own. Which is a shame, because
you’re not getting these big engines coming out of academia any more.”11
Which set of players is more powerful in the market? The answer is very dependent upon traffic,
the commodity that fuels the online advertising industry. Large access-based search distributors,
such as AOL or Terra Lycos, generate their own traffic, primarily through the use of default
home pages for their access subscribers, and by providing other services such as email or chat
which ensures regular visits from subscribers. Other distributors, like traditional and local media
outlets such as CNN, gain traffic from huge offline promotions on their other media products.
This traffic gives large distributors a strong bargaining chip with the search providers, for whom
traffic is the major commodity. Search engine providers give such large traffic sources major
reductions on the purchase of such services, in exchange for being able to advertise directly to
the distributor’s customers. In the words of the managing director of one large search site:
“[W]hat’s happened is that people are now selling … search results tied to a
deal on paid-for placement, so you’ll get the paid-for placement quite clearly
linked, indicated it’s a paid-for placement, it’s not part of the search results,
hidden away, it’s clearly separated, but they will sell the two as a package, so as
a distributor of search one can make quite a lot of money out of it, one won’t
necessarily be paying for the search or paying a very subsidised rate, plus one
will be paid for the paid-for search.”
This dynamic – reimbursement by the large search providers for advertisements on a distributor’s
site – is extremely important when in comes to evaluating the relative positions of distributors
and providers. Advertising run by search providers can be a major source of income for both
small and large distributors, a factor which helps to consolidate the influence of the two major
search providers on the market as a whole, as we shall see below.
In this section we have seen that the search engine industry is dominated by large companies with
one of two complementary business models: either a search model, dependent upon advertising for
revenue; or an access model, dependent upon the sale of services, particularly Internet access, to
consumers. Companies that operate with the search model are search providers and own their own
technology. These companies sell results and advertisements to those operating with the access
model, who are search distributors. Other companies are also search distributors, most notably
other online media companies such as CNN or the BBC.
11 In addition to documentary analysis, this paper is based on data collected from seven interviews with engineers and others at search
providers and search distributors, including Yahoo!, MSN, AskJeeves, Lycos Europe and Nutch as well as former employees of
Excite and Webcrawler. These interviews are part of an ongoing research project. The author would like to thank the interviewees
for their kind participation in the research.
02 September 2004
New Media? A Political Economy of Search Engines
E.J. Van Couvering
Page 10

---