Hourly

Analysis

of

a

Very

Large

Topically

Categorized

Web

Query

Log

We

review

a

query

log

of

hundreds

of

millions

of

queries

that

constitute

the

total

query

traffic

for

an

entire

week

of

a

general-purpose

commercial

web

search

service

Previously

query

logs

have

been

studied

from

a

single

cumulative

view

In

contrast

our

analysis

shows

changes

in

popularity

and

uniqueness

of

topically

categorized

queries

across

the

hours

of

the

day

We

examine

query

traffic

on

an

hourly

basis

by

matching

it

against

lists

of

queries

that

have

been

topically

pre-categorized

by

human

editors

This

represents

of

the

query

traffic

We

show

that

query

traffic

from

particular

topical

categories

differs

both

from

the

query

stream

as

a

whole

and

from

other

categories

This

analysis

provides

valuable

insight

for

improving

retrieval

effectiveness

and

efficiency

It

is

also

relevant

to

the

development

of

enhanced

query

disambiguation

routing

and

caching

algorithms

H.

-LSB-

Information

Storage

and

Retrieval

-RSB-

Online

Information

Services

Web-based

services

Measurement

Human

Factors

Understanding

how

queries

change

over

time

is

critical

to

developing

effective

efficient

search

services

We

are

unaware

of

any

log

analysis

that

studies

differences

in

the

query

stream

over

the

hours

in

a

day

much

less

how

those

differences

are

manifested

within

topical

categories

We

focus

on

Circadian

changes

in

popularity

and

uniqueness

of

topical

categories

Emphasis

on

changing

query

stream

characteristics

over

this

longitudinal

-LRB-

time

-RRB-

aspect

of

query

logs

distinguishes

this

work

from

prior

static

log

analysis

surveyed

in

-LSB-

-RSB-

We

began

with

the

hypothesis

that

there

are

very

different

characteristics

during

peak

hours

and

off-peak

hours

during

a

day

After

reviewing

a

week

's

worth

of

data

hundreds

of

millions

of

queries

we

have

found

not

surprisingly

that

The

number

of

queries

issued

is

substantially

lower

during

non-peak

hours

than

peak

hours

However

we

knew

little

about

how

often

queries

are

repeated

from

one

hour

of

the

day

to

the

next

After

examining

the

behavior

of

millions

of

queries

from

one

hour

of

the

day

to

the

next

we

have

found

the

less

obvious

result

The

average

number

of

query

repetitions

in

an

hour

does

not

change

significantly

on

an

hourly

basis

throughout

the

day

Most

queries

appear

no

more

than

several

times

per

hour

These

queries

consistently

account

for

a

large

portion

of

total

query

volume

throughout

the

course

of

the

day

The

queries

received

during

peak

hours

are

more

similar

to

each

other

than

their

non-peak

hour

counterparts

We

also

analyze

the

queries

representing

different

topics

using

a

topical

categorization

of

our

query

stream

These

cover

approximately

of

the

total

query

volume

We

hypothesized

that

traffic

behavior

for

some

categories

would

change

over

time

and

that

others

would

remain

stable

For

different

categories

we

examined

their

traffic

characteristics

Some

topical

categories

vary

substantially

more

in

popularity

than

others

as

we

move

through

an

average

day

Some

topics

are

more

popular

during

particular

times

of

the

day

while

others

have

a

more

constant

level

of

interest

over

time

The

query

sets

for

different

categories

have

differing

similarity

over

time

The

level

of

similarity

between

the

actual

query

sets

received

within

topical

categories

varies

differently

according

to

category

This

leads

us

to

believe

that

predictive

algorithms

that

are

able

to

estimate

the

likelihood

of

a

query

being

repeated

may

well

be

possible

This

could

have

a

significant

impact

on

future

cache

management

and

load-balancing

algorithms

Such

algorithms

could

improve

retrieval

effectiveness

by

assisting

in

query

disambiguation

making

it

easier

to

determine

what

information

need

is

being

expressed

by

a

query

at

a

given

time

They

could

also

assist

research

in

search

efficiency

that

takes

into

account

query

arrival-rate

s

-LSB-

-RSB-

Our

analysis

covers

the

entirety

of

the

tens

of

millions

of

queries

each

day

in

the

search

log

from

America

Online

over

a

complete

week

in

December

This

represents

a

population

of

tens

of

millions

of

users

searching

for

a

wide

variety

of

topics

Section

reviews

the

prior

work

in

query

log

analysis

Section

describes

our

analysis

of

overall

query

traffic

Section

describes

our

analysis

of

trends

in

categorized

queries

Finally

in

Section

we

present

our

conclusions

and

directions

for

future

work

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

or

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

SIGIR

July

Sheffield

South

Yorkshire

UK

Copyright

ACM

Examinations

of

search

engine

evaluation

indicate

that

performance

likely

varies

over

time

due

to

differences

in

query

sets

and

collections

-LSB-

-RSB-

Although

the

change

in

collections

over

time

has

been

studied

-LRB-

e.g.

the

growth

of

the

web

-RRB-

-LSB-

-RSB-

analysis

of

users

queries

has

been

primarily

limited

to

the

investigation

of

a

small

set

of

available

query

logs

that

provide

a

snapshot

of

their

query

stream

over

a

fixed

period

of

time

Prior

work

can

be

partitioned

into

static

query

log

analysis

and

some

recent

disclosures

by

web

search

engines

Query

log

analysis

can

be

partitioned

into

large-scale

log

analysis

small-scale

log

analysis

and

some

other

applications

of

log

analysis

such

as

categorization

and

query

clustering

Jansen

and

Pooch

provide

a

framework

for

static

log

analysis

but

do

not

address

analysis

of

changes

in

a

query

stream

over

time

-LSB-

-RSB-

Given

that

most

search

engines

receive

on

the

order

of

between

tens

and

hundreds

of

millions

of

queries

a

day

-LSB-

-RSB-

current

and

future

log

analysis

efforts

should

use

increasingly

larger

query

sets

to

ensure

that

prior

assumptions

still

hold

Previous

studies

measured

overall

aspects

of

users

queries

from

static

web

query

logs

In

the

only

large-scale

study

-LRB-

all

others

involve

only

a

few

million

queries

-RRB-

Silverstein

concludes

that

users

typically

view

only

the

top

ten

search

results

and

that

they

generally

enter

short

queries

from

a

static

analysis

of

an

AltaVista

query

log

from

six

weeks

in

consisting

of

million

non-empty

queries

-LSB-

-RSB-

He

also

found

that

only

of

queries

appear

more

than

three

times

the

top

queries

represent

of

the

total

query

volume

and

in

of

sessions

users

do

not

revise

their

queries

Additionally

co-occurrence

analysis

of

the

most

frequent

queries

showed

that

the

most

correlated

terms

are

often

constituents

of

phrases

No

time-based

or

topic-based

analysis

of

this

query

load

was

reported

it

does

not

provide

insight

into

how

or

when

any

usage

or

topical

interest

changes

occur

Other

studies

examine

the

effect

of

advanced

query

operators

on

the

search

service

coverage

of

Google

MSN

and

AOL

finding

that

in

general

they

had

little

effect

-LSB-

-RSB-

These

overall

statistics

do

not

provide

any

insight

into

temporal

changes

in

the

query

log

but

do

provide

some

insight

into

how

people

use

search

services

Jansen

et

al

also

provide

analysis

of

query

frequency

-LSB-

-RSB-

-LSB-

-RSB-

Their

findings

indicate

that

the

majority

-LRB-

-RRB-

of

query

terms

from

the

Excite

log

of

more

than

queries

are

used

only

once

and

a

large

majority

-LRB-

-RRB-

occur

three

times

or

less

These

studies

show

that

neither

queries

nor

their

component

terms

follow

a

Zipfian

distribution

as

the

number

of

rare

infrequently

repeated

queries

and

terms

is

disproportionately

large

Other

studies

have

focused

on

user

behavior

at

the

query

session

level

and

found

varying

results

with

some

estimating

reformulated

queries

constituting

of

queries

in

a

log

-LSB-

-RSB-

-LSB-

-RSB-

Wang

et

al

examined

a

log

of

more

than

queries

to

a

university

search

engine

from

-LSB-

-RSB-

They

find

trends

in

the

number

of

queries

received

by

season

month

and

day

We

extend

upon

this

work

by

examining

the

larger

community

of

general

web

searchers

and

analyzing

trends

corresponding

to

hour

of

day

Several

studies

examine

query

categories

in

small

static

logs

Spink

et

al

analyzed

logs

totaling

more

than

one

million

queries

submitted

to

the

Excite

web

search

engine

during

single

days

in

and

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

They

classified

approximately

queries

from

each

log

into

topical

categories

and

found

that

although

search

topics

have

changed

over

the

years

users

behaviors

have

not

Ross

and

Wolfram

categorized

the

top

term

pairs

from

the

one

million

query

Excite

log

into

subject

areas

to

show

commonalities

of

terms

in

categorie

s

-LSB-

-RSB-

Jansen

et

al

used

lists

of

terms

to

identify

image

audio

and

video

queries

and

measure

their

presence

in

the

one

million

query

Excite

log

-LSB-

-RSB-

In

order

to

examine

the

differences

in

queries

from

users

in

different

countries

Spink

et

al

examined

a

query

log

from

the

FAST

web

search

engine

during

believed

to

be

used

largely

by

Europeans

at

that

time

classifying

queries

from

it

into

the

same

topical

categories

They

found

differences

between

FAST

and

Excite

in

the

topics

searched

fo

r

-LSB-

-RSB-

Other

work

manually

grouped

queries

by

task

Broder

defines

queries

as

informational

navigational

or

transactional

and

presents

a

study

of

AltaVista

users

via

a

popup

survey

and

manual

categorization

of

queries

from

a

log

-LSB-

-RSB-

Beitzel

et

al

implicitly

categorized

queries

from

a

search

log

as

navigational

by

matching

them

to

edited

titles

in

web

directories

to

automatically

evaluate

navigational

web

search

-LSB-

-RSB-

Xie

and

Wolfram

automatically

categorized

query

terms

by

using

results

from

web

search

engines

to

assign

the

terms

to

broad

subject

categorie

s

-LSB-

-RSB-

Several

studies

of

query

caching

examine

query

frequency

distributions

from

a

static

log

focusing

on

the

average

likelihood

of

an

arbitrary

query

being

repeated

over

the

entire

fixed-length

log

Lempel

and

Moran

evaluated

the

performance

of

caching

strategies

over

a

log

of

seven

million

queries

to

AltaVista

in

and

found

that

the

frequencies

of

queries

in

their

log

followed

a

power

law

-LSB-

-RSB-

Eiron

and

McCurley

compared

query

vocabulary

from

a

log

of

nearly

million

queries

posed

to

a

corporate

intranet

to

the

vocabulary

of

web

page

anchor

text

and

found

that

the

frequency

of

queries

and

query

terms

follows

a

tail-heavy

power

law

-LSB-

-RSB-

Xie

and

O'Hallaron

studied

query

logs

from

the

Vivisimo

meta-search

engine

of

queries

over

one

month

in

in

comparison

to

the

Excite

log

of

million

over

one

day

in

and

found

that

although

as

in

other

studies

over

half

of

the

queries

are

never

repeated

the

frequencies

of

queries

that

are

repeated

do

follow

a

Zipfian

distribution

-LSB-

-RSB-

Saraiva

et

al

evaluated

a

two-level

caching

scheme

on

a

log

of

over

queries

to

a

Brazilian

search

engine

and

found

that

query

frequencies

follow

a

Zipf-like

distri

bution

-LSB-

-RSB-

Markatos

simulated

the

effect

of

several

types

of

query

caches

on

an

Excite

query

log

of

approximately

one

million

queries

and

found

that

traditional

caching

methods

provide

significant

improvements

in

efficiency

-LSB-

-RSB-

Although

traditional

MRU-style

caches

obviously

enhance

throughput

by

exploiting

temporal

locality

at

the

minute-to

minute

level

these

studies

do

not

examine

changes

in

the

query

stream

according

to

the

hour

of

the

day

that

may

be

leveraged

in

enhanced

cache

design

It

is

well

known

that

different

users

represent

the

same

information

need

with

different

query

terms

making

query

clustering

attractive

when

examining

groups

of

related

queries

However

as

Raghavan

and

Sever

have

shown

traditional

similarity

measures

are

unsuitable

for

finding

query-to-query

similarity

-LSB-

-RSB-

Wen

et

al

incorporated

click-through

to

cluster

users

queries

-LSB-

-RSB-

In

evaluating

their

system

they

analyzed

a

random

subset

of

queries

from

a

single

month

of

their

approximately

queries-per-week

traffic

They

found

that

the

most

popular

queries

represent

only

clusters

of

queries

using

differing

sets

of

query

terms

Many

web

search

services

have

begun

to

offer

views

of

the

most

popular

and/or

changing

-LRB-

becoming

drastically

more

or

less

popular

-RRB-

queries

AOL

Member

Trends

Yahoo

Buzz

Index

Lycos

The

Lycos

with

Aaron

Schatz

Google

Zeitgeist

AltaVista

Top

Queries

Ask

Jeeves

Fast

-LRB-

AllTheWeb

-RRB-

These

views

necessarily

incorporate

a

temporal

aspect

often

showing

popular

queries

for

the

current

time

period

and

those

that

are

consistently

popular

Some

also

break

down

popularity

by

topical

categories

Systems

seeking

to

display

changing

queries

must

address

the

issue

of

relative

versus

absolute

change

in

a

query

's

frequency

to

find

queries

whose

change

is

interesting

not

simply

a

query

that

went

from

frequency

one

to

two

-LRB-

a

jump

-RRB-

or

one

that

went

from

to

-LRB-

a

absolute

change

-RRB-

We

examine

a

search

log

consisting

of

hundreds

of

millions

of

queries

from

a

major

commercial

search

service

over

the

seven-day

period

from

through

This

log

represents

queries

from

approximately

million

users

We

preprocess

the

queries

to

normalize

the

query

strings

by

removing

any

case

differences

replacing

any

punctuation

with

white

space

-LRB-

stripping

advanced

search

operators

from

the

approximately

of

queries

containing

them

-RRB-

and

compressing

white

space

to

single

spaces

The

average

query

length

is

terms

for

popular

queries

and

terms

over

all

queries

On

average

users

view

only

one

page

of

results

of

the

time

two

pages

and

three

or

more

of

the

time

First

we

examine

trends

in

the

query

stream

as

a

whole

and

then

focus

on

trends

related

to

queries

manually

categorized

into

topical

categories

We

begin

our

analysis

of

the

overall

stream

by

examining

how

the

volume

of

query

traffic

changes

as

we

move

from

peak

to

non-peak

hours

We

show

the

percentage

of

the

day

's

total

and

distinct

number

of

queries

for

each

hour

in

the

day

on

average

over

our

seven-day

peri

od

in

Figure

-LRB-

all

times

in

our

query

log

are

Eastern

Standard

Time

-RRB-

Only

of

the

day

's

total

queries

appear

from

whereas

of

the

day

's

queries

appear

from

Perhaps

more

interestingly

the

ratio

of

distinct

to

total

queries

in

a

given

hour

is

nearly

constant

throughout

the

day

This

shows

that

the

average

number

of

times

a

query

is

repeated

is

virtually

constant

over

the

hours

in

a

day

remaining

near

with

only

a

standard

deviation

Although

the

average

repetition

of

queries

remains

nearly

constant

we

can

examine

this

in

greater

detail

by

measuring

the

frequency

distribution

of

queries

at

various

hours

in

the

day

as

seen

in

Figure

From

this

analysis

it

is

clear

that

the

vast

majority

of

queries

in

an

hour

appear

only

one

to

five

times

and

that

these

rare

queries

consistently

account

for

large

portions

of

the

total

query

volume

throughout

the

course

of

the

day

Figure

Although

we

have

shown

that

the

query

distribution

does

not

change

substantially

over

the

course

of

a

day

this

does

not

provide

insight

into

how

the

sets

of

queries

vary

from

one

hour

to

the

next

To

examine

this

we

measure

the

overlap

between

the

sets

of

queries

entered

during

those

hours

We

use

traditional

set

and

bag

overlap

measures

as

gi

ven

in

Equation

and

Equation

respectively

Distinct

overlap

measures

the

similarity

between

the

sets

of

unique

queries

from

each

hour

while

overall

-LRB-

bag

-RRB-

overlap

measures

the

similarity

of

their

frequency

distributions

by

incorporating

the

number

of

times

each

query

appears

in

an

hour

-RRB-

-LRB-

A

q

C

i

While

these

measures

examine

the

similarity

of

the

sets

of

queries

received

in

an

hour

and

the

number

of

times

they

are

entered

they

do

not

incorporate

the

relative

popularity

or

ranking

of

queries

within

the

query

sets

To

examine

this

we

also

measure

the

Pearson

correlation

of

the

queries

frequencies

As

can

be

seen

from

Equation

-LRB-

where

-RRB-

-LRB-

A

q

C

is

the

mean

number

of

query

repetitions

in

period

A

and

-RRB-

-LRB-

A

q

C

s

is

the

standard

deviation

of

all

the

query

frequencies

in

period

A

-RRB-

this

measures

the

degree

of

linear

correlation

between

the

frequencies

of

the

queries

in

each

hour

so

two

hours

that

had

exactly

the

same

queries

with

exactly

the

same

frequencies

would

have

a

correlation

of

one

Note

that

this

normalizes

for

the

effect

of

differing

query

volume

i.e.

the

correlation

of

two

hours

with

exactly

the

same

underlying

query

distributions

simply

scaled

by

a

constant

would

also

have

a

correlation

of

one

Figure

Percentage

of

Average

Daily

Query

Traffic

at

Each

Hour

Hour

of

Day

P

e

r

c

en

t

a

g

e

o

f

D

a

i

l

y

Qu

er

y

T

r

af

f

i

c

Average

Total

Queries

Average

Distinct

Queries

Frequency

Distribution

of

Selected

Hours

from

Frequency

Ranges

P

erc

en

t

a

g

e

o

f

T

o

t

a

l

Q

u

er

i

e

s

B

A

B

A

B

A

overlap

dist

-RRB-

-LRB-

Equation

Distinct

Overlap

of

Query

Sets

from

Hours

A

and

B

B

A

q

i

i

B

q

i

A

q

i

B

A

q

i

i

i

i

i

i

B

q

C

A

q

C

B

q

C

A

q

C

B

q

C

A

q

C

B

A

overlap

-RRB-

-RRB-

-LRB-

-RRB-

-LRB-

min

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-LRB-

min

-LRB-

-RRB-

-LRB-

Equation

Overall

Overlap

of

Query

Sets

from

Hours

A

and

B

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-LRB-

B

q

C

A

q

C

n

i

i

i

B

A

s

s

B

q

C

B

q

C

A

q

C

A

q

C

n

r

Equation

Pearson

Correlation

of

Query

Frequencies

from

Hours

A

and

B

Figure

In

Figure

we

examine

the

average

level

of

overlap

and

correlation

between

the

query

sets

received

during

the

same

hour

for

each

day

over

our

week

As

measuring

overlap

over

the

set

of

all

queries

appearing

in

our

week

would

be

computationally

expensive

we

use

the

set

of

all

the

tens

of

millions

of

queries

in

the

day

after

our

seven-day

period

as

an

independent

sample

and

measure

overlap

at

each

hour

in

our

week

of

the

queries

matching

those

in

that

sample

Although

we

previously

saw

that

the

frequency

distribution

of

queries

does

not

substantially

change

across

hours

of

the

day

Figure

shows

that

the

similarity

between

the

actual

queries

that

are

received

during

each

hour

does

in

fact

change

This

trend

seems

to

follow

query

volume

which

is

apparent

if

we

sort

the

same

overlap

data

by

query

volume

as

is

done

in

Figure

Clearly

as

query

volume

increases

the

queries

that

compose

that

traffic

are

more

likely

to

be

similar

across

samples

of

those

peak

time

periods

This

finding

is

consistent

with

prior

analyses

of

web

query

caches

showing

they

significantly

improve

performance

under

heavy

load

The

more

redundancy

they

are

able

to

detect

the

more

caching

algorithms

are

able

to

enhance

throughput

Although

the

prior

work

primarily

measures

the

effect

of

this

redundancy

in

cache

performance

it

is

obvious

that

redundancy

must

exist

and

be

detected

for

caching

to

succeed

By

examining

the

overall

query

stream

by

hour

we

are

able

to

infer

the

effectiveness

of

general

caching

algorithms

at

those

times

Figure

In

Section

we

analyzed

the

entire

query

log

However

this

blanket

view

of

the

query

traffic

does

not

provide

insight

into

the

characteristics

of

particular

categories

of

queries

that

might

be

exploited

for

enhanced

efficiency

or

effectiveness

For

example

a

search

provider

who

returns

specialized

results

for

entertainment

queries

can

not

determine

from

general

query

traffic

alone

whether

a

given

query

is

more

likely

to

be

referring

to

entertainment

related

content

or

how

to

best

process

and

cache

that

query

The

remainder

of

our

analysis

focuses

on

trends

relating

to

topical

category

of

queries

Our

query

set

is

categorized

simply

by

exactly

matching

queries

to

one

of

the

lists

corresponding

to

each

category

These

lists

are

manually

constructed

by

editors

who

categorize

real

users

queries

generate

likely

queries

and

import

lists

of

phrases

likely

to

be

queries

in

a

category

-LRB-

e.g.

cities

in

the

US

for

the

US

Sites

category

-RRB-

Queries

that

match

at

least

one

category

list

comprise

of

the

total

query

traffic

on

average

This

represents

millions

of

queries

per

day

Figure

To

verify

that

our

defined

category

lists

sufficiently

cover

the

topics

in

the

query

stream

we

manually

classified

a

random

sample

of

queries

assigning

them

to

Other

if

they

did

not

intuitively

fit

into

an

existing

category

as

can

be

seen

in

Figure

To

determine

the

number

of

queries

required

to

achieve

a

representative

sample

we

calculate

the

necessary

sample

size

in

queries

ss

-LRB-

z

-RRB-

where

z

is

the

confidence

level

value

is

Sorted

Average

Overlap

Characteristics

from

that

Matched

Each

Hour

Hour

of

Day

P

e

rcen

t

a

g

e

Overlap

Distinct

Overlap

Pearson

Sampled

Categorized

Query

Stream

Breakdown

Personal

Finance

Computing

Research

Learn

Entertainment

Games

Holidays

Home

US

Sites

Porn

Shopping

Sports

T

ravel

Other

Health

Average

Overlap

Characteristics

of

Matching

Queries

from

Hour

of

Day

Overlap

Distinct

Overlap

Pearson

the

sample

standard

deviation

and

is

the

error

rate

By

setting

our

confidence

level

to

and

error

rate

to

we

require

a

sample

of

queries

The

relative

percentages

for

each

category

of

the

approximately

of

query

volume

that

match

any

category

list

over

our

week

-LRB-

s

ee

Figure

-RRB-

are

within

the

error

rate

of

those

from

our

manually

categorized

sample

This

shows

that

our

lists

are

a

reasonable

representation

of

these

topical

categories

We

focus

on

a

subset

of

these

categories

and

examine

music

and

movies

independent

of

other

entertainment

queries

The

relative

size

of

each

category

list

we

used

is

given

in

Figure

Obviously

not

all

queries

listed

actually

match

those

entered

by

users

especially

when

the

category

contains

large

imported

lists

of

phrases

Figure

Although

we

have

shown

that

our

lists

are

a

fair

representation

of

the

topics

in

the

query

stream

this

does

not

indicate

what

portion

of

the

frequency

distribution

of

that

stream

they

represent

To

determine

this

we

measured

the

average

proportion

of

queries

matching

any

category

list

that

appear

at

various

frequencies

each

hour

and

compared

them

to

the

average

overall

hourly

frequency

distribution

of

the

query

stream

-LRB-

see

Figure

-RRB-

Unsurprisingly

this

comparison

shows

that

queries

in

the

category

lists

represent

more

popular

repeated

queries

than

average

although

the

general

shape

of

the

distributions

is

similar

Figure

Trends

in

Category

Popularity

We

begin

our

temporal

analysis

of

topical

categories

by

measuring

their

relative

popularity

over

the

hours

in

a

day

First

we

examine

the

percent

of

total

query

volume

matching

a

selected

group

of

category

lists

as

can

be

seen

in

Figure

It

is

clear

that

different

topical

categories

are

more

and

less

popular

at

different

times

of

the

day

Personal

finance

for

example

becomes

more

popular

from

while

music

queries

become

less

popular

Although

it

is

difficult

to

compare

the

relative

level

of

popularity

shift

from

one

category

to

another

due

to

the

differences

in

scale

of

each

of

their

percentages

of

the

query

stream

it

is

clear

that

some

categories

popularity

changes

more

drastically

throughout

the

day

than

others

Figure

In

order

to

quantify

this

we

calculated

the

KL-divergence

-LRB-

Equation

-RRB-

between

the

likelihood

of

receiving

any

query

at

a

particular

time

and

the

likelihood

of

receiving

a

query

in

a

particular

category

as

can

be

seen

in

Figure

This

reveals

that

the

top

three

categories

in

terms

of

popularity

are

pornography

entertainment

and

music

-RRB-

-LRB-

-RRB-

-LRB-

log

-RRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-LRB-

-LRB-

t

c

q

p

t

q

p

t

q

p

t

c

q

p

t

q

p

D

q

Equation

KL-Divergence

of

Query

Occurrence

Likelihood

for

Category

c

and

Total

Stream

at

Time

t

Figure

Comparing

these

divergences

to

the

proportion

of

categorized

queries

in

each

category

in

Figure

quickly

illustrates

that

divergence

is

not

correlated

with

the

number

of

queries

categorized

in

each

category

Also

shown

in

Figure

is

the

average

percentage

of

the

entire

query

volume

and

distinct

queries

that

match

each

category

Although

the

categories

that

cover

the

largest

portions

of

the

query

stream

also

have

the

most

relative

popularity

fluctuation

this

correlation

does

not

continue

throughout

all

categories

Relative

Percentage

of

Categorized

Queries

Shopp

ing

Com

puti

ng

Trave

l

Hom

e

Hea

lth

Gov

ernm

ent

Games

Rese

arch

L

earn

ing

Porn

Holiday

s

Spor

ts

Mov

ies

Pers

onal

Fina

nce

Ente

rtain

men

t

US

S

ites

Music

P

e

r

c

e

n

t

a

ge

of

C

a

t

e

gor

i

z

e

d

Q

u

e

r

i

e

s

Hourly

Frequency

Distribution

of

Matching

Queries

vs.

All

Queries

Averaged

over

Days

and

Categories

Frequency

Ranges

P

e

r

cent

a

g

e

of

A

v

er

age

T

o

t

a

M

a

t

c

hi

ng

Q

u

er

i

e

s

Avg

Matching

Queries

Avg

Queries

Category

Percentage

of

Entire

Query

Stream

and

Divergence

from

Likelihood

of

any

Query

at

Each

Hour

Com

puti

ng

Spo

rts

Holi

day

s

Rese

arch

and

L

earni

ng

Hea

lth

Gam

es

US

S

ites

Shopp

ing

Gov

ernm

ent

Mov

ies

Tra

vel

Pers

ona

l

Fin

ance

Home

Music

Ente

rtain

men

t

Porn

Category

KL-Divergence

of

query

stream

Distinct

of

query

stream

Categorical

Percent

over

Time

Hour

of

Day

Entertainment

Games

Health

Personal

Finance

Shopping

Music

USSites

Porn

We

drilled

down

into

the

highly

fluctuating

categories

and

examined

the

behavior

of

the

queries

with

the

most

highly

fluctuating

frequencies

in

each

category

From

this

we

hoped

to

gain

some

insight

into

the

reasons

why

certain

categories

fluctuate

and

the

effect

of

terms

and

queries

with

very

high

flux

on

those

categories

For

example

the

three

most

changing

queries

for

the

entertainment

category

on

average

over

our

week

were

Table

Top

Three

Fluctuating

Entertainment

Queries

gwyneth

paltrow

paris

hilton

orlando

bloom

All

three

of

these

queries

are

specifically

related

to

recent

events

in

US

popular

culture

the

actress

Gwyneth

Paltrow

recently

married

in

secret

and

the

news

of

her

nuptials

broke

during

the

week

we

analyzed

Hilton

Hotel

heiress

Paris

Hilton

has

been

a

popular

topic

recently

she

starred

in

a

prime

time

reality

TV

show

entitled

The

Simple

Life

Also

popular

is

Orlando

Bloom

the

actor

who

portrays

a

popular

character

in

the

Lord

of

the

Rings

trilogy

As

the

final

installment

of

the

series

was

released

in

US

theatres

during

the

week

prior

to

our

query

log

it

is

no

surprise

to

see

his

name

as

a

top-changing

query

Drilling

down

further

we

pinpointed

some

of

the

specific

instances

where

these

popular

queries

jumped

the

most

For

example

in

the

afternoon

of

Friday

December

the

popularity

of

the

query

gwyneth

paltrow

skyrocketed

From

it

occurred

once

from

it

occurred

times

and

from

it

occurred

times

The

top

changing

-LRB-

on

average

-RRB-

twenty-five

queries

after

normalization

in

the

Entertainment

and

Music

categories

are

shown

in

Table

Table

Top

Fluctuating

Queries

from

Music

and

Entertainment

Music

Entertainment

lyrics

music

britney

spears

furniture

love

hilary

duff

good

charlotte

sloppy

seconds

jessica

simpson

eminem

christina

aguilera

simple

plan

justin

timberlake

free

music

linkin

park

michael

jackson

beyonce

jennifer

lopez

cent

kinky

napster

chic

tupac

blink

gwyneth

paltrow

paris

hilton

orlando

bloom

espn

disney

johnny

depp

much

music

disney

channel

hgtv

disneychannel

com

www

disneychanel

com

katie

holmes

pictures

pamela

anderson

cartoon

network

hilary

duff

fake

chad

michael

murray

vivica

a

fox

disneychannel

care

bears

sailor

moon

www

cartoonnetwork

com

days

of

our

lives

charmed

tom

welling

We

also

looked

at

some

of

the

most

frequently

changing

terms

to

see

how

they

relate

to

the

change

of

entire

queries

containing

those

terms

Some

excellent

examples

of

this

behavior

in

the

Entertainment

category

include

the

terms

pictures

-LRB-

the

tenth-most

changing

term

-RRB-

and

duff

-LRB-

the

th

most

changing

term

-RRB-

We

looked

at

the

popularity

change

-LRB-

i.e.

change

in

frequency

-RRB-

for

queries

containing

these

terms

and

found

that

several

of

them

also

exhibited

large

changes

over

time

For

example

on

the

afternoon

of

December

th

from

noon

to

EST

the

query

hilary

duff

changed

from

an

initial

frequency

of

from

to

a

peak

of

-LRB-

from

-RRB-

and

then

stabilized

around

for

the

rest

of

the

evening

similar

spikes

in

frequency

for

this

query

occurred

at

similar

times

during

other

days

in

our

period

of

study

Trends

in

Uniqueness

of

Queries

Within

Categories

Although

we

have

shown

that

different

categories

have

differing

trends

of

popularity

over

the

hours

of

a

day

this

does

not

provide

insight

into

how

the

sets

of

queries

within

those

categories

change

throughout

the

day

In

order

to

examine

this

we

return

to

the

overlap

measures

used

in

Sec

tion

Overlap

distinct

overlap

and

the

Pearson

correlation

of

query

frequencies

for

Personal

Finance

and

Music

are

show

n

in

Figure

a

nd

Figure

Figure

Although

the

uniqueness

of

queries

in

categories

in

general

appears

to

be

correlated

with

that

of

the

entire

query

stream

-LRB-

Figure

-RRB-

that

of

particular

categories

appears

to

be

substantially

different

from

one

to

the

next

For

example

if

we

compare

the

overlap

characteristics

of

personal

finance

with

those

of

music

we

see

they

are

quite

different

Not

only

does

personal

finance

have

generally

higher

overlap

but

it

has

a

much

higher

overall

overlap

than

distinct

overlap

whereas

they

are

nearly

equal

for

music

Other

categories

with

generally

high

overlap

and

distinct

overlap

are

shopping

computing

and

travel

Also

the

correlation

of

frequencies

of

personal

finance

queries

is

very

high

all

day

indicating

searchers

are

entering

the

same

queries

roughly

the

same

relative

amount

of

times

this

is

clearly

not

true

for

music

Some

categories

have

a

high

Pearson

correlation

This

indicates

that

a

significant

portion

of

the

queries

in

these

categories

is

often

ranked

similarly

by

frequency

These

categories

are

pornography

travel

research

and

learning

and

computing

and

their

Pearson

correlations

are

illustrate

d

in

Figure

Personal

Finance

Overlap

Overlap

Dist

Olap

Pearson

Figure

It

is

clear

that

some

categories

have

very

similarly

ranked

queries

by

frequency

throughout

the

day

while

others

vary

dramatically

according

to

query

volume

Referring

back

to

Figure

and

Figure

uniqueness

of

queries

in

particular

categories

does

not

appear

to

be

correlated

with

the

number

of

queries

in

their

respective

category

lists

the

proportion

of

the

query

stream

they

represent

or

the

number

of

distinct

queries

they

match

Figure

This

type

of

data

is

potentially

of

great

use

to

query

caching

algorithms

For

example

if

it

is

known

a

priori

that

queries

for

certain

categories

are

similarly

ranked

throughout

the

day

they

can

be

given

higher

priority

in

a

query-caching

scheme

Similarly

queries

in

categories

whose

rankings

change

vastly

over

time

might

be

given

low

caching

priority

This

study

focuses

on

investigating

the

nature

of

changes

in

the

query

stream

of

a

very

large

search

service

over

time

Understanding

how

users

queries

change

over

time

is

critical

to

developing

effective

efficient

search

systems

and

to

engineering

representative

test

sets

and

evaluations

that

drive

this

development

In

this

study

we

find

trends

over

time

that

are

stable

despite

continuing

fluctuation

in

query

volume

Although

the

average

query

is

repeated

only

twice

during

any

given

hour

of

the

day

the

total

query

traffic

varies

both

in

magnitude

from

one

hour

to

the

next

and

also

in

degree

of

overlap

and

correlation

in

popularity

of

the

queries

that

are

received

In

addition

we

also

find

that

the

frequency

distribution

of

an

hour

's

worth

of

queries

remains

constant

throughout

the

day

Also

at

the

most

general

level

we

find

that

query

volume

is

highest

and

query

sets

are

most

stable

during

peak

hours

of

the

day

This

study

further

investigates

changes

in

the

query

stream

over

time

by

examining

the

nature

of

changes

in

popularity

of

particular

topical

categories

For

this

we

use

a

set

of

topical

categories

created

by

human

editors

that

represents

approximately

of

the

average

query

traffic

We

show

that

popularity

of

some

of

these

categories

fluctuates

considerably

while

other

categories

remain

relatively

stable

over

the

hours

in

a

day

Additionally

we

show

that

the

overlap

and

correlation

in

popularity

of

the

queries

within

each

topical

category

varies

quite

differently

over

the

course

of

the

day

Extending

this

analysis

to

investigate

changes

in

the

very

rare

queries

not

often

matched

by

our

category

lists

would

provide

insight

into

whether

those

are

changing

similarly

to

more

popular

queries

One

method

for

approaching

this

might

be

to

incorporate

automatic

query

classification

methods

to

extend

our

basic

lists

This

study

is

the

gateway

to

a

large

and

diverse

body

of

future

work

Integrating

this

knowledge

of

Circadian

changes

in

the

query

stream

by

category

will

likely

yield

improved

query

disambiguation

query

caching

and

load

balancing

algorithms

-LSB-

-RSB-

Beitzel

S.

Jensen

E.

Chowdhury

A.

and

Grossman

D.

Using

Titles

and

Category

Names

from

Editor-driven

Taxonomies

for

Automatic

Evaluation

In

Proceedings

of

CIKM

-LRB-

New

Orleans

LA

November

-RRB-

ACM

Press

-LSB-

-RSB-

Broder

A

A

Taxonomy

of

Web

Search

SIGIR

Forum

-LRB-

-RRB-

-LRB-

Fall

-RRB-

-LSB-

-RSB-

Chowdhury

A.

G.

Pass

Operational

Requirements

for

Scalable

Search

Systems

In

Proceedings

of

CIKM

-LRB-

New

Orleans

LA

November

-RRB-

ACM

Press

-LSB-

-RSB-

Eastman

C.

B.

Jansen

Coverage

Relevance

and

Ranking

The

Impact

of

Query

Operators

on

Web

Search

Engine

Results

ACM

Transactions

on

Information

Systems

Vol

No.

October

Pages

-LSB-

-RSB-

Eiron

N.

K.

McCurley

Analysis

of

Anchor

Text

for

Web

Search

In

Proceedings

of

SIGIR

-LRB-

Toronto

Canada

July

-RRB-

ACM

Press

-LSB-

-RSB-

Hawking

D.

Craswell

N.

and

Griffiths

K.

Which

Search

Engine

is

Best

at

Finding

Online

Services

In

Proceedings

of

-LRB-

Hong

Kong

May

-RRB-

Posters

Actual

poster

available

as

ter.pdf

-LSB-

-RSB-

Jansen

B.

and

Pooch

U

A

review

of

Web

searching

studies

and

a

framework

for

future

research

Journal

of

the

American

Society

for

Information

Science

and

Technology

-LRB-

-RRB-

-LSB-

-RSB-

Jansen

B.

Spink

A.

and

Saracevic

T.

Real

life

real

users

and

real

needs

a

study

and

analysis

of

user

queries

on

the

web

Information

Processing

and

Management

-LRB-

-RRB-

-LRB-

-RRB-

-LSB-

-RSB-

Jansen

B.J.

Goodrum

A.

Spink

A.

Searching

for

multimedia

video

audio

and

image

Web

queries

World

Wide

Web

-LRB-

-RRB-

-LSB-

-RSB-

Lawrence

S.

and

Giles

C.L.

Searching

the

World

Wide

Web

Science

-LRB-

-RRB-

Pearson

Correlations

of

Frequencies

for

Categories

Personal

Finance

Music

Movies

Porn

Computing

Games

Entertainment

Government

Music

Overlap

Overlap

Dist

Olap

Pearson

-LSB-

-RSB-

Lempel

R.

and

Moran

S.

Predictive

caching

and

prefetching

of

query

results

in

search

engines

In

Proceedings

of

-LRB-

Budapest

May

-RRB-

-LSB-

-RSB-

Markatos

E.P.

On

Caching

Search

Engine

Query

Results

In

the

Proceedings

of

the

International

Web

Caching

and

Content

Delivery

Workshop

May

-LSB-

-RSB-

Raghavan

V.

and

Sever

H.

On

the

Reuse

of

Past

Optimal

Queries

In

Proc

of

the

SIGIR

Conference

Seattle

WA

July

-LSB-

-RSB-

Ross

N.

and

Wolfram

D.

End

user

searching

on

the

Internet

An

analysis

of

term

pair

topics

submitted

to

the

Excite

search

engine

Journal

of

the

American

Society

for

Information

Science

-LRB-

-RRB-

-LSB-

-RSB-

Saraiva

P.

Moura

E.

Ziviani

N.

Meira

W.

Fonseca

R.

Riberio-Neto

B.

Rank-preserving

two-level

caching

for

scalable

search

engines

In

Proc

of

the

SIGIR

Conference

New

Orleans

LA

September

-LSB-

-RSB-

Silverstein

C.

Henzinger

M.

Marais

H.

and

Moricz

M.

Analysis

of

a

very

large

web

search

engine

query

log

SIGIR

Forum

-LRB-

-RRB-

-LRB-

Fall

-RRB-

-LSB-

-RSB-

Spink

A.

Ozmutlu

S.

Ozmutlu

H.C.

and

Jansen

B.J.

U.S.

versus

European

web

searching

trends

SIGIR

Forum

-LRB-

-RRB-

-LSB-

-RSB-

Spink

A.

Jansen

B.J.

Wolfram

D.

and

Saracevic

T.

From

E-sex

to

e-commerce

Web

search

changes

IEEE

Computer

-LRB-

-RRB-

-LSB-

-RSB-

Spink

A.

Wolfram

D.

Jansen

B.J.

and

Saracevic

T.

Searching

the

Web

The

Public

and

Their

Queries

Journal

of

the

American

Society

of

Information

Science

-LRB-

-RRB-

-LSB-

-RSB-

Spink

A.

Jansen

B.J.

and

Saracevic

T.

Vox

populi

The

public

searching

of

the

web

Journal

of

the

American

Society

of

Information

Science

-LRB-

-RRB-

-LSB-

-RSB-

Spink

A.

Jansen

B.J.

and

Ozmultu

H.C.

Use

of

query

reformulation

and

relevance

feedback

by

Excite

users

Internet

Research

Electronic

Networking

Applications

and

Policy

-LRB-

-RRB-

-LSB-

-RSB-

Sullivan

D.

Searches

Per

Day

Search

Engine

Watch

February

-LSB-

-RSB-

Wang

P.

Berry

M.

and

Yang

Y.

Mining

longitudinal

web

queries

Trends

and

patterns

Journal

of

the

American

Society

for

Information

Science

and

Technology

-LRB-

-RRB-

June

-LSB-

-RSB-

J.

Wen

J.

Nie

H.

Zhang

Query

Clustering

using

User

Logs

ACM

Transactions

on

Information

Systems

Vol

No.

January

-LSB-

-RSB-

Wolfram

D.

H.

Xie

Subject

categorization

of

query

terms

for

exploring

Web

users

search

interests

Journal

of

the

American

Society

for

Information

Science

v.

n.

June

-LSB-

-RSB-

Xie

Y.

O'Hallaron

D.

Locality

in

Search

Engine

Queries

and

Its

Implications

for

Caching

Infocom

O O O O O O O O O B I O O O B I O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I I O O O O O B I O O B I I O O O O O B I O O B I O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

query log analysis
query log
topical categories
web search
query stream
log analysis
query traffic
frequency distribution