Impedance

Coupling

in

Content-targeted

Advertising

The

current

boom

of

the

Web

is

associated

with

the

revenues

originated

from

on-line

advertising

While

search-based

advertising

is

dominant

the

association

of

ads

with

a

Web

page

-LRB-

during

user

navigation

-RRB-

is

becoming

increasingly

important

In

this

work

we

study

the

problem

of

associating

ads

with

a

Web

page

referred

to

as

content-targeted

advertising

from

a

computer

science

perspective

We

assume

that

we

have

access

to

the

text

of

the

Web

page

the

keywords

declared

by

an

advertiser

and

a

text

associated

with

the

advertiser

's

business

Using

no

other

information

and

operating

in

fully

automatic

fashion

we

propose

ten

strategies

for

solving

the

problem

and

evaluate

their

effectiveness

Our

methods

indicate

that

a

matching

strategy

that

takes

into

account

the

semantics

of

the

problem

-LRB-

referred

to

as

AAK

for

ads

and

keywords

-RRB-

can

yield

gains

in

average

precision

figures

of

compared

to

a

trivial

vector-based

strategy

Further

a

more

sophisticated

impedance

coupling

strategy

which

expands

the

text

of

the

Web

page

to

reduce

vocabulary

impedance

with

regard

to

an

advertisement

can

yield

extra

gains

in

average

precision

of

These

are

first

results

They

suggest

that

great

accuracy

in

content-targeted

advertising

can

be

attained

with

appropriate

algorithms

H.

-LSB-

Information

Storage

and

Retrieval

-RSB-

Information

Search

and

Retrieval

I.

-LSB-

Pattern

Recognition

-RSB-

Applications

Text

processing

Algorithms

Experimentation

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

to

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

SIGIR

August

Salvador

Brazil

Copyright

ACM

The

emergence

of

the

Internet

has

opened

up

new

marketing

opportunities

In

fact

a

company

has

now

the

possibility

of

showing

its

advertisements

-LRB-

ads

-RRB-

to

millions

of

people

at

a

low

cost

During

the

's

many

companies

invested

heavily

on

advertising

in

the

Internet

with

apparently

no

concerns

about

their

investment

return

-LSB-

-RSB-

This

situation

radically

changed

in

the

following

decade

when

the

failure

of

many

Web

companies

led

to

a

dropping

in

supply

of

cheap

venture

capital

and

a

considerable

reduction

in

on-line

advertising

investments

-LSB-

-RSB-

It

was

clear

then

that

more

effective

strategies

for

on-line

advertising

were

required

For

that

it

was

necessary

to

take

into

account

short-term

and

long-term

interests

of

the

users

related

to

their

information

needs

-LSB-

-RSB-

As

a

consequence

many

companies

intensified

the

adoption

of

intrusive

techniques

for

gathering

information

of

users

mostly

without

their

consent

-LSB-

-RSB-

This

raised

privacy

issues

which

stimu-lated

the

research

for

less

invasive

measures

-LSB-

-RSB-

More

recently

Internet

information

gatekeepers

as

for

example

search

engines

recommender

systems

and

comparison

shopping

services

have

employed

what

is

called

paid

placement

strategies

-LSB-

-RSB-

In

such

methods

an

advertiser

company

is

given

prominent

positioning

in

advertisement

lists

in

return

for

a

placement

fee

Amongst

these

methods

the

most

popular

one

is

a

non-intrusive

technique

called

keyword

targeted

marketing

-LSB-

-RSB-

In

this

technique

keywords

extracted

from

the

user

's

search

query

are

matched

against

keywords

associated

with

ads

provided

by

advertisers

A

ranking

of

the

ads

which

also

takes

into

consideration

the

amount

that

each

advertiser

is

willing

to

pay

is

computed

The

top

ranked

ads

are

displayed

in

the

search

result

page

together

with

the

answers

for

the

user

query

The

success

of

keyword

targeted

marketing

has

motivated

information

gatekeepers

to

offer

their

advertisement

services

in

different

contexts

For

example

as

shown

in

Figure

relevant

ads

could

be

shown

to

users

directly

in

the

pages

of

information

portals

The

motivation

is

to

take

advantage

of

the

users

immediate

information

interests

at

browsing

time

The

problem

of

matching

ads

to

a

Web

page

that

is

browsed

which

we

also

refer

to

as

content-targeted

advertising

-LSB-

-RSB-

is

different

from

that

of

keyword

marketing

In

this

case

instead

of

dealing

with

users

keywords

we

have

to

use

the

contents

of

a

Web

page

to

decide

which

ads

to

display

Figure

Example

of

content-based

advertising

in

the

page

of

a

newspaper

The

middle

slice

of

the

page

shows

the

beginning

of

an

article

about

the

launch

of

a

DVD

movie

At

the

bottom

slice

we

can

see

advertisements

picked

for

this

page

by

Google

's

content-based

advertising

system

AdSense

It

is

important

to

notice

that

paid

placement

advertising

strategies

imply

some

risks

to

information

gatekeepers

For

instance

there

is

the

possibility

of

a

negative

impact

on

their

credibility

which

at

long

term

can

demise

their

market

share

-LSB-

-RSB-

This

makes

investments

in

the

quality

of

ad

recommendation

systems

even

more

important

to

minimize

the

possibility

of

exhibiting

ads

unrelated

to

the

user

's

interests

By

investing

in

their

ad

systems

information

gatekeepers

are

investing

in

the

maintenance

of

their

credibility

and

in

the

reinforcement

of

a

positive

user

attitude

towards

the

advertisers

and

their

ads

-LSB-

-RSB-

Further

that

can

translate

into

higher

clickthrough

rates

that

lead

to

an

increase

in

revenues

for

information

gatekeepers

and

advertisers

with

gains

to

all

parts

-LSB-

-RSB-

In

this

work

we

focus

on

the

problem

of

content-targeted

advertising

We

propose

new

strategies

for

associating

ads

with

a

Web

page

Five

of

these

strategies

are

referred

to

as

matching

strategies

They

are

based

on

the

idea

of

matching

the

text

of

the

Web

page

directly

to

the

text

of

the

ads

and

its

associated

keywords

Five

other

strategies

which

we

here

introduce

are

referred

to

as

impedance

coupling

strategies

They

are

based

on

the

idea

of

expanding

the

Web

page

with

new

terms

to

facilitate

the

task

of

matching

ads

and

Web

pages

This

is

motivated

by

the

observation

that

there

is

frequently

a

mismatch

between

the

vocabulary

of

a

Web

page

and

the

vocabulary

of

an

advertisement

We

say

that

there

is

a

vocabulary

impedance

problem

and

that

our

technique

provides

a

positive

effect

of

impedance

coupling

by

reducing

the

vocabulary

impedance

Further

all

our

strategies

rely

on

information

that

is

already

available

to

information

gatekeepers

that

operate

keyword

targeted

advertising

systems

Thus

no

other

data

from

the

advertiser

is

required

Using

a

sample

of

a

real

case

database

with

over

ads

and

Web

pages

selected

for

testing

we

evaluate

our

ad

recommendation

strategies

First

we

evaluate

the

five

matching

strategies

They

match

ads

to

a

Web

page

using

a

standard

vector

model

and

provide

what

we

may

call

trivial

solutions

Our

results

indicate

that

a

strategy

that

matches

the

ad

plus

its

keywords

to

a

Web

page

requiring

the

keywords

to

appear

in

the

Web

page

provides

improvements

in

average

precision

figures

of

roughly

relative

to

a

strategy

that

simply

matches

the

ads

to

the

Web

page

Such

strategy

which

we

call

AAK

-LRB-

for

ads

and

keywords

-RRB-

is

then

taken

as

our

baseline

Following

we

evaluate

the

five

impedance

coupling

strategies

They

are

based

on

the

idea

of

expanding

the

ad

and

the

Web

page

with

new

terms

to

reduce

the

vocabulary

impedance

between

their

texts

Our

results

indicate

that

it

is

possible

to

generate

extra

improvements

in

average

precision

figures

of

roughly

relative

to

the

AAK

strategy

The

paper

is

organized

as

follows

In

section

we

introduce

five

matching

strategies

to

solve

content-targeted

advertising

In

section

we

present

our

impedance

coupling

strategies

In

section

we

describe

our

experimental

methodology

and

datasets

and

discuss

our

results

In

section

we

discuss

related

work

In

section

we

present

our

conclusions

Keyword

advertising

relies

on

matching

search

queries

to

ads

and

its

associated

keywords

Context-based

advertising

which

we

address

here

relies

on

matching

ads

and

its

associated

keywords

to

the

text

of

a

Web

page

Given

a

certain

Web

page

p

which

we

call

triggering

page

our

task

is

to

select

advertisements

related

to

the

contents

of

p.

Without

loss

of

generality

we

consider

that

an

advertisement

a

i

is

composed

of

a

title

a

textual

description

and

a

hyperlink

To

illustrate

for

the

first

ad

by

Google

shown

in

Figure

the

title

is

Star

Wars

Trilogy

Full

the

description

is

Get

this

popular

DVD

free

Free

w

free

shopping

Sign

up

now

and

the

hyperlink

points

to

the

site

"www.freegiftworld.com

Advertisements

can

be

grouped

by

advertisers

in

groups

called

campaigns

such

that

a

campaign

can

have

one

or

more

advertisements

Given

our

triggering

page

p

and

a

set

A

of

ads

a

simple

way

of

ranking

a

i

A

with

regard

to

p

is

by

matching

the

contents

of

p

to

the

contents

of

a

i

For

this

we

use

the

vector

space

model

-LSB-

-RSB-

as

discussed

in

the

immediately

following

In

the

vector

space

model

queries

and

documents

are

represented

as

weighted

vectors

in

an

n-dimensional

space

Let

w

iq

be

the

weight

associated

with

term

t

i

in

the

query

q

and

w

ij

be

the

weight

associated

with

term

t

i

in

the

document

d

j

Then

q

-LRB-

w

w

w

iq

w

nq

-RRB-

and

d

j

-LRB-

w

w

w

ij

w

nj

-RRB-

are

the

weighted

vectors

used

to

represent

the

query

q

and

the

document

d

j

These

weights

can

be

computed

using

classic

tf-idf

schemes

In

such

schemes

weights

are

taken

as

the

product

between

factors

that

quantify

the

importance

of

a

term

in

a

document

-LRB-

given

by

the

term

frequency

or

tf

factor

-RRB-

and

its

rarity

in

the

whole

collection

-LRB-

given

by

the

inverse

document

factor

or

idf

factor

-RRB-

see

-LSB-

-RSB-

for

details

The

ranking

of

the

query

q

with

regard

to

the

document

d

j

is

computed

by

the

cosine

similarity

formula

that

is

the

cosine

of

the

angle

between

the

two

corresponding

vectors

sim

-LRB-

q

d

j

-RRB-

q

d

j

q

d

j

P

n

i

w

iq

w

ij

qP

n

i

w

iq

qP

n

i

w

ij

-LRB-

-RRB-

By

considering

p

as

the

query

and

a

i

as

the

document

we

can

rank

the

ads

with

regard

to

the

Web

page

p

This

is

our

first

matching

strategy

It

is

represented

by

the

function

AD

given

by

AD

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

a

i

-RRB-

where

AD

stands

for

direct

match

of

the

ad

composed

by

title

and

description

and

sim

-LRB-

p

a

i

-RRB-

is

computed

according

to

Eq

-LRB-

-RRB-

In

our

second

method

we

use

other

source

of

evidence

provided

by

the

advertisers

the

keywords

With

each

advertisement

a

i

an

advertiser

associates

a

keyword

k

i

which

may

be

composed

of

one

or

more

terms

We

denote

the

association

between

an

advertisement

a

i

and

a

keyword

k

i

as

the

pair

-LRB-

a

i

k

i

-RRB-

K

where

K

is

the

set

of

associations

made

by

the

advertisers

In

the

case

of

keyword

targeted

advertising

such

keywords

are

used

to

match

the

ads

to

the

user

queries

In

here

we

use

them

to

match

ads

to

the

Web

page

p

This

provides

our

second

method

for

ad

matching

given

by

KW

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

k

i

-RRB-

where

-LRB-

a

i

k

i

-RRB-

K

and

KW

stands

for

match

the

ad

keywords

We

notice

that

most

of

the

keywords

selected

by

advertisers

are

also

present

in

the

ads

associated

with

those

keywords

For

instance

in

our

advertisement

test

collection

this

is

true

for

of

the

ads

Thus

instead

of

using

the

keywords

as

matching

devices

we

can

use

them

to

emphasize

the

main

concepts

in

an

ad

in

an

attempt

to

improve

our

AD

strategy

This

leads

to

our

third

method

of

ad

matching

given

by

AD

KW

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

a

i

k

i

-RRB-

where

-LRB-

a

i

k

i

-RRB-

K

and

AD

KW

stands

for

match

the

ad

and

its

keywords

Finally

it

is

important

to

notice

that

the

keyword

k

i

associated

with

a

i

could

not

appear

at

all

in

the

triggering

page

p

even

when

a

i

is

highly

ranked

However

if

we

assume

that

k

i

summarizes

the

main

topic

of

a

i

according

to

an

advertiser

viewpoint

it

can

be

interesting

to

assure

its

presence

in

p

This

reasoning

suggests

that

requiring

the

occurrence

of

the

keyword

k

i

in

the

triggering

page

p

as

a

condition

to

associate

a

i

with

p

might

lead

to

improved

results

This

leads

to

two

extra

matching

strategies

as

follows

ANDKW

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

a

i

-RRB-

if

k

i

p

if

otherwise

AD

ANDKW

-LRB-

p

a

i

-RRB-

AAK

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

a

i

k

i

-RRB-

if

k

i

p

if

otherwise

where

-LRB-

a

i

k

i

-RRB-

K

ANDKW

stands

for

match

the

ad

keywords

and

force

their

appearance

and

AD

ANDKW

-LRB-

or

AAK

for

ads

and

keywords

-RRB-

stands

for

match

the

ad

its

keywords

and

force

their

appearance

As

we

will

see

in

our

results

the

best

among

these

simple

methods

is

AAK

Thus

it

will

be

used

as

baseline

for

our

impedance

coupling

strategies

which

we

now

discuss

Two

key

issues

become

clear

as

one

plays

with

the

content-targeted

advertising

problem

First

the

triggering

page

normally

belongs

to

a

broader

contextual

scope

than

that

of

the

advertisements

Second

the

association

between

a

good

advertisement

and

the

triggering

page

might

depend

on

a

topic

that

is

not

mentioned

explicitly

in

the

triggering

page

The

first

issue

is

due

to

the

fact

that

Web

pages

can

be

about

any

subject

and

that

advertisements

are

concise

in

nature

That

is

ads

tend

to

be

more

topic

restricted

than

Web

pages

The

second

issue

is

related

to

the

fact

that

as

we

later

discuss

most

advertisers

place

a

small

number

of

advertisements

As

a

result

we

have

few

terms

describing

their

interest

areas

Consequently

these

terms

tend

to

be

of

a

more

general

nature

For

instance

a

car

shop

probably

would

prefer

to

use

car

instead

of

super

sport

to

describe

its

core

business

topic

As

a

consequence

many

specific

terms

that

appear

in

the

triggering

page

find

no

match

in

the

advertisements

To

make

matters

worst

a

page

might

refer

to

an

entity

or

subject

of

the

world

through

a

label

that

is

distinct

from

the

label

selected

by

an

advertiser

to

refer

to

the

same

entity

A

consequence

of

these

two

issues

is

that

vocabularies

of

pages

and

ads

have

low

intersection

even

when

an

ad

is

related

to

a

page

We

cite

this

problem

from

now

on

as

the

vocabulary

impedance

problem

In

our

experiments

we

realized

that

this

problem

limits

the

final

quality

of

direct

matching

strategies

Therefore

we

studied

alternatives

to

reduce

the

referred

vocabulary

impedance

For

this

we

propose

to

expand

the

triggering

pages

with

new

terms

Figure

illustrates

our

intuition

We

already

know

that

the

addition

of

keywords

-LRB-

selected

by

the

advertiser

-RRB-

to

the

ads

leads

to

improved

results

We

say

that

a

keyword

reduces

the

vocabulary

impedance

by

providing

an

alternative

matching

path

Our

idea

is

to

add

new

terms

-LRB-

words

-RRB-

to

the

Web

page

p

to

also

reduce

the

vocabulary

impedance

by

providing

a

second

alternative

matching

path

We

refer

to

our

expansion

technique

as

impedance

coupling

For

this

we

proceed

as

follows

expansion

terms

keyword

vocabulary

impedance

triggering

page

p

ad

Figure

Addition

of

new

terms

to

a

Web

page

to

reduce

the

vocabulary

impedance

An

advertiser

trying

to

describe

a

certain

topic

in

a

concise

way

probably

will

choose

general

terms

to

characterize

that

topic

To

facilitate

the

matching

between

this

ad

and

our

triggering

page

p

we

need

to

associate

new

general

terms

with

p.

For

this

we

assume

that

Web

documents

similar

to

the

triggering

page

p

share

common

topics

Therefore

by

inspecting

the

vocabulary

of

these

similar

documents

we

might

find

good

terms

for

better

characterizing

the

main

topics

in

the

page

p

We

now

describe

this

idea

using

a

Bayesian

network

model

-LSB-

-RSB-

depicted

in

Figure

R

D

D

D

j

D

k

T

T

T

T

i

T

m

Figure

Bayesian

network

model

for

our

impedance

coupling

technique

In

our

model

which

is

based

on

the

belief

network

in

-LSB-

-RSB-

the

nodes

represent

pieces

of

information

in

the

domain

With

each

node

is

associated

a

binary

random

variable

which

takes

the

value

to

mean

that

the

corresponding

entity

-LRB-

a

page

or

terms

-RRB-

is

observed

and

thus

relevant

in

our

computations

In

this

case

we

say

that

the

information

was

observed

Node

R

represents

the

page

r

a

new

representation

for

the

triggering

page

p.

Let

N

be

the

set

of

the

k

most

similar

documents

to

the

triggering

page

including

the

triggering

page

p

itself

in

a

large

enough

Web

collection

C.

Root

nodes

D

through

D

k

represent

the

documents

in

N

that

is

the

triggering

page

D

and

its

k

nearest

neighbors

D

through

D

k

among

all

pages

in

C

There

is

an

edge

from

node

D

j

to

node

R

if

document

d

j

is

in

N

Nodes

T

through

T

m

represent

the

terms

in

the

vocabulary

of

C

There

is

an

edge

from

node

D

j

to

a

node

T

i

if

term

t

i

occurs

in

document

d

j

In

our

model

the

observation

of

the

pages

in

N

leads

to

the

observation

of

a

new

representation

of

the

triggering

page

p

and

to

a

set

of

terms

describing

the

main

topics

associated

with

p

and

its

neighbors

Given

these

definitions

we

can

now

use

the

network

to

determine

the

probability

that

a

term

t

i

is

a

good

term

for

representing

a

topic

of

the

triggering

page

p

In

other

words

we

are

interested

in

the

probability

of

observing

the

final

evidence

regarding

a

term

t

i

given

that

the

new

representation

of

the

page

p

has

been

observed

P

-LRB-

T

i

R

-RRB-

This

translates

into

the

following

equation

P

-LRB-

T

i

R

-RRB-

P

-LRB-

R

-RRB-

X

d

P

-LRB-

T

i

d

-RRB-

P

-LRB-

R

d

-RRB-

P

-LRB-

d

-RRB-

-LRB-

-RRB-

where

d

represents

the

set

of

states

of

the

document

nodes

Since

we

are

interested

just

in

the

states

in

which

only

a

single

document

d

j

is

observed

and

P

-LRB-

d

-RRB-

can

be

regarded

as

a

constant

we

can

rewrite

Eq

-LRB-

-RRB-

as

P

-LRB-

T

i

R

-RRB-

P

-LRB-

R

-RRB-

k

X

j

P

-LRB-

T

i

d

j

-RRB-

P

-LRB-

R

d

j

-RRB-

-LRB-

-RRB-

where

d

j

represents

the

state

of

the

document

nodes

in

which

only

document

d

j

is

observed

and

is

a

constant

To

simplify

our

notation

we

represent

the

probabilities

P

-LRB-

X

-RRB-

as

P

-LRB-

X

-RRB-

and

P

-LRB-

X

-RRB-

as

P

-LRB-

X

-RRB-

associated

with

P

-LRB-

d

j

-RRB-

Eq

-LRB-

-RRB-

is

the

general

equation

to

compute

the

probability

that

a

term

t

i

is

related

to

the

triggering

page

We

now

define

the

probabilities

P

-LRB-

T

i

d

j

-RRB-

and

P

-LRB-

R

d

j

-RRB-

as

follows

P

-LRB-

T

i

d

j

-RRB-

w

ij

-LRB-

-RRB-

P

-LRB-

R

d

j

-RRB-

-LRB-

-RRB-

j

sim

-LRB-

r

d

j

-RRB-

j

k

-LRB-

-RRB-

where

is

a

normalizing

constant

w

ij

is

the

weight

associated

with

term

t

i

in

the

document

d

j

and

sim

-LRB-

p

d

j

-RRB-

is

given

by

Eq

-LRB-

-RRB-

i.e.

is

the

cosine

similarity

between

p

and

d

j

The

weight

w

ij

is

computed

using

a

classic

tf-idf

scheme

and

is

zero

if

term

t

i

does

not

occur

in

document

d

j

Notice

that

P

-LRB-

T

i

d

j

-RRB-

P

-LRB-

T

i

d

j

-RRB-

and

P

-LRB-

R

d

j

-RRB-

P

-LRB-

R

d

j

-RRB-

By

defining

the

constant

it

is

possible

to

determine

how

important

should

be

the

influence

of

the

triggering

page

p

to

its

new

representation

r.

By

substituting

Eq

-LRB-

-RRB-

and

Eq

-LRB-

-RRB-

into

Eq

-LRB-

-RRB-

we

obtain

P

-LRB-

T

i

R

-RRB-

-LRB-

-LRB-

-RRB-

w

k

X

j

w

ij

sim

-LRB-

r

d

j

-RRB-

-RRB-

-LRB-

-RRB-

where

is

a

normalizing

constant

We

use

Eq

-LRB-

-RRB-

to

determine

the

set

of

terms

that

will

compose

r

as

illustrated

in

Figure

Let

t

top

be

the

top

ranked

term

according

to

Eq

-LRB-

-RRB-

The

set

r

is

composed

of

the

terms

t

i

such

that

P

-LRB-

T

i

R

-RRB-

P

-LRB-

T

top

R

-RRB-

where

is

a

given

threshold

In

our

experiments

we

have

used

Notice

that

the

set

r

might

contain

terms

that

already

occur

in

p

That

is

while

we

will

refer

to

the

set

r

as

expansion

terms

it

should

be

clear

that

p

r

By

using

we

simply

consider

the

terms

originally

in

page

p.

By

increasing

we

relax

the

context

of

the

page

p

adding

terms

from

neighbor

pages

turning

page

p

into

its

new

representation

r

This

is

important

because

sometimes

a

topic

apparently

not

important

in

the

triggering

page

offers

a

good

opportunity

for

advertising

For

example

consider

a

triggering

page

that

describes

a

congress

in

London

about

digital

photography

Although

London

is

probably

not

an

important

topic

in

this

page

advertisements

about

hotels

in

London

would

be

appropriate

Thus

adding

hotels

to

page

p

is

important

This

suggests

using

that

is

preserving

the

contents

of

p

and

using

the

terms

in

r

to

expand

p

In

this

paper

we

examine

both

approaches

Thus

in

our

sixth

method

we

match

r

the

set

of

new

expansion

terms

directly

to

the

ads

as

follows

AAK

T

-LRB-

p

a

i

-RRB-

AAK

-LRB-

r

a

i

-RRB-

where

AAK

T

stands

for

match

the

ad

and

keywords

to

the

set

r

of

expansion

terms

In

our

seventh

method

we

match

an

expanded

page

p

to

the

ads

as

follows

AAK

EXP

-LRB-

p

a

i

-RRB-

AAK

-LRB-

p

r

a

i

-RRB-

where

AAK

EXP

stands

for

match

the

ad

and

keywords

to

the

expanded

triggering

page

To

improve

our

ad

placement

methods

other

external

source

that

we

can

use

is

the

content

of

the

page

h

pointed

to

by

the

advertisement

's

hyperlink

that

is

its

landing

page

After

all

this

page

comprises

the

real

target

of

the

ad

and

perhaps

could

present

a

more

detailed

description

of

the

product

or

service

being

advertised

Given

that

the

advertisement

a

i

points

to

the

landing

page

h

i

we

denote

this

association

as

the

pair

-LRB-

a

i

h

i

-RRB-

H

where

H

is

the

set

of

associations

between

the

ads

and

the

pages

they

point

to

Our

eighth

method

consists

of

matching

the

triggering

page

p

to

the

landing

pages

pointed

to

by

the

advertisements

as

follows

H

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

h

i

-RRB-

where

-LRB-

a

i

h

i

-RRB-

H

and

H

stands

for

match

the

hyperlink

pointed

to

by

the

ad

We

can

also

combine

this

information

with

the

more

promising

methods

previously

described

AAK

and

AAK

EXP

as

follows

Given

that

-LRB-

a

i

h

i

-RRB-

H

and

-LRB-

a

i

k

i

-RRB-

K

we

have

our

last

two

methods

AAK

H

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

a

i

h

i

k

i

-RRB-

if

k

i

p

if

otherwise

AAK

EXP

H

-LRB-

p

a

i

-RRB-

sim

-LRB-

p

r

a

i

h

i

k

i

-RRB-

if

k

i

-LRB-

p

r

-RRB-

if

otherwise

where

AAK

H

stands

for

match

ads

and

keywords

also

considering

the

page

pointed

by

the

ad

and

AAH

EXP

H

stands

for

match

ads

and

keywords

with

expanded

triggering

page

also

considering

the

page

pointed

by

the

ad

Notice

that

other

combinations

were

not

considered

in

this

study

due

to

space

restrictions

These

other

combinations

led

to

poor

results

in

our

experimentation

and

for

this

reason

were

discarded

To

evaluate

our

ad

placement

strategies

we

performed

a

series

of

experiments

using

a

sample

of

a

real

case

ad

collection

with

advertisements

advertisers

and

keywords

The

advertisements

are

grouped

in

campaigns

with

an

average

of

campaigns

per

advertiser

For

the

strategies

AAK

T

and

AAK

EXP

we

had

to

generate

a

set

of

expansion

terms

For

that

we

used

a

database

of

Web

pages

crawled

by

the

TodoBR

search

engine

-LSB-

-RSB-

-LRB-

-RRB-

This

database

is

composed

of

pages

of

the

Brazilian

Web

under

the

domain

br

For

the

strategies

H

AAK

H

and

AAK

EXP

H

we

also

crawled

the

pages

pointed

to

by

the

advertisers

No

other

filtering

method

was

applied

to

these

pages

besides

the

removal

of

HTML

tags

Since

we

are

initially

interested

in

the

placement

of

advertisements

in

the

pages

of

information

portals

our

test

collection

was

composed

of

pages

extracted

from

a

Brazilian

newspaper

These

are

our

triggering

pages

They

were

crawled

in

such

a

way

that

only

the

contents

of

their

articles

was

preserved

As

we

have

no

preferences

for

particular

Data

in

portuguese

provided

by

an

on-line

advertisement

company

that

operates

in

Brazil

topics

the

crawled

pages

cover

topics

as

diverse

as

politics

economy

sports

and

culture

For

each

of

our

triggering

pages

we

selected

the

top

three

ranked

ads

provided

by

each

of

our

ad

placement

strategies

Thus

for

each

triggering

page

we

select

no

more

than

ads

These

top

ads

were

then

inserted

in

a

pool

for

that

triggering

page

Each

pool

contained

an

average

of

advertisements

All

advertisements

in

each

pool

were

submitted

to

a

manual

evaluation

by

a

group

of

users

The

average

number

of

relevant

advertisements

per

page

pool

was

Notice

that

we

adopted

the

same

pooling

method

used

to

evaluate

the

TREC

Web-based

collection

-LSB-

-RSB-

To

quantify

the

precision

of

our

results

we

used

average

figures

-LSB-

-RSB-

Since

we

are

not

able

to

evaluate

the

entire

ad

collection

recall

values

are

relative

to

the

set

of

evaluated

advertisements

Tuning

Idf

factors

We

start

by

analyzing

the

impact

of

different

idf

factors

in

our

advertisement

collection

Idf

factors

are

important

because

they

quantify

how

discriminative

is

a

term

in

the

collection

In

our

ad

collection

idf

factors

can

be

computed

by

taking

ads

advertisers

or

campaigns

as

documents

To

exemplify

consider

the

computation

of

ad

idf

for

a

term

t

i

that

occurs

times

in

a

collection

of

ads

Then

the

inverse

document

frequency

of

t

i

is

given

by

idf

i

log

Hence

we

can

compute

ad

advertiser

or

campaign

idf

factors

As

we

observe

in

Figure

for

the

AD

strategy

the

best

ranking

is

obtained

by

the

use

of

campaign

idf

that

is

by

calculating

our

idf

factor

so

that

it

discriminates

campaigns

Similar

results

were

obtained

for

all

the

other

methods

precision

recall

Campaign

idf

Advertiser

idf

Ad

idf

Figure

Precision-recall

curves

obtained

for

the

AD

strategy

using

ad

advertiser

and

campaign

idf

factors

This

reflects

the

fact

that

terms

might

be

better

discriminators

for

a

business

topic

than

for

an

specific

ad

This

effect

can

be

accomplished

by

calculating

the

factor

relative

to

idf

advertisers

or

campaigns

instead

of

ads

In

fact

campaign

idf

factors

yielded

the

best

results

Thus

they

will

be

used

in

all

the

experiments

reported

from

now

on

Results

Matching

Strategies

Figure

displays

the

results

for

the

matching

strategies

presented

in

Section

As

shown

directly

matching

the

contents

of

the

ad

to

the

triggering

page

-LRB-

AD

strategy

-RRB-

is

not

so

effective

The

reason

is

that

the

ad

contents

are

very

noisy

It

may

contain

messages

that

do

not

properly

describe

the

ad

topics

such

as

requisitions

for

user

actions

-LRB-

e.g

visit

our

site

-RRB-

and

general

sentences

that

could

be

applied

to

any

product

or

service

-LRB-

e.g

we

delivery

for

the

whole

country

-RRB-

On

the

other

hand

an

advertiser

provided

keyword

summarizes

well

the

topic

of

the

ad

As

a

consequence

the

KW

strategy

is

superior

to

the

AD

and

AD

KW

strategies

This

situation

changes

when

we

require

the

keywords

to

appear

in

the

target

Web

page

By

filtering

out

ads

whose

keywords

do

not

occur

in

the

triggering

page

much

noise

is

discarded

This

makes

ANDKW

a

better

alternative

than

KW

Further

in

this

new

situation

the

contents

of

the

ad

becomes

useful

to

rank

the

most

relevant

ads

making

AD

ANDKW

-LRB-

or

AAK

for

ads

and

keywords

-RRB-

the

best

among

all

described

methods

For

this

reason

we

adopt

AAK

as

our

baseline

in

the

next

set

of

experiments

precision

recall

AAK

ANDKW

KW

AD

Figure

Comparison

among

our

five

matching

strategies

AAK

-LRB-

ads

and

keywords

-RRB-

is

superior

Table

illustrates

average

precision

figures

for

Figure

We

also

present

actual

hits

per

advertisement

slot

We

call

hit

an

assignment

of

an

ad

-LRB-

to

the

triggering

page

-RRB-

that

was

considered

relevant

by

the

evaluators

We

notice

that

our

AAK

strategy

provides

a

gain

in

average

precision

of

relative

to

the

trivial

AD

strategy

This

shows

that

careful

consideration

of

the

evidence

related

to

the

problem

does

pay

off

Impedance

Coupling

Strategies

Table

shows

top

ranked

terms

that

occur

in

a

page

covering

Argentinean

wines

produced

using

grapes

derived

from

the

Bordeaux

region

of

France

The

p

column

includes

the

top

terms

for

this

page

ranked

according

to

our

tf-idf

weighting

scheme

The

r

column

includes

the

top

ranked

expansion

terms

generated

according

to

Eq

-LRB-

-RRB-

Notice

that

the

expansion

terms

not

only

emphasize

important

terms

of

the

target

page

-LRB-

by

increasing

their

weights

-RRB-

such

as

wines

and

Methods

Hits

average

total

score

gain

-LRB-

-RRB-

AD

AD

KW

KW

ANDKW

AD

ANDKW

-LRB-

AAK

-RRB-

Table

Average

precision

figures

corresponding

to

Figure

for

our

five

matching

strategies

Columns

labelled

and

indicate

total

of

hits

in

first

second

and

third

advertisement

slots

respectively

The

AAK

strategy

provides

improvements

of

relative

to

the

AD

strategy

Rank

p

r

term

score

term

score

argentina

wines

obtained

wine

class

whites

whites

red

french

grape

origin

bordeaux

france

acideness

grape

argentina

sweet

aroma

country

blanc

wines

Table

Top

ranked

terms

for

the

triggering

page

p

according

to

our

tf-idf

weighting

scheme

and

top

ranked

terms

for

r

the

expansion

terms

for

p

generated

according

to

Eq

-LRB-

-RRB-

Ranking

scores

were

normalized

in

order

to

sum

up

to

Terms

marked

with

are

not

shared

by

the

sets

p

and

r.

whites

but

also

reveal

new

terms

related

to

the

main

topic

of

the

page

such

as

aroma

and

red

Further

they

avoid

some

uninteresting

terms

such

as

obtained

and

country

Figure

illustrates

our

results

when

the

set

r

of

expansion

terms

is

used

They

show

that

matching

the

ads

to

the

terms

in

the

set

r

instead

of

to

the

triggering

page

p

-LRB-

AAK

T

strategy

-RRB-

leads

to

a

considerable

improvement

over

our

baseline

AAK

The

gain

is

even

larger

when

we

use

the

terms

in

r

to

expand

the

triggering

page

-LRB-

AAK

EXP

method

-RRB-

This

confirms

our

hypothesis

that

the

triggering

page

could

have

some

interesting

terms

that

should

not

be

completely

discarded

Finally

we

analyze

the

impact

on

the

ranking

of

using

the

contents

of

pages

pointed

by

the

ads

Figure

displays

our

results

It

is

clear

that

using

only

the

contents

of

the

pages

pointed

by

the

ads

-LRB-

H

strategy

-RRB-

yields

very

poor

results

However

combining

evidence

from

the

pages

pointed

by

the

ads

with

our

baseline

yields

improved

results

Most

important

combining

our

best

strategy

so

far

-LRB-

AAK

EXP

-RRB-

with

pages

pointed

by

ads

-LRB-

AAK

EXP

H

strategy

-RRB-

leads

to

superior

results

This

happens

because

the

two

additional

sources

of

evidence

expansion

terms

and

pages

pointed

by

the

ads

are

distinct

and

complementary

providing

extra

and

valuable

information

for

matching

ads

to

a

Web

page

precision

recall

AAK

Figure

Impact

of

using

a

new

representation

for

the

triggering

page

one

that

includes

expansion

terms

precision

recall

AAK

H

Figure

Impact

of

using

the

contents

of

the

page

pointed

by

the

ad

-LRB-

the

hyperlink

-RRB-

Figure

and

Table

summarize

all

results

described

in

this

section

In

Figure

we

show

precision-recall

curves

and

in

Table

we

show

average

figures

We

also

present

actual

hits

per

advertisement

slot

and

gains

in

average

precision

relative

to

our

baseline

AAK

We

notice

that

the

highest

number

of

hits

in

the

first

slot

was

generated

by

the

method

AAK

EXP

However

the

method

with

best

overall

retrieval

performance

was

AAK

EXP

H

yielding

a

gain

in

average

precision

figures

of

roughly

over

the

baseline

-LRB-

AAK

-RRB-

Performance

Issues

In

a

keyword

targeted

advertising

system

ads

are

assigned

at

query

time

thus

the

performance

of

the

system

is

a

very

important

issue

In

content-targeted

advertising

systems

we

can

associate

ads

with

a

page

at

publishing

-LRB-

or

updating

-RRB-

time

Also

if

a

new

ad

comes

in

we

might

consider

assigning

this

ad

to

already

published

pages

in

offline

mode

That

is

we

might

design

the

system

such

that

its

performance

depends

fundamentally

on

the

rate

that

new

pages

precision

recall

AAK

H

Figure

Comparison

among

our

ad

placement

strategies

Methods

Hits

average

total

score

gain

-LRB-

-RRB-

H

AAK

AAK

H

AAK

T

AAK

EXP

AAK

EXP

H

Table

Results

for

our

impedance

coupling

strategies

are

published

and

the

rate

that

ads

are

added

or

modified

Further

the

data

needed

by

our

strategies

-LRB-

page

crawling

page

expansion

and

ad

link

crawling

-RRB-

can

be

gathered

and

processed

offline

not

affecting

the

user

experience

Thus

from

this

point

of

view

the

performance

is

not

critical

and

will

not

be

addressed

in

this

work

Several

works

have

stressed

the

importance

of

relevance

in

advertising

For

example

in

-LSB-

-RSB-

it

was

shown

that

advertisements

that

are

presented

to

users

when

they

are

not

interested

on

them

are

viewed

just

as

annoyance

Thus

in

order

to

be

effective

the

authors

conclude

that

advertisements

should

be

relevant

to

consumer

concerns

at

the

time

of

exposure

The

results

in

-LSB-

-RSB-

enforce

this

conclusion

by

pointing

out

that

the

more

targeted

the

advertising

the

more

effective

it

is

Therefore

it

is

not

surprising

that

other

works

have

addressed

the

relevance

issue

For

instance

in

-LSB-

-RSB-

it

is

proposed

a

system

called

ADWIZ

that

is

able

to

adapt

online

advertisement

to

a

user

's

short-term

interests

in

a

non-intrusive

way

Contrary

to

our

work

ADWIZ

does

not

directly

use

the

content

of

the

page

viewed

by

the

user

It

relies

on

search

keywords

supplied

by

the

user

to

search

engines

and

on

the

URL

of

the

page

requested

by

the

user

On

the

other

hand

in

-LSB-

-RSB-

the

authors

presented

an

intrusive

approach

in

which

an

agent

sits

between

advertisers

and

the

user

's

browser

allowing

a

banner

to

be

placed

into

the

currently

viewed

page

In

spite

of

having

the

opportunity

to

use

the

page

's

content

the

agent

infers

relevance

based

on

category

information

and

user

's

private

information

collected

along

the

time

In

-LSB-

-RSB-

the

authors

provide

a

comparison

between

the

ranking

strategies

used

by

Google

and

Overture

for

their

keyword

advertising

systems

Both

systems

select

advertisements

by

matching

them

to

the

keywords

provided

by

the

user

in

a

search

query

and

rank

the

resulting

advertisement

list

according

to

the

advertisers

willingness

to

pay

In

particular

Google

approach

also

considers

the

clickthrough

rate

of

each

advertisement

as

an

additional

evidence

for

its

relevance

The

authors

conclude

that

Google

's

strategy

is

better

than

that

used

by

Overture

As

mentioned

before

the

ranking

problem

in

keyword

advertising

is

different

from

that

of

content-targeted

advertising

Instead

of

dealing

with

keywords

provided

by

users

in

search

queries

we

have

to

deal

with

the

contents

of

a

page

which

can

be

very

diffuse

Finally

the

work

in

-LSB-

-RSB-

focuses

on

improving

search

engine

results

in

a

TREC

collection

by

means

of

an

automatic

query

expansion

method

based

on

kNN

-LSB-

-RSB-

Such

method

resembles

our

expansion

approach

presented

in

section

Our

method

is

different

from

that

presented

by

-LSB-

-RSB-

They

expand

user

queries

applied

to

a

document

collection

with

terms

extracted

from

the

top

k

documents

returned

as

answer

to

the

query

in

the

same

collection

In

our

case

we

use

two

collections

an

advertisement

and

a

Web

collection

We

expand

triggering

pages

with

terms

extracted

from

the

Web

collection

and

then

we

match

these

expanded

pages

to

the

ads

from

the

advertisement

collection

By

doing

this

we

emphasize

the

main

topics

of

the

triggering

pages

increasing

the

possibility

of

associating

relevant

ads

with

them

In

this

work

we

investigated

ten

distinct

strategies

for

associating

ads

with

a

Web

page

that

is

browsed

-LRB-

content-targeted

advertising

-RRB-

Five

of

our

strategies

attempt

to

match

the

ads

directly

to

the

Web

page

Because

of

that

they

are

called

matching

strategies

The

other

five

strategies

recognize

that

there

is

a

vocabulary

impedance

problem

among

ads

and

Web

pages

and

attempt

to

solve

the

problem

by

expanding

the

Web

pages

and

the

ads

with

new

terms

Because

of

that

they

are

called

impedance

coupling

strategies

Using

a

sample

of

a

real

case

database

with

over

thousand

ads

we

evaluated

our

strategies

For

the

five

matching

strategies

our

results

indicated

that

planned

consideration

of

additional

evidence

-LRB-

such

as

the

keywords

provided

by

the

advertisers

-RRB-

yielded

gains

in

average

precision

figures

-LRB-

for

our

test

collection

-RRB-

of

This

was

obtained

by

a

strategy

called

AAK

-LRB-

for

ads

and

keywords

-RRB-

which

is

taken

as

the

baseline

for

evaluating

our

more

advanced

impedance

coupling

strategies

For

our

five

impedance

coupling

strategies

the

results

indicate

that

additional

gains

in

average

precision

of

-LRB-

now

relative

to

the

AAK

strategy

-RRB-

are

possible

These

were

generated

by

expanding

the

Web

page

with

new

terms

-LRB-

obtained

using

a

sample

Web

collection

containing

over

five

million

pages

-RRB-

and

the

ads

with

the

contents

of

the

page

they

point

to

-LRB-

a

hyperlink

provided

by

the

advertisers

-RRB-

These

are

first

time

results

that

indicate

that

high

quality

content-targeted

advertising

is

feasible

and

practical

This

work

was

supported

in

part

by

the

GERINDO

project

grant

MCT/CNPq/CT-INFO

by

CNPq

grant

-LRB-

Berthier

Ribeiro-Neto

-RRB-

and

by

CNPq

grant

-LRB-

Edleno

Silva

de

Moura

-RRB-

Marco

Cristo

is

supported

by

Fucapi

Manaus

AM

Brazil

-LSB-

-RSB-

The

Google

adwords

Google

content-targeted

advertising

November

-LSB-

-RSB-

R.

Baeza-Yates

and

B.

Ribeiro-Neto

Modern

Information

Retrieval

Addison-Wesley-Longman

edition

-LSB-

-RSB-

H.

K.

Bhargava

and

J.

Feng

Paid

placement

strategies

for

internet

search

engines

In

Proceedings

of

the

eleventh

international

conference

on

World

Wide

Web

pages

ACM

Press

-LSB-

-RSB-

E.

P.

Chan

S.

Garcia

and

S.

Roukos

ad

hoc

retrieval

using

k

nearest-neighbors

re-scoring

In

The

Fifth

Text

REtrieval

Conference

-LRB-

-RRB-

National

Institute

of

Standards

and

Technology

-LRB-

NIST

-RRB-

November

-LSB-

-RSB-

J.

Feng

H.

K.

Bhargava

and

D.

Pennock

Comparison

of

allocation

rules

for

paid

placement

advertising

in

search

engines

In

Proceedings

of

the

international

conference

on

Electronic

commerce

pages

ACM

Press

-LSB-

-RSB-

D.

Hawking

N.

Craswell

and

P.

B.

Thistlewaite

Overview

of

very

large

collection

track

In

The

Seventh

Text

REtrieval

Conference

-LRB-

-RRB-

pages

Gaithersburg

Maryland

USA

November

-LSB-

-RSB-

Y.

Kohda

and

S.

Endo

Ubiquitous

advertising

on

the

www

merging

advertisement

on

the

browser

Comput

Netw

ISDN

Syst.

-LRB-

-RRB-

-LSB-

-RSB-

M.

Langheinrich

A.

Nakamura

N.

Abe

T.

Kamba

and

Y.

Koseki

Unintrusive

customization

techniques

for

web

advertising

Comput

Networks

-LRB-

-RRB-

-LSB-

-RSB-

T.

P.

Novak

and

D.

L.

Hoffman

New

metrics

for

new

media

toward

the

development

of

web

measurement

standards

World

Wide

Web

J.

-LRB-

-RRB-

-LSB-

-RSB-

J.

Pearl

Probabilistic

Reasoning

in

Intelligent

Systems

Networks

of

plausible

inference

Morgan

Kaufmann

Publishers

edition

-LSB-

-RSB-

B.

Ribeiro-Neto

and

R.

Muntz

A

belief

network

model

for

IR

In

Proceedings

of

the

Annual

International

ACM

SIGIR

Conference

on

Research

and

Development

in

Information

Retrieval

pages

Zurich

Switzerland

August

-LSB-

-RSB-

A.

Silva

E.

Veloso

P.

Golgher

B.

Ribeiro-Neto

A.

Laender

and

N.

Ziviani

CobWeb

a

crawler

for

the

brazilian

web

In

Proceedings

of

the

String

Processing

and

Information

Retrieval

Symposium

-LRB-

SPIRE

-RRB-

pages

Cancun

Mexico

September

-LSB-

-RSB-

H.

Turtle

and

W.

B.

Croft

Evaluation

of

an

inference

network-based

retrieval

model

ACM

Transactions

on

Information

Systems

-LRB-

-RRB-

July

-LSB-

-RSB-

C.

Wang

P.

Zhang

R.

Choi

and

M.

Daeredita

Understanding

consumers

attitude

toward

advertising

In

Eighth

Americas

Conference

on

Information

Systems

pages

August

-LSB-

-RSB-

M.

Weideman

Ethical

issues

on

content

distribution

to

digital

consumers

via

paid

placement

as

opposed

to

website

visibility

in

search

engine

results

In

The

Seventh

ETHICOMP

International

Conference

on

the

Social

and

Ethical

Impacts

of

Information

and

Communication

Technologies

pages

Troubador

Publishing

Ltd

April

-LSB-

-RSB-

M.

Weideman

and

T.

Haig-Smith

An

investigation

into

search

engines

as

a

form

of

targeted

advert

delivery

In

Proceedings

of

the

annual

research

conference

of

the

South

African

institute

of

computer

scientists

and

information

technologists

on

Enablement

through

technology

pages

South

African

Institute

for

Computer

Scientists

and

Information

Technologists

-LSB-

-RSB-

Y.

Yang

Expert

network

Effective

and

efficient

learning

from

human

decisions

in

text

categorization

and

retrieval

In

W.

B.

Croft

and

e.

C.

J.

van

Rijsbergen

editors

Proceedings

of

the

annual

international

ACM

SIGIR

conference

on

Research

and

development

in

information

retrieval

pages

Springer-Verlag

B I O O B O O O O O B O O O O O O O O B O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O B O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

web
impedance coupling
triggering page
advertising
advertisements
knn
matching

content-targeted advertising
bayesian networks


Information

Retrieval

for

Language

Tutoring

An

Overview

of

the

REAP

Project

H.

-LSB-

Information

Search

and

Retrieval

-RSB-

Retrieval

models

Algorithms

Design

Human

Factors

Typical

Web

search

engines

are

designed

to

run

short

queries

against

a

huge

collection

of

hyperlinked

documents

quickly

and

cheaply

and

are

often

tuned

for

the

types

of

queries

people

submit

most

often

-LSB-

-RSB-

Many

other

types

of

applications

exist

for

which

large

open

collections

like

the

Web

would

be

a

valuable

resource

However

these

applications

may

require

much

more

advanced

support

from

information

retrieval

technology

than

is

currently

available

In

particular

an

application

may

have

to

describe

more

complex

information

needs

with

a

varied

set

of

properties

and

data

models

including

aspects

of

the

user

's

context

and

goals

In

this

paper

we

present

an

overview

of

one

such

application

the

REAP

project

whose

main

purpose

is

to

provide

reader-specific

practice

for

improved

reading

comprehension

-LRB-

REAP

stands

for

REAder-specific

Practice

-RRB-

A

key

component

of

REAP

is

an

advanced

search

model

that

can

find

documents

satisfying

a

set

of

diverse

and

possibly

complex

lexical

constraints

including

a

passage

's

topic

reading

level

-LRB-

e.g.

grade

-RRB-

use

of

syntax

-LRB-

simple

vs.

complex

sentence

structures

-RRB-

and

vocabulary

that

is

known

or

unknown

to

the

student

Searching

is

performed

on

a

database

of

documents

automatically

gathered

from

the

Web

which

have

been

analyzed

and

annotated

with

a

rich

set

of

linguistic

metadata

The

Web

is

a

potentially

valuable

resource

for

providing

reading

material

of

interest

to

the

student

because

of

its

extent

variety

and

currency

for

popular

topics

Here

we

describe

the

high-level

design

of

the

REAP

information

retrieval

system

including

document

database

requirements

and

construction

annotations

and

a

brief

description

of

the

retrieval

model

Database

Construction

Our

goal

is

to

present

passages

that

are

interesting

to

students

whether

they

are

on

current

topics

such

as

pop

stars

or

sports

events

or

related

to

particular

classroom

projects

To

this

end

we

use

the

Web

as

our

source

of

reading

practice

materials

because

of

its

extent

variety

and

currency

of

information

We

want

coverage

of

topics

in

the

database

to

be

deeper

in

areas

that

are

more

likely

to

be

of

interest

to

students

Coverage

of

other

areas

is

intended

to

be

broad

but

more

shallow

We

therefore

gather

documents

for

the

database

using

focused

crawling

-LSB-

-RSB-

The

current

prototype

uses

a

page

's

reading

difficulty

to

set

priority

for

all

links

from

that

page

equally

based

on

the

distance

from

the

target

reading

level

range

We

plan

to

explore

more

refined

use

of

annotations

to

direct

the

crawl

on

a

link-by-link

basis

In

our

prototype

we

collected

million

pages

based

on

an

initial

set

of

seed

pages

acquired

from

the

Google

Kids

Directory

-LSB-

-RSB-

Our

goal

is

to

have

at

least

million

pages

that

focus

on

material

for

grades

through

The

document

database

must

be

large

enough

that

the

most

important

lexical

constraints

are

satisfied

by

at

least

a

small

number

of

pages

Data

annotation

is

currently

performed

off-line

at

indexing

time

The

specific

annotations

for

REAP

are

described

in

Section

Once

the

documents

are

acquired

they

are

indexed

using

an

extended

version

of

the

Lemur

IR

Toolkit

-LSB-

-RSB-

We

chose

Lemur

because

of

its

support

for

language

model-based

retrieval

its

extensibility

and

its

support

for

incremental

indexing

which

is

important

for

efficient

updates

to

the

database

Annotations

are

currently

stored

as

Lemur

properties

but

later

versions

will

take

advantage

of

the

enhancements

planned

for

support

of

rich

document

structure

described

in

Section

Linguistic

Annotations

In

addition

to

the

underlying

text

the

following

linguistic

annotations

are

specified

as

features

to

be

indexed

Basic

text

difficulty

within

a

document

section

or

region

This

is

calculated

using

a

new

method

based

on

a

mixture

of

language

models

-LSB-

-RSB-

that

is

more

reliable

for

Web

pages

and

other

non-traditional

documents

than

typical

reading

difficulty

measures

Grammatical

structure

This

includes

part-of-speech

tags

for

individual

words

as

well

as

higher-level

parse

structures

up

to

sentence

level

Document-level

attributes

such

as

title

metadata

keywords

and

ratings

Topic

category

This

would

involve

broad

categories

such

as

fiction/non-fiction

-LSB-

-RSB-

or

specific

topics

perhaps

based

on

Open

Directory

Named

entity

tags

We

use

BBN

's

Identifinder

-LSB-

-RSB-

for

high-precision

tagging

of

proper

names

We

may

also

look

at

more

advanced

attributes

such

as

text

coherence

and

cohesion

-LSB-

-RSB-

Query

and

Retrieval

Models

A

typical

information

need

for

the

REAP

system

might

be

described

as

follows

Find

a

Web

page

about

soccer

in

American

English

with

reading

difficulty

around

the

Grade

level

The

text

should

use

both

passive

and

active-voice

sentence

constructions

and

should

introduce

about

new

vocabulary

relative

to

the

student

's

known-vocabulary

model

The

page

's

topic

is

less

important

than

finding

pages

that

practice

the

words

for

example

an

article

on

another

sport

that

satisfies

the

other

constraints

would

also

be

acceptable

Information

needs

in

REAP

will

be

modeled

as

mixtures

of

multiple

word

histograms

representing

different

sources

of

evidence

as

well

as

document-level

or

passage-level

constraints

on

attributes

such

as

reading

difficulty

There

is

precedent

for

using

word

histograms

to

specify

information

needs

indeed

query

expansion

is

one

example

of

this

More

specifically

related

work

includes

language

model-based

techniques

such

as

relevance

models

-LSB-

-RSB-

No

current

Web-based

search

engine

is

able

to

make

use

of

combinations

of

lexical

constraints

and

language

models

in

this

way

on

such

a

large

scale

To

support

this

we

are

making

extensions

to

Lemur

that

include

Retrieval

models

for

rich

document

structure

which

includes

nested

fields

of

different

datatypes

where

each

field

may

be

associated

with

its

own

language

model

More

detailed

retrieval

models

in

which

we

skew

language

models

towards

the

appropriate

grade

level

topic

or

style

The

use

of

user

model

descriptions

as

context

for

a

query

User

Profiles

In

the

current

prototype

we

model

a

reader

's

topic

interests

reading

level

and

vocabulary

acquisition

goals

using

simple

language

models

For

example

we

model

the

curriculum

as

a

word

histogram

Although

crude

this

captures

word-frequency

information

associated

with

general

reading

difficulty

as

well

as

capturing

topics

that

are

the

focus

of

the

curriculum

at

each

grade

level

We

plan

to

add

more

complex

aspects

to

user

profiles

including

more

specific

lexical

constraints

such

as

grammar

constructs

and

text

novelty

The

models

can

be

updated

incrementally

as

the

student

's

interests

evolve

and

they

make

progress

through

the

curriculum

Evaluation

of

the

end-to-end

REAP

system

will

be

via

a

series

of

three

year-long

studies

with

both

adults

and

children

The

adult

studies

will

provide

feedback

on

vocabulary

matching

and

comprehension

and

the

child

studies

will

test

the

hypothesis

that

children

will

read

adaptively

to

texts

that

vary

in

vocabulary

demands

where

those

texts

that

closely

reflect

the

reader

's

interests

and

comprehension

can

be

used

to

support

improved

comprehension

and

vocabulary

growth

The

REAP

project

is

intended

to

advance

the

state

of

the

art

in

information

retrieval

as

well

as

research

in

reading

comprehension

by

bringing

together

practical

user

models

of

student

interests

vocabulary

knowledge

and

growth

and

other

aspects

of

reading

with

interesting

material

from

large

open

collections

like

the

World

Wide

Web

This

type

of

system

is

a

valuable

new

research

tool

for

educational

psychologists

and

learning

scientists

because

it

gives

much

greater

control

over

how

instructional

materials

are

selected

This

in

turn

allows

testing

of

instructional

hypotheses

such

as

the

effect

of

vocabulary

stretch

which

have

been

impractical

to

test

in

the

past

The

work

also

has

direct

application

to

other

areas

of

language

learning

such

as

English

as

a

Second

Language

training

More

broadly

however

we

believe

the

REAP

project

is

a

important

first

step

toward

enabling

richer

user

and

task

models

than

currently

available

with

ad-hoc

search

systems

We

thank

our

collaborators

Maxine

Eskenazi

Charles

Perfetti

and

Jonathan

Brown

John

Cho

and

Alexandros

Ntoulas

of

UCLA

for

their

crawler

code

and

the

anonymous

reviewers

This

work

was

supported

by

U.S.

Dept.

of

Education

grant

Any

opinions

findings

conclusions

or

recommendations

expressed

in

this

material

are

the

authors

and

do

not

necessarily

reflect

those

of

the

sponsors

-LSB-

-RSB-

Bikel

D.

M.

Miller

S.

Schwartz

R.

Weischedel

R.

M.

Nymbol

A

high-performance

learning

name-finder

In

Proceedings

of

the

Conference

on

Applied

Natural

Language

Processing

-LSB-

-RSB-

Broder

A

A

taxonomy

of

Web

search

In

SIGIR

Forum

-LRB-

-RRB-

-LSB-

-RSB-

Chakrabarti

S.

van

der

Berg

M.

Dom

B.

Focused

crawling

a

new

approach

to

topic-specific

web

resource

discovery

In

Proc

of

the

International

World-Wide

Web

Conference

-LRB-

-RRB-

-LSB-

-RSB-

Collins-Thompson

K.

Callan

J

A

language

modeling

approach

to

predicting

reading

difficulty

Proceedings

of

HLT/NAACL

Boston

USA

-LSB-

-RSB-

Finn

A.

Kushmerick

N.

Smyth

B.

Fact

or

fiction

Content

classification

for

digital

libraries

Joint

DELOS-NSF

Workshop

on

Personalisation

and

Recommender

Systems

in

Digital

Libraries

-LRB-

Dublin

-RRB-

-LSB-

-RSB-

Foltz

P.

W.

Kintsch

W.

Landauer

T.

K.

Analysis

of

text

coherence

using

Latent

Semantic

Analysis

Discourse

Processes

-LRB-

-RRB-

-LSB-

-RSB-

Google

Kids

Directory

-LSB-

-RSB-

Lavrenko

V.

and

Croft

B.

Relevance-based

language

models

In

Proc

of

the

Annual

International

ACM

SIGIR

Conference

New

Orleans

-LSB-

-RSB-

Ogilvie

P.

and

Callan

J.

Experiments

using

the

Lemur

Toolkit

In

Proc.of

the

Text

Retrieval

Conference

TREC

NIST

Special

Publication

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

information retrieval
reading comprehension
user model
searching