Ranking

Target

Objects

of

Navigational

Queries

Web

navigation

plays

an

important

role

in

exploring

public

interconnected

data

sources

such

as

life

science

data

A

navigational

query

in

the

life

science

graph

produces

a

result

graph

which

is

a

layered

directed

acyclic

graph

-LRB-

DAG

-RRB-

Traversing

the

result

paths

in

this

graph

reaches

a

target

object

set

-LRB-

TOS

-RRB-

The

challenge

for

ranking

the

target

objects

is

to

provide

recommendations

that

reflect

the

relative

importance

of

the

retrieved

object

as

well

as

its

relevance

to

the

specific

query

posed

by

the

scientist

We

present

a

metric

layered

graph

PageRank

-LRB-

lgPR

-RRB-

to

rank

target

objects

based

on

the

link

structure

of

the

result

graph

LgPR

is

a

modification

of

PageRank

it

avoids

random

jumps

to

respect

the

path

structure

of

the

result

graph

We

also

outline

a

metric

layered

graph

ObjectRank

-LRB-

lgOR

-RRB-

which

extends

the

metric

ObjectRank

to

layered

graphs

We

then

present

an

initial

evaluation

of

lgPR

We

perform

experiments

on

a

real-world

graph

of

life

sciences

objects

from

NCBI

and

report

on

the

ranking

distribution

produced

by

lgPR

We

compare

lgPR

with

PageRank

In

order

to

understand

the

characteristics

of

lgPR

an

expert

compared

the

Top

K

target

objects

-LRB-

publications

in

the

PubMed

source

-RRB-

produced

by

lgPR

and

a

word-based

ranking

method

that

uses

text

features

extracted

from

an

external

source

-LRB-

such

as

Entrez

Gene

-RRB-

to

rank

publications

H.

-LSB-

Information

Storage

and

Retrieval

-RSB-

On-line

Information

Services

Web-based

Services

H.

-LSB-

Database

Management

-RSB-

Database

Applications

Data

Mining

Algorithms

Experimentation

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

to

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

WIDM

November

Arlington

Virginia

USA

Copyright

ACM

The

last

few

years

have

seen

an

explosion

in

the

number

of

public

Web

accessible

data

sources

Web

services

and

semantic

Web

applications

While

this

has

occurred

in

many

domains

biologists

have

taken

the

lead

in

making

life

science

data

public

and

biologists

spend

a

considerable

amount

of

time

navigating

through

the

contents

of

these

sources

to

obtain

information

that

is

critical

to

their

research

Providing

meaningful

answers

to

queries

on

life

science

data

sources

poses

some

unique

challenges

First

information

about

a

scientific

entity

e.g.

genes

proteins

sequences

and

publications

may

be

available

in

a

large

number

of

autonomous

sources

and

several

sources

may

provide

different

descriptions

of

some

entity

such

as

a

protein

Second

the

links

between

scientific

objects

-LRB-

links

between

data

entries

in

the

different

sources

-RRB-

are

important

in

this

domain

since

they

capture

significant

knowledge

about

the

relationship

and

interactions

between

these

objects

Third

interconnected

data

entries

can

be

modeled

as

a

large

complex

graph

Queries

could

be

expressed

as

regular

expression

navigational

queries

and

can

more

richly

express

a

user

's

needs

compared

to

simpler

keyword

based

queries

Consider

the

following

navigational

query

Retrieve

publications

related

to

the

gene

tnf

that

are

reached

by

traversing

one

intermediate

-LRB-

protein

or

sequence

-RRB-

entry

This

query

expresses

the

scientist

's

need

to

expand

a

search

for

gene

related

publications

beyond

those

publications

whose

text

directly

addresses

the

tnf

gene

while

still

limiting

the

search

to

publications

that

are

closely

linked

to

gene

entries

Consider

gene

sources

OMIM

Gene

and

Entrez

Gene

protein

sources

NCBI

Protein

and

SwissProt

sequences

in

NCBI

Nucleotide

and

biomedical

publications

in

PubMed

Figure

represents

the

results

of

evaluating

this

navigational

query

against

these

sources

The

result

is

a

layered

DAG

we

refer

to

it

as

a

result

graph

-LRB-

RG

-RRB-

All

paths

in

this

directed

result

graph

-LRB-

RG

-RRB-

start

with

data

entries

in

the

sources

OMIM

Gene

or

Entrez

Gene

this

is

the

first

layer

They

visit

one

intermediate

data

entry

in

sources

NCBI

Protein

Swiss

Prot

or

NCBI

Nucleotide

-LRB-

second

layer

-RRB-

and

they

terminate

in

a

publication

data

entry

in

PubMed

-LRB-

final

layer

-RRB-

The

query

returns

all

objects

in

PubMed

that

are

reached

tnf

keyword

tnf

keyword

NCBI

Nucleotide

Swiss

Prot

PubMed

NCBI

Protein

OMIM

Gene

Entrez

Gene

Figure

An

example

of

a

result

graph

-LRB-

RG

-RRB-

by

traversing

results

paths

these

PubMed

entries

are

re-ferred

to

as

the

target

object

set

-LRB-

TOS

-RRB-

reached

by

traversing

the

result

paths

of

the

RG

In

contrast

a

keyword

based

query

would

not

have

been

able

to

specify

the

set

of

target

publications

Navigational

queries

the

RG

and

the

target

object

set

-LRB-

TOS

-RRB-

that

answers

the

query

are

defined

in

the

paper

It

is

difficult

for

a

user

to

explore

all

target

objects

in

a

reasonable

amount

of

time

and

it

is

important

to

provide

a

ranking

of

the

TOS

As

is

well

known

word

based

ranking

methods

are

very

good

at

identifying

the

most

relevant

results

typically

using

features

extracted

from

the

contents

of

the

target

objects

For

example

-LSB-

-RSB-

produces

a

ranking

of

documents

in

PubMed

that

are

most

relevant

to

a

gene

In

contrast

PageRank

-LSB-

-RSB-

focuses

on

the

importance

of

the

target

object

and

importance

is

transferred

from

other

important

objects

via

the

link

structure

A

recent

technique

ObjectRank

-LSB-

-RSB-

addresses

both

relevance

and

importance

it

exploits

schema

knowledge

to

determine

the

correct

authority

transfer

between

important

pages

We

note

that

there

is

also

research

on

ranking

paths

-LSB-

-RSB-

For

term-based

query

dependent

ranking

we

refer

to

-LSB-

-RSB-

The

focus

of

this

paper

is

to

produce

a

ranking

method

to

select

the

best

target

objects

in

the

RG

that

answer

the

navigational

query

Our

ideal

ranking

must

identify

target

objects

that

are

both

relevant

and

important

The

ranking

must

also

be

query

dependent

since

we

must

guarantee

that

the

target

objects

that

are

ranked

indeed

occur

in

the

RG

and

answer

the

navigational

query

Further

both

relevance

and

importance

must

be

determined

with

respect

to

the

objects

in

TOS

rather

than

with

respect

to

all

the

data

entries

-LRB-

as

is

the

case

with

PageRank

-RRB-

We

propose

two

ranking

metrics

for

the

layered

graph

RG

they

are

layered

graph

PageRank

-LRB-

lgPR

-RRB-

and

layered

graph

ObjectRank

-LRB-

lgOR

-RRB-

lgPR

extends

PageRank

by

distinguishing

different

roles

-LRB-

intermediate

node

answer

node

-RRB-

which

can

be

played

by

the

same

node

in

the

result

graph

It

does

not

perform

random

jumps

so

as

to

respect

the

RG

Our

second

metric

lgOR

is

an

extension

to

ObjectRank

due

to

space

limitations

we

only

discuss

it

briefly

We

report

on

our

preliminary

evaluation

of

lgPR

on

a

real

dataset

from

NCBI/NIH

For

some

navigational

queries

we

apply

lgPR

to

the

corresponding

RG

and

use

the

ranking

distribution

for

lgPR

to

illustrate

that

lgPR

indeed

discriminates

among

the

TOS

objects

We

also

apply

the

original

PR

metric

to

the

object

graph

of

life

science

data

-LRB-

against

which

we

evaluate

the

query

-RRB-

We

compare

with

applying

lgPR

to

the

actual

RG

to

illustrate

that

lgPR

and

PR

produce

dissimilar

rankings

Finally

we

report

on

an

initial

user

experiment

We

consider

a

set

of

complex

queries

typical

of

a

scientist

searching

for

gene

related

PubMed

publications

and

the

Top

K

results

of

a

word

based

ranking

technique

-LRB-

Iowa

-RRB-

that

has

been

shown

to

be

accurate

in

answering

gene

queries

-LSB-

-RSB-

We

compare

the

Iowa

Top

K

publications

with

the

lgPR

Top

K

publications

for

some

sample

gene

related

queries

using

criteria

that

reflect

both

relevance

and

importance

We

use

these

criteria

to

understand

the

characteristics

of

lgPR

The

paper

is

organized

as

follows

Section

describes

the

data

model

navigational

query

language

and

layered

DAG

result

graph

Section

presents

PageRank

lgPR

ObjectRank

and

lgOR

reports

on

preliminary

results

of

an

experimental

study

with

NCBI

data

and

concludes

We

briefly

describe

a

data

model

and

navigational

query

language

for

the

life

science

graph

Details

in

-LSB-

-RSB-

Data

Model

for

the

Life

Science

Graph

The

data

model

comprises

three

levels

ontology

source

and

data

-LRB-

Figure

-RRB-

At

the

ontology

level

a

domain

ontol-Gene

Marker

Publication

Nucleotide

Protein

Disease

OMIM

Gene

NCBI

Gene

PubMed

OMIM

Disease

Swiss

Prot

NCBI

Protein

NCBI

Nucleotide

UniSTS

Ontology

Level

Source

Level

Mappings

Data

Level

Mappings

Figure

A

Data

Model

for

the

Life

Science

Graph

ogy

describes

the

universe

of

discourse

e.g.

a

gene

a

tein

etc.

and

the

relationships

among

them

An

ontology

graph

OG

-LRB-

C

L

C

-RRB-

models

the

domain

ontology

where

nodes

in

C

represent

classes

and

edges

in

L

C

correspond

to

relationships

among

classes

For

example

genes

and

publications

are

classes

in

OG

and

the

association

discuss

relates

publications

with

genes

In

this

paper

we

only

consider

one

type

of

link

isRelatedTo

to

capture

the

semantics

of

a

relationship

therefore

we

omit

all

link

labels

At

the

source

level

a

source

graph

SG

-LRB-

S

L

S

-RRB-

describes

data

sources

and

links

that

implement

logical

classes

-LRB-

C

-RRB-

and

associations

-LRB-

L

C

-RRB-

in

OG

respectively

For

example

PubMed

and

Entrez

Gene

are

sources

that

implement

the

logical

classes

publications

and

genes

respectively

A

mapping

defines

logical

classes

in

C

in

terms

of

the

sources

in

S

that

implement

them

A

link

between

sources

represents

a

hyper-link

a

service

or

an

application

that

connects

these

two

sources

At

the

data

level

a

Data

Graph

is

a

graph

-LRB-

D

L

D

-RRB-

where

D

is

a

set

of

data

entries

and

L

D

is

a

set

of

references

between

entries

A

mapping

m

S

establishes

which

data

entries

in

D

are

published

by

source

S.

Navigational

Query

Language

We

define

a

query

as

a

path

expression

over

the

alphabet

C

in

OG

where

each

class

occurrence

can

optionally

be

an-notated

with

a

Boolean

expression

The

simplest

Boolean

expression

is

the

comparison

of

a

Field

to

a

particular

value

In

this

paper

a

field

can

be

either

source

or

Object

content

and

the

relational

operators

can

be

for

source

and

contains

for

Object

content

A

condition

over

source

and

the

relational

operator

-LRB-

source

name-of-source

-RRB-

restricts

the

query

to

some

specific

sources

that

implement

the

class

A

condition

on

Object

content

and

the

relational

operator

contains

specifies

the

set

of

keywords

that

must

occur

within

objects

in

the

Data

Graph

The

symbol

is

a

wild-card

matching

any

class

and

the

represents

any

relationship

The

query

Retrieve

publications

that

are

related

to

the

gene

tnf

or

aliases

in

human

in

OMIM

or

Enrtez

Gene

and

are

reached

by

traversing

one

intermediate

resource

is

expressed

in

the

navigational

query

language

as

follows

Q

Gene

-LSB-

Object

content

contains

-LCB-

tnf

and

aliases

in

human

-RCB-

and

source

OMIM

or

Entrez

Gene

-RSB-

Publication

The

answer

to

a

query

Q

is

defined

at

the

three

levels

of

the

data

model

It

comprises

three

sets

of

paths

OG

-LRB-

Q

-RRB-

SG

-LRB-

Q

-RRB-

and

DG

-LRB-

Q

-RRB-

The

meaning

of

query

Q

with

respect

to

the

ontology

graph

OG

OG

-LRB-

Q

-RRB-

is

the

set

of

simple

paths

in

OG

that

correspond

to

words

in

the

language

induced

by

the

regular

expression

Q

The

meaning

of

the

query

with

respect

to

the

source

graph

SG

SG

-LRB-

Q

-RRB-

is

the

set

of

all

simple

paths

in

SG

that

correspond

to

mappings

of

the

paths

in

OG

-LRB-

Q

-RRB-

Finally

the

answer

for

query

Q

with

respect

to

the

data

graph

DG

DG

-LRB-

Q

-RRB-

is

the

set

of

simple

paths

in

DG

that

are

the

result

of

mapping

the

paths

in

SG

-LRB-

Q

-RRB-

using

mapping

function

m

S

A

simple

path

does

not

repeat

-LRB-

revisit

-RRB-

the

same

class

data

source

or

data

entry

-LRB-

in

the

same

path

-RRB-

The

queries

that

are

presented

in

this

section

are

typical

queries

posed

by

researchers

At

present

there

are

no

query

evaluation

engines

to

answer

navigational

queries

and

researchers

must

rely

on

manual

navigation

via

browsers

or

they

must

write

scripts

the

latter

involves

labor

to

keep

writing

the

scripts

and

the

scripts

may

be

inefficient

in

answering

these

queries

Result

Graph

The

union

of

paths

in

DG

-LRB-

Q

-RRB-

is

the

result

graph

RG

We

note

that

for

our

query

language

all

the

paths

that

satisfy

a

query

are

of

the

same

length

i.e.

all

the

paths

in

the

sets

OG

-LRB-

Q

-RRB-

SG

-LRB-

Q

-RRB-

and

DG

-LRB-

Q

-RRB-

are

of

the

same

length

We

model

a

result

graph

RG

Q

-LRB-

D

RG

L

RG

-RRB-

for

a

query

Q

as

a

layered

directed

acyclic

graph

comprising

k

layers

L

L

k

where

k

is

determined

by

the

query

The

set

of

nodes

D

RG

corresponds

to

the

union

of

the

data

entries

that

appear

in

the

paths

in

DG

-LRB-

Q

-RRB-

L

RG

represents

the

links

among

these

data

entries

A

layer

L

i

is

composed

of

the

union

of

the

data

entries

in

the

paths

DG

-LRB-

Q

-RRB-

that

appear

in

the

i-th

position

of

the

paths

The

data

entries

in

the

k-th

layer

are

called

the

target

objects

and

they

form

the

target

object

set

-LRB-

TOS

-RRB-

of

the

RG

Note

that

since

the

result

graph

has

multiple

paths

and

since

a

source

may

occur

in

different

layers

of

these

paths

the

same

data

entry

may

appear

multiple

times

in

the

different

layers

depending

on

its

connectivity

to

other

data

entries

In

this

case

each

occurrence

of

the

data

entry

is

represented

independently

within

each

layer/path

in

which

it

occurs

The

result

graph

framework

distinguishes

the

different

roles

-LRB-

intermediate

node

answer

node

-RRB-

which

can

be

played

by

the

same

node

in

the

result

graph

Figure

is

a

layered

RG

for

the

following

query

Retrieve

publications

related

to

the

gene

tnf

traversing

one

intermediate

source

it

has

three

layers

The

first

layer

corresponds

to

the

genes

in

the

sources

OMIM

Gene

and

Entrez

Gene

that

are

related

to

the

keyword

tnf

The

second

layer

are

the

entries

in

the

sources

NCBI

Protein

Swiss

Prot

or

NCBI

Nucleotide

that

are

reached

by

objects

in

the

first

layer

Finally

the

target

objects

in

the

third

layer

-LRB-

TOS

-RRB-

are

the

publications

in

PubMed

that

are

linked

to

the

objects

in

the

second

layer

We

briefly

describe

the

PageRank

metric

-LSB-

-RSB-

and

then

discuss

our

metric

lgPR

for

layered

DAGs

We

briefly

discuss

the

ObjectRank

metric

-LSB-

-RSB-

and

our

extension

lgOR

PageRank

PageRank

assumes

that

links

between

pages

confer

authority

A

link

from

page

i

to

page

j

is

evidence

that

i

is

suggesting

that

j

is

important

The

importance

from

page

i

that

is

contributed

to

page

j

is

inversely

proportional

to

the

outdegree

of

i.

Let

N

i

be

the

outdegree

of

page

i

The

corresponding

random

walk

on

the

directed

web

graph

can

be

expressed

by

a

transition

matrix

A

as

follows

A

-LSB-

i

j

-RSB-

N

i

if

there

is

an

edge

from

i

to

j

otherwise

Let

E

be

an

arbitrary

vector

over

the

webpages

representing

the

initial

probability

of

visiting

a

page

Let

d

be

the

probability

of

following

a

link

from

a

page

and

let

-LRB-

d

-RRB-

be

the

probability

of

a

random

jump

to

a

page

The

PageRank

ranking

vector

R

dA

R

-LRB-

d

-RRB-

E.

R

converges

for

the

web

graph

with

any

E

since

generally

the

web

graph

is

aperiodic

and

irreducible

-LSB-

-RSB-

PageRank

can

not

be

directly

applied

to

a

layered

graph

A

Markov

Chain

is

irreducible

if

and

only

if

the

graph

contains

only

one

strongly

connected

component

RG

is

not

outgoing

links

with

respect

to

the

query

There

are

several

potential

ways

to

extend

PageRank

for

RG

First

one

can

ignore

links

that

point

to

pages

without

outgoing

edges

since

these

pages

do

not

affect

the

ranking

of

other

pages

-LSB-

-RSB-

However

we

are

specifically

interested

in

obtaining

a

ranking

for

the

TOS

or

the

objects

in

the

last

layer

of

the

layered

result

graph

RG

with

no

outgoing

links

we

can

not

ignore

these

pages

Another

possibility

is

modifying

the

transition

matrix

probability

so

that

one

takes

a

random

jump

from

a

node

in

the

TOS

-LSB-

-RSB-

This

will

ensure

that

the

graph

will

be

irreducible

and

aperiodic

However

this

would

arbitrarily

modify

RG

whose

structure

is

determined

by

the

query

modifying

RG

will

not

assure

that

it

answers

the

query

To

summarize

the

extensions

to

PageRank

in

the

literature

can

not

be

applied

to

the

problem

of

ranking

the

target

object

set

TOS

of

RG

Layered

Graph

PageRank

-LRB-

lgPR

-RRB-

We

describe

layered

graph

PageRank

to

rank

the

TOS

The

Metric

Table

lists

the

symbols

used

to

compute

lgPR

Symbol

Meaning

RG

-LRB-

V

RG

E

RG

-RRB-

Result

Graph

a

layered

DAG

with

objects

V

RG

and

edges

E

RG

e

E

RG

an

edge

in

E

RG

R

ranking

vector

for

objects

in

RG

R

ini

initial

ranking

vector

A

lg

the

transition

matrix

for

objects

in

RG

k

the

number

of

layers

in

the

result

graph

OutDeg

RG

-LRB-

u

p

-RRB-

outdegree

from

object

u

at

layer

p

-LRB-

across

multiple

link

types

to

objects

in

layer

p

Table

Symbols

used

by

lgPR

The

layered

DAG

result

graph

RG

is

represented

by

a

transition

matrix

A

lg

to

be

defined

next

Note

that

an

object

in

the

object

graph

may

occur

in

multiple

paths

of

the

result

graph

in

different

layers

it

will

be

replicated

in

the

transition

matrix

for

each

occurrence

Each

object

u

at

layer

p

will

have

an

entry

in

the

transition

matrix

to

some

object

v

at

layer

q

We

denote

the

occurrence

of

them

as

u

p

and

v

q

respectively

The

ranking

vector

R

is

defined

by

a

transition

matrix

A

lg

and

initial

ranking

vector

R

ini

is

as

follows

R

A

k

lg

R

ini

-LRB-

k

l

A

lg

-RRB-

R

ini

We

pick

R

ini

as

follows

the

entry

for

an

object

in

R

ini

is

if

this

object

is

a

link

in

start

layer

and

otherwise

The

transition

matrix

A

lg

is

computed

as

follows

A

lg

-LSB-

i

p

u

j

q

v

-RSB-

OutDeg

RG

-LRB-

u

p

-RRB-

if

OutDeg

RG

-LRB-

u

p

-RRB-

and

e

-LRB-

u

p

v

q

-RRB-

E

RG

otherwise

Note

that

we

define

the

outdegree

of

each

object

in

RG

to

only

consider

those

edges

that

actually

occur

in

RG

and

link

to

objects

in

the

next

layer

This

reflect

the

probability

that

a

user

follows

an

object

path

in

the

RG

In

contrast

PageRank

considers

all

outgoing

edges

from

a

page

Unlike

PageRank

lgPR

differentiates

the

occurrence

of

a

data

entry

in

different

layers

as

well

as

the

links

to

entries

in

subsequent

layers

lgPR

is

thus

able

to

reflect

the

role

of

objects

and

links

-LRB-

from

the

entire

graph

of

data

entries

-RRB-

in

answering

a

navigational

query

Suppose

an

object

a

occurs

in

an

intermediate

layer

as

well

as

in

the

TOS

of

the

RG

It

is

possible

that

a

is

able

to

convey

authority

to

other

objects

in

the

TOS

However

a

may

not

rank

very

high

in

the

TOS

for

this

query

This

characteristic

is

unique

to

lgPR

Thus

the

score

associated

with

the

object

is

query

dependent

to

reflect

the

role

played

by

the

object

in

the

result

graph

Convergence

Property

This

transition

matrix

A

lg

is

neither

irreducible

nor

aperiodic

as

all

rows

for

target

objects

contain

only

's

The

matrix

A

is

a

nilpotent

matrix

and

the

number

of

layers

is

the

index

We

provide

two

defintions

-LRB-

details

in

-LSB-

-RSB-

-RRB-

Definition

A

square

matrix

A

is

a

nilpotent

matrix

if

there

exists

some

positive

integer

k

such

that

A

k

but

A

k

Integer

k

is

known

as

the

index

of

A.

Definition

Let

k

be

the

index

of

A.

-LCB-

A

k

x

A

k

x

Ax

x

-RCB-

form

a

Jordan

Chain

where

x

is

any

vector

such

that

A

k

x

A

characteristic

of

a

nilpotent

matrix

is

that

its

only

eigenvalue

is

The

consequence

is

that

any

vector

x

is

an

eigenvector

of

A

as

long

as

Ax

From

the

previous

definition

-LCB-

A

k

lg

R

ini

A

k

lg

R

ini

A

lg

R

ini

A

lg

-RCB-

forms

a

Jordan

Chain

since

A

k

lg

R

ini

We

show

following

two

lemmas

without

providing

proof

in

this

paper

Lemma

Jordan

chain

-LCB-

A

k

lg

R

ini

A

k

lg

R

ini

A

lg

R

ini

A

lg

-RCB-

is

a

linearly

independent

set

Lemma

-LCB-

A

k

lg

R

ini

A

k

lg

R

ini

A

lg

R

ini

A

lg

-RCB-

consists

of

a

sequence

of

ranking

vectors

In

R

ini

only

objects

in

layer

have

non-zero

scores

In

ranking

vector

A

m

lg

R

ini

only

objects

in

layer

m

receive

non-zero

scores

The

final

ranking

vector

by

lgPR

is

the

first

eigenvector

in

the

Jordan

Chain

given

the

above

initial

ranking

vector

R

ini

and

the

transition

matrix

A

lg

While

the

traditional

PageRank

algorithm

converges

on

a

ranking

in

multiple

iterations

lgPR

can

be

computed

in

exactly

k

iterations

Note

that

because

RG

is

a

layered

DAG

we

can

use

link

matrices

each

of

which

represents

links

between

neighboring

layers

instead

of

the

single

transition

matrix

A

lg

for

the

entire

graph

We

also

use

keywords

to

filter

query

answers

at

each

iteration

Layered

Graph

ObjectRank

-LRB-

lgOR

-RRB-

PR

is

computed

a

priori

on

the

complete

data

graph

and

is

independent

of

the

RG

A

recent

technique

ObjectRank

-LSB-

-RSB-

extends

PageRank

to

consider

relevance

of

query

keywords

It

exploits

schema

knowledge

to

determine

the

correct

authority

transfer

in

a

schema

graph

In

ObjectRank

the

authority

flows

between

objects

according

to

semantic

connections

It

does

so

by

determining

an

authority

weight

for

each

edge

in

their

schema

graph

The

ranking

is

-LRB-

keyword

-RRB-

query

dependent

Due

to

space

limitations

we

do

not

provide

the

details

of

the

ObjectRank

metric

Instead

we

briefly

describe

how

irreducible

since

the

last

layer

in

RG

contains

nodes

with

no

the

transition

matrix

for

lgPR

can

be

extended

to

consider

the

authority

weights

associated

with

edges

that

occur

in

RG

Consider

a

metric

layered

graph

ObjectRank

-LRB-

lgOR

-RRB-

The

difference

from

lgPR

is

the

transition

matrix

A

OG

It

is

as

follows

A

-LSB-

i

p

u

j

q

v

-RSB-

-LRB-

e

E

RG

-RRB-

if

e

-LRB-

u

p

v

q

-RRB-

E

RG

otherwise

-LRB-

e

E

RG

-RRB-

-LRB-

e

ESG

-RRB-

OutDeg

-LRB-

u

p

e

ESG

-RRB-

if

OutDeg

-LRB-

u

p

e

E

SG

-RRB-

if

OutDeg

-LRB-

u

p

e

E

SG

-RRB-

Let

the

edge

between

u

p

and

v

q

map

to

an

edge

E

SG

in

the

SG

-LRB-

E

SG

-RRB-

represents

the

authority

transfer

weight

associated

with

E

SG

OutDeg

-LRB-

u

p

e

E

SG

-RRB-

is

the

outdegree

in

RG

of

type

E

SG

As

discussed

in

-LSB-

-RSB-

the

success

of

ObjectRank

depends

on

correctly

determining

the

authority

weight

to

be

associated

with

each

link

Figure

-LRB-

next

section

-RRB-

illustrates

the

source

graph

that

we

use

in

our

evaluation

of

navigational

queries

For

lgOR

to

be

successful

an

authority

weight

may

have

to

be

associated

with

each

link

in

each

result

path

-LRB-

type

-RRB-

in

the

RG

Experiments

with

users

to

determine

the

correct

authority

weights

for

lgOR

is

planned

for

future

work

Currently

the

importance

is

computed

after

query

evaluation

We

compute

result

graph

first

then

ranking

for

the

reason

that

the

transition

matrix

is

defined

in

terms

of

outdegree

in

the

RG

This

motivates

further

research

of

combination

of

two

problems

whose

ideal

solution

is

to

ranking

objects

during

query

evaluation

We

report

on

experiments

on

real

world

data

We

show

that

the

lgPR

ranking

distribution

has

the

ability

to

differentiate

among

the

target

objects

of

the

RG

and

it

is

different

from

PageRank

A

user

compared

the

Top

K

results

of

lgPR

and

a

word

based

ranking

-LRB-

Iowa

-RRB-

-LSB-

-RSB-

using

criteria

that

reflect

both

importance

and

relevance

to

determine

their

characteristics

Experiment

Setting

NCBI/NIH

is

the

gatekeeper

for

biological

data

produced

using

federal

funds

in

the

US

We

consider

a

source

graph

SG

of

data

sources

and

links

Figure

presents

the

source

graph

used

in

this

task.We

used

several

hundred

keywords

to

sample

data

from

these

sources

-LRB-

the

EFetch

utility

-RRB-

and

followed

links

to

the

other

sources

-LRB-

the

ELink

utility

-RRB-

We

created

a

data

graph

of

approximately

million

objects

and

million

links

We

note

that

several

objects

are

machine

predicted

objects

so

it

is

not

uncommon

that

they

have

no

links

The

object

identifiers

for

the

data

entries

-LRB-

nodes

of

the

data

graph

-RRB-

and

the

pair

of

object

identifiers

-LRB-

links

-RRB-

were

stored

in

a

relational

database

Table

identifies

the

queries

and

keywords

that

were

used

in

this

experiment

The

symbols

g

p

n

s

refer

to

classes

gene

publication

nucleotide

and

SNP

respectively

Note

that

is

the

wild

card

and

can

match

all

the

classes

and

sources

-LRB-

in

the

source

graph

-RRB-

For

each

navigational

query

the

source

paths

that

answer

the

query

were

determined

using

an

algorithm

described

in

Class

PubMed

PmId

Title

Class

Author

Name

Class

Journal

Name

ClassYear

Year

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

Class

Lash

Terms

TermId

Descrip

-LRB-

-RRB-

-LRB-

-RRB-

Class

Entrez

Gene

EGId

-LRB-

-RRB-

-LRB-

-RRB-

Class

Geo

GeoId

ClassOMIM

CddId

Class

UniSTS

USId

Class

UniGene

UGId

Class

Entrez

Protein

EPId

Class

dbSNP

dbSID

Class

CDD

CddId

Class

Entrez

Nucleotide

NuId

-LRB-

-RRB-

Figure

Source

Graph

for

User

Evaluation

Queries

g.n.p

g.s.p

g.n.s.p

g.s.n.p

g.s.g.n.p

g.s.n.g.p

g.

p

g.

p

g.

p

parkinson

disease

aging

cancer

Keywords

diabetes

flutamide

stress

degenerative

joint

tnf

insulin

fluorouracil

osteoarthritis

sarcoma

Table

Experiment

setting

-LSB-

-RSB-

Evaluating

the

paths

in

the

data

graph

for

each

source

path

was

implemented

by

SQL

queries

Since

a

result

graph

RG

could

involve

multiple

source

paths

whose

computation

may

overlap

we

applied

several

multiple

query

optimization

techniques

The

SQL

queries

were

executed

on

Enterprise

Server

installed

on

a

GHz

Intel

Xeon

processor

with

GB

RAM

The

execution

time

for

these

queries

varied

considerably

depending

on

the

size

and

shape

of

RG

If

we

consider

the

query

g.n.p

with

keyword

degenerative

joint

used

to

filter

g

one

source

path

was

ranked

in

approximately

second

However

the

query

-LRB-

g.

p

-RRB-

with

the

keyword

aging

used

to

filter

g

created

a

very

large

result

graph

and

the

execution

time

for

this

was

approximately

seconds

Typically

the

We

note

that

computing

the

high

scoring

TOS

objects

of

the

RG

efficiently

is

a

related

but

distinct

optimization

problem

lgPR

Distribution

We

report

on

the

query

-LRB-

g.

p

-RRB-

i.e.

paths

from

genes

to

publications

via

one

intermediate

source

Figures

and

report

on

the

distribution

of

scores

produced

by

the

lgPR

metric

for

the

target

objects

in

TOS

for

some

representative

queries

The

first

bars

represent

scores

in

the

range

-LRB-

-RRB-

to

-LRB-

-RRB-

and

the

last

bar

represents

the

range

-LRB-

-RRB-

Fig

shows

that

a

small

number

of

objects

have

very

high

score

and

the

majority

have

a

low

score

As

expected

many

queries

and

keywords

produced

distributions

that

were

similar

to

Figure

Most

of

the

objects

in

TOS

in

this

case

approx.

objects

had

a

very

low

score

and

less

than

object

had

a

score

in

the

range

-LRB-

-RRB-

However

we

made

an

interesting

observation

that

some

queries

produced

distributions

that

were

similar

to

Figure

In

this

case

while

many

of

the

results

-LRB-

approx.

-RRB-

had

low

scores

in

the

range

-LRB-

-RRB-

objects

had

scores

in

the

range

-LRB-

-RRB-

and

objects

had

scores

in

between

lgPR

score

Number

of

objects

Figure

Histogram

for

query

g

-LSB-

Object

content

contains

aging

-RSB-

p

lgPR

score

Number

of

objects

Figure

Histogram

for

query

g

-LSB-

Object

content

contains

degenerative

joint

-RSB-

p

Finally

we

compared

the

ranking

produced

by

lgPR

and

PageRank

We

apply

PageRank

to

the

entire

data

graph

of

million

objects

and

million

links

described

in

section

For

the

three

sample

queries

-LRB-

described

in

the

next

section

-RRB-

there

are

no

PubMed

ID

's

in

common

to

the

Top

for

each

of

the

queries

except

that

the

top

of

query

with

Lash

term

allele

have

PubMed

publications

in

common

and

top

of

same

query

have

in

common

We

speculate

that

the

link

structure

of

the

RG

is

distinct

compared

to

the

link

structure

of

the

data

graph

hence

applying

lgPR

to

the

RG

results

in

dissimilar

ranking

compared

to

a

priori

applying

PageRank

to

the

entire

data

graph

We

summarize

that

the

lgPR

score

can

both

identify

those

objects

with

a

very

low

ranking

that

may

not

be

of

interest

to

the

user

However

it

can

also

be

used

to

discriminate

amongst

objects

in

the

TOS

whose

ranking

has

a

much

lower

variation

of

scores

Finally

lgPR

ranking

is

not

the

same

as

that

produced

by

PageRank

applied

to

the

entire

data

graph

User

Evaluation

In

our

user

evaluation

of

lgPR

we

consider

a

set

of

complex

queries

typical

of

a

scientist

searching

for

gene

related

PubMed

publications

and

the

Top

K

results

of

a

word

based

ranking

technique

-LRB-

Iowa

-RRB-

that

has

been

shown

to

be

accurate

in

answering

gene

queries

-LSB-

-RSB-

We

compare

the

Iowa

Top

K

publications

with

the

lgPR

Top

K

publications

for

some

sample

gene

related

queries

We

use

criteria

that

reflect

both

relevance

and

importance

to

identify

characteristics

of

lgPR

Researchers

are

particularly

interested

in

genetic

and

phe-notypic

variations

associated

with

genes

these

phenomena

are

often

studied

in

the

context

of

diseases

in

a

chromoso-mal

region

identified

by

a

genomic

marker

-LRB-

a

unique

known

sequence

-RRB-

associated

with

the

disease

Genetic

and

pheno-typic

knowledge

are

described

using

terms

of

the

Lash

controlled

vocabulary

-LSB-

-RSB-

We

focus

on

a

branch

of

the

Lash

vocabulary

that

relates

to

phenotypes

and

population

genetics

Terms

of

interest

include

linkage

disequilibrium

quantitative

trait

locus

and

allele

Figure

presents

a

portion

of

the

Lash

controlled

vocabulary

-LRB-

term

hierarchy

-RRB-

LD

is

not

listed

as

the

synonym

to

the

term

linkage

disequilibrium

because

LD

may

often

refer

to

another

concept

In

the

following

experiment

we

did

not

consider

the

plural

form

of

some

terms

such

as

alleles

to

allele

but

this

can

be

extended

in

the

future

studies

EPIGENETIC

ALTERATION

GENOMIC

SEGMENT

LOSS

GENOMIC

SEGMENT

GAIN

GENOMIC

SEQUENCE

ALTERATION

PHENOTYPIC

ASSOCIATION

-LRB-

synonym

phenotype

trait

-RRB-

-LRB-

a

-RRB-

locus

association

-LRB-

synonym

locus

loci

-RRB-

i.

linkage

ii

quantitative

trait

locus

-LRB-

synonym

QTL

-RRB-

-LRB-

b

-RRB-

allelic

association

-LRB-

synonym

allele

-RRB-

i.

linkage

disequilibrium

Figure

Branch

in

Hierarchical

controlled

vocabulary

of

genetics

terms

-LRB-

Lash

Controlled

Vocabulary

-RRB-

The

navigational

query

used

in

our

evaluation

experiment

can

be

described

in

English

as

follows

Return

all

publications

in

PubMed

that

are

linked

to

an

Entrez

Gene

entry

that

is

related

to

the

human

gene

TNF

-LRB-

or

its

aliases

-RRB-

The

entry

in

PubMed

must

contain

an

STS

marker

and

a

term

from

the

Lash

controlled

vocabulary

We

used

the

query

term

TNF

AND

-LSB-

TAXID

-RSB-

to

sample

data

from

Entrez

Gene

We

then

followed

paths

to

PubMed

Table

reports

on

the

number

of

entries

in

Entrez

Gene

as

well

as

the

cardinality

of

the

TOS

for

some

sample

queries

We

briefly

describe

the

word-based

ranking

method

-LRB-

Iowa

-RRB-

that

focuses

on

ranking

documents

retrieved

by

PubMed

Note

the

Taxonomy

ID

for

human

is

-LSB-

-RSB-

and

term

-LSB-

TAXID

-RSB-

was

used

to

select

human

genes

We

use

g

-LSB-

tnf

and

aliases

in

human

-RSB-

to

denote

g

-LSB-

Object

content

contains

-LCB-

tnf

and

aliases

in

human

-RCB-

-RSB-

the

entries

in

the

first

column

of

Table

are

similar

Query

Cardinality

of

TOS

g

-LSB-

tnf

and

aliases

in

human

-RSB-

g

-LSB-

tnf

and

aliases

in

human

-RSB-

p

-LSB-

STS

marker

and

allele

-RSB-

g

-LSB-

tnf

and

aliases

in

human

-RSB-

p

-LSB-

STS

marker

and

linkage

disequilibrium

-RSB-

g

-LSB-

tnf

and

aliases

in

human

-RSB-

p

-LSB-

STS

marker

and

quantitative

trait

locus

-RSB-

Table

Cardinality

of

TOS

for

human

gene

queries

-LSB-

-RSB-

so

that

relevant

documents

are

ranked

higher

than

non-relevant

documents

This

method

relies

on

using

post-retrieval

queries

-LRB-

ranking

queries

-RRB-

au-tomatically

generated

from

an

external

source

viz.

Entrez

Gene

-LRB-

Locus

Link

-RRB-

to

rank

retrieved

documents

The

research

shows

that

ranking

queries

generated

from

a

combination

of

the

Official

Gene

Symbol

Official

Gene

Name

Alias

Symbols

Entrez

Summary

and

Protein

Products

-LRB-

optional

-RRB-

were

very

effective

in

ranking

relevant

documents

higher

in

the

retrieved

list

Documents

and

ranking

queries

are

represented

using

the

traditional

vector-space

representation

commonly

used

in

information

retrieval

Given

a

gene

the

cosine

similarity

score

between

the

ranking

query

vector

for

the

gene

and

each

document

vector

is

computed

Cosine

scores

are

in

the

-LSB-

-RSB-

range

and

documents

assigned

a

higher

score

are

ranked

higher

than

documents

with

a

lower

score

In

the

absence

of

summary

and

protein

product

information

ranking

queries

generated

from

the

gene

symbol

name

and

aliases

are

used

to

rank

retrieved

documents

In

this

experiment

study

we

are

working

on

the

Bio

Web

documents

alone

We

use

the

following

criteria

to

compare

the

Top

K

results

from

Iowa

and

lgPR

to

understand

basic

characteristics

of

the

two

methods

Criteria

labeled

R

appear

to

judge

the

relevance

of

the

paper

and

those

labeled

I

appear

to

judge

importance

Some

criteria

appear

to

judge

both

and

are

labeled

R

I.

R

Does

the

title

or

abstract

of

the

article

contain

the

term

TNF

or

its

aliases

in

human

Does

the

article

discuss

immune

response

R

I

Does

the

article

contain

any

disease

related

terms

Does

the

article

contain

any

genomic

components

-LRB-

genes

markers

snps

sequences

etc.

-RRB-

R

I

Does

the

article

discuss

biological

processes

related

to

the

Lash

terms

R

I

What

is

the

connectivity

of

the

article

to

gene

entries

in

Entrez

Gene

that

are

related

to

TNF

Note

that

as

shown

in

Table

there

are

Entrez

Gene

entries

that

are

related

to

human

gene

TNF

Each

PubMed

publication

was

reached

by

following

a

result

path

through

the

result

graph

RG

that

started

with

one

of

these

Entrez

Gene

entries

However

some

PubMed

publications

may

have

been

reached

along

multiple

paths

in

the

RG

reflecting

much

greater

connectivity

I

What

is

the

category

of

the

article

-LRB-

review

survey

etc.

-RRB-

Does

the

article

address

some

specific

topics

or

is

it

a

broad

brush

article

I

Where

did

the

article

appear

What

is

the

journal

impact

factor

Has

the

article

been

highly

cited

Top

Rel

Imp

Criteria

PMID

-LRB-

-RRB-

-LRB-

-RRB-

H

M

H

L

L

L

H

L

H

M

H

H

H

H

H

L

H

H

H

M

H

L

H

H

H

L

H

L

H

H

H

M

H

L

H

H

H

M

H

L

H

L

H

H

H

L

H

L

H

H

H

H

H

L

H

L

H

L

H

H

Table

Relevance

and

Importance

of

Top

Puli-cations

Reported

by

Iowa

Ranking

Method

Top

Rel

Imp

Criteria

PMID

-LRB-

-RRB-

-LRB-

-RRB-

H

H

H

H

H

H

H

H

H

H

H

H

M

H

H

L

H

L

H

H

H

H

H

L

H

H

H

L

H

L

H

H

H

H

H

L

L

H

H

L

H

H

H

H

H

H

H

L

H

H

H

L

H

H

H

H

H

H

H

H

Table

Relevance

and

Importance

of

Top

Puli-cations

Reported

by

lgPR

Ranking

Method

Tables

and

report

Top

publications

in

PubMed

that

are

linked

to

an

Entrez

Gene

entry

that

is

related

to

human

gene

TNF

and

contain

the

term

linkage

disequilibrium

The

first

column

reports

the

PubMed

identifiers

-LRB-

PMIDs

-RRB-

of

the

Top

publications

returned

by

the

Iowa

and

the

lgPR

ranking

methods

The

human

evaluation

results

are

reported

in

the

fourth

to

the

ninth

columns

using

the

the

six

criteria

listed

above

An

H

represents

the

publication

is

highly

matched

to

the

correspoinding

criteria

-LRB-

M

and

L

represents

medium

and

low

respectively

-RRB-

An

H

indicates

The

PubMed

entry

is

linked

to

the

human

gene

TNF

with

Entrez

Gene

identifier

GeneID

The

publication

contains

both

diseases

related

terms

and

genomic

components

The

publication

contains

multiple

Lash

terms

The

connectivity

is

high

if

there

are

more

than

five

related

gene

entries

linked

to

the

publication

A

research

article

considered

more

important

than

a

review

or

a

survey

and

a

more

specific

topic

is

better

The

article

is

published

in

a

journal

with

the

impact

factor

higher

than

or

the

article

is

cited

by

ten

or

more

publications

We

then

score

the

relevance

-LRB-

rel

-RRB-

and

the

importance

-LRB-

imp

-RRB-

in

the

second

and

the

third

columns

by

combining

the

number

of

H

and

M

reported

in

the

six

criteria

Criteria

weighs

twice

compared

to

the

other

five

criteria

We

use

a

number

between

and

in

which

indicates

the

corresponding

PubMed

entry

is

highly

relevant

or

highly

important

to

the

given

query

While

both

rankings

appear

to

identify

good

documents

Iowa

appears

to

favor

relevant

documents

based

on

their

word

content

lgPR

appears

to

exploit

the

link

structure

of

the

RG

and

have

higher

in-terconnectivity

to

TNF

related

entries

in

Entrez

gene

The

publications

retrieved

by

lgPR

are

more

likely

to

contain

diseases

related

terms

or

genomic

components

The

Iowa

ranking

has

a

primary

focus

on

the

relevance

of

documents

-LRB-

based

on

document

contents

it

is

not

able

to

differentiate

the

importance

of

these

relevant

documents

In

contrast

lgPR

has

a

primary

focus

on

importance

-LRB-

based

on

the

link

structure

of

the

result

graph

-RRB-

it

is

not

able

to

differentiate

the

relevance

of

important

documents

We

conclude

that

further

study

is

needed

to

determine

how

we

can

exploit

the

characteristics

of

both

methods

There

is

no

intersection

between

two

sets

of

Top

publications

returned

by

these

two

ranking

methods

The

first

common

PMID

is

which

is

ranked

in

the

Iowa

method

and

by

the

lgPR

method

We

have

defined

a

model

for

life

science

sources

The

answer

to

a

navigational

query

are

the

target

objects

-LRB-

TOS

-RRB-

of

a

layered

graph

Result

Graph

-LRB-

RG

-RRB-

We

define

two

ranking

metrics

layered

graph

PageRank

-LRB-

lgPR

-RRB-

and

layered

graph

ObjectRank

-LRB-

lgOR

-RRB-

We

also

report

on

the

results

of

experiments

on

real

world

data

from

NCBI/NIH

We

show

that

the

ranking

distribution

of

lgPR

indeed

discriminates

among

the

TOS

objects

of

the

RG

The

lgPR

distribution

is

not

the

same

as

applying

PageRank

a

priori

to

the

data

graph

We

perform

a

user

experiment

on

complex

queries

typical

of

a

scientist

searching

for

gene

related

PubMed

publications

and

the

Top

K

results

of

a

word

based

ranking

technique

-LRB-

Iowa

-RRB-

that

has

been

shown

to

be

accurate

in

answering

gene

queries

the

query

Using

criteria

that

judge

both

relevance

and

importance

we

explore

the

characteristics

of

these

two

rankings

Our

preliminary

evaluation

indicates

there

may

be

a

benefit

or

a

meta-ranking

We

briefly

presented

layered

graph

ObjectRank

-LRB-

lgOR

-RRB-

which

is

an

extension

to

ObjectRank

The

challenge

of

ObjectRank

is

determining

the

correct

authority

weight

for

each

edge

For

lgOR

we

need

to

find

the

weight

for

the

edges

that

occur

in

RG

Experiments

with

users

to

determine

the

correct

authority

weights

for

lgOR

is

planned

for

future

work

We

expect

that

IR

techniques

can

be

used

to

determine

authority

weights

-LSB-

-RSB-

Andrey

Balmin

Vagelis

Hristidis

and

Yannis

Papakonstantinou

Objectrank

Authority-based

keyword

search

in

databases

In

VLDB

pages

-LSB-

-RSB-

Magdalini

Eirinaki

Michalis

Vazirgiannis

and

Dimitris

Kapogiannis

Web

path

recommendations

based

on

page

ranking

and

markov

models

In

WIDM

Proceedings

of

the

annual

ACM

international

workshop

on

Web

information

and

data

management

pages

New

York

NY

USA

ACM

Press

-LSB-

-RSB-

Taher

H.

Haveliwala

Topic-sensitive

pagerank

In

WWW

Proceedings

of

the

international

conference

on

World

Wide

Web

pages

New

York

NY

USA

ACM

Press

-LSB-

-RSB-

Homo

sapiens

in

NCBI

Taxonomy

Browser

mode

Info

id

-LSB-

-RSB-

Sepandar

D.

Kamvar

Taher

H.

Haveliwala

Christopher

D.

Manning

and

Gene

H.

Golub

Extrapolation

methods

for

accelerating

pagerank

computations

In

WWW

pages

-LSB-

-RSB-

Z.

Lacroix

L.

Raschid

and

M.-E

Vidal

Semantic

model

ot

integrate

biological

resources

In

International

Workshop

on

Semantic

Web

and

Databases

-LRB-

SWDB

-RRB-

Atlanta

Georgia

USA

April

-LSB-

-RSB-

Alex

Lash

Woei-Jyh

Lee

and

Louiqa

Raschid

A

methodology

to

enhance

the

semantics

of

links

between

PubMed

publications

and

markers

in

the

human

genome

In

Fifth

IEEE

Symposium

on

Bioinformatics

and

Bioengineering

-LRB-

BIBE

-RRB-

pages

Minneapolis

Minnesota

USA

October

-LSB-

-RSB-

Carl

D.

Meyer

Matrix

Analysis

and

Applied

Linear

Algebra

Society

for

Industrial

and

Applied

Mathmatics

-LSB-

-RSB-

G.

Mihaila

F.

Naumann

L.

Raschid

and

M.

Vidal

A

data

model

and

query

language

to

explore

enhanced

links

and

paths

in

life

sciences

data

sources

Proceedings

of

the

Workshop

on

Web

and

Databases

WebDB

Maryland

USA

-LSB-

-RSB-

Rajeev

Motwani

and

Prabhakar

Raghavan

Randomized

algorithms

Cambridge

University

Press

New

York

NY

USA

-LSB-

-RSB-

Lawrence

Page

Sergey

Brin

Rajeev

Motwani

and

Terry

Winograd

The

pagerank

citation

ranking

Bringing

order

to

the

web

Technical

report

Stanford

Digital

Library

Technologies

Project

-LSB-

-RSB-

Matthew

Richardson

and

Pedro

Domingos

Combining

link

and

content

information

in

web

search

In

Web

Dynamics

Web

Dynamics

Adapting

to

Change

in

Content

Size

Topology

and

Use

pages

Springer

-LSB-

-RSB-

Aditya

Kumar

Sehgal

and

Padmini

Srinivasan

Retrieval

with

gene

queries

BMC

Bioinformatics

-LSB-

-RSB-

Maria-Esther

Vidal

Louiqa

Raschid

Natalia

M

arquez

Marelis

C

ardenas

and

Yao

Wu

Query

rewriting

in

the

semantic

web

In

InterDB

B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O