Ranking

Flows

from

Sampled

Traffic

Most

of

the

theoretical

work

on

sampling

has

addressed

the

inversion

of

general

traffic

properties

such

as

flow

size

distribution

average

flow

size

or

total

number

of

flows

In

this

paper

we

make

a

step

towards

understanding

the

impact

of

packet

sampling

on

individual

flow

properties

We

study

how

to

detect

and

rank

the

largest

flows

on

a

link

To

this

end

we

develop

an

analytical

model

that

we

validate

on

real

traces

from

two

networks

First

we

study

a

blind

ranking

method

where

only

the

number

of

sampled

packets

from

each

flow

is

known

Then

we

propose

a

new

method

protocol-aware

ranking

where

we

make

use

of

the

packet

sequence

number

-LRB-

when

available

in

transport

header

-RRB-

to

infer

the

number

of

non-sampled

packets

from

a

flow

and

hence

to

improve

the

ranking

Surprisingly

our

analytical

and

experimental

results

indicate

that

a

high

sampling

rate

-LRB-

and

even

more

depending

on

the

number

of

top

flows

to

be

ranked

-RRB-

is

required

for

a

correct

blind

ranking

of

the

largest

flows

The

sampling

rate

can

be

reduced

by

an

order

of

magnitude

if

one

just

aims

at

detecting

these

flows

or

by

using

the

protocol-aware

method

C.

-LSB-

Computer

Systems

Organization

-RSB-

PERFORMANCE

OF

SYSTEMS

Measurement

techniques

Measurement

Performance

Experimentation

This

work

was

done

while

the

author

was

visiting

Intel

Research

Cambridge

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

to

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

CoNEXT

October

Toulouse

France

Copyright

ACM

The

list

of

the

top

users

or

applications

is

one

of

the

most

useful

statistics

to

be

extracted

from

network

traffic

Network

operators

use

the

knowledge

of

the

most

popular

destinations

to

identify

emerging

markets

and

applications

or

to

locate

where

to

setup

new

Points

of

Presence

Content

delivery

networks

use

the

popularity

of

sites

to

define

caching

and

replication

strategies

In

traffic

engineering

the

identification

of

heavy

hitters

in

the

network

can

be

used

to

treat

and

route

them

differently

across

the

network

-LSB-

-RSB-

Keeping

track

of

the

network

prefixes

that

generate

most

traffic

is

also

of

great

importance

for

anomaly

detection

A

variation

in

the

pattern

of

the

most

common

applications

may

be

used

as

a

warning

sign

and

trigger

careful

inspection

of

the

packet

streams

However

the

ability

to

identify

the

top

users

in

a

packet

stream

is

limited

by

the

network

monitoring

technology

Capturing

and

processing

all

packets

on

high

speed

links

still

remains

a

challenge

for

today

's

network

equipment

-LSB-

-RSB-

In

this

context

a

common

solution

is

to

sample

the

packet

stream

to

reduce

the

load

on

the

monitoring

system

and

to

simplify

the

task

of

sorting

the

list

of

items

The

underlying

assumption

in

this

approach

is

that

the

sampling

process

does

not

alter

the

properties

of

the

data

distribution

Sampled

traffic

data

is

then

used

to

infer

properties

of

the

original

data

-LRB-

this

operation

is

called

inversion

-RRB-

The

inversion

of

sampled

traffic

is

however

an

error-prone

procedure

that

often

requires

a

deep

study

of

the

data

distribution

to

evaluate

how

the

sampling

rate

impacts

the

accuracy

of

the

metric

of

interest

Although

the

inversion

may

be

simple

for

aggregate

link

statistics

-LRB-

e.g.

to

estimate

the

number

of

packets

transmitted

on

a

link

it

is

usually

sufficient

to

multiply

the

number

of

sampled

packets

by

the

inverse

of

the

sampling

rate

-RRB-

it

is

much

harder

for

the

properties

of

individual

connections

or

flows

-LSB-

-RSB-

For

these

reasons

in

this

paper

we

address

this

simple

and

so

far

unanswered

question

which

sampling

rate

is

needed

to

correctly

detect

and

rank

the

flows

that

carry

the

most

packets

We

define

the

problem

as

follows

Consider

a

traffic

monitor

that

samples

packets

independently

of

each

other

with

probability

p

-LRB-

random

sampling

-RRB-

and

classifies

them

into

sampled

flows

At

the

end

of

the

measurement

period

the

monitor

processes

the

list

of

sampled

flows

ranks

them

based

on

their

size

in

packets

and

returns

an

ordered

list

of

the

t

largest

flows

We

are

interested

in

knowing

-LRB-

i

-RRB-

whether

the

ordered

list

contains

all

the

actual

largest

flows

in

the

original

packet

stream

-LRB-

detection

-RRB-

and

-LRB-

ii

-RRB-

if

the

items

in

the

list

appear

in

the

correct

order

-LRB-

ranking

-RRB-

We

build

an

analytical

model

and

define

a

performance

metric

that

evaluates

the

accuracy

of

identification

and

ranking

of

the

largest

flows

We

consider

a

flow

to

consist

of

a

single

TCP

connection

However

our

results

are

general

and

can

be

applied

to

alternative

definitions

of

flow

as

well

We

evaluate

two

approaches

to

sort

the

list

of

flows

-LRB-

i

-RRB-

Blind

where

the

sampled

flows

are

ranked

just

based

on

their

sampled

size

This

method

can

be

applied

to

any

definition

of

flow

-LRB-

ii

-RRB-

Protocol-aware

where

we

make

use

of

additional

information

in

the

packet

header

-LRB-

e.g.

the

sequence

number

in

TCP

packets

-RRB-

to

infer

the

number

of

non-sampled

packets

between

sampled

ones

This

method

can

only

be

applied

to

flow

definitions

that

preserve

the

protocol

level

details

The

contributions

of

this

work

are

the

following

-LRB-

-RRB-

We

perform

an

analytical

study

of

the

problem

of

ranking

two

sampled

flows

and

compute

the

probability

that

they

are

misranked

We

propose

a

Gaussian

approximation

to

make

the

problem

numerically

tractable

-LRB-

-RRB-

We

introduce

the

protocol-aware

ranking

method

that

uses

protocol

level

information

to

complement

the

flow

statistics

and

render

the

detection

and

ranking

of

the

largest

flows

more

accurate

-LRB-

-RRB-

Based

on

the

model

for

the

ranking

of

two

flows

we

propose

a

general

model

to

study

the

detection

and

ranking

problem

given

a

generic

flow

size

distribution

We

define

a

performance

metric

and

evaluate

the

impact

of

several

metric

's

parameter

on

the

accuracy

of

the

ranking

-LRB-

-RRB-

We

validate

our

findings

on

measurement

data

using

publicly-available

packet-level

traces

Our

results

indicate

that

a

surprisingly

high

sampling

rate

is

required

to

obtain

a

good

accuracy

with

the

blind

approach

-LRB-

and

even

more

depending

on

the

number

of

flows

of

interest

-RRB-

As

for

the

protocol-aware

approach

it

allows

to

reduce

the

required

sampling

rate

by

an

order

of

magnitude

compared

to

the

blind

approach

The

paper

is

structured

as

follows

Next

we

discuss

the

related

literature

In

Section

and

we

present

our

model

Section

analyzes

the

model

numerically

and

Section

validates

it

on

real

packet-level

traces

Section

concludes

the

paper

and

provides

perspectives

for

our

future

research

The

inversion

of

sampled

traffic

has

been

extensively

studied

in

the

literature

The

main

focus

has

been

on

the

inversion

of

aggregate

flow

properties

such

as

flow

size

distribution

-LSB-

-RSB-

average

flow

size

or

total

number

of

flows

-LSB-

-RSB-

on

a

given

network

link

Duffield

et

al.

-LSB-

-RSB-

study

the

problem

of

flow

splitting

and

propose

estimators

for

the

total

number

of

flows

and

for

the

average

flow

size

in

the

original

traffic

stream

-LSB-

-RSB-

study

the

inversion

of

the

flow

size

distribution

with

two

different

methods

They

both

show

that

the

major

difficulty

comes

from

the

number

of

flows

that

are

not

sampled

at

all

and

that

need

to

be

estimated

with

an

auxiliary

method

As

an

auxiliary

method

-LSB-

-RSB-

propose

the

use

of

the

SYN

flag

in

the

TCP

header

to

mark

the

beginning

of

a

flow

-LSB-

-RSB-

shows

that

periodic

and

random

sampling

provide

roughly

the

same

result

on

high

speed

links

and

so

random

sampling

can

be

used

for

mathematical

analysis

due

to

its

appealing

features

-LSB-

-RSB-

finds

the

sampling

rate

that

assures

a

bounded

error

on

the

estimation

of

the

size

of

flows

contributing

to

more

than

some

predefined

percentage

of

the

traffic

volume

-LSB-

-RSB-

studies

whether

the

number

of

sampled

packets

is

a

good

estimator

for

the

detection

of

large

flows

without

considering

its

impact

on

the

flow

ranking

Given

the

potential

applications

of

finding

the

list

of

top

users

it

does

not

come

as

a

surprise

that

there

has

been

a

significant

effort

in

the

research

community

to

find

ways

to

track

frequent

items

in

a

data

stream

-LSB-

-RSB-

However

this

problem

has

usually

been

addressed

from

a

memory

requirement

standpoint

All

the

works

in

the

literature

assume

that

if

the

algorithm

and

the

memory

size

is

well

chosen

the

largest

flows

can

be

detected

and

ranked

with

a

high

precision

However

in

the

presence

of

packet

sampling

even

if

the

methods

rank

correctly

the

set

of

sampled

flows

there

is

no

guarantee

that

the

sampled

rank

corresponds

to

the

original

rank

The

problem

we

address

in

this

paper

complements

these

works

as

it

focuses

on

the

impact

of

sampling

on

the

flow

ranking

In

this

section

we

study

the

probability

to

misrank

two

flows

of

original

sizes

S

and

S

in

packets

This

probability

is

the

basis

for

the

general

model

for

detecting

and

ranking

the

largest

flows

that

we

will

present

later

Indeed

the

detection

and

ranking

of

the

largest

flows

can

be

transformed

into

a

problem

of

ranking

over

a

set

of

flow

pairs

Without

loss

of

generality

we

assume

S

S

We

consider

a

random

sampling

of

rate

p.

Let

s

and

s

denote

the

sizes

in

packets

of

both

flows

after

sampling

The

two

sampled

flows

are

misranked

if

-LRB-

i

-RRB-

s

is

larger

than

s

or

-LRB-

ii

-RRB-

both

flows

are

not

sampled

i.e.

their

sampled

sizes

equal

to

zero

By

combining

-LRB-

i

-RRB-

and

-LRB-

ii

-RRB-

one

can

see

that

the

necessary

condition

for

a

good

ranking

is

to

sample

at

least

one

packet

from

the

larger

flow

-LRB-

i.e.

the

smaller

of

the

two

flows

can

disappear

after

sampling

-RRB-

The

probability

to

misrank

the

two

flows

can

then

be

written

as

P

m

-LRB-

S

S

-RRB-

P

-LCB-

s

s

-RCB-

For

the

case

S

S

we

consider

the

two

flows

as

misranked

if

s

s

or

if

both

flows

are

not

sampled

at

all

i.e.

s

s

We

compute

and

study

the

misranking

probability

of

two

flows

of

given

sizes

in

the

rest

of

this

section

First

we

consider

the

blind

ranking

method

where

only

the

number

of

sampled

packets

from

a

flow

is

known

For

this

method

we

express

the

misranking

probability

as

a

double

sum

of

binomials

then

we

present

a

Gaussian

approximation

to

make

the

problem

tractable

numerically

Second

we

consider

the

protocol-aware

ranking

method

for

which

we

calculate

a

numerical-tractable

closed-form

expression

of

the

misraking

probability

Note

that

the

misranking

probability

is

a

symmetric

function

i.e.

P

m

-LRB-

S

S

-RRB-

P

m

-LRB-

S

S

-RRB-

Blind

ranking

With

this

method

s

and

s

represent

the

number

of

sampled

packets

from

flows

S

and

S

Under

our

assumptions

these

two

variables

are

distributed

according

to

a

binomial

distribution

of

probability

p.

Hence

we

can

write

for

S

S

P

m

-LRB-

S

S

-RRB-

P

-LCB-

s

s

-RCB-

S

i

b

p

-LRB-

i

S

-RRB-

i

j

b

p

-LRB-

j

S

-RRB-

-LRB-

-RRB-

b

p

-LRB-

i

S

-RRB-

is

the

probability

density

function

of

a

binomial

distribution

of

probability

p

i.e.

the

probability

of

obtaining

i

successes

out

of

S

trials

We

have

b

p

-LRB-

i

S

-RRB-

S

i

p

i

-LRB-

p

-RRB-

S-i

for

i

S

and

b

p

-LRB-

i

S

-RRB-

for

i

and

i

S

The

probability

to

misrank

two

flows

of

equal

sizes

is

given

by

P

-LCB-

s

s

or

s

s

-RCB-

P

-LCB-

s

s

-RCB-

S

i

b

p

-LRB-

i

S

-RRB-

Unfortunately

the

above

expression

for

the

misranking

probability

is

numerically

untractable

since

it

involves

two

sums

of

binomials

For

large

flows

of

order

S

packets

the

number

of

operations

required

to

compute

such

a

probability

is

on

the

order

of

O

-LRB-

S

-RRB-

assuming

that

the

complexity

of

the

binomial

computation

is

on

the

order

of

O

-LRB-

S

-RRB-

The

problem

becomes

much

more

complex

if

one

has

to

sum

over

all

possible

flow

sizes

-LRB-

i.e.

O

-LRB-

S

-RRB-

-RRB-

For

this

reason

we

propose

next

a

Gaussian

approximation

to

the

problem

of

blind

ranking

that

is

accurate

and

easy

to

compute

We

use

this

approximation

to

study

the

ranking

performance

as

a

function

of

the

sampling

rate

and

the

flow

sizes

Gaussian

approximation

to

blind

ranking

Consider

a

flow

made

of

S

packets

and

sampled

at

rate

p

The

sampled

size

follows

a

binomial

distribution

However

it

is

well

known

that

the

binomial

distribution

can

be

approximated

by

a

Normal

-LRB-

or

Gaussian

-RRB-

distribution

when

p

is

small

and

when

the

product

pS

is

on

the

order

of

one

-LRB-

flows

for

which

on

average

at

least

few

packets

are

sampled

-RRB-

-LSB-

pages

-RSB-

We

assume

that

this

is

the

case

for

the

largest

flows

and

we

consider

the

sampled

size

of

a

flow

as

distributed

according

to

a

Normal

distribution

of

average

pS

and

of

variance

p

-LRB-

p

-RRB-

S.

Using

this

approximation

one

can

express

the

misranking

probability

for

the

blind

ranking

problem

in

the

following

simple

form

Proposition

For

any

two

flows

of

sizes

S

and

S

packets

-LRB-

S

S

-RRB-

the

Gaussian

approximation

gives

P

m

-LRB-

S

S

-RRB-

erf

c

S

S

-LRB-

-RRB-

-LRB-

S

S

-RRB-

-LRB-

-RRB-

where

erfc

-LRB-

x

-RRB-

-LRB-

-RRB-

x

e

u

du

is

the

complementary

error

cumulative

function

Proof

Consider

two

flows

of

sizes

S

and

S

in

packets

such

that

S

S

Their

sampled

versions

s

and

s

both

follow

Normal

distributions

of

averages

pS

and

pS

and

of

variances

p

-LRB-

p

-RRB-

S

and

p

-LRB-

p

-RRB-

S

We

know

that

the

sum

of

two

Normal

variables

is

a

Normal

variable

So

the

difference

s

s

follows

a

Normal

distribution

of

average

p

-LRB-

S

S

-RRB-

and

of

variance

p

-LRB-

p

-RRB-

-LRB-

S

S

-RRB-

We

have

then

this

approximation

for

the

misranking

probability

P

m

-LRB-

S

S

-RRB-

P

-LCB-

s

s

-RCB-

P

V

p

-LRB-

S

S

-RRB-

p

-LRB-

p

-RRB-

-LRB-

S

S

-RRB-

erfc

S

S

-LRB-

-RRB-

-LRB-

S

S

-RRB-

-LRB-

-RRB-

V

is

a

standard

Normal

random

variable

Given

the

symmetry

of

the

misranking

probability

one

can

take

the

absolute

value

of

S

S

in

-LRB-

-RRB-

and

get

the

expression

stated

in

the

proposition

which

is

valid

for

all

S

and

S

For

S

S

one

can

safely

approximate

the

misranking

probability

to

be

equal

to

This

approximation

is

however

of

little

importance

given

the

very

low

probability

of

having

two

flows

of

equal

sizes

especially

when

they

are

large

Protocol-aware

ranking

Packets

can

carry

in

their

transport

header

an

increasing

sequence

number

A

typical

example

is

the

byte

sequence

number

in

the

TCP

header

Another

example

could

be

the

sequence

number

in

the

header

of

the

Real

Time

Protocol

-LRB-

RTP

-RRB-

-LSB-

-RSB-

One

can

use

this

sequence

number

when

available

to

infer

the

number

of

non-sampled

packets

-LRB-

or

bytes

in

the

case

of

TCP

-RRB-

between

sampled

ones

and

hence

to

improve

the

accuracy

of

ranking

The

size

of

the

sampled

flow

in

this

case

is

no

longer

the

number

of

packets

collected

but

rather

the

number

of

packets

that

exist

between

the

first

and

last

sampled

packets

from

the

flow

Although

this

solution

is

limited

to

flows

whose

packet

carry

a

sequence

number

we

believe

that

the

study

of

this

ranking

method

is

important

given

the

widespread

use

of

the

TCP

protocol

Our

objective

is

to

understand

how

the

use

of

protocol-level

information

can

supplement

the

simple

and

more

general

blind

method

and

if

it

is

worth

the

additional

overhead

it

introduces

-LRB-

i.e.

storing

two

sequence

numbers

per

flow

record

-RRB-

In

the

following

we

calculate

the

misranking

probability

of

two

flows

of

given

sizes

when

using

the

protocol-aware

method

This

probability

will

be

used

later

in

the

general

ranking

problem

The

main

contribution

of

this

section

is

a

closed-form

expression

for

the

misranking

probability

that

is

numerically

tractable

without

the

need

for

any

approximation

Let

S

be

the

size

of

a

flow

in

packets

Let

s

b

s

b

S

denote

the

-LRB-

packet

-RRB-

sequence

number

carried

by

the

first

sampled

packet

and

let

s

e

s

e

S

S

s

b

denote

the

sequence

number

carried

by

the

last

sampled

packet

Given

s

b

and

s

e

one

can

estimate

the

size

of

the

sampled

flow

in

packets

to

s

s

e

s

b

The

error

in

this

estimation

comes

from

the

non-sampled

packets

that

are

transmitted

before

s

b

and

after

s

e

We

give

next

the

distribution

of

s

which

is

needed

for

the

computation

of

the

misranking

probability

then

we

state

our

main

result

Before

presenting

the

analysis

note

that

this

new

flow

size

estimator

only

counts

the

packets

that

are

transmitted

with

distinct

sequence

numbers

In

the

case

of

TCP

this

corresponds

to

the

number

of

bytes

received

at

the

application

layer

rather

then

the

number

of

bytes

carried

over

the

network

It

is

equivalent

to

assuming

that

the

probability

of

sampling

a

retransmitted

-LRB-

or

duplicated

-RRB-

packet

is

negligible

This

is

a

reasonable

assumption

if

the

loss

rate

is

low

We

will

address

this

aspect

in

more

detail

in

Section

Consider

a

flow

of

size

S

in

packets

Using

the

above

definition

for

s

the

sampled

flow

has

a

size

of

i

packets

i

with

probability

P

-LCB-

s

i

-RCB-

S-i

k

P

-LCB-

s

b

k

-RCB-

P

-LCB-

s

e

k

i

-RCB-

We

have

P

-LCB-

s

b

k

-RCB-

-LRB-

p

-RRB-

p

and

P

-LCB-

s

e

k

i

-RCB-

-LRB-

p

-RRB-

S-k-i

p

This

gives

P

-LCB-

s

i

-RCB-

S-i

k

-LRB-

p

-RRB-

p

-LRB-

p

-RRB-

S-k-i

p

p

-LRB-

p

-RRB-

S-i

-LRB-

S

i

-RRB-

-LRB-

-RRB-

As

for

i

we

have

P

-LCB-

s

-RCB-

-LRB-

p

-RRB-

S

for

S

And

for

i

we

have

P

-LCB-

s

-RCB-

p

-LRB-

p

-RRB-

S

for

S

It

is

easy

to

prove

that

the

cumulative

distribution

of

s

is

the

following

for

all

values

of

S

P

-LCB-

s

i

-RCB-

p

-LRB-

p

-RRB-

S-i

-LRB-

S

i

-RRB-

-LRB-

p

-RRB-

S-i

-LRB-

-RRB-

We

come

now

to

the

misranking

probability

which

we

recall

is

a

symmetric

function

For

S

S

we

have

P

m

-LRB-

S

S

-RRB-

P

-LCB-

s

s

-RCB-

S

i

P

-LCB-

s

i

-RCB-

i

j

P

-LCB-

s

j

-RCB-

-LRB-

-RRB-

And

for

S

S

we

have

P

m

-LRB-

S

S

-RRB-

S

i

P

-LCB-

s

i

-RCB-

-LRB-

-RRB-

Our

main

result

is

the

following

Proposition

For

S

S

the

misranking

probability

is

equal

to

P

m

-LRB-

S

S

-RRB-

-LRB-

p

-RRB-

S

-LRB-

p

-RRB-

S

p

-LRB-

p

-RRB-

S

S

-LSB-

p

-LRB-

p

-RRB-

S

S

-LRB-

p

-RRB-

S

-RSB-

p

F

-LRB-

p

p

-RRB-

xy

p

F

-LRB-

p

p

-RRB-

x

where

F

-LRB-

x

y

-RRB-

xy

S

S

x

S

y

S

xy

S

S

-LRB-

-LRB-

xy

-RRB-

S

-RRB-

-LRB-

xy

-RRB-

For

S

S

S

the

misranking

probability

is

equal

to

P

m

-LRB-

S

S

-RRB-

p

-LRB-

p

-RRB-

-LRB-

-RRB-

S

p

G

-LRB-

p

p

-RRB-

xy

where

G

-LRB-

x

y

-RRB-

xy

x

y

x

y

-LRB-

xy

-LRB-

xy

-RRB-

S

-RRB-

-LRB-

xy

-RRB-

Proof

One

can

validate

the

results

by

plugging

-LRB-

-RRB-

and

-LRB-

-RRB-

into

-LRB-

-RRB-

and

-LRB-

-RRB-

Note

that

the

main

gain

of

writing

the

misraking

probability

in

such

a

condensed

form

is

a

complexity

that

drops

from

O

-LRB-

S

-RRB-

in

-LRB-

-RRB-

to

O

-LRB-

S

-RRB-

in

our

final

result

This

gain

comes

from

the

closed-form

expression

for

the

cumulative

distribution

in

-LRB-

-RRB-

and

from

introducing

the

two

functions

F

-LRB-

x

y

-RRB-

and

G

-LRB-

x

y

-RRB-

These

two

latter

functions

transform

two

series

whose

complexity

is

O

-LRB-

S

-RRB-

into

a

closed-form

expression

whose

complexity

is

O

-LRB-

S

-RRB-

We

solve

the

derivatives

in

the

above

equations

using

the

symbolic

toolbox

of

matlab

which

gives

explicit

expressions

for

the

misranking

probability

These

expressions

are

simple

to

compute

but

span

on

multiple

lines

so

we

omit

them

for

lack

of

space

Analysis

of

the

misranking

probability

The

blind

case

We

use

the

Gaussian

approximation

to

study

how

the

misranking

probability

varies

with

the

sampling

rate

and

with

the

sizes

of

both

flows

in

particular

their

difference

The

study

of

the

impact

of

the

flow

sizes

is

important

to

understand

the

relation

between

flow

size

distribution

and

ranking

of

the

largest

flows

The

misranking

probability

is

a

decreasing

function

of

the

sampling

rate

It

moves

to

zero

when

p

moves

to

and

to

when

p

approaches

zero

Therefore

there

exists

one

sampling

rate

that

leads

to

some

desired

misranking

probability

and

any

lower

sampling

rate

results

in

larger

error

We

study

now

how

the

misranking

probability

varies

with

the

sizes

of

both

flows

Take

S

S

k

k

a

positive

integer

From

-LRB-

-RRB-

and

for

fixed

k

the

misranking

probability

increases

with

S

and

S

-LRB-

erfc

-LRB-

x

-RRB-

is

an

increasing

function

in

x

-RRB-

This

indicates

that

it

is

more

difficult

to

rank

correctly

two

flows

different

by

k

packets

as

their

sizes

increase

in

absolute

terms

The

result

is

different

if

we

take

the

size

of

one

flow

equal

to

times

the

size

of

the

second

i.e.

S

S

Here

-LRB-

S

S

-RRB-

S

S

is

equal

to

S

-LRB-

-RRB-

which

increases

with

S

Hence

the

misranking

probability

given

in

-LRB-

-RRB-

decreases

when

S

increases

We

conclude

that

when

the

two

flow

sizes

maintain

the

same

proportion

it

is

easier

to

obtain

a

correct

ranking

when

they

are

large

in

absolute

terms

We

can

now

generalize

the

result

above

One

may

think

that

the

larger

the

flows

the

better

the

ranking

of

their

sampled

versions

Our

last

two

examples

indicate

that

this

is

not

always

the

case

The

ranking

accuracy

depends

on

the

relative

difference

of

the

flow

sizes

In

general

to

have

a

better

ranking

the

difference

between

the

two

flow

sizes

must

increase

with

the

flow

sizes

and

the

increase

must

be

larger

than

a

certain

threshold

This

threshold

is

given

by

-LRB-

-RRB-

the

difference

must

increase

at

least

as

the

square

root

of

the

flow

sizes

This

is

an

interesting

finding

In

the

context

of

the

general

ranking

problem

it

can

be

interpreted

as

follows

Suppose

that

the

flow

size

has

a

cumulative

distribution

function

y

F

-LRB-

x

-RRB-

As

we

move

to

the

tail

of

the

distribution

the

size

of

the

flows

to

be

ranked

increases

The

ranking

performance

improves

if

the

difference

between

flow

sizes

increases

faster

than

x

This

is

equivalent

to

saying

that

dx/dy

should

increase

with

x

faster

than

x.

All

common

distributions

satisfy

this

condition

at

least

at

their

tails

For

example

with

the

exponential

distribution

we

have

dx/dy

e

x

-LRB-

is

the

average

-RRB-

while

for

the

Pareto

distribution

we

have

dx/dy

x

-LRB-

is

the

shape

-RRB-

The

protocol-aware

case

The

first

difference

with

the

blind

case

is

in

the

estimation

error

-LRB-

S

s

s

b

S

s

e

-RRB-

which

can

be

safely

assumed

to

be

independent

of

the

flow

size

for

large

flows

-LRB-

only

dependent

on

p

-RRB-

This

means

that

if

two

large

flows

keep

the

same

distance

between

them

while

their

sizes

increase

their

ranking

maintains

the

same

accuracy

Their

ranking

improves

if

the

difference

between

their

sizes

increases

as

well

and

it

deteriorates

if

the

difference

between

their

sizes

decreases

So

in

contrast

to

the

blind

case

the

threshold

for

the

ranking

here

to

improve

is

that

the

larger

flow

should

have

its

size

increasing

a

little

faster

than

the

smaller

one

In

the

context

of

the

general

ranking

problem

where

flow

sizes

are

distributed

according

to

a

cumulative

distribution

function

y

F

-LRB-

x

-RRB-

and

when

the

top

flows

become

larger

the

protocol-aware

ranking

improves

if

the

derivative

dx/dy

increases

with

x

This

is

equivalent

to

saying

that

the

function

F

-LRB-

x

-RRB-

should

be

concave

which

is

satisfied

by

most

common

distributions

at

their

tail

For

blind

ranking

concavity

was

The

Gaussian

approximation

does

not

account

for

the

case

p

where

the

misranking

probability

should

be

equal

to

based

on

our

definition

Because

we

are

more

and

more

focusing

on

large

flows

or

because

the

number

of

available

flows

for

ranking

increases

not

enough

to

obtain

a

better

ranking

the

derivative

dx/dy

had

to

increase

faster

than

x

So

in

conclusion

the

condition

to

have

a

better

ranking

when

we

move

to

the

tail

of

the

flow

size

distribution

is

less

strict

with

the

protocol-aware

method

which

is

an

indication

of

its

good

performance

The

second

difference

with

the

blind

case

is

in

the

relation

between

the

ranking

accuracy

and

the

sampling

rate

Consider

two

large

flows

of

sizes

S

and

S

in

packets

and

let

s

and

s

denote

their

sampled

sizes

The

coefficient

of

variation

of

the

difference

s

s

is

an

indication

on

how

well

the

ranking

performs

-LRB-

a

small

coefficient

of

variation

results

in

better

ranking

-RRB-

It

is

easy

to

prove

that

this

coefficient

of

variation

scales

as

for

protocol-aware

ranking

and

as

for

blind

ranking

This

is

again

an

important

finding

It

tells

that

when

the

sampling

rate

is

very

small

blind

ranking

could

-LRB-

asymptotically

-RRB-

perform

better

than

protocol-aware

ranking

Our

numerical

and

experimental

results

will

confirm

this

finding

We

generalize

the

previous

model

from

the

ranking

of

two

flows

to

the

detection

and

ranking

of

the

top

t

flows

t

N

The

misranking

probability

P

m

-LRB-

S

S

-RRB-

pre-viously

calculated

is

the

basis

for

this

generalization

Let

N

t

denote

the

total

number

of

flows

available

in

the

measurement

period

before

sampling

We

want

the

sampled

list

of

top

t

flows

to

match

the

list

of

top

t

flows

in

the

original

traffic

Two

criteria

are

considered

to

decide

whether

this

match

is

accurate

First

we

require

the

two

lists

to

be

identical

This

corresponds

to

the

ranking

problem

The

second

less

constrained

criterion

requires

the

two

lists

to

contain

the

same

flows

regardless

of

their

relative

order

within

the

list

This

corresponds

to

the

detection

problem

For

both

problems

the

quality

of

the

result

is

expressed

as

a

function

of

the

sampling

rate

p

the

flow

size

distribution

the

number

of

flows

to

rank

t

and

the

total

number

of

flows

N

Performance

metric

In

order

to

evaluate

the

accuracy

of

detection

and

ranking

we

need

to

define

a

performance

metric

that

is

easy

to

compute

and

that

focuses

on

the

largest

flows

A

flow

at

the

top

of

the

list

can

be

misranked

with

a

neighboring

large

flow

or

a

distant

small

flow

We

want

our

metric

to

differentiate

between

these

two

cases

and

to

penalize

more

the

latter

one

a

flow

replaced

by

the

flow

in

the

sampled

top

list

is

worse

than

the

flow

being

replaced

by

the

flow

We

also

want

our

metric

to

be

zero

when

the

detection

and

ranking

of

the

top

flows

are

correct

We

introduce

our

performance

metric

using

the

ranking

problem

The

performance

metric

for

the

detection

problem

is

a

straightforward

extension

Let

's

form

all

flow

pairs

where

the

first

element

of

a

pair

is

a

flow

in

the

top

t

and

the

second

element

is

anywhere

in

the

sorted

list

of

the

N

original

flows

The

number

of

these

pairs

is

equal

to

N

N

N

t

-LRB-

t

-RRB-

We

then

count

the

pairs

in

this

set

that

are

misranked

after

sampling

and

we

take

the

sum

as

our

metric

for

ranking

accuracy

This

For

S

S

we

are

interested

in

P

-LCB-

s

s

-RCB-

According

to

Tchebychev

inequality

this

probability

can

be

supposed

to

behave

like

VAR

-LSB-

s

s

-RSB-

E

-LSB-

s

s

-RSB-

which

is

the

square

of

the

coefficient

of

variation

sum

indicates

how

good

the

ranking

is

at

the

top

of

the

list

It

is

equal

to

zero

when

the

ranking

is

correct

When

the

ranking

is

not

correct

it

takes

a

value

proportional

to

the

original

rank

of

the

flows

that

have

taken

a

slot

in

the

top-t

list

For

example

if

the

top

flow

is

replaced

by

its

immediate

successor

in

the

list

the

metric

will

return

a

ranking

error

of

Instead

if

the

same

flow

is

replaced

by

a

distant

flow

say

the

the

metric

will

return

an

error

of

Also

note

that

our

metric

does

not

account

for

any

misranking

of

flows

outside

the

list

of

top

t

flows

For

any

two

flows

n

and

m

such

that

n

m

t

the

fact

that

n

takes

the

position

of

m

does

not

add

anything

to

our

performance

metric

since

our

metric

requires

at

least

one

element

of

a

flow

pair

to

be

in

the

original

list

of

top

t

flows

In

the

detection

problem

we

are

no

longer

interested

in

comparing

flow

pairs

whose

both

elements

are

in

the

top

t

list

We

are

only

interested

in

the

ranking

between

flows

in

the

top

t

list

and

those

outside

the

list

Therefore

our

detection

metric

is

defined

as

the

number

of

misranked

flow

pairs

where

the

first

element

of

a

pair

is

in

the

list

of

top

t

flows

and

the

second

element

is

outside

this

list

-LRB-

non

top

t

-RRB-

The

above

metrics

return

one

value

for

each

realization

of

flow

sizes

and

of

sampled

packets

Given

that

we

want

to

account

for

all

realizations

we

define

the

performance

metrics

as

the

number

of

misranked

flow

pairs

averaged

over

all

possible

values

of

flow

sizes

in

the

original

list

of

N

flows

and

over

all

sampling

runs

We

deem

the

ranking/detection

as

acceptable

when

our

metric

takes

a

value

below

one

-LRB-

i.e.

on

average

less

than

one

flow

pair

is

misranked

-RRB-

In

addition

to

the

above

our

metrics

have

the

advantage

of

being

easily

and

exactly

calculable

Performance

metrics

based

on

probabilities

-LRB-

e.g.

-LSB-

-RSB-

-RRB-

require

lot

of

assumptions

that

make

them

only

suitable

for

computing

bounds

but

not

exact

values

Computation

of

the

performance

metric

for

the

ranking

problem

Consider

a

flow

of

i

packets

belonging

to

the

list

of

top

t

flows

in

the

original

traffic

-LRB-

before

sampling

-RRB-

First

we

compute

the

probability

that

this

flow

is

misranked

with

another

flow

of

general

size

and

general

position

Denote

this

probability

by

P

mt

-LRB-

i

-RRB-

where

m

stands

for

misranking

and

t

for

top

Then

we

average

over

all

values

of

i

to

get

P

This

latter

function

gives

us

the

probability

that

on

average

the

top

t-th

flow

is

misranked

with

another

flow

Thus

our

performance

metric

which

is

defined

as

the

average

number

of

misranked

flow

pairs

where

at

least

one

element

of

a

pair

is

in

the

top

t

is

equal

to

-LRB-

t

-RRB-

t

P

mt

Next

we

compute

the

value

of

P

mt

Let

p

i

denote

the

probability

that

the

size

of

a

general

flow

is

equal

to

i

packets

and

P

i

denote

the

flow

size

complementary

cumulative

distribution

i.e.

P

i

j

i

p

j

For

a

large

number

of

flows

N

and

a

high

degree

of

multiplexing

we

consider

safe

to

assume

that

flow

sizes

are

independent

of

each

other

-LRB-

see

-LSB-

-RSB-

for

a

study

of

the

flow

size

correlation

on

a

IP

backbone

link

-RRB-

A

flow

of

size

i

belongs

to

the

list

of

top

t

flows

if

the

number

of

flows

in

the

original

total

list

with

a

size

larger

than

i

is

less

or

equal

than

t

Since

each

flow

can

be

larger

than

i

with

probability

P

i

independently

of

the

other

flows

we

can

write

the

probability

that

a

flow

of

size

i

belongs

to

the

list

of

the

top

t

flows

Note

that

the

distribution

of

the

size

of

a

flow

at

the

top

of

the

list

is

different

from

that

of

a

generic

flow

as

P

t

-LRB-

i

t

N

-RRB-

k

b

P

i

-LRB-

k

N

-RRB-

where

b

P

i

-LRB-

k

N

-RRB-

is

the

probability

to

obtain

k

successes

out

of

N

trials

P

i

being

the

probability

of

a

success

The

probability

that

the

t-th

largest

flow

has

a

size

of

i

packets

is

equal

to

P

t

-LRB-

i

-RRB-

p

i

P

t

-LRB-

i

t

N

-RRB-

P

t

-LRB-

t

N

-RRB-

P

t

-LRB-

t

N

-RRB-

is

the

probability

that

a

flow

of

general

size

is

among

the

top

t

in

the

original

total

list

which

is

simply

equal

to

t/N

Using

the

above

notation

one

can

write

the

misranking

probability

between

a

top

t

flow

of

original

size

i

packets

and

any

other

flow

as

follows

P

mt

-LRB-

i

-RRB-

P

t

-LRB-

i

t

N

-RRB-

j

p

j

P

t

-LRB-

i

t

N

-RRB-

P

m

-LRB-

j

i

-RRB-

j

i

p

j

P

t

-LRB-

i

t

N

-RRB-

P

m

-LRB-

i

j

-RRB-

-LRB-

-RRB-

In

this

expression

we

sum

over

all

possible

original

sizes

of

the

other

flow

-LRB-

the

variable

j

-RRB-

and

we

separate

the

case

when

this

other

flow

is

smaller

than

i

from

the

case

when

it

is

larger

than

i

P

m

-LRB-

i

j

-RRB-

is

the

misranking

probability

of

two

flows

of

sizes

i

and

j

packets

which

we

calculated

in

the

previous

section

for

the

two

ranking

methods

P

mt

is

then

equal

to

i

P

t

-LRB-

i

-RRB-

P

mt

-LRB-

i

-RRB-

For

protocol-aware

ranking

P

m

-LRB-

i

j

-RRB-

is

given

explicitly

in

Proposition

and

can

be

easily

computed

For

blind

ranking

we

use

the

Gaussian

approximation

summarized

in

Proposition

which

we

recall

holds

when

at

least

one

of

the

two

flows

to

be

compared

is

large

Computation

of

the

performance

metric

for

the

detection

problem

Consider

the

probability

that

a

flow

among

the

top

t

is

swapped

with

a

flow

that

does

belong

to

the

top

t.

Let

P

mt

denote

this

probability

Following

the

same

approach

described

in

Section

we

can

write

P

mt

P

t

i

j

p

i

p

j

P

t

-LRB-

j

i

t

N

-RRB-

P

m

-LRB-

j

i

-RRB-

To

get

this

expression

for

P

mt

we

sum

over

all

possible

values

for

the

size

of

the

flow

in

the

top

t

-LRB-

index

i

-RRB-

and

all

possible

values

for

the

size

of

the

other

flow

not

among

the

top

t

-LRB-

index

j

-RRB-

In

this

expression

p

i

and

p

j

represent

the

probability

that

the

size

of

a

flow

is

equal

to

i

or

j

packets

respectively

P

m

-LRB-

j

i

-RRB-

is

the

probability

that

two

flows

of

sizes

i

and

j

are

misranked

it

is

given

by

the

Gaussian

approximation

described

in

Proposition

for

the

blind

method

and

the

result

stated

in

Proposition

for

the

protocol-aware

method

P

t

-LRB-

j

i

t

N

-RRB-

is

the

joint

probability

that

a

flow

of

size

i

belongs

to

the

list

of

the

top

t

flows

while

another

flow

of

size

j

does

not

belong

to

it

-LRB-

i.e.

it

is

in

the

bottom

N

t

flows

-RRB-

P

t

is

the

joint

probability

that

a

flow

of

any

size

belongs

to

the

list

of

the

top

t

flows

while

another

flow

of

any

size

does

not

belong

to

this

list

It

is

equal

to

t

-LRB-

N

t

-RRB-

-LRB-

N

-LRB-

N

-RRB-

-RRB-

We

now

compute

P

t

-LRB-

j

i

t

N

-RRB-

for

j

i

i.e.

the

probability

that

flow

i

belongs

to

the

top

list

while

flow

j

does

not

The

number

of

flows

larger

than

i

should

be

smaller

than

t

while

the

number

of

flows

larger

than

j

should

be

larger

than

t

The

probability

that

a

flow

size

is

larger

than

In

the

case

j

i

at

most

t

flows

can

be

larger

than

i

packets

if

we

want

the

flow

of

size

i

to

be

in

the

top

t.

Trace

Jussieu

Abilene

Link

speed

GigE

-LRB-

Gbps

-RRB-

-LRB-

Gbps

-RRB-

Duration

hours

minutes

TCP

connections

Packets

Table

Summary

of

the

traces

i

is

P

i

k

i

p

k

The

probability

that

it

is

larger

than

j

is

P

j

k

j

p

k

The

probability

that

a

flow

size

is

between

j

and

i

given

that

it

is

smaller

than

i

is

-LRB-

P

j

P

i

-RRB-

-LRB-

P

i

-RRB-

We

call

it

P

j

i

It

follows

that

P

t

-LRB-

j

i

t

N

-RRB-

k

b

P

i

-LRB-

k

N

-RRB-

N

l

b

P

j

i

-LRB-

l

N

k

-RRB-

The

first

sum

accounts

for

the

probability

to

see

less

than

t

flows

above

i

packets

The

second

sum

accounts

for

the

probability

to

see

more

than

t

flows

above

j

given

that

k

flows

-LRB-

k

t

-RRB-

were

already

seen

above

i.

For

t

P

t

-LRB-

j

i

t

N

-RRB-

is

no

other

than

P

t

-LRB-

i

t

N

-RRB-

and

both

P

mt

and

P

mt

are

equal

-LRB-

i.e.

the

ranking

and

the

detection

problems

are

the

same

-RRB-

Once

P

mt

is

computed

we

multiply

it

by

the

total

number

of

flow

pairs

whose

one

element

is

in

the

top

t

and

the

other

one

is

not

This

total

number

is

equal

to

t

-LRB-

N

t

-RRB-

Our

metric

for

the

detection

problem

is

the

result

of

this

multiplication

As

for

the

ranking

problem

we

want

this

metric

to

be

less

than

one

for

the

detection

of

the

top

t

flows

to

be

accurate

We

analyze

now

the

accuracy

of

identifying

and

ranking

the

largest

flows

in

a

packet

stream

for

both

the

blind

and

protocol-aware

methods

Our

metrics

require

the

following

input

p

i

the

flow

size

distribution

and

N

the

total

number

of

flows

observed

on

the

link

during

the

measurement

period

To

derive

realistic

values

for

these

two

quantities

we

consider

two

publicly

available

packet-level

traces

The

first

trace

is

Abilene-I

collected

by

NLANR

-LSB-

-RSB-

on

an

-LRB-

Gbps

-RRB-

link

on

the

Abilene

Network

-LSB-

-RSB-

The

second

trace

has

been

collected

by

the

Metropolis

project

-LSB-

-RSB-

on

a

Gigabit

Ethernet

access

link

from

the

Jussieu

University

campus

in

Paris

to

the

Renater

Network

-LSB-

-RSB-

Table

summarizes

the

characteristics

of

the

two

traces

We

model

the

flow

size

distribution

in

the

traces

with

Pareto

We

opted

for

Pareto

since

it

is

known

to

be

appropriate

to

model

flow

sizes

in

the

Internet

due

to

its

heavy

tailed

feature

-LSB-

-RSB-

Note

that

it

is

not

our

goal

to

find

an

accurate

approximation

of

the

distribution

of

flow

sizes

in

our

traces

but

rather

to

find

a

general

well-known

distribution

that

approaches

the

actual

flow

size

In

this

section

we

analyze

a

wide

range

of

parameters

while

Section

focuses

on

the

performance

we

observe

in

the

two

packet-level

traces

The

Pareto

distribution

is

continuous

with

a

complementary

cumulative

distribution

function

given

by

P

-LCB-

S

x

-RCB-

-LRB-

x/a

-RRB-

is

a

parameter

describing

the

shape

of

the

distribution

and

a

is

a

parameter

describing

its

scale

The

Pareto

random

variable

takes

values

larger

than

a

and

has

an

average

value

equal

to

a

-LRB-

-RRB-

The

tail

of

the

Pareto

distribution

becomes

heavier

as

decreases

We

use

our

traces

to

derive

an

indicative

value

of

the

shape

parameter

To

this

end

we

compute

the

empirical

complementary

cumulative

distribution

of

flow

sizes

and

we

Flow

size

in

Kbytes

Complementary

CDF

Abilene

trace

x

Flow

size

in

Kbytes

Complementary

CDF

Jussieu

trace

x

Figure

Empirical

flow

size

distribution

plot

it

on

a

log-log

scale

A

heavy-tailed

distribution

of

shape

parameter

decays

linearly

on

a

log-log

scale

at

rate

The

empirical

distributions

are

shown

in

Figure

The

plots

show

that

equal

to

suits

the

Abilene

trace

and

equal

to

suits

the

Jussieu

one

This

means

that

the

flow

size

distribution

has

a

heavier

tail

in

the

Jussieu

trace

Then

we

compute

the

average

flow

size

in

packets

to

get

the

starting

point

a

for

the

Pareto

distribution

As

an

average

flow

size

we

measure

Kbytes

and

packets

on

the

Abilene

trace

and

Kbytes

and

packets

on

the

Jussieu

trace

The

total

number

of

flows

N

is

set

by

taking

a

measurement

interval

equal

to

one

minute

then

multiplying

this

interval

by

the

average

arrival

rate

of

flows

per

second

on

each

trace

This

gives

N

Kflows

for

the

Abilene

trace

and

N

Kflows

for

the

Jussieu

one

In

the

rest

of

this

section

all

figures

plot

the

ranking

metric

versus

the

packet

sampling

rate

p

on

a

log-log

scale

We

vary

p

from

to

Each

figure

shows

different

lines

that

correspond

to

different

combinations

of

t

and

N

We

are

interested

in

the

regions

where

the

value

of

the

metric

is

below

one

indicating

that

the

ranking

is

accurate

on

average

To

ease

the

interpretation

of

results

in

the

figures

we

plot

the

horizontal

line

of

ordinate

Blind

ranking

Impact

of

the

number

of

flows

of

interest

The

first

parameter

we

study

is

t

the

number

of

largest

flows

to

rank

The

purpose

is

to

show

how

many

flows

can

be

detected

and

ranked

correctly

for

a

given

sampling

rate

We

set

N

and

the

average

flow

size

to

the

values

described

before

The

performance

of

blind

ranking

the

top

t

flows

is

shown

in

Figure

for

both

traces

We

observe

that

the

larger

the

number

of

top

flows

of

interest

the

more

difficult

it

is

to

detect

and

rank

them

correctly

In

particular

with

a

sampling

rate

on

the

order

of

it

is

possible

to

rank

at

most

the

top

one

or

two

flows

As

we

focus

at

larger

values

of

t

the

required

sampling

rate

to

get

a

correct

ranking

increases

well

above

Note

that

with

a

sampling

rate

on

the

order

of

it

is

almost

impossible

to

detect

even

the

largest

flow

We

also

observe

that

the

ranking

on

the

Jussieu

trace

behaves

slightly

better

than

that

on

the

Abilene

trace

The

Jussieu

trace

has

a

heavier

tail

for

its

flow

size

distribution

and

so

the

probability

to

get

larger

flows

at

the

top

of

the

list

is

higher

which

makes

the

ranking

more

accurate

This

will

be

made

clear

next

as

we

will

study

the

impact

of

the

shape

parameter

Impact

of

the

flow

size

distribution

We

consider

the

blind

ranking

of

the

top

flows

varying

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Abilene

trace

N

beta

blind

ranking

t

t

t

t

t

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Jussieu

trace

N

beta

blind

ranking

t

t

t

t

t

Figure

Performance

of

blind

ranking

varying

the

number

t

of

top

flows

of

interest

the

shape

parameter

for

the

Pareto

distribution

among

five

distinct

values

and

Note

that

for

the

Pareto

distribution

is

known

to

be

heavy

tailed

-LRB-

infinite

variance

-RRB-

The

other

parameters

of

the

model

-LRB-

N

and

the

average

flow

size

-RRB-

are

set

as

before

The

values

taken

by

our

metric

are

shown

in

Figure

for

both

traces

We

can

make

the

following

observations

from

the

figure

Given

a

sampling

rate

the

ranking

accuracy

improves

as

becomes

smaller

i.e.

the

tail

of

the

flow

size

distribution

becomes

heavier

Indeed

when

the

distribution

tail

becomes

heavier

the

probability

to

obtain

larger

flows

at

the

top

of

the

list

increases

and

since

it

is

simpler

to

blindly

rank

larger

flows

-LRB-

for

distributions

satisfying

the

square

root

condition

see

Section

-RRB-

the

ranking

becomes

more

accurate

The

ranking

is

never

correct

unless

the

sampling

rate

is

very

high

In

our

setting

one

needs

to

sample

at

more

than

to

obtain

an

average

number

of

misranked

flow

pairs

below

one

for

a

value

of

equal

to

-LRB-

i.e

heavy

tailed

distribution

-RRB-

and

at

more

than

for

a

value

of

equal

to

-LRB-

i.e.

pronounced

heavy

tailed

distribution

-RRB-

For

larger

values

of

-LRB-

i.e.

lighter

tail

-RRB-

the

sampling

rate

needs

to

be

as

high

as

Impact

of

the

total

number

of

flows

Another

important

parameter

in

the

ranking

problem

is

N

the

total

number

of

flows

available

during

the

measurement

period

When

N

increases

the

flows

at

the

top

of

the

list

should

become

larger

and

therefore

as

we

saw

in

Section

the

blind

ranking

accuracy

should

improve

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Abilene

trace

N

t

flows

blind

ranking

beta

beta

beta

beta

beta

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Jussieu

trace

N

t

flows

blind

ranking

beta

beta

beta

beta

beta

Figure

Performance

of

blind

ranking

varying

the

shape

parameter

of

the

flow

size

distribution

for

flow

size

distributions

satisfying

the

square

root

condition

-LRB-

in

particular

the

Pareto

distribution

we

are

considering

here

-RRB-

N

varies

with

the

utilization

of

the

monitored

link

the

higher

the

utilization

the

larger

the

number

of

flows

N

can

also

vary

with

the

duration

of

the

measurement

period

the

longer

we

wait

before

ranking

and

reporting

results

the

larger

the

number

of

flows

We

study

the

impact

of

N

on

the

blind

ranking

accuracy

We

take

the

same

value

of

N

used

in

the

previous

sections

and

computed

over

one

minute

measurement

period

-LRB-

Kflows

for

the

Abilene

trace

and

Kflows

for

the

Jussieu

trace

-RRB-

then

we

multiply

it

by

some

constant

factor

ranging

from

-LRB-

times

fewer

flows

-RRB-

to

-LRB-

times

more

flows

-RRB-

Results

are

shown

in

Figure

The

lines

in

the

figures

correspond

to

a

factor

value

equal

to

and

In

these

figures

we

consider

the

ranking

of

the

top

flows

with

the

values

of

and

average

flow

size

set

from

the

traces

Clearly

the

ranking

accuracy

improves

as

N

increases

However

in

our

setting

this

improvement

is

still

not

enough

to

allow

a

perfect

ranking

One

can

always

imagine

increasing

N

-LRB-

e.g.

by

increasing

the

measurement

period

-RRB-

until

the

top

t

flows

are

extremely

large

and

hence

perfectly

detected

and

ranked

Protocol-aware

ranking

Protocol-aware

ranking

takes

advantage

of

the

information

carried

in

the

transport

header

of

the

sampled

packets

to

infer

the

number

of

non-sampled

packets

of

a

flow

We

use

our

model

to

check

whether

this

improvement

exists

and

to

evaluate

it

Remember

that

we

are

always

in

the

context

of

low

retransmission

and

duplication

rates

which

is

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Abilene

trace

beta

t

blind

ranking

N

N

N

M

N

M

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Jussieu

trace

beta

t

blind

ranking

N

N

N

N

Figure

Performance

of

blind

ranking

varying

the

total

number

of

flows

sary

to

remove

the

discrepancy

between

carried

data

volume

-LRB-

throughput

-RRB-

and

application

data

volume

-LRB-

goodput

-RRB-

Using

the

previous

values

for

N

and

average

flow

size

we

reproduce

Figure

but

this

time

for

the

protocol-aware

case

This

leads

to

Figure

which

illustrates

the

impact

of

the

number

of

largest

flows

to

rank

For

lack

of

space

we

omit

the

other

figures

We

compare

this

new

figure

to

its

counterpart

in

the

blind

case

We

make

the

following

two

observations

-LRB-

i

-RRB-

The

protocol-aware

method

improves

the

accuracy

of

the

largest

flows

ranking

by

an

order

of

magnitude

for

high

sampling

rates

-LRB-

above

-RRB-

For

example

for

the

Abilene

trace

a

sampling

rate

on

the

order

of

was

necessary

to

detect

and

rank

the

largest

flows

with

the

blind

method

Now

with

the

protocol-aware

method

a

sampling

rate

on

the

order

of

is

sufficient

The

same

conclusion

applies

to

the

Jussieu

trace

A

sampling

rate

on

the

order

of

is

needed

With

the

protocol-aware

method

it

becomes

on

the

order

of

-LRB-

ii

-RRB-

The

protocol-aware

method

does

not

improve

the

performance

when

applied

at

low

sampling

rates

-LRB-

above

-RRB-

This

can

be

clearly

seen

if

we

compare

the

plots

between

both

figures

for

sampling

rates

below

This

results

confirms

our

observations

in

Section

Largest

flows

detection

To

illustrate

the

difference

between

ranking

and

detection

we

consider

the

same

scenario

as

in

Section

We

plot

the

detection

metric

as

a

function

of

the

sampling

rate

for

different

values

of

t

-LRB-

the

number

of

top

flows

of

interest

-RRB-

and

for

both

Abilene

and

Jussieu

traces

This

gives

Figure

for

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Abilene

trace

N

beta

protocol-aware

ranking

t

t

t

t

t

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Jussieu

trace

N

beta

protocol-aware

ranking

t

t

t

t

t

Figure

Performance

of

protocol-aware

ranking

varying

the

number

t

of

top

flows

of

interest

blind

ranking

and

Figure

for

protocol-aware

ranking

A

comparison

between

these

results

and

their

counterparts

in

Figure

and

respectively

shows

a

significant

improvement

in

the

detection

case

for

both

ranking

methods

All

plots

are

shifted

down

by

an

order

of

magnitude

For

example

in

the

case

of

blind

ranking

the

required

sampling

rate

to

correctly

rank

the

top

flows

was

around

for

the

Abilene

trace

and

for

the

Jussieu

trace

Now

with

blind

detection

it

is

around

and

respectively

Another

example

is

with

the

protocol-aware

method

where

a

sampling

rate

around

was

required

to

rank

the

largest

flows

-LRB-

Figure

-RRB-

whereas

now

a

sampling

rate

around

is

sufficient

to

only

detect

them

The

same

gain

can

be

observed

if

we

reconsider

the

other

scenarios

in

Section

-LRB-

not

presented

here

for

lack

of

space

-RRB-

Also

note

how

in

the

detection

case

the

protocol

aware

method

allows

a

better

accuracy

for

high

sampling

rates

when

compared

to

the

blind

method

For

low

sampling

rates

-LRB-

e.g.

below

-RRB-

the

accuracy

does

not

improve

In

this

section

we

present

the

results

of

running

random

sampling

experiments

directly

on

the

packet

traces

We

use

the

traces

described

in

Section

and

compute

the

performance

metrics

defined

in

Section

In

our

traces

we

consider

only

TCP

packets

Since

TCP

sequence

numbers

count

bytes

we

express

the

flow

sizes

in

bytes

instead

of

packets

throughout

this

section

Our

experiments

are

meant

to

address

four

major

issues

that

arise

when

we

move

from

the

analytical

study

to

a

real

network

setting

-LRB-

i

-RRB-

how

to

deal

with

invalid

TCP

sequence

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Abilene

trace

N

beta

blind

detection

t

t

t

t

t

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Jussieu

trace

N

beta

blind

detection

t

t

t

t

t

Figure

Only

detecting

the

largest

flows

Performance

of

blind

ranking

varying

the

number

t

of

top

flows

of

interest

numbers

in

the

packet

stream

-LRB-

ii

-RRB-

the

importance

of

flow

size

distributions

and

duration

of

the

measurement

interval

-LRB-

iii

-RRB-

the

impact

of

packet

loss

rates

on

individual

flows

lost

packets

trigger

retransmissions

by

the

TCP

senders

-LRB-

iv

-RRB-

the

variability

of

the

detection/ranking

performance

across

multiple

bins

and

packet

sampling

patterns

Implementation

of

protocol-aware

ranking

The

protocol-aware

method

depends

on

TCP

sequence

numbers

to

perform

the

ranking

For

a

given

flow

it

keeps

track

of

the

lowest

and

highest

sequence

number

observed

-LRB-

taking

care

of

packets

that

wrap

around

the

sequence

number

space

-RRB-

s

b

and

s

e

respectively

Note

that

an

actual

implementation

of

this

method

would

just

require

two

bit

fields

per

flow

to

store

the

two

sequence

numbers

At

the

end

of

the

measurement

period

we

compute

the

difference

between

the

highest

and

lowest

sequence

numbers

for

each

sampled

flow

and

we

use

the

obtained

values

to

rank

flows

We

then

compare

this

ranking

with

the

one

obtained

by

counting

all

the

bytes

each

flow

transmits

in

the

original

non

sampled

traffic

In

order

to

discard

invalid

packets

carrying

incorrect

sequence

numbers

that

would

corrupt

the

ranking

we

implement

a

simple

heuristic

to

update

s

e

and

s

b

A

sampled

packet

with

sequence

number

S

s

e

causes

an

update

s

e

S

if

S

s

e

-LRB-

MTU

-RRB-

p

The

same

rule

applies

to

the

updates

of

s

b

This

way

we

set

a

limit

on

the

maximum

distance

in

the

sequence

space

between

two

sampled

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Abilene

trace

N

beta

protocol-aware

detection

t

t

t

t

t

Packet

sampling

rate

-LRB-

-RRB-

Average

number

of

misranked

flow

pairs

Jussieu

trace

N

beta

protocol-aware

detection

t

t

t

t

t

Figure

Only

detecting

the

largest

flows

Performance

of

protocol-aware

ranking

varying

the

number

t

of

top

flows

of

interest

ets

This

distance

is

inversely

proportional

to

the

sampling

rate

and

depends

on

the

Maximum

Transmission

Unit

Furthermore

we

use

the

parameter

that

allows

to

make

this

threshold

more

or

less

permissive

in

order

to

account

for

the

randomness

of

the

sampling

process

and

for

other

transport-layer

events

-LRB-

e.g.

packet

retransmissions

when

the

TCP

window

is

large

-RRB-

We

have

run

several

experiments

with

different

values

of

and

the

results

have

shown

little

sensitivity

to

values

of

All

the

results

in

this

Section

are

derived

with

Flow

size

distribution

and

measurement

interval

As

shown

in

Figure

flow

size

distributions

do

not

follow

a

perfect

Pareto

Furthermore

the

measurement

interval

itself

plays

a

major

role

in

shaping

the

distribution

it

caps

the

size

of

the

largest

flows

that

is

not

unbounded

but

now

depends

on

the

link

speed

Indeed

network

operators

often

run

measurements

using

a

binning

method

where

packets

are

sampled

for

a

time

interval

classified

into

flows

ranked

and

then

reported

At

the

end

of

the

interval

the

memory

is

cleared

and

the

operation

is

repeated

for

the

next

measurement

interval

With

this

binning

method

all

flows

active

at

the

end

of

the

measurement

interval

are

truncated

so

that

not

all

sampled

packets

of

the

truncated

flow

are

considered

at

the

same

time

for

the

ranking

The

truncation

may

therefore

penalize

large

flows

and

alter

the

tail

of

the

flow

size

distribution

-LRB-

where

flows

are

of

large

size

and

probably

last

longer

than

the

measurement

interval

-RRB-

Each

experiment

consists

of

the

following

We

run

packet

sampling

rate

average

number

of

misranked

flow

pairs

Jussieu

trace

blind

ranking

top

top

top

top

top

packet

sampling

rate

average

number

of

misranked

flow

pairs

Jussieu

trace

protocol-aware

ranking

top

top

top

top

top

Figure

Performance

of

blind

and

protocol-aware

ranking

on

Jussieu

trace

-LRB-

measurement

interval

-RRB-

dom

sampling

on

the

packet

traces

and

classify

the

sampled

packets

into

flows

At

the

end

of

each

measurement

interval

-LRB-

set

to

or

minutes

-RRB-

we

collect

the

flows

and

rank

them

by

the

number

of

bytes

sampled

for

each

flow

We

compare

the

ranking

before

and

after

sampling

using

our

performance

metric

-LRB-

Section

-RRB-

For

each

sampling

rate

we

conduct

runs

and

we

calculate

averages

The

results

of

the

experiments

confirm

the

numerical

results

of

the

previous

section

In

the

interest

of

space

we

plot

the

results

of

two

representative

experiments

on

which

we

make

several

observations

The

difference

between

numerical

and

experimental

results

especially

at

low

sampling

rates

is

caused

by

the

non

perfect

match

of

the

empirical

flow

size

distribution

with

Pareto

-LRB-

Figure

-RRB-

Figure

shows

the

performance

of

ranking

flows

on

the

Jussieu

trace

when

the

measurement

bin

is

We

consider

a

wide

range

of

sampling

rates

from

to

and

study

the

performance

when

ranking

the

top

and

flows

in

the

packet

stream

The

top

graph

in

Figure

is

derived

using

the

blind

method

while

the

bottom

graph

shows

the

performance

of

the

protocol-aware

methods

These

results

are

very

similar

to

the

numerical

results

For

sampling

rates

above

protocol-aware

ranking

gives

approximately

an

order

of

magnitude

gain

on

the

performance

when

compared

to

blind

ranking

When

the

sampling

rate

is

lower

than

however

the

performance

of

the

two

methods

is

similar

Overall

the

blind

method

requires

a

sampling

rate

of

to

correctly

identify

the

largest

flow

in

the

packet

stream

The

same

sampling

rate

allows

to

correctly

rank

the

largest

flows

when

using

the

protocol-aware

method

Impact

of

loss

rate

In

the

analysis

of

the

protocol-aware

method

in

Section

we

made

the

assumption

of

negligible

number

of

retransmissions

for

all

the

flows

in

the

packet

stream

A

retransmitted

packet

may

cause

inconsistency

between

the

blind

and

protocol-aware

method

depending

on

the

location

of

the

monitoring

point

Indeed

the

blind

method

counts

the

total

number

of

bytes

sent

by

the

flow

while

the

protocol-aware

method

considers

only

the

data

sent

by

the

transport

layer

Therefore

if

the

packet

is

lost

before

the

monitoring

point

the

blind

and

protocol-aware

method

will

have

a

consistent

view

of

the

number

of

bytes

sent

Instead

if

the

packet

is

lost

after

the

monitoring

point

the

blind

method

may

count

this

packet

twice

The

impact

of

packet

losses

on

the

detection

and

ranking

of

the

largest

flows

depends

on

the

metric

used

to

estimate

the

size

of

the

flows

If

flow

sizes

are

estimated

according

to

the

total

number

of

bytes

sent

-LRB-

i.e.

the

throughput

-RRB-

then

the

protocol-aware

method

may

incur

in

an

underestimation

error

that

is

independent

of

the

sampling

rate

-LRB-

it

will

occur

even

if

all

packets

are

sampled

-RRB-

On

the

other

hand

if

the

flow

sizes

are

estimated

according

to

the

transport

data

sent

-LRB-

i.e.

the

goodput

-RRB-

then

the

blind

method

may

incur

in

an

overestimation

error

independently

of

the

sampling

rate

To

illustrate

the

effect

of

packet

loss

rates

we

plot

in

Figure

the

performance

of

detecting

the

largest

flows

in

the

Abilene

trace

when

the

measurement

bin

is

minutes

and

the

flow

sizes

are

measured

using

the

total

number

of

bytes

sent

over

the

link

The

top

graph

shows

the

performance

of

the

blind

method

while

the

bottom

graph

presents

the

results

for

the

protocol-aware

method

We

can

make

the

following

observations

The

protocol-aware

method

keeps

performing

better

than

the

blind

method

when

the

sampling

rate

is

above

At

lower

sampling

rates

the

blind

method

performs

better

although

it

presents

very

large

errors

For

sampling

rates

above

the

curve

relative

to

the

detection

of

the

flows

in

the

protocol-aware

method

flattens

to

a

value

around

This

is

due

to

the

presence

of

a

few

flows

that

experience

a

high

loss

rate

when

compared

to

other

flows

Increasing

the

sampling

rate

does

not

help

the

protocol-aware

method

in

detecting

the

largest

flows

when

the

volume

of

bytes

sent

is

used

to

define

the

flow

size

However

the

protocol-aware

method

can

correctly

detect

the

flows

when

their

size

is

defined

in

terms

of

transport

data

-LRB-

see

Figure

-RRB-

In

summary

the

network

operator

has

to

choose

the

metric

of

interest

that

depends

on

the

application

For

example

for

anomaly

detection

or

traffic

engineering

a

metric

that

counts

the

number

of

bytes

sent

may

be

more

appropriate

Instead

for

dimensioning

caches

and

proxies

the

metric

that

considers

the

size

of

the

objects

transferred

may

be

preferred

This

latter

metric

suits

more

the

protocol-aware

method

Variability

of

the

results

A

last

important

aspect

that

we

need

to

address

is

the

variability

of

the

results

across

multiple

measurement

intervals

and

different

realizations

of

the

sampling

process

Indeed

moving

from

one

measurement

interval

to

another

packet

sampling

rate

average

number

of

misranked

flow

pairs

Abilene

trace

blind

detection

top

top

top

top

top

packet

sampling

rate

average

number

of

misranked

flow

pairs

Abilene

trace

protocol-aware

detection

top

top

top

top

top

Figure

Performance

of

blind

-LRB-

top

-RRB-

and

protocol-aware

-LRB-

bottom

-RRB-

detection

on

Abilene

trace

-LRB-

measurement

interval

-RRB-

the

composition

of

flows

varies

and

with

it

the

flow

size

distribution

Moreover

the

sampling

process

may

get

lucky

in

certain

cases

and

provide

good

results

The

opposite

is

also

possible

Figure

shows

the

average

performance

over

sampling

experiments

of

the

detection

of

the

flows

in

the

Abilene

trace

over

the

measurement

intervals

The

error

bars

indicate

the

standard

deviation

across

the

experiments

As

usual

the

top

graph

refers

to

the

blind

method

while

the

bottom

graph

presents

the

protocol-aware

method

results

As

we

can

see

the

average

performance

shows

limited

variability

A

sampling

rate

of

gives

poor

results

for

all

bins

while

increasing

the

sampling

rates

consistently

helps

With

a

sampling

rate

of

the

performance

metric

-LRB-

i.e.

average

number

of

misranked

flow

pairs

-RRB-

for

the

blind

method

is

always

below

while

the

protocol-aware

method

is

always

below

Looking

at

the

standard

deviation

we

observe

large

values

for

the

blind

method

and

much

smaller

values

for

the

protocol-aware

method

This

indicates

that

the

blind

method

is

more

sensitive

to

the

sampling

process

than

the

protocol-aware

method

The

explanation

is

given

in

Section

where

we

showed

that

that

the

blind

method

presents

a

larger

error

for

large

flow

sizes

-LRB-

expect

when

the

sampling

rate

is

very

low

-RRB-

We

study

the

problem

of

detection

and

ranking

the

largest

flows

from

a

traffic

sampled

at

the

packet

level

The

study

is

packet

sampling

rate

average

number

of

misranked

flow

pairs

Abilene

trace

protocol-aware

detect

top

top

top

top

top

Figure

Performance

of

protocol-aware

detection

on

Abilene

trace

-LRB-

measurement

interval

-RRB-

when

using

actual

amount

of

data

sent

by

the

transport

layer

application

done

with

stochastic

tools

and

real

packet-level

traces

We

find

that

the

ranking

accuracy

is

strongly

dependent

on

the

sampling

rate

the

flow

size

distribution

the

total

number

of

flows

and

the

number

of

largest

flows

to

be

detected

and

ranked

By

changing

all

these

parameters

we

conclude

that

ranking

the

largest

flows

requires

a

high

sampling

rate

-LRB-

and

even

more

-RRB-

One

can

reduce

the

required

sampling

rate

by

only

detecting

the

largest

flows

without

considering

their

relative

order

We

also

introduce

a

new

method

for

flow

ranking

that

exploits

the

information

carried

in

transport

header

By

analysis

and

experimentation

we

demonstrate

that

this

new

technique

allows

to

reduce

the

required

sampling

rate

by

an

order

of

magnitude

We

are

currently

exploring

two

possible

future

directions

for

this

work

First

we

want

to

study

the

accuracy

of

the

ranking

when

the

sampled

traffic

is

fed

into

one

of

the

mechanisms

proposed

in

-LSB-

-RSB-

for

sorting

flows

with

reduced

memory

requirements

Second

we

are

exploring

the

use

of

adaptive

schemes

that

set

the

sampling

rate

based

on

the

characteristics

of

the

observed

traffic

We

wish

to

thank

NLANR

-LSB-

-RSB-

-LSB-

-RSB-

and

the

Metropolis

project

-LSB-

-RSB-

for

making

available

the

packet

traces

used

in

this

work

-LSB-

-RSB-

Abilene

Advanced

networking

for

leading-edge

research

and

education

-LSB-

-RSB-

C.

Barakat

P.

Thiran

G.

Iannaccone

C.

Diot

and

P.

Owezarski

Modeling

Internet

backbone

traffic

at

the

flow

level

IEEE

Transactions

on

Signal

Processing

-LRB-

Special

Issue

on

Signal

Processing

in

Networking

-RRB-

-LRB-

-RRB-

Aug.

-LSB-

-RSB-

M.

Charikar

K.

Chen

and

M.

Farach-Colton

Finding

frequent

items

in

data

streams

In

Proceedings

of

ICALP

-LSB-

-RSB-

B.

Y.

Choi

J.

Park

and

Z.

Zhang

Adaptive

packet

sampling

for

flow

volume

measurement

Technical

Report

University

of

Minnesota

-LSB-

-RSB-

G.

Cormode

and

S.

Muthukrishnan

What

's

hot

and

what

's

not

Tracking

most

frequent

items

dynamically

In

Proceedings

of

ACM

PODS

June

-LSB-

-RSB-

M.

Crovella

and

A.

Bestravos

Self-similarity

in

the

World

Wide

Web

traffic

Evidence

and

possible

causes

IEEE/ACM

Transactions

on

Networking

-LRB-

-RRB-

Dec.

time

-LRB-

sec

-RRB-

average

number

of

swapped

flow

pairs

Abilene

trace

detection

sampling

sampling

sampling

time

-LRB-

sec

-RRB-

average

number

of

swapped

flow

pairs

Abilene

trace

detection

sampling

sampling

sampling

Figure

Performance

of

blind

-LRB-

top

-RRB-

and

protocol-aware

-LRB-

bottom

-RRB-

detection

over

multiple

intervals

-LRB-

Abilene

trace

-RRB-

Vertical

bars

show

the

standard

deviation

over

multiple

experiments

-LSB-

-RSB-

E.

Demaine

A.

Lopez-Ortiz

and

I.

Munro

Frequency

estimation

of

internet

packet

streams

with

limited

space

In

Proceedings

of

Annual

European

Symposium

on

Algorithms

-LSB-

-RSB-

N.

G.

Duffield

C.

Lund

and

M.

Thorup

Properties

and

prediction

of

flow

statistics

from

sampled

packet

streams

In

Proceedings

of

ACM

Sigcomm

Internet

Measurement

Workshop

Nov.

-LSB-

-RSB-

N.

G.

Duffield

C.

Lund

and

M.

Thorup

Estimating

flow

distributions

from

sampled

flow

statistics

In

Proceedings

of

ACM

Sigcomm

Aug.

-LSB-

-RSB-

C.

Estan

and

G.

Varghese

New

directions

in

traffic

measurement

and

accounting

In

Proceedings

of

ACM

Sigcomm

Aug.

-LSB-

-RSB-

N.

Hohn

and

D.

Veitch

Inverting

sampled

traffic

In

Proceedings

of

ACM

Sigcomm

Internet

Measurement

Conference

Oct.

-LSB-

-RSB-

J.

Jedwab

P.

Phaal

and

B.

Pinna

Traffic

estimation

for

the

largest

sources

on

a

network

using

packet

sampling

with

limited

storage

Technical

Report

HP

Laboratories

Mar.

-LSB-

-RSB-

Metropolis

METROlogie

Pour

l'Internet

et

ses

services

owe/METROPOLIS/metropolis

eng.html

-LSB-

-RSB-

T.

Mori

M.

Uchida

R.

Kawahara

J.

Pan

and

S.

Goto

Identifying

elephant

flows

through

periodically

sampled

packets

In

Proceedings

of

ACM

Sigcomm

Internet

Measurement

Conference

Oct.

-LSB-

-RSB-

NLANR

National

Laboratory

for

Applied

Network

Research

-LSB-

-RSB-

Packet

Sampling

Working

Group

Internet

Engineering

Task

Force

-LSB-

-RSB-

K.

Papagiannaki

N.

Taft

and

C.

Diot

Impact

of

flow

dynamics

on

traffic

engineering

design

principles

In

Proceedings

of

IEEE

Infocom

Hong

Kong

China

Mar.

-LSB-

-RSB-

Renater

-LSB-

-RSB-

H.

Schulzrinne

S.

Casner

R.

Frederick

and

V.

Jacobson

RTP

A

transport

protocol

for

real-time

applications

RFC

Jan.

-LSB-

-RSB-

A.

Shaikh

J.

Rexford

and

K.

G.

Shin

Load-sensitive

routing

of

long-lived

IP

flows

In

Proceedings

of

ACM

Sigcomm

Sept.

-LSB-

-RSB-

M.

Spiegel

Theory

and

Problems

of

Probability

and

Statistics

McGraw-Hill

performance evaluation
validation with real traces
largest flow detection and ranking