Distance

Measures

for

Retrieval

In

visual

information

retrieval

the

careful

choice

of

suitable

proximity

measures

is

a

crucial

success

factor

The

evaluation

presented

in

this

paper

aims

at

showing

that

the

distance

measures

suggested

by

the

group

for

the

visual

descriptors

can

be

beaten

by

general-purpose

measures

Eight

visual

descriptors

were

selected

and

distance

measures

implemented

Three

media

collections

were

created

and

assessed

performance

indicators

developed

and

more

than

tests

performed

Additionally

a

quantisation

model

was

developed

to

be

able

to

use

predicate-based

distance

measures

on

continuous

data

as

well

The

evaluation

shows

that

the

distance

measures

recommended

in

the

are

among

the

best

but

that

other

measures

perform

even

better

H.

-LSB-

Information

Storage

and

Retrieval

-RSB-

Information

Search

and

Retrieval

Information

filtering

Query

formulation

Retrieval

models

Algorithms

Measurement

Experimentation

Performance

Theory

The

standard

defines

among

others

a

set

of

descriptors

for

visual

media

Each

descriptor

consists

of

a

feature

extraction

mechanism

a

description

-LRB-

in

binary

and

XML

format

-RRB-

and

guidelines

that

define

how

to

apply

the

descriptor

on

different

kinds

of

media

-LRB-

e.g.

on

temporal

media

-RRB-

The

descriptors

have

been

carefully

designed

to

meet

partially

complementary

requirements

of

different

application

domains

archival

browsing

retrieval

etc.

-LSB-

-RSB-

In

the

following

we

will

exclusively

deal

with

the

visual

descriptors

in

the

context

of

media

retrieval

The

visual

descriptors

fall

in

five

groups

colour

texture

shape

motion

and

others

-LRB-

e.g.

face

description

-RRB-

and

sum

up

to

basic

descriptors

For

retrieval

applications

a

rule

for

each

descriptor

is

mandatory

that

defines

how

to

measure

the

similarity

of

two

descriptions

Common

rules

are

distance

functions

like

the

Euclidean

distance

and

the

Mahalanobis

distance

Unfortunately

the

standard

does

not

include

distance

measures

in

the

normative

part

because

it

was

not

designed

to

be

-LRB-

and

should

not

exclusively

understood

to

be

-RRB-

retrieval-specific

However

the

authors

give

recommendations

which

distance

measure

to

use

on

a

particular

descriptor

These

recommendations

are

based

on

accurate

knowledge

of

the

descriptors

behaviour

and

the

description

structures

In

the

present

study

a

large

number

of

successful

distance

measures

from

different

areas

-LRB-

statistics

psychology

medicine

social

and

economic

sciences

etc.

-RRB-

were

implemented

and

applied

on

data

vectors

to

verify

whether

or

not

the

recommended

distance

measures

are

really

the

best

for

any

reasonable

class

of

media

objects

From

the

tests

and

the

recommendations

it

does

not

become

clear

how

many

and

which

distance

measures

have

been

tested

on

the

visual

descriptors

and

the

test

datasets

The

hypothesis

is

that

analytically

derived

distance

measures

may

be

good

in

general

but

only

a

quantitative

analysis

is

capable

to

identify

the

best

distance

measure

for

a

specific

feature

extraction

method

The

paper

is

organised

as

follows

Section

gives

a

minimum

of

background

information

on

the

descriptors

and

distance

measurement

in

visual

information

retrieval

-LRB-

VIR

see

-LSB-

-RSB-

-LSB-

-RSB-

-RRB-

Section

gives

an

overview

over

the

implemented

distance

measures

Section

describes

the

test

setup

including

the

test

data

and

the

implemented

evaluation

methods

Finally

Section

presents

the

results

per

descriptor

and

over

all

descriptors

The

visual

part

of

the

standard

defines

several

descriptors

Not

all

of

them

are

really

descriptors

in

the

sense

that

they

extract

properties

from

visual

media

Some

of

them

are

just

structures

for

descriptor

aggregation

or

localisation

The

basic

descriptors

are

Color

Layout

Color

Structure

Dominant

Color

Scalable

Color

Edge

Histogram

Homogeneous

Texture

Texture

Browsing

Region-based

Shape

Contour-based

Shape

Camera

Motion

Parametric

Motion

and

Motion

Activity

Other

descriptors

are

based

on

low-level

descriptors

or

semantic

information

Group-of-Frames/Group-of-Pictures

Color

-LRB-

based

on

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

or

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

MIR

November

Berkeley

California

USA

Copyright

ACM

Scalable

Color

-RRB-

Shape

-LRB-

based

on

mesh

information

-RRB-

Motion

Trajectory

-LRB-

based

on

object

segmentation

-RRB-

and

Face

Recognition

-LRB-

based

on

face

extraction

-RRB-

Descriptors

for

spatiotemporal

aggregation

and

localisation

are

Spatial

Coordinates

Grid

Layout

Region

Locator

-LRB-

spatial

-RRB-

Time

Series

Temporal

Interpolation

-LRB-

temporal

-RRB-

and

SpatioTemporal

Locator

-LRB-

combined

-RRB-

Finally

other

structures

exist

for

colour

spaces

colour

quantisation

and

multiple

views

of

objects

These

additional

structures

allow

combining

the

basic

descriptors

in

multiple

ways

and

on

different

levels

But

they

do

not

change

the

characteristics

of

the

extracted

information

Consequently

structures

for

aggregation

and

localisation

were

not

considered

in

the

work

described

in

this

paper

Similarity

measurement

on

visual

data

Generally

similarity

measurement

on

visual

information

aims

at

imitating

human

visual

similarity

perception

Unfortunately

human

perception

is

much

more

complex

than

any

of

the

existing

similarity

models

-LRB-

it

includes

perception

recognition

and

subjectivity

-RRB-

The

common

approach

in

visual

information

retrieval

is

measuring

dis-similarity

as

distance

Both

query

object

and

candidate

object

are

represented

by

their

corresponding

feature

vectors

The

distance

between

these

objects

is

measured

by

computing

the

distance

between

the

two

vectors

Consequently

the

process

is

independent

of

the

employed

querying

paradigm

-LRB-

e.g.

query

by

example

-RRB-

The

query

object

may

be

natural

-LRB-

e.g.

a

real

object

-RRB-

or

artificial

-LRB-

e.g.

properties

of

a

group

of

objects

-RRB-

Goal

of

the

measurement

process

is

to

express

a

relationship

between

the

two

objects

by

their

distance

Iteration

for

multiple

candidates

allows

then

to

define

a

partial

order

over

the

candidates

and

to

address

those

in

a

-LRB-

to

be

defined

-RRB-

neighbourhood

being

similar

to

the

query

object

At

this

point

it

has

to

be

mentioned

that

in

a

multi-descriptor

environment

especially

in

we

are

only

half

way

towards

a

statement

on

similarity

If

multiple

descriptors

are

used

-LRB-

e.g.

a

descriptor

scheme

-RRB-

a

rule

has

to

be

defined

how

to

combine

all

distances

to

a

global

value

for

each

object

Still

distance

measurement

is

the

most

important

first

step

in

similarity

measurement

Obviously

the

main

task

of

good

distance

measures

is

to

reorganise

descriptor

space

in

a

way

that

media

objects

with

the

highest

similarity

are

nearest

to

the

query

object

If

distance

is

defined

minimal

the

query

object

is

always

in

the

origin

of

distance

space

and

similar

candidates

should

form

clusters

around

the

origin

that

are

as

large

as

possible

Consequently

many

well

known

distance

measures

are

based

on

geometric

assumptions

of

descriptor

space

-LRB-

e.g.

Euclidean

distance

is

based

on

the

metric

axioms

-RRB-

Unfortunately

these

measures

do

not

fit

ideally

with

human

similarity

perception

-LRB-

e.g.

due

to

human

subjectivity

-RRB-

To

overcome

this

shortage

researchers

from

different

areas

have

developed

alternative

models

that

are

mostly

predicate-based

-LRB-

descriptors

are

assumed

to

contain

just

binary

elements

e.g.

Tversky

's

Feature

Contrast

Model

-LSB-

-RSB-

-RRB-

and

fit

better

with

human

perception

In

the

following

distance

measures

of

both

groups

of

approaches

will

be

considered

The

distance

measures

used

in

this

work

have

been

collected

from

various

areas

-LRB-

Subsection

-RRB-

Because

they

work

on

differently

quantised

data

Subsection

sketches

a

model

for

unification

on

the

basis

of

quantitative

descriptions

Finally

Subsection

introduces

the

distance

measures

as

well

as

their

origin

and

the

idea

they

implement

Sources

Distance

measurement

is

used

in

many

research

areas

such

as

psychology

sociology

-LRB-

e.g.

comparing

test

results

-RRB-

medicine

-LRB-

e.g.

comparing

parameters

of

test

persons

-RRB-

economics

-LRB-

e.g.

comparing

balance

sheet

ratios

-RRB-

etc.

Naturally

the

character

of

data

available

in

these

areas

differs

significantly

Essentially

there

are

two

extreme

cases

of

data

vectors

-LRB-

and

distance

measures

-RRB-

predicate-based

-LRB-

all

vector

elements

are

binary

e.g.

-LCB-

-RCB-

-RRB-

and

quantitative

-LRB-

all

vector

elements

are

continuous

e.g.

-LSB-

-RSB-

-RRB-

Predicates

express

the

existence

of

properties

and

represent

high-level

information

while

quantitative

values

can

be

used

to

measure

and

mostly

represent

low-level

information

Predicates

are

often

employed

in

psychology

sociology

and

other

human-related

sciences

and

most

predicate-based

distance

measures

were

therefore

developed

in

these

areas

Descriptions

in

visual

information

retrieval

are

nearly

ever

-LRB-

if

they

do

not

integrate

semantic

information

-RRB-

quantitative

Consequently

mostly

quantitative

distance

measures

are

used

in

visual

information

retrieval

The

goal

of

this

work

is

to

compare

the

distance

measures

with

the

most

powerful

distance

measures

developed

in

other

areas

Since

descriptions

are

purely

quantitative

but

some

of

the

most

sophisticated

distance

measures

are

defined

exclusively

on

predicates

a

model

is

mandatory

that

allows

the

application

of

predicate-based

distance

measures

on

quantitative

data

The

model

developed

for

this

purpose

is

presented

in

the

next

section

Quantisation

model

The

goal

of

the

quantisation

model

is

to

redefine

the

set

operators

that

are

usually

used

in

predicate-based

distance

measures

on

continuous

data

The

first

in

visual

information

retrieval

to

follow

this

approach

were

Santini

and

Jain

who

tried

to

apply

Tversky

's

Feature

Contrast

Model

-LSB-

-RSB-

to

content-based

image

retrieval

-LSB-

-RSB-

-LSB-

-RSB-

They

interpreted

continuous

data

as

fuzzy

predicates

and

used

fuzzy

set

operators

Unfortunately

their

model

suffered

from

several

shortcomings

they

described

in

-LSB-

-RSB-

-LSB-

-RSB-

-LRB-

for

example

the

quantitative

model

worked

only

for

one

specific

version

of

the

original

predicate-based

measure

-RRB-

The

main

idea

of

the

presented

quantisation

model

is

that

set

operators

are

replaced

by

statistical

functions

In

-LSB-

-RSB-

the

authors

could

show

that

this

interpretation

of

set

operators

is

reasonable

The

model

offers

a

solution

for

the

descriptors

considered

in

the

evaluation

It

is

not

specific

to

one

distance

measure

but

can

be

applied

to

any

predicate-based

measure

Below

it

will

be

shown

that

the

model

does

not

only

work

for

predicate

data

but

for

quantitative

data

as

well

Each

measure

implementing

the

model

can

be

used

as

a

substitute

for

the

original

predicate-based

measure

Generally

binary

properties

of

two

objects

-LRB-

e.g.

media

objects

-RRB-

can

exist

in

both

objects

-LRB-

denoted

as

a

-RRB-

in

just

one

-LRB-

b

c

-RRB-

or

in

of

them

-LRB-

d

-RRB-

The

operator

needed

for

these

relationships

are

UNION

MINUS

and

NOT

In

the

quantisation

model

they

are

replaced

as

follows

-LRB-

see

-LSB-

-RSB-

for

further

details

-RRB-

k

jk

ik

jk

ik

k

k

j

i

else

x

x

M

if

x

x

s

s

X

X

a

-LRB-

-RRB-

-LRB-

-RRB-

k

jk

ik

jk

ik

k

k

j

i

k

ik

jk

ik

jk

k

k

i

j

k

jk

ik

jk

ik

k

k

j

i

else

x

x

if

x

x

M

s

s

X

X

d

else

x

x

M

if

x

x

s

s

X

X

c

else

x

x

M

if

x

x

s

s

X

X

b

with

-LRB-

-RRB-

-LSB-

-RSB-

-LRB-

-RRB-

-LCB-

-RCB-

min

max

max

min

R

p

k

i

x

where

else

p

if

p

M

k

i

x

where

else

p

if

p

M

x

x

M

x

x

x

with

x

X

i

k

ik

i

k

ik

ik

ik

i

a

selects

properties

that

are

present

in

both

data

vectors

-LRB-

X

i

X

j

representing

media

objects

-RRB-

b

and

c

select

properties

that

are

present

in

just

one

of

them

and

d

selects

properties

that

are

present

in

neither

of

the

two

data

vectors

Every

property

is

selected

by

the

extent

to

which

it

is

present

-LRB-

a

and

d

mean

b

and

c

difference

-RRB-

and

only

if

the

amount

to

which

it

is

present

exceeds

a

certain

threshold

-LRB-

depending

on

the

mean

and

standard

deviation

over

all

elements

of

descriptor

space

-RRB-

The

implementation

of

these

operators

is

based

on

one

assumption

It

is

assumed

that

vector

elements

measure

on

interval

scale

That

means

each

element

expresses

that

the

measured

property

is

more

or

less

present

-LRB-

not

at

all

M

fully

present

-RRB-

This

is

true

for

most

visual

descriptors

and

all

descriptors

A

natural

origin

as

it

is

assumed

here

-LRB-

-RRB-

is

not

needed

Introducing

p

-LRB-

called

discriminance-defining

parameter

-RRB-

for

the

thresholds

has

the

positive

consequence

that

a

b

c

d

can

then

be

controlled

through

a

single

parameter

p

is

an

additional

criterion

for

the

behaviour

of

a

distance

measure

and

determines

the

thresholds

used

in

the

operators

It

expresses

how

accurate

data

items

are

present

-LRB-

quantisation

-RRB-

and

consequently

how

accurate

they

should

be

investigated

p

can

be

set

by

the

user

or

automatically

Interesting

are

the

limits

M

p

In

this

case

all

elements

-LRB-

properties

-RRB-

are

assumed

to

be

continuous

-LRB-

high

quantisation

-RRB-

In

consequence

all

properties

of

a

descriptor

are

used

by

the

operators

Then

the

distance

measure

is

not

discriminant

for

properties

p

In

this

case

all

properties

are

assumed

to

be

predicates

In

consequence

only

binary

elements

-LRB-

predicates

-RRB-

are

used

by

the

operators

-LRB-

quantisation

-RRB-

The

distance

measure

is

then

highly

discriminant

for

properties

Between

these

limits

a

distance

measure

that

uses

the

quantisation

model

is

depending

on

p

more

or

less

discriminant

for

properties

This

means

it

selects

a

subset

of

all

available

description

vector

elements

for

distance

measurement

For

both

predicate

data

and

quantitative

data

it

can

be

shown

that

the

quantisation

model

is

reasonable

If

description

vectors

consist

of

binary

elements

only

p

should

be

used

as

follows

-LRB-

for

example

p

can

easily

be

set

automatically

-RRB-

-LRB-

-RRB-

min

p

g

e

p

In

this

case

a

b

c

d

measure

like

the

set

operators

they

replace

For

example

Table

shows

their

behaviour

for

two

one-dimensional

feature

vectors

X

i

and

X

j

As

can

be

seen

the

statistical

measures

work

like

set

operators

Actually

the

quantisation

model

works

accurate

on

predicate

data

for

any

p.

To

show

that

the

model

is

reasonable

for

quantitative

data

the

following

fact

is

used

It

is

easy

to

show

that

for

predicate

data

some

quantitative

distance

measures

degenerate

to

predicate-based

measures

For

example

the

L

metric

-LRB-

Manhattan

metric

-RRB-

degenerates

to

the

Hamming

distance

-LRB-

from

-LSB-

-RSB-

without

weights

-RRB-

distance

Hamming

c

b

x

x

L

k

jk

ik

If

it

can

be

shown

that

the

quantisation

model

is

able

to

reconstruct

the

quantitative

measure

from

the

degenerated

predicate-based

measure

the

model

is

obviously

able

to

extend

predicate-based

measures

to

the

quantitative

domain

This

is

easy

to

illustrate

For

purely

quantitative

feature

vectors

p

should

be

used

as

follows

-LRB-

again

p

can

easily

be

set

automatically

-RRB-

p

Then

a

and

d

become

continuous

functions

k

jk

ik

k

k

jk

ik

k

jk

ik

k

k

jk

ik

x

x

M

s

where

s

d

true

M

x

x

x

x

s

where

s

a

true

M

x

x

M

b

and

c

can

be

made

continuous

for

the

following

expressions

-LRB-

-RRB-

-LRB-

-RRB-

k

jk

ik

k

k

k

ik

jk

ik

jk

k

k

ik

jk

ik

jk

k

jk

ik

jk

ik

k

k

jk

ik

jk

ik

x

x

s

where

s

c

b

else

x

x

if

x

x

s

where

s

c

x

x

M

x

x

M

else

x

x

if

x

x

s

where

s

b

x

x

M

x

x

M

Table

Quantisation

model

on

predicate

vectors

X

i

X

j

a

b

c

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

k

ik

jk

k

k

k

jk

ik

k

k

x

x

s

where

s

b

c

x

x

s

where

s

c

b

This

means

for

sufficiently

high

p

every

predicate-based

distance

measure

that

is

either

not

using

b

and

c

or

just

as

b

c

b-c

or

c-b

can

be

transformed

into

a

continuous

quantitative

distance

measure

For

example

the

Hamming

distance

-LRB-

again

without

weights

-RRB-

L

x

x

x

x

s

where

s

c

b

k

jk

ik

k

jk

ik

k

k

The

quantisation

model

successfully

reconstructs

the

L

metric

and

no

distance

measure-specific

modification

has

to

be

made

to

the

model

This

demonstrates

that

the

model

is

reasonable

In

the

following

it

will

be

used

to

extend

successful

predicate-based

distance

measures

on

the

quantitative

domain

The

major

advantages

of

the

quantisation

model

are

-LRB-

-RRB-

it

is

application

domain

independent

-LRB-

-RRB-

the

implementation

is

straightforward

-LRB-

-RRB-

the

model

is

easy

to

use

and

finally

-LRB-

-RRB-

the

new

parameter

p

allows

to

control

the

similarity

measurement

process

in

a

new

way

-LRB-

discriminance

on

property

level

-RRB-

Implemented

measures

For

the

evaluation

described

in

this

work

next

to

predicate-based

-LRB-

based

on

the

quantisation

model

-RRB-

and

quantitative

measures

the

distance

measures

recommended

in

the

standard

were

implemented

-LRB-

all

together

different

distance

measures

-RRB-

Table

summarises

those

predicate-based

measures

that

performed

best

in

the

evaluation

-LRB-

in

sum

predicate-based

measures

were

investigated

-RRB-

For

these

measures

K

is

the

number

of

predicates

in

the

data

vectors

X

i

and

X

j

In

the

sum

is

used

for

Tversky

's

f

-LRB-

-RRB-

-LRB-

as

Tversky

himself

does

in

-LSB-

-RSB-

-RRB-

and

are

weights

for

element

b

and

c

In

-LSB-

-RSB-

the

author

's

investigated

Tversky

's

Feature

Contrast

Model

and

found

to

be

the

optimum

parameters

Some

of

the

predicate-based

measures

are

very

simple

-LRB-

e.g.

-RRB-

but

have

been

heavily

exploited

in

psychological

research

Pattern

difference

-LRB-

-RRB-

a

very

powerful

measure

is

used

in

the

statistics

package

SPSS

for

cluster

analysis

is

a

correlation

coefficient

for

predicates

developed

by

Pearson

Table

shows

the

best

quantitative

distance

measures

that

were

used

and

are

metric-based

and

were

implemented

as

representatives

for

the

entire

group

of

Minkowski

distances

The

w

i

are

weights

In

i

i

are

mean

and

standard

deviation

for

the

elements

of

descriptor

X

i

In

m

is

-LRB-

-RRB-

the

Canberra

metric

is

a

normalised

form

of

Similarly

Clark

's

divergence

coefficient

is

a

normalised

version

of

is

a

further-developed

correlation

coefficient

that

is

invariant

against

sign

changes

This

measure

is

used

even

though

its

particular

properties

are

of

minor

importance

for

this

application

domain

Finally

is

a

measure

that

takes

the

differences

between

adjacent

vector

elements

into

account

This

makes

it

structurally

different

from

all

other

measures

Obviously

one

important

distance

measure

is

missing

The

Mahalanobis

distance

was

not

considered

because

different

descriptors

would

require

different

covariance

matrices

and

for

some

descriptors

it

is

simply

impossible

to

define

a

covariance

matrix

If

the

identity

matrix

was

used

in

this

case

the

Mahalanobis

distance

would

degenerate

to

a

Minkowski

distance

Additionally

the

recommended

distances

were

implemented

with

the

following

parameters

In

the

distance

measure

of

the

Color

Layout

descriptor

all

weights

were

set

to

-LRB-

as

in

all

other

implemented

measures

-RRB-

In

the

distance

measure

of

the

Dominant

Color

descriptor

the

following

parameters

were

used

d

T

w

w

-LRB-

as

recommended

-RRB-

In

the

Homogeneous

Texture

descriptor

's

distance

all

-LRB-

-RRB-

k

were

set

to

and

matching

was

done

rotation

and

scale-invariant

Important

Some

of

the

measures

presented

in

this

section

are

distance

measures

while

others

are

similarity

measures

For

the

tests

it

is

important

to

notice

that

all

similarity

measures

were

inverted

to

distance

measures

Subsection

describes

the

descriptors

-LRB-

including

parameters

-RRB-

and

the

collections

-LRB-

including

ground

truth

information

-RRB-

that

were

used

in

the

evaluation

Subsection

discusses

the

evaluation

method

that

was

implemented

and

Subsection

sketches

the

test

environment

used

for

the

evaluation

process

Test

data

For

the

evaluation

eight

descriptors

were

used

All

colour

descriptors

Color

Layout

Color

Structure

Dominant

Color

Scalable

Color

all

texture

descriptors

Edge

Histogram

Homogeneous

Texture

Texture

Browsing

and

one

shape

descriptor

Region-based

Shape

Texture

Browsing

was

used

even

though

the

standard

suggests

that

it

is

not

suitable

for

retrieval

The

other

basic

shape

descriptor

Contour-based

Shape

was

not

used

because

it

produces

structurally

different

descriptions

that

can

not

be

transformed

to

data

vectors

with

elements

measuring

on

interval-scales

The

motion

descriptors

were

not

used

because

they

integrate

the

temporal

dimension

of

visual

media

and

would

only

be

comparable

if

the

basic

colour

texture

and

shape

descriptors

would

be

aggregated

over

time

This

was

not

done

Finally

no

high-level

descriptors

were

used

-LRB-

Localisation

Face

Recognition

etc.

see

Subsection

-RRB-

because

to

the

author

's

opinion

the

behaviour

of

the

basic

descriptors

on

elementary

media

objects

should

be

evaluated

before

conclusions

on

aggregated

structures

can

be

drawn

Table

Predicate-based

distance

measures

No

Measure

Comment

c

b

a

Feature

Contrast

Model

Tversky

-LSB-

-RSB-

a

No

of

co-occurrences

c

b

Hamming

distance

K

a

Russel

-LSB-

-RSB-

c

b

a

Kulczvnski

-LSB-

-RSB-

K

bc

Pattern

difference

-LSB-

-RSB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

d

c

d

b

c

a

b

a

bc

ad

Pearson

-LSB-

-RSB-

The

Texture

Browsing

descriptions

had

to

be

transformed

from

five

bins

to

an

eight

bin

representation

in

order

that

all

elements

of

the

descriptor

measure

on

an

interval

scale

A

Manhattan

metric

was

used

to

measure

proximity

-LRB-

see

-LSB-

-RSB-

for

details

-RRB-

Descriptor

extraction

was

performed

using

the

reference

implementation

In

the

extraction

process

each

descriptor

was

applied

on

the

entire

content

of

each

media

object

and

the

following

extraction

parameters

were

used

Colour

in

Color

Structure

was

quantised

to

bins

For

Dominant

Color

colour

space

was

set

to

YCrCb

default

quantisation

was

used

and

the

default

value

for

spatial

coherency

was

used

Homogeneous

Texture

was

quantised

to

components

Scalable

Color

values

were

quantised

to

sizeof

-LRB-

int

-RRB-

bits

and

bins

were

used

Finally

Texture

Browsing

was

used

with

five

components

These

descriptors

were

applied

on

three

media

collections

with

image

content

the

Brodatz

dataset

-LRB-

images

pixel

-RRB-

a

subset

of

the

Corel

dataset

-LRB-

images

pixel

portrait

and

landscape

-RRB-

and

a

dataset

with

coats-of-arms

images

-LRB-

images

pixel

-RRB-

Figure

shows

examples

from

the

three

collections

Designing

appropriate

test

sets

for

a

visual

evaluation

is

a

highly

difficult

task

-LRB-

for

example

see

the

TREC

video

report

-LSB-

-RSB-

-RRB-

Of

course

for

identifying

the

best

distance

measure

for

a

descriptor

it

should

be

tested

on

an

infinite

number

of

media

objects

But

this

is

not

the

aim

of

this

study

It

is

just

evaluated

if

for

likely

image

collections

better

proximity

measures

than

those

suggested

by

the

group

can

be

found

Collections

of

this

relatively

small

size

were

used

in

the

evaluation

because

the

applied

evaluation

methods

are

above

a

certain

minimum

size

invariant

against

collection

size

and

for

smaller

collections

it

is

easier

to

define

a

high-quality

ground

truth

Still

the

average

ratio

of

ground

truth

size

to

collection

size

is

at

least

Especially

no

collection

from

the

dataset

was

used

in

the

evaluation

because

the

evaluations

should

show

how

well

the

descriptors

and

the

recommended

distance

measures

perform

on

unknown

material

When

the

descriptor

extraction

was

finished

the

resulting

XML

descriptions

were

transformed

into

a

data

matrix

with

lines

-LRB-

media

objects

-RRB-

and

columns

-LRB-

descriptor

elements

-RRB-

To

be

usable

with

distance

measures

that

do

not

integrate

domain

knowledge

the

elements

of

this

data

matrix

were

normalised

to

-LSB-

-RSB-

For

the

distance

evaluation

next

to

the

normalised

data

matrix

human

similarity

judgement

is

needed

In

this

work

the

ground

truth

is

built

of

twelve

groups

of

similar

images

-LRB-

four

for

each

dataset

-RRB-

Group

membership

was

rated

by

humans

based

on

semantic

criterions

Table

summarises

the

twelve

groups

and

the

underlying

descriptions

It

has

to

be

noticed

that

some

of

these

groups

-LRB-

especially

and

-RRB-

are

much

harder

to

find

with

low-level

descriptors

than

others

Evaluation

method

Usually

retrieval

evaluation

is

performed

based

on

a

ground

truth

with

recall

and

precision

-LRB-

see

for

example

-LSB-

-RSB-

-LSB-

-RSB-

-RRB-

In

multi-descriptor

environments

this

leads

to

a

problem

because

the

resulting

recall

and

precision

values

are

strongly

influenced

by

the

method

used

to

merge

the

distance

values

for

one

media

object

Even

though

it

is

nearly

impossible

to

say

how

big

the

influence

of

a

single

distance

measure

was

on

the

resulting

recall

and

precision

values

this

problem

has

been

almost

ignored

so

far

In

Subsection

it

was

stated

that

the

major

task

of

a

distance

measure

is

to

bring

the

relevant

media

objects

as

close

to

the

origin

-LRB-

where

the

query

object

lies

-RRB-

as

possible

Even

in

a

multi-descriptor

environment

it

is

then

simple

to

identify

the

similar

objects

in

a

large

distance

space

Consequently

it

was

decided

to

Table

Quantitative

distance

measures

No

Measure

Comment

No

Measure

Comment

k

jk

ik

i

x

x

w

City

block

distance

-LRB-

L

-RRB-

-LRB-

-RRB-

k

jk

ik

i

x

x

w

Euclidean

distance

-LRB-

L

-RRB-

k

jk

ik

jk

ik

x

x

x

x

Canberra

metric

Lance

Williams

-LSB-

-RSB-

-LRB-

-RRB-

k

jk

ik

jk

ik

x

x

x

x

K

Divergence

coefficient

Clark

-LSB-

-RSB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

k

k

j

jk

i

ik

k

j

jk

i

ik

x

x

x

x

Correlation

coefficient

k

ik

k

jk

ik

k

ik

k

k

jk

k

ik

jk

ik

x

m

Km

x

x

m

Km

x

x

x

m

Km

x

x

Cohen

-LSB-

-RSB-

k

k

jk

ik

k

jk

ik

x

x

x

x

Angular

distance

Gower

-LSB-

-RSB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

K

k

jk

jk

ik

ik

x

x

x

x

Meehl

Index

-LSB-

-RSB-

Table

Ground

truth

information

Coll

No

Images

Description

Regular

chequered

patterns

Dark

white

noise

Moon-like

surfaces

Brodatz

Water-like

surfaces

Humans

in

nature

-LRB-

difficult

-RRB-

Images

with

snow

-LRB-

mountains

skiing

-RRB-

Animals

in

nature

-LRB-

difficult

-RRB-

Corel

Large

coloured

flowers

Bavarian

communal

arms

All

Bavarian

arms

-LRB-

difficult

-RRB-

Dark

objects

light

unsegmented

shield

Arms

Major

charges

on

blue

or

red

shield

use

indicators

measuring

the

distribution

in

distance

space

of

candidates

similar

to

the

query

object

for

this

evaluation

instead

of

recall

and

precision

Identifying

clusters

of

similar

objects

-LRB-

based

on

the

given

ground

truth

-RRB-

is

relatively

easy

because

the

resulting

distance

space

for

one

descriptor

and

any

distance

measure

is

always

one-dimensional

Clusters

are

found

by

searching

from

the

origin

of

distance

space

to

the

first

similar

object

grouping

all

following

similar

objects

in

the

cluster

breaking

off

the

cluster

with

the

first

un-similar

object

and

so

forth

For

the

evaluation

two

indicators

were

defined

The

first

measures

the

average

distance

of

all

cluster

means

to

the

origin

distance

avg

clusters

no

size

cluster

distance

clusters

no

i

i

size

cluster

j

ij

d

i

where

distance

ij

is

the

distance

value

of

the

j-th

element

in

the

i-th

cluster

CLUSTERS

i

i

CLUSTERS

i

size

cluster

j

ij

size

cluster

distance

distance

avg

i

is

the

number

of

found

clusters

and

i

is

the

size

of

the

i-th

cluster

The

resulting

indicator

is

normalised

by

the

distribution

characteristics

of

the

distance

measure

-LRB-

-RRB-

Additionally

the

standard

deviation

is

used

In

the

evaluation

process

this

measure

turned

out

to

produce

valuable

results

and

to

be

relatively

robust

against

parameter

p

of

the

quantisation

model

In

Subsection

we

noted

that

p

affects

the

discriminance

of

a

predicate-based

distance

measure

The

smaller

p

is

set

the

larger

are

the

resulting

clusters

because

the

quantisation

model

is

then

more

discriminant

against

properties

and

less

elements

of

the

data

matrix

are

used

This

causes

a

side-effect

that

is

measured

by

the

second

indicator

more

and

more

un-similar

objects

come

out

with

exactly

the

same

distance

value

as

similar

objects

-LRB-

a

problem

that

does

not

exist

for

large

p

's

-RRB-

and

become

indiscernible

from

similar

objects

Consequently

they

are

-LRB-

false

-RRB-

cluster

members

This

phenomenon

-LRB-

conceptually

similar

to

the

false

negatives

indicator

-RRB-

was

named

cluster

pollution

and

the

indicator

measures

the

average

cluster

pollution

over

all

clusters

clusters

no

doubles

no

cp

clusters

no

i

size

cluster

j

ij

i

where

ij

is

the

number

of

indiscernible

un-similar

objects

associated

with

the

j-th

element

of

cluster

i.

Remark

Even

though

there

is

a

certain

influence

it

could

be

proven

in

-LSB-

-RSB-

that

no

significant

correlation

exists

between

parameter

p

of

the

quantisation

model

and

cluster

pollution

Test

environment

As

pointed

out

above

to

generate

the

descriptors

the

reference

implementation

in

version

was

used

-LRB-

provided

by

TU

Munich

-RRB-

Image

processing

was

done

with

Adobe

Photoshop

and

normalisation

and

all

evaluations

were

done

with

Perl

The

querying

process

was

performed

in

the

following

steps

-LRB-

-RRB-

random

selection

of

a

ground

truth

group

-LRB-

-RRB-

random

selection

of

a

query

object

from

this

group

-LRB-

-RRB-

distance

comparison

for

all

other

objects

in

the

dataset

-LRB-

-RRB-

clustering

of

the

resulting

distance

space

based

on

the

ground

truth

and

finally

-LRB-

-RRB-

evaluation

For

each

combination

of

dataset

and

distance

measure

queries

were

issued

and

evaluations

were

aggregated

over

all

datasets

and

descriptors

The

next

section

shows

the

partially

surprising

results

In

the

results

presented

below

the

first

indicator

from

Subsection

was

used

to

evaluate

distance

measures

In

a

first

step

parameter

p

had

to

be

set

in

a

way

that

all

measures

are

equally

discriminant

Distance

measurement

is

fair

if

the

following

condition

holds

true

for

any

predicate-based

measure

d

P

and

any

continuous

measure

d

C

-LRB-

-RRB-

-LRB-

-RRB-

C

P

d

cp

p

d

cp

Then

it

is

guaranteed

that

predicate-based

measures

do

not

create

larger

clusters

-LRB-

with

a

higher

number

of

similar

objects

-RRB-

for

the

price

of

higher

cluster

pollution

In

more

than

test

queries

the

optimum

value

was

found

to

be

p

Results

are

organised

as

follows

Subsection

summarises

the

Figure

Test

datasets

Left

Brodatz

dataset

middle

Corel

dataset

right

coats-of-arms

dataset

best

distance

measures

per

descriptor

Section

shows

the

best

overall

distance

measures

and

Section

points

out

other

interesting

results

-LRB-

for

example

distance

measures

that

work

particularly

good

on

specific

ground

truth

groups

-RRB-

Best

measure

per

descriptor

Figure

shows

the

evaluation

results

for

the

first

indicator

For

each

descriptor

the

best

measure

and

the

performance

of

the

recommendation

are

shown

The

results

are

aggregated

over

the

tested

datasets

On

first

sight

it

becomes

clear

that

the

recommendations

are

mostly

relatively

good

but

never

the

best

For

Color

Layout

the

difference

between

and

the

best

measure

the

Meehl

index

-LRB-

-RRB-

is

just

and

the

measure

has

a

smaller

standard

deviation

The

reason

why

the

Meehl

index

is

better

may

be

that

this

descriptors

generates

descriptions

with

elements

that

have

very

similar

variance

Statistical

analysis

confirmed

that

-LRB-

see

-LSB-

-RSB-

-RRB-

For

Color

Structure

Edge

Histogram

Homogeneous

Texture

Region-based

Shape

and

Scalable

Color

by

far

the

best

measure

is

pattern

difference

-LRB-

-RRB-

Psychological

research

on

human

visual

perception

has

revealed

that

in

many

situation

differences

between

the

query

object

and

a

candidate

weigh

much

stronger

than

common

properties

The

pattern

difference

measure

implements

this

insight

in

the

most

consequent

way

In

the

author

's

opinion

the

reason

why

pattern

difference

performs

so

extremely

well

on

many

descriptors

is

due

to

this

fact

Additional

advantages

of

pattern

difference

are

that

it

usually

has

a

very

low

variance

and

because

it

is

a

predicate-based

measure

its

discriminance

-LRB-

and

cluster

structure

-RRB-

can

be

tuned

with

parameter

p

The

best

measure

for

Dominant

Color

turned

out

to

be

Clark

's

Divergence

coefficient

-LRB-

-RRB-

This

is

a

similar

measure

to

pattern

difference

on

the

continuous

domain

The

Texture

Browsing

descriptor

is

a

special

problem

In

the

standard

it

is

recommended

to

use

it

exclusively

for

browsing

After

testing

it

for

retrieval

on

various

distance

measures

the

author

supports

this

opinion

It

is

very

difficult

to

find

a

good

distance

measure

for

Texture

Browsing

The

proposed

Manhattan

metric

for

example

performs

very

bad

The

best

measure

is

predicate-based

-LRB-

-RRB-

It

works

on

common

properties

-LRB-

a

d

-RRB-

but

produces

clusters

with

very

high

cluster

pollution

For

this

descriptor

the

second

indicator

is

up

to

eight

times

higher

than

for

predicate-based

measures

on

other

descriptors

Best

overall

measures

Figure

summarises

the

results

over

all

descriptors

and

media

collections

The

diagram

should

give

an

indication

on

the

general

potential

of

the

investigated

distance

measures

for

visual

information

retrieval

It

can

be

seen

that

the

best

overall

measure

is

a

predicate-based

one

The

top

performance

of

pattern

difference

-LRB-

-RRB-

proves

that

the

quantisation

model

is

a

reasonable

method

to

extend

predicate-based

distance

measures

on

the

continuous

domain

The

second

best

group

of

measures

are

the

recommendations

which

have

a

slightly

higher

mean

but

a

lower

standard

deviation

than

pattern

difference

The

third

best

measure

is

the

Meehl

index

-LRB-

-RRB-

a

measure

developed

for

psychological

applications

but

because

of

its

characteristic

properties

tailor-made

for

certain

-LRB-

homogeneous

-RRB-

descriptors

Minkowski

metrics

are

also

among

the

best

measures

the

average

mean

and

variance

of

the

Manhattan

metric

-LRB-

-RRB-

and

the

Euclidean

metric

-LRB-

-RRB-

are

in

the

range

of

Of

course

these

measures

do

not

perform

particularly

well

for

any

of

the

descriptors

Remarkably

for

a

predicate-based

measure

Tversky

's

Feature

Contrast

Model

-LRB-

-RRB-

is

also

in

the

group

of

very

good

measures

-LRB-

even

though

it

is

not

among

the

best

-RRB-

that

ends

with

the

correlation

coefficient

The

other

measures

either

have

a

significantly

higher

mean

or

a

very

large

standard

deviation

Other

interesting

results

Distance

measures

that

perform

in

average

worse

than

others

may

in

certain

situations

-LRB-

e.g.

on

specific

content

-RRB-

still

perform

better

For

Color

Layout

for

example

is

a

very

good

measure

on

colour

photos

It

performs

as

good

as

and

has

a

lower

standard

deviation

For

artificial

images

the

pattern

difference

and

the

Hamming

distance

produce

comparable

results

as

well

If

colour

information

is

available

in

media

objects

pattern

difference

performs

well

on

Dominant

Color

-LRB-

just

worse

-RRB-

and

in

case

of

difficult

ground

truth

-LRB-

group

-RRB-

the

Meehl

index

is

as

strong

as

MP

MP

MP

MP

MP

MP

MP

Color

Layout

Color

Structure

Dominant

Color

Edge

Histogram

Homog

Texture

Region

Shape

Scalable

Color

Texture

Browsing

Figure

Results

per

measure

and

descriptor

The

horizontal

axis

shows

the

best

measure

and

the

performance

of

the

recommendation

for

each

descriptor

The

vertical

axis

shows

the

values

for

the

first

indicator

-LRB-

smaller

value

better

cluster

structure

-RRB-

Shades

have

the

following

meaning

black

-LRB-

good

cases

-RRB-

black

dark

grey

-LRB-

average

-RRB-

and

black

dark

grey

light

grey

-LRB-

bad

-RRB-

The

evaluation

presented

in

this

paper

aims

at

testing

the

recommended

distance

measures

and

finding

better

ones

for

the

basic

visual

descriptors

Eight

descriptors

were

selected

distance

measures

were

implemented

media

collections

were

created

and

assessed

performance

indicators

were

defined

and

more

than

tests

were

performed

To

be

able

to

use

predicate-based

distance

measures

next

to

quantitative

measures

a

quantisation

model

was

defined

that

allows

the

application

of

predicate-based

measures

on

continuous

data

In

the

evaluation

the

best

overall

distance

measures

for

visual

content

as

extracted

by

the

visual

descriptors

turned

out

to

be

the

pattern

difference

measure

and

the

Meehl

index

-LRB-

for

homogeneous

descriptions

-RRB-

Since

these

two

measures

perform

significantly

better

than

the

recommendations

they

should

be

further

tested

on

large

collections

of

image

and

video

content

-LRB-

e.g.

from

-LSB-

-RSB-

-RRB-

The

choice

of

the

right

distance

function

for

similarity

measurement

depends

on

the

descriptor

the

queried

media

collection

and

the

semantic

level

of

the

user

's

idea

of

similarity

This

work

offers

suitable

distance

measures

for

various

situations

In

consequence

the

distance

measures

identified

as

the

best

will

be

implemented

in

the

open

based

visual

information

retrieval

framework

VizIR

-LSB-

-RSB-

The

author

would

like

to

thank

Christian

Breiteneder

for

his

valuable

comments

and

suggestions

for

improvement

The

work

presented

in

this

paper

is

part

of

the

VizIR

project

funded

by

the

Austrian

Scientific

Research

Fund

FWF

under

grant

no

-LSB-

-RSB-

Clark

P.S.

An

extension

of

the

coefficient

of

divergence

for

use

with

multiple

characters

Copeia

-LRB-

-RRB-

-LSB-

-RSB-

Cohen

J

A

profile

similarity

coefficient

invariant

over

variable

reflection

Psychological

Bulletin

-LRB-

-RRB-

-LSB-

-RSB-

Del

Bimbo

A.

Visual

information

retrieval

Morgan

Kaufmann

Publishers

San

Francisco

CA

-LSB-

-RSB-

Eidenberger

H.

and

Breiteneder

C

A

framework

for

visual

information

retrieval

In

Proceedings

Visual

Information

Systems

Conference

-LRB-

HSinChu

Taiwan

March

-RRB-

LNCS

Springer

Verlag

-LSB-

-RSB-

Eidenberger

H.

and

Breiteneder

C.

Visual

similarity

measurement

with

the

Feature

Contrast

Model

In

Proceedings

SPIE

Storage

and

Retrieval

for

Media

Databases

Conference

-LRB-

Santa

Clara

CA

January

-RRB-

SPIE

Vol

-LSB-

-RSB-

Eidenberger

H.

How

good

are

the

visual

features

In

Proceedings

SPIE

Visual

Communications

and

Image

Processing

Conference

-LRB-

Lugano

Switzerland

July

-RRB-

SPIE

Vol

-LSB-

-RSB-

Gower

J.G.

Multivariate

analysis

and

multidimensional

geometry

The

Statistician

-LRB-

-RRB-

-LSB-

-RSB-

Lance

G.N.

and

Williams

W.T.

Mixed

data

classificatory

programs

Agglomerative

Systems

Australian

Comp

Journal

-LRB-

-RRB-

-LSB-

-RSB-

Manjunath

B.S.

Ohm

J.R.

Vasudevan

V.V.

and

Yamada

A.

Color

and

texture

descriptors

In

Special

Issue

on

IEEE

Transactions

on

Circuits

and

Systems

for

Video

Technology

-LRB-

June

-RRB-

-LSB-

-RSB-

Meehl

P.

E

The

problem

is

epistemology

not

statistics

Replace

significance

tests

by

confidence

intervals

and

quantify

accuracy

of

risky

numerical

predictions

In

Harlow

L.L.

Mulaik

S.A.

and

Steiger

J.H.

-LRB-

Eds

-RRB-

What

if

there

were

no

significance

tests

Erlbaum

Mahwah

NJ

-LSB-

-RSB-

Pearson

K.

On

the

coefficients

of

racial

likeness

Biometrica

-LRB-

-RRB-

-LSB-

-RSB-

Santini

S.

and

Jain

R.

Similarity

is

a

geometer

Multimedia

Tools

and

Application

-LRB-

-RRB-

-LSB-

-RSB-

Santini

S.

and

Jain

R.

Similarity

measures

IEEE

Transactions

on

Pattern

Analysis

and

Machine

Intelligence

-LRB-

September

-RRB-

-LSB-

-RSB-

Sint

P.P.

Similarity

structures

and

similarity

measures

Austrian

Academy

of

Sciences

Press

Vienna

Austria

-LRB-

in

German

-RRB-

-LSB-

-RSB-

Smeaton

A.F.

and

Over

P

The

video

track

report

NIST

Special

Publication

SP

-LRB-

March

-RRB-

available

from

VIDEO.OVER.pdf

-LRB-

last

visited

-RRB-

-LSB-

-RSB-

Smeulders

A.W.M.

Worring

M.

Santini

S.

Gupta

A.

and

Jain

R.

Content-based

image

retrieval

at

the

end

of

the

early

years

IEEE

Transactions

on

Pattern

Analysis

and

Machine

Intelligence

-LRB-

December

-RRB-

-LSB-

-RSB-

Tversky

A.

Features

of

similarity

Psychological

Review

-LRB-

July

-RRB-

MP

Figure

Overall

results

-LRB-

ordered

by

the

first

indicator

-RRB-

The

vertical

axis

shows

the

values

for

the

first

indicator

-LRB-

smaller

value

better

cluster

structure

-RRB-

Shades

have

the

following

meaning

black

black

dark

grey

and

black

dark

grey

light

grey

O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

similarity measurement
distance measurement
quantisation
distance measure
visual information retrieval
similarity perception

content-based image retrieval
content-based video retrieval
mpeg-7 descriptors
human similarity perception
mpeg-7


Development of E-commerce Statistics and the Implications This text has analyzed the development of E-commerce in some developed countries such as Canada , U.S.A. , Japan , etc and put forward several suggestions on how to set up the system of E-commerce in our country taking the national conditions of our country into account . Since the 1990s , the rapid development of e-commerce has brought extensive , enormous and far-reaching influence on the economy of the countries all over the world . E-commerce has already become the contemporary chief trend of economic and social development . As representatives of advanced productivity of new economic period , the level of its development has already become important signs of measuring the modernization level and comprehensive strength of countries and cities ; it has become important means to make changeover in the economic system and reform the style of economic , promote the upgrading of the industrial structure , promote the modernized level of the city and strengthen international competitiveness . So , the governments all over the world have paid close attention to the development of E-commerce Statistics . Though the development of informationization in our country is very quick , it still has great disparity with the developed countries for relatively late start . Our country is still in the interim facing the double task of informationization and industrialization at present . So , in order to carry on an instant , accurate Statistics to the development level of E-commerce and set up perfect E-commerce Statistics system , we must understand , absorb and bring in the theories and methods of E-commerce Statistics from the main foreign countries to make E-commerce Statistics become effective guarantee in leading and promoting e-commerce in a healthy way , combining social source and promoting national power . We have chosen some representative countries in the world and analyzed the development of E-commerce Statistics in these countries . 2.1 Definitions of e-commerce in main Developed countries . The definition of e-commerce is the standard of carrying on E-commerce Statistics , but there are various kinds of definition of E-commerce in the world because of visual angles are different . So , it is necessary for each country to make a distinct , standard , practical , wide meaningful and measurable definition which should be suitable for each field and can be amenable to time . 2.1.1 Definition of e-commerce in OECD -LRB- Organization for Economic Cooperation and Development -RRB- There are broadly-defined e-commerce and the narrowly-defined e-commerce . The broadly-defined e-commerce means the activity of electronic transaction on item and service , no matter the transaction occurred between enterprise , family , government and other public or individual organizations . It uses network as a intermediary . Goods and service should be ordered on the network but the payment and goods service do n't need to carry on the net ; the narrowly-defined e-commerce is only referred to trade activity carrying on through Internet . It is worth pointing out that the source of OECD definition of e-commerce is the Canada official Statistical department . 2.1.2 Definition of e-commerce in Canada The e-commerce definition of Canada official Statistical department is : E-commerce is the transaction which based on the computer network , including the transformation of ownership , transformation of tangible and intangible assets right to use . It consists of B2B -LRB- business to business -RRB- , B2C -LRB- business to consumer -RRB- , B2G -LRB- business to government -RRB- and G2C -LRB- government to consumer -RRB- . But the transaction taking place inside enterprises will not be included in the e-commerce Statistics . 2.1.3 Definition of e-commerce in the U.S.A. Definition of e-commerce in the U.S.A. is defined by the U.S.A. general survey bureau who divides e-commerce into three parts from angle of the overall situation : e-commerce infrastructure ; electronic affairs ; e-commerce . E-commerce infrastructure is the economic facility or equipment which is used for supporting electronic affairs or electronic transaction activities . Electronic affairs include the affairs managed by computer network in company , government and other non - profit organization . E-commerce refers to goods or service transaction activity completed on computer network . 70 2.2 Overview and Characteristic of Main Country 's e-commerce Statistics 2.2.1 Overviews of e-commerce Statistical Surveys 2.2.1.1 Overviews of Canadian e-commerce Statistical Survey The e-commerce Statistics in Canada is an official activity that was presiding over by government and implemented concretely by State Statistics Bureau of Canada . Up till now , Canada has implemented four pieces of different e-commerce Statistics . a -RRB- `` Net-banking operation and bank service Statistics survey on internet and e-commerce application in financial department '' , this investigation is an irregular survey ; its respondents are enterprises of the financial field and its nature is a separate investigation ; b -RRB- `` Annually Statistical survey on internet application in family '' , is a fixed annual Statistical survey . It is a supplemented investigation and its respondents are families . c -RRB- `` The Statistics survey on communication technology and e - commerce '' , is an irregular Statistical survey ; it is a supplement investigation and the respondents are enterprises in `` the standard industry of North America classifies '' d -RRB- `` Annually Statistical survey on e-commerce and relevant technology '' , is a fixed annual Statistical survey ; it is a supplemental investigation and the respondents are enterprises in `` the standard industry of North America classifies '' 2.2.1.2 Overviews of e-commerce Statistical survey in U.S.A. U.S.A. is one of the countries that e-commerce and e-commerce Statistical survey launched earliest in the world . The U.S.A. general survey bureau is the principal organ responsible for e-commerce Statistical survey . The annually Statistical survey adopted by U.S.A. general survey bureau is consisted of annual sample investigation of commerce , annual sample investigation of manufacturing industry , annual sample investigation of retailing business and annual sample investigation of service trade . The method taken in these investigations is dividing layer and sampling . The concrete method in e-commerce Statistical survey is joining the questions of e-commerce into the existing questionnaire except the annually sample investigation of manufacturing industry which is joining the supplementary questionnaire . These respondents investigating are enterprises and the enterprise e-commerce activity , business procedure and sales amount are investigated on the foundation of existing investigation . 2.2.1.3 Overview of e-commerce Statistical survey in other countries 2.2.1.3.1 Overview of e-commerce Statistical survey in Japan In Japan , the departments in chare of e-commerce Statistical survey is Statistics Bureau , Ministry of Internal Affairs and Communication , Japan , but other departments participate in the e-commerce Statistical survey such as Statistics Bureau of Cabinet , Statistics Bureau of Ministry of Economics and Industry , etc. . So , there are more than forty kinds of official investigation on e-commerce which involve every aspect of e-commerce but have great differences in purpose , frequency and content . These investigations launch around three departments including enterprises , governments and families . 2.2.1.3.2 Overview of e-commerce Statistical Survey in S.Korean Statistics bureau of the S.Korean began the official e-commerce Statistical survey since April of 2000 . The investigation mainly concentrates on B2C -LRB- business to consumers -RRB- and B2B -LRB- business to business -RRB- . The investigation on B2G -LRB- business to government -RRB- lags behind slightly , which began since the first quarter of 2001 . 2.2.2 Characteristics of the e-commerce Statistical Survey in each country . a -RRB- The organizers carrying on e-commerce Statistical survey in the above-mentioned countries are all official departments , or implemented by cooperating with other relevant government departments -LRB- as Japan -RRB- . The Statistical survey presided over by the government can not only strengthen its Fairness and dependability , but also give the survey authoritativeness . b -RRB- The investigations almost are not specially but supplementary . The main reasons are high cost of special investigations and not perfect e-commerce Statistical systems of each country which have not reach the level of special investigation . c -RRB- The above-mentioned countries confirm the content of investigation not only consulting the content that OECD recommends , but also considering the development level and characteristics of the national e-commerce . It is worth pointing out those indexes of Statistical survey in Singapore , Canada and U.S.A. are comprehensive and have involved the preferential investigation content that OECD recommends . d -RRB- Most investigations take the annually survey as the core , but there also are monthly , quarter , general survey and irregular surveys . The industries included in monthly and quarterly investigation are not more than on generally , such as `` monthly trade sample investigation of retail business '' in U.S.A. and `` the investigation on family consumption trend '' , etc. e -RRB- Most countries adopt the sample investigations , but other method as census and census combine with sample investigation are also adopted . The method of sampling is mainly used and the following two kinds are used less . The e-commerce in our country is still in the elementary stage and the e-commerce Statistical survey is just start too . There are just some semi-official or unofficial departments and organization trying to carry on e-commerce Statistical survey but not a formal , overall , official survey on e-commerce in our country . For instance : `` Statistical Reports on the Internet Development in China '' , `` CII research and calculating on e-commerce total index system in China '' , `` Statistical survey on intranet and e-commerce development level '' , `` investigation on e-commerce developing in enterprises '' , etc. . Most of above mentioned investigations are irregular , even once only , lack unified consideration and ca n't form a system except `` Statistical Reports on the Internet Development in China '' which hold regularly and establishes its own system to a certain 71 extent . Meanwhile , the unofficial survey is very apt to the systemic deviation and utility nature for it is not mandatory ; even affect the fairness , accuracy and representativeness of the investigation result . 3.2 Implications of e-commerce Statistical survey in China According to the experience of some foreign countries that carrying on e-commerce Statistical and the development of e-commerce Statistical in our country , we consider that if we want to set up a comparatively perfect e-commerce Statistical survey system , we should accomplish the following several points at least : 3.2.1 Attach importance to the definition of e-commerce The kind , range and respondents are all fixed according to the definition of e-commerce which is prerequisite of e-commerce Statistical survey . There is not an authoritative definition of e-commerce in our country so that the key problem we met is how to define e-commerce when we carrying on the e-commerce Statistical survey . We consider that open principle should be followed when defining e-commerce according to its characteristic of appearing late and excessive growth , in order to perfect it constantly with the development of e-commerce . 3.2.2 The government should take charge of e-commerce Statistical survey We could understand the development of e-commerce prompt and accurate , find the questions existing in e-commerce and predict the development trend according to the e-commerce Statistical . It is obvious that e-commerce Statistical is important to the sound development of e-commerce . E-commerce could be promoted by just and accurate Statistical survey but the unilateral and utilitarian Statistical survey will mislead even hamper it . However , the e-commerce Statistical survey of our country lacks the authoritativeness and mandatory at present even affected the fairness and accuracy of Statistical survey . So , the Statistical survey of e-commerce in our country should be included in the official Statistical development plan as early as possible and we should set up the official survey system of e-commerce in order to make the e-commerce Statistical survey authoritative and promote the development of it . 3.2.3 Accelerate the research of the e-commerce Statistical theory The problem which should be considered first in research on Statistical theory of e-commerce is to keep the continuity with traditional Statistical . The e-commerce Statistical is not produced without foundation after all but is the extension on the network of traditional Statistical so that the basic theories of traditional Statistical are still suitable for the e-commerce Statistical survey . Secondly , we should make further research on Statistical method , Statistical caliber and Statistical range of e-commerce , and then set up the index system of e-commerce Statistical as soon as possible . Moreover , e-commerce has the characteristics of crossing over the limit of region . We should try our best to keep the harmony with the world on research in e-commerce Statistical theories for the overall and perfect system of e-commerce Statistical need the joint efforts of countries all over the world . 3.2.4 The service of e-commerce Statistical survey should be comprehensively and pointed . The e-commerce Statistical survey should serve not only for the macroscopically strategic policies of countries but also for the micro operation of enterprises . Meanwhile , there should be different surveys to conform to the different respondents in order to offer the personalized service of the Statistical survey . Only in that way can we offer the good development environment for e-commerce and reflect the value of e-commerce Statistical survey . -LSB- 1 -RSB- . Seminar of `` research on e-commerce Statistical survey and application '' . `` Statistical Surveys and Application of E-Commerce in Canada '' -LSB- J -RSB- . China Statistics , 2003 , 3 2003 , 4 -LSB- 2 -RSB- . Seminar of `` research on e-commerce Statistical survey and application '' . `` Survey of e-commerce development in S.Korean '' . -LSB- J -RSB- . China Statistics , 2003 , 5 -LSB- 3 -RSB- . Seminar of `` research on e-commerce Statistical survey and application '' . `` Statistical Surveys and Application of E-Commerce in Japan '' -LSB- J -RSB- . China Statistics , 2003 , 6 -LSB- 4 -RSB- . Seminar of `` research on e-commerce Statistical survey and application '' . `` Statistical Surveys and Application of E-Commerce in the U.S.A. '' -LSB- J -RSB- . China Statistics , 2003 , 7 -LSB- 5 -RSB- . `` Research paper of e-commerce development all over the world '' , translated by Juanying Zhu , Bingzhi Yang United Nations Trade and Development Board -LSB- M -RSB- .2003 . -LSB- 6 -RSB- . `` The application of IT in Statisticals '' Feng Cui , -LSB- M -RSB- Lixin Accounting Publishing House ,2003 72

O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O B O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O B O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

e-commerce statistics
statistical methods
statistical survey
development stage
authority
measurement
definition of e-commerce
e-commerce