Autonomous

and

Distributed

Node

Recovery

in

Wireless

Sensor

Networks

Intrusion

or

misbehaviour

detection

systems

are

an

important

and

widely

accepted

security

tool

in

computer

and

wireless

sensor

networks

Their

aim

is

to

detect

misbehaving

or

faulty

nodes

in

order

to

take

appropriate

countermeasures

thus

limiting

the

damage

caused

by

adversaries

as

well

as

by

hard

or

software

faults

So

far

however

once

detected

misbehaving

nodes

have

just

been

isolated

from

the

rest

of

the

sensor

network

and

hence

are

no

longer

usable

by

running

applications

In

the

presence

of

an

adversary

or

software

faults

this

proceeding

will

inevitably

lead

to

an

early

and

complete

loss

of

the

whole

network

For

this

reason

we

propose

to

no

longer

expel

misbehaving

nodes

but

to

recover

them

into

normal

operation

In

this

paper

we

address

this

problem

and

present

a

formal

specification

of

what

is

considered

a

secure

and

correct

node

recovery

algorithm

together

with

a

distributed

algorithm

that

meets

these

properties

We

discuss

its

requirements

on

the

soft

and

hardware

of

a

node

and

show

how

they

can

be

fulfilled

with

current

and

upcoming

technologies

The

algorithm

is

evaluated

analytically

as

well

as

by

means

of

extensive

simulations

and

the

findings

are

compared

to

the

outcome

of

a

real

implementation

for

the

BTnode

sensor

platform

The

results

show

that

recovering

sensor

nodes

is

an

expensive

though

feasible

and

worthwhile

task

Moreover

the

proposed

program

code

update

algorithm

is

not

only

secure

but

also

fair

and

robust

C.

-LSB-

Computer-communication

networks

-RSB-

General-Security

and

protection

Algorithms

Reliability

Security

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

to

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

SASN

October

Alexandria

Virginia

USA

Copyright

ACM

Wireless

sensor

networks

-LRB-

WSNs

-RRB-

consist

of

many

wireless

communicating

sensor

nodes

Essentially

these

are

mi-crocontrollers

including

a

communication

unit

and

a

power

supply

as

well

as

several

attached

sensors

to

examine

the

environment

Sensor

nodes

typically

have

very

limited

computing

and

storage

capacities

and

can

only

communicate

with

their

direct

neighbourhood

In

addition

WSNs

have

to

work

unattended

most

of

the

time

as

their

operation

area

can

not

or

must

not

be

visited

Reasons

can

be

that

the

area

is

inhospitable

unwieldy

or

ecologically

too

sensitive

for

human

visitation

or

that

manual

maintenance

would

just

be

too

expensive

More

and

more

WSN

applications

are

supposed

to

operate

in

hostile

environments

where

their

communication

might

be

overheard

and

nodes

can

be

removed

or

manipu-lated

Regarding

attacks

on

sensor

networks

one

differentiates

between

so

called

outsider

and

insider

attacks

-LSB-

-RSB-

In

the

former

a

potential

attacker

tries

to

disclose

or

influence

a

confidential

outcome

without

participating

in

its

computation

for

instance

by

intercepting

modifying

or

adding

messages

In

the

latter

by

contrast

an

attacker

appears

as

an

adequate

member

of

the

WSN

by

either

plausibly

impersonating

regular

nodes

or

by

capturing

and

compromising

them

Cryptographic

methods

such

as

encrypting

or

signing

messages

are

an

effective

protection

against

attacks

from

outside

the

network

but

are

of

only

limited

help

against

insider

attacks

Once

an

adversary

possesses

one

or

several

valid

node

identities

-LRB-

including

the

associated

keys

-RRB-

it

can

actively

participate

in

the

operations

of

the

WSN

and

influence

the

computed

results

Intrusion

or

misbehaviour

detection

systems

-LRB-

IDS

-RRB-

on

the

other

hand

are

an

important

and

widely

accepted

security

tool

against

insider

attacks

-LSB-

-RSB-

They

allow

for

the

detection

of

malicious

or

failed

nodes

and

the

application

of

appropriate

countermeasures

So

far

however

once

detected

misbehaving

nodes

have

just

been

isolated

from

the

rest

of

the

network

and

hence

are

no

longer

usable

by

running

applications

In

the

presence

of

an

adversary

or

software

faults

this

proceding

will

inevitably

result

in

an

early

and

complete

loss

of

the

whole

network

Therefore

not

only

the

detection

of

misbehaving

nodes

is

important

but

also

the

selection

and

application

of

effective

countermeasures

Their

aim

must

not

be

to

simply

expel

suspected

nodes

but

to

recover

them

into

correct

operation

In

combination

the

advantages

of

an

IDS

together

with

the

appropriate

ery

measures

are

manifold

Not

only

do

they

help

in

case

of

program

faults

-LRB-

e.g.

deadlocks

or

crashes

-RRB-

but

even

if

an

attacker

manages

to

capture

a

node

and

to

abuse

it

for

his

own

purposes

there

is

a

chance

that

the

aberrant

behaviour

of

this

node

will

be

detected

and

the

node

be

recovered

thus

nullifying

the

attack

However

due

to

the

size

of

sensor

networks

both

the

IDS

functionality

as

well

as

the

recovery

measures

should

be

autonomously

executed

by

the

involved

nodes

in

a

distributed

and

cooperative

manner

and

without

the

need

for

central

instances

with

extended

functionality

Motivated

by

the

above

mentioned

insights

this

paper

focuses

on

autonomous

an

distributed

node

recovery

in

wireless

sensor

networks

and

proposes

three

alternative

countermeasures

to

node

expelling

namely

to

switch

a

node

off

to

restart

it

and

to

update

its

program

code

We

formally

specify

what

we

consider

a

secure

and

correct

recovery

algorithm

present

a

distributed

algorithm

which

meets

these

properties

and

reason

why

it

can

help

to

extend

the

overall

lifetime

of

a

sensor

network

In

addition

we

discuss

the

limitations

of

the

proposed

countermeasures

show

which

hard-and

software

parts

of

a

corrupted

node

must

still

work

correctly

to

make

them

applicable

and

explain

how

this

can

be

achieved

with

current

and

upcoming

technologies

More

precisely

the

contributions

of

this

paper

are

as

follows

We

propose

to

no

longer

expel

misbehaving

nodes

but

to

either

-LRB-

i

-RRB-

switch

them

off

-LRB-

ii

-RRB-

restart

them

or

-LRB-

iii

-RRB-

update

their

program

code

We

give

a

formal

specification

of

a

secure

and

correct

recovery

algorithm

We

present

a

provably

secure

and

robust

distributed

node

recovery

algorithm

We

discuss

the

requirements

on

the

soft

and

hardware

of

a

node

in

order

to

make

the

countermeasures

applicable

and

show

how

they

can

be

fulfilled

with

current

and

upcoming

technologies

The

algorithm

is

evaluated

analytically

as

well

as

by

means

of

extensive

simulations

and

the

findings

are

compared

to

the

outcome

of

a

real

implementation

for

the

BTnode

sensor

platform

The

results

show

that

recovering

sensor

nodes

is

an

expensive

though

feasible

and

worthwhile

task

Moreover

the

proposed

program

code

update

algorithm

is

not

only

provably

secure

but

also

fair

and

robust

It

distributes

the

update

load

equally

over

all

participating

nodes

and

terminates

as

long

as

at

least

one

of

them

remains

correct

The

rest

of

this

paper

is

organised

as

follows

Section

presents

the

related

work

in

the

area

of

intrusion

detection

and

node

recovery

in

wireless

sensor

networks

Section

states

the

required

definitions

and

assumptions

Section

specifies

the

proposed

recovery

algorithm

whose

correctness

is

proven

in

section

The

algorithm

is

evaluated

in

section

and

section

concludes

the

paper

Related

Work

In

this

section

we

present

related

work

in

the

area

of

intrusion

detection

in

wireless

sensor

networks

Additionally

related

work

regarding

program

code

updates

in

sensor

networks

is

also

discussed

as

we

propose

program

code

updates

as

a

mean

to

recover

nodes

Intrusion

Detection

In

recent

years

intrusion

detection

systems

for

wireless

sensor

networks

have

become

a

major

research

issue

and

several

approaches

have

been

proposed

However

to

our

best

knowledge

the

only

countermeasure

applied

so

far

was

to

-LRB-

logically

-RRB-

exclude

malicious

nodes

Khalil

Bagchi

and

Nina-Rotaru

present

a

distributed

IDS

where

nodes

monitor

the

communication

among

their

neighbours

-LSB-

-RSB-

For

each

monitored

node

a

malignity

counter

is

maintained

and

incremented

whenever

the

designated

node

misbehaves

Once

a

counter

exceeds

a

predefined

threshold

an

according

alert

is

sent

to

all

neighbours

and

if

enough

alerts

are

received

the

accused

node

is

revoked

from

the

neighbourhood

list

Hsin

and

Liu

suggest

a

two-phase

timeout

system

for

neighbour

monitoring

which

uses

active

probing

to

reduce

the

probability

of

false-positives

-LSB-

-RSB-

A

rule-based

IDS

which

proceeds

in

three

phases

is

proposed

by

da

Silva

et

al.

-LSB-

-RSB-

In

the

first

phase

messages

are

overheard

and

the

collected

information

is

filtered

and

ordered

In

the

second

phase

the

detection

rules

are

applied

to

the

gathered

data

and

each

inconsistency

counted

as

a

failure

Finally

in

the

third

phase

the

number

or

failures

is

compared

to

the

expected

amount

of

occasional

failures

and

if

too

high

an

intrusion

alert

is

raised

Inverardi

Mostarda

and

Navarra

introduce

a

framework

which

enables

the

automatic

translation

of

IDS

specifications

into

program

code

-LSB-

-RSB-

The

so

generated

code

is

then

installed

on

the

sensor

nodes

in

order

to

locally

detect

violations

of

the

node

interaction

policies

In

the

approach

by

Herbert

et

al.

-LSB-

-RSB-

predefined

correctness

properties

-LRB-

invariants

-RRB-

are

associated

with

conditions

of

individual

nodes

or

the

whole

network

and

program

code

to

verify

these

invariants

is

automatically

inserted

during

compilation

A

reputation-based

IDS

framework

for

WSNs

where

sensor

nodes

maintain

a

reputation

for

other

nodes

is

presented

by

Ganeriwal

and

Srivastava

-LSB-

-RSB-

It

uses

a

Bayesian

formula-tion

for

reputation

representation

update

and

integration

Program

Code

Update

The

main

difference

between

the

already

available

reprogramming

algorithms

and

the

proposed

recovery

measures

are

that

the

former

focus

on

the

propagation

of

new

program

releases

among

all

nodes

of

the

network

whereas

the

aim

of

the

latter

is

the

local

and

autonomous

update

of

a

single

node

Furthermore

most

reprogramming

mechanisms

do

not

care

about

security

at

all

or

rely

on

expensive

public

key

cryptography

Kulkarni

and

Wang

propose

a

multihop

reprogramming

service

for

wireless

sensor

networks

which

uses

a

sender

selection

algorithm

to

avoid

collisions

-LSB-

-RSB-

Impala

a

middleware

system

for

managing

sensor

systems

is

presented

by

Liu

and

Martonosi

-LSB-

-RSB-

Its

modular

architecture

supports

updates

to

the

running

system

An

application

consists

of

several

modules

which

are

independently

transferred

an

update

is

complete

if

all

its

modules

have

been

received

Jeong

and

Culler

introduce

an

efficient

incremental

network

programming

mechanism

-LSB-

-RSB-

Thanks

to

the

usage

of

the

Rsync

algorithm

only

incremental

changes

to

the

new

program

must

be

transferred

A

secure

dissemination

algorithm

to

distribute

new

program

releases

among

nodes

is

presented

by

Dutta

et

al.

-LSB-

-RSB-

Program

binaries

are

propagated

as

a

sequence

of

data

blocks

of

which

the

first

is

authenticated

with

the

private

key

of

the

base

station

and

the

subsequent

ones

by

means

of

a

hash

chain

In

order

to

improve

the

fault

tolerance

of

the

sensor

network

nodes

use

a

grenade

timer

to

reboot

period-ically

During

the

boot

process

neighboring

nodes

are

asked

whether

a

new

program

release

is

available

and

if

so

its

download

is

initiated

In

this

section

we

define

our

assumptions

regarding

the

observation

of

nodes

and

the

network

communication

model

We

specify

the

capabilities

of

a

potential

adversary

explain

what

we

consider

a

correct

recover

algorithm

and

discuss

the

requirements

on

the

hard

and

software

of

a

sensor

node

Intrusion

and

Misbehaviour

Detection

Troughout

this

paper

we

assume

that

the

network

is

divided

into

N

C

so

called

observation

clusters

C

i

-LRB-

V

i

E

i

-RRB-

i

N

C

of

size

n

n

V

i

Within

a

cluster

each

node

is

connected

to

each

other

-LRB-

v

i

v

j

V

k

v

i

v

j

-LCB-

v

i

v

j

-RCB-

E

k

-RRB-

and

observes

the

behaviour

of

its

cluster

neighbours

For

the

actual

monitoring

of

the

neighbours

an

arbitrary

IDS

can

be

used

as

long

as

each

node

ends

up

with

an

-LRB-

individual

-RRB-

decision

about

whether

a

certain

node

behaves

correct

or

malicious

The

set

of

malicious

nodes

in

a

cluster

is

denoted

by

M

i

and

their

number

by

t

t

M

i

n.

Network

Model

In

the

following

p

s

-LRB-

p

r

-RRB-

denotes

the

probability

that

the

sending

-LRB-

receiving

-RRB-

of

a

message

fails

Thus

for

p

s

p

l

the

resulting

probability

for

an

unsuccessful

transmission

-LRB-

packet

loss

ratio

PLR

-RRB-

is

p

l

-LRB-

p

s

-RRB-

-LRB-

p

r

-RRB-

p

s

p

r

p

s

p

r

Additionally

we

assume

that

there

exists

a

constant

upper

bound

p

O

-LRB-

-RRB-

on

the

transmission

time

of

a

message

Adversary

Model

We

consider

an

omnipresent

but

computationally

bounded

adversary

who

can

perform

both

outsider

as

well

as

insider

attacks

This

means

that

a

potential

adversary

is

able

to

intercept

and

create

arbitrary

messages

but

unable

to

decrypt

or

authenticate

messages

for

which

he

does

not

possess

the

required

keys

We

further

assume

that

nodes

can

be

either

logically

-LRB-

i.e.

by

exploiting

a

software

bug

-RRB-

or

physically

captured

However

the

time

to

compromise

a

node

physically

is

considered

non-negligible

-LRB-

i.e.

it

takes

some

time

to

move

from

node

to

node

and

to

perform

the

physical

manipulations

-RRB-

and

to

not

significantly

decrease

with

the

number

of

already

captured

nodes

Hard

and

Software

Requirements

To

all

presented

recovery

measures

applies

that

they

are

only

applicable

if

at

least

the

therefore

needed

systems

of

the

corrupt

node

in

the

following

denoted

as

the

recovery

system

still

work

correctly

In

order

to

achieve

this

one

has

to

make

sure

that

the

recovery

system

is

logically

and

if

feasible

physically

protected

Logical

Protection

of

the

Recovery

System

Logical

protection

means

that

it

should

not

be

possible

for

a

running

application

to

prevent

the

execution

of

the

recovery

procedures

That

is

if

the

program

code

running

on

a

node

has

crashed

or

been

corrupted

by

an

adversary

-LRB-

e.g.

by

exploiting

a

security

hole

-RRB-

this

should

not

affect

the

integrity

and

availability

of

the

recovery

system

One

mechanism

to

achieve

this

is

to

set

up

a

hardware

interrupt

which

can

not

be

suppressed

or

redirected

by

the

application

and

by

locating

the

dedicated

interrupt

routine

in

a

write

protected

memory

area

Consequently

on

each

interrupt

request

control

is

handed

over

to

the

immutable

interrupt

routine

an

thus

to

the

recovery

system

A

simple

variant

of

this

mechanism

in

which

a

grenade

timer

period-ically

reboots

the

system

and

the

bootloader

is

located

in

read

only

memory

-LRB-

ROM

-RRB-

is

used

by

Dutta

et

al.

-LSB-

-RSB-

Another

approach

would

be

to

misuse

some

additionally

available

MCUs

-LSB-

-RSB-

for

example

the

ARM

CPU

on

the

ARM-based

Bluetooth

module

on

the

BTnode

Some

of

these

MCUs

are

powerful

enough

to

take

on

additional

tasks

like

monitoring

the

main

MCUs

activities

or

rewriting

the

application

memory

In

case

of

the

BTnode

that

extra

MCU

is

directly

responsible

for

communication

and

thus

it

would

be

guaranteed

that

it

has

access

to

all

received

packets

as

well

On

more

advanced

systems

mechanisms

as

provided

by

Intel

's

protected

mode

-LRB-

e.g.

isolated

memory

areas

privilege

levels

etc.

-RRB-

could

be

used

to

protect

the

recovery

system

more

efficiently

Current

technologies

such

as

ARM

's

TrustZone

-LSB-

-RSB-

for

embedded

devices

or

Intel

's

LaGrande

technology

-LSB-

-RSB-

go

even

further

and

enable

a

comprehensive

protection

of

the

CPU

memory

and

peripherals

from

software

attacks

Physical

Protection

of

the

Recovery

System

The

physical

protection

of

current

sensor

node

platforms

is

very

poor

because

of

their

focus

on

simple

maintenance

-LSB-

-RSB-

However

although

it

is

generally

agreed

that

entirely

tamper-proof

sensor

nodes

would

be

too

expensive

current

trends

in

the

hardware

development

of

embedded

devices

indicate

that

some

level

of

physical

protection

will

be

available

in

the

near

future

-LSB-

-RSB-

Security

mechanisms

regarding

the

packaging

of

sensor

nodes

as

for

instance

those

proposed

by

FIPS

level

-LSB-

-RSB-

could

already

significantly

increase

the

cost

for

an

adversary

For

integrity

and

not

confidentiality

is

the

main

concern

with

the

recovery

module

it

has

only

to

be

protected

against

manipulations

but

not

against

unintended

disclosure

or

side-channel

attacks

In

fact

it

would

be

sufficient

to

have

mechanisms

which

render

a

node

useless

if

the

case

of

the

recovery

system

was

opened

complete

tamper

resistance

is

not

required

Correct

Node

Recover

Algorithms

A

node

recovery

algorithm

for

a

cluster

C

i

-LRB-

V

i

E

i

-RRB-

is

considered

correct

if

the

following

liveness

and

safety

properties

hold

If

all

correct

nodes

-LRB-

V

i

M

i

-RRB-

accuse

a

node

m

V

i

to

be

faulty

or

malicious

its

recovery

process

will

finally

be

initiated

with

high

probability

Once

the

recovery

process

for

a

node

m

V

i

has

been

initiated

it

will

eventually

terminate

as

long

as

there

remain

at

least

k

correct

nodes

V

i

M

i

If

no

more

than

correct

nodes

-LRB-

i.e.

a

minority

-RRB-

accuse

another

correct

node

v

V

i

M

i

the

recovery

process

will

not

be

initiated

After

the

recovery

process

a

node

m

V

i

must

either

-LRB-

i

-RRB-

be

halted

-LRB-

ii

-RRB-

contain

the

same

program

code

as

before

or

-LRB-

iii

-RRB-

contain

the

correct

program

code

The

two

liveness

properties

and

ensure

that

each

malicious

node

is

recovered

if

its

aberrant

behaviour

is

detected

by

enough

neighbours

Safety

property

is

required

to

make

sure

that

a

node

is

only

recovered

if

a

majority

of

correct

nodes

accuses

it

and

property

ensures

that

things

are

not

worsened

by

applying

the

recovery

process

In

this

section

we

present

a

distributed

node

recovery

algorithm

which

is

autonomously

executed

within

an

observation

cluster

The

supported

recovery

measures

are

node

shutdown

node

restart

and

program

code

update

As

long

as

the

recovery

module

of

an

otherwise

faulty

or

malicious

node

is

still

intact

it

is

tried

to

recover

it

by

restarting

it

or

updating

its

program

code

or

to

at

least

eliminate

its

interfering

influence

by

turning

it

off

If

a

node

does

not

respond

to

any

of

these

measures

it

is

still

possible

to

logically

expell

it

preferably

by

means

of

a

reliable

majority

decision

-LSB-

-RSB-

to

avoid

inconsistencies

among

the

cluster

members

Description

of

the

Recovery

Procedure

The

proposed

recovery

algorithm

consists

of

two

phases

In

the

first

so

called

accusation

phase

nodes

accuse

all

neighbours

which

are

regarded

as

being

malicious

If

a

node

is

accused

by

at

least

two

third

of

its

neighbours

it

initiates

the

second

so

called

recovery

phase

during

which

the

actual

countermeasures

are

executed

To

simplify

the

cooperative

program

code

update

the

program

memory

of

a

node

is

divided

into

F

frames

f

i

i

F

of

size

f

s.

Additionally

for

each

frame

f

i

its

corresponding

hash

value

h

i

h

-LRB-

f

i

-RRB-

is

computed

Accusation

Phase

Recovery

Phase

Round

Round

Round

Figure

Schematic

depiction

of

a

recovery

procedure

which

performs

a

program

update

as

the

countermeasure

Accusation

Phase

Nodes

which

conclude

that

one

of

their

neighbours

behaves

maliciously

send

it

an

authenticated

accusation

mes

sage

The

proposed

countermeasure

depends

on

the

observed

aberration

and

can

be

either

of

type

shutdown

reset

or

update

if

the

node

should

be

halted

restarted

or

its

program

code

updated

respectively

Accusation

messages

have

to

be

ac-knowledged

and

are

resent

up

to

r

times

otherwise

For

simplicity

it

is

assumed

that

nodes

can

accuse

their

neighbours

at

any

time

However

if

the

recovery

module

is

only

active

from

time

to

time

nodes

could

of

course

also

actively

ask

for

-LRB-

pending

-RRB-

accusations

In

case

that

a

program

update

is

requested

the

accusation

messages

also

include

a

list

of

the

sender

's

F

frame

hash

values

h

i

They

represent

the

current

state

of

its

program

memory

and

are

required

to

deduce

the

correct

program

code

Therefore

for

each

frame

f

i

not

only

its

hash

value

h

i

but

also

a

counter

c

i

which

is

initialised

with

zero

is

stored

Upon

reception

of

a

accusation

message

each

included

hash

value

is

compared

to

the

already

stored

one

and

if

they

are

equal

c

i

is

incremented

by

one

If

they

differ

and

c

i

the

counter

is

decremented

by

one

otherwise

-LRB-

i.e.

they

are

not

equal

and

c

i

-RRB-

the

stored

hash

value

is

replaced

with

the

received

value

This

procedure

ensures

that

for

n

every

h

i

will

contain

the

hash

value

of

the

correct

program

code

frame

after

-LRB-

-RRB-

accusations

have

been

received

-LRB-

see

Proof

-RRB-

Recovery

Phase

When

a

node

m

has

received

-LRB-

-RRB-

accusations

of

a

certain

recovery

type

the

corresponding

measure

is

initiated

In

the

non

trivial

case

of

a

distributed

program

code

update

the

correct

program

code

has

therefore

to

be

down-loaded

from

the

neighboring

nodes

Otherwise

the

node

is

just

rebooted

or

shutdown

and

no

further

communication

or

coordination

is

required

The

autonomous

program

code

transfer

is

performed

in

rounds

of

which

each

starts

with

the

broadcasting

of

an

authenticated

update

request

message

by

the

accused

node

m.

Essentially

the

message

contains

a

list

of

so

called

frame

descriptors

-LRB-

u

i

Q

i

-RRB-

consisting

of

a

node

id

u

i

and

a

set

of

requested

frame

numbers

Q

i

-LCB-

r

r

r

Q

-RCB-

Upon

reception

of

a

valid

request

a

node

v

seeks

for

descriptors

which

contain

its

own

id

-LRB-

i.e.

u

i

v

-RRB-

If

present

for

each

requested

frame

number

r

j

Q

i

the

corresponding

program

code

frame

is

sent

back

to

m

with

an

update

message

All

received

program

code

fragments

f

i

in

turn

are

verified

by

m

using

the

stored

hash

values

h

i

Valid

code

fragments

are

copied

into

the

program

memory

and

the

frame

marked

as

updated

If

for

a

duration

of

round

no

update

messages

arrive

although

there

are

still

some

outstanding

frames

a

new

update

request

message

is

broadcasted

and

the

next

round

initiated

As

soon

as

all

frames

have

been

received

the

node

is

rebooted

and

thus

the

new

program

code

activated

In

order

to

distribute

the

transfer

load

equally

among

all

participating

nodes

and

to

ensure

that

the

update

procedure

terminates

if

at

least

one

correct

node

is

available

the

frame

descriptors

are

determined

as

follows

First

the

n

participating

nodes

are

ordered

such

that

id

-LRB-

v

-RRB-

id

-LRB-

v

-RRB-

id

-LRB-

v

-RRB-

Next

the

F

memory

frames

are

divided

into

n

sectors

of

length

l

F

Finally

to

each

node

one

such

fragment

is

assigned

per

update

round

in

a

round

robin

fashion

Thus

in

round

i

node

v

j

is

responsible

for

the

segment

s

j

i

mod

-LRB-

n

-RRB-

that

is

for

the

frames

sl

to

min

-LRB-

-LRB-

s

-RRB-

l

F

-RRB-

In

the

first

round

for

example

the

first

node

is

responsible

for

the

first

l

frames

the

second

node

for

the

second

l

frames

and

so

on

In

the

second

round

however

the

assignment

is

rotated

by

one

and

thus

the

outstanding

frames

of

the

first

sector

are

now

requested

from

the

second

node

This

process

has

to

be

continued

until

all

required

frames

have

been

received

On

most

sensor

node

platforms

new

code

is

not

directly

written

into

program

memory

but

into

a

therefore

available

Flash

memory

and

installed

during

a

subsequent

reboot

Extensions

and

Optimisations

Even

though

not

all

but

only

the

subset

of

modified

program

code

frames

has

to

be

requested

updating

a

node

is

still

a

time

consuming

and

expensive

task

Consequently

the

amount

of

update

load

that

a

specific

node

can

cause

should

be

restricted

for

instance

by

limiting

the

number

of

update

messages

that

are

sent

to

it

To

further

reduce

the

load

for

the

participating

nodes

the

F

hash

values

h

i

in

an

accusation

message

can

be

replaced

by

the

hash

value

h

h

-LRB-

h

h

h

F

-RRB-

Once

the

correct

value

h

has

been

determinded

using

the

corresponding

counter

c

in

analogy

to

the

above

mentioned

algorithm

the

actual

hash

values

can

be

requested

from

the

neighbours

in

a

second

step

and

verified

with

h

In

order

to

decrease

the

total

number

of

required

accusation

messages

more

than

one

recovery

measure

per

message

should

be

allowed

Alternatively

the

measures

could

be

hierarchically

organised

having

the

type

update

also

counting

as

a

reboot

or

shutdown

request

Algorithms

Listing

Algorithm

for

an

accusing

node

v.

v

a

r

a

c

c

r

e

t

r

i

e

s

-LSB-

n

-RSB-

-LCB-

-RCB-

a

c

c

f

a

i

l

e

d

-LSB-

n

-RSB-

-LCB-

false

false

-RCB-

n

u

m

u

p

d

a

t

e

s

-LSB-

n

-RSB-

-LCB-

-RCB-

upon

m

i

s

b

e

h

a

v

i

o

r

d

e

t

e

c

t

i

o

n

o

f

n

o

d

e

m

c

h

o

o

s

e

an

a

p

p

r

o

p

r

i

a

t

e

a

c

c

u

s

a

t

i

o

n

t

y

p

e

a

m

i

f

a

m

acc

update

s

e

n

d

a

c

c

u

s

a

t

i

o

n

v

m

a

m

-LCB-

h

-LRB-

f

-RRB-

h

-LRB-

f

F

-RRB-

-RCB-

t

o

m

e

l

s

e

s

e

n

d

a

c

c

u

s

a

t

i

o

n

v

m

a

m

t

o

m

s

t

a

r

t

t

i

m

e

r

A

m

upon

r

e

c

e

p

t

i

o

n

o

f

a

c

c

u

s

a

t

i

o

n

a

c

k

m

a

f

r

o

m

m

s

t

o

p

t

i

m

e

r

A

m

a

c

c

r

e

t

r

i

e

s

-LSB-

m

-RSB-

upon

t

i

m

e

o

u

t

o

f

t

i

m

e

r

A

m

i

f

a

c

c

r

e

t

r

i

e

s

-LSB-

m

-RSB-

max

acc

retries

a

c

c

r

e

t

r

i

e

s

-LSB-

m

-RSB-

a

c

c

r

e

t

r

i

e

s

-LSB-

m

-RSB-

s

e

n

d

a

c

c

u

s

a

t

i

o

n

v

m

a

m

-LCB-

h

-LRB-

f

-RRB-

h

-LRB-

f

F

-RRB-

-RCB-

t

o

m

s

t

a

r

t

t

i

m

e

r

A

m

e

l

s

e

a

c

c

f

a

i

l

e

d

-LSB-

m

-RSB-

true

upon

r

e

c

e

p

t

i

o

n

o

f

u

p

d

a

t

e

r

e

q

u

e

s

t

m

R

f

r

o

m

m

i

f

n

u

m

u

p

d

a

t

e

s

-LSB-

m

-RSB-

m

a

x

u

p

d

a

t

e

s

and

-LRB-

u

-LCB-

r

r

k

-RCB-

-RRB-

R

n

u

m

u

p

d

a

t

e

s

-LSB-

m

-RSB-

n

u

m

u

p

d

a

t

e

s

-LSB-

m

-RSB-

r

i

i

k

s

e

n

d

u

p

d

a

t

e

v

m

r

i

f

r

i

t

o

m

Listing

Algorithm

for

the

accused

node

m.

v

a

r

u

p

d

a

t

i

n

g

false

n

u

m

a

c

c

r

e

s

e

t

n

u

m

a

c

c

u

p

d

a

t

e

n

u

m

a

c

c

s

h

u

t

d

o

w

n

s

t

a

r

t

n

o

d

e

a

c

c

r

e

s

e

t

r

e

c

v

d

-LSB-

n

-RSB-

-LCB-

-RCB-

a

c

c

u

p

d

a

t

e

r

e

c

v

d

-LSB-

n

-RSB-

-LCB-

-RCB-

a

c

c

s

h

u

t

d

o

w

r

e

c

v

d

-LSB-

n

-RSB-

-LCB-

-RCB-

f

r

a

m

e

u

p

d

a

t

e

d

-LSB-

F

-RSB-

-LCB-

false

false

-RCB-

f

r

a

m

e

d

i

g

e

s

t

-LSB-

F

-RSB-

-LCB-

h

-LRB-

f

-RRB-

h

-LRB-

f

F

-RRB-

-RCB-

f

r

a

m

e

c

o

u

n

t

-LSB-

F

-RSB-

-LCB-

-RCB-

upon

r

e

c

e

p

t

i

o

n

o

f

a

c

c

u

s

a

t

i

o

n

v

m

acc

reset

f

r

o

m

v

s

e

n

d

a

c

c

u

s

a

t

i

o

n

a

c

k

m

acc

reset

t

o

v

i

f

n

o

t

u

p

d

a

t

i

n

g

and

n

o

t

a

c

c

r

e

s

e

t

r

e

c

v

d

-LSB-

v

-RSB-

a

c

c

r

e

s

e

t

r

e

c

v

d

-LSB-

v

-RSB-

true

n

u

m

a

c

c

r

e

s

e

t

n

u

m

a

c

c

r

e

s

e

t

i

f

n

o

t

u

p

d

a

t

i

n

g

and

n

u

m

a

c

c

r

e

s

e

t

-LRB-

-RRB-

r

e

s

e

t

n

o

d

e

upon

r

e

c

e

p

t

i

o

n

o

f

a

c

c

u

s

a

t

i

o

n

v

m

acc

shutdown

f

r

o

m

v

s

e

n

d

a

c

c

u

s

a

t

i

o

n

a

c

k

m

acc

shutdown

t

o

v

i

f

n

o

t

u

p

d

a

t

i

n

g

and

n

o

t

a

c

c

s

h

u

t

d

o

w

n

r

e

c

v

d

-LSB-

v

-RSB-

a

c

c

s

h

u

t

d

o

w

n

r

e

c

v

d

-LSB-

v

-RSB-

true

n

u

m

a

c

c

s

h

u

t

d

o

w

n

n

u

m

a

c

c

s

h

u

t

d

o

w

n

i

f

n

o

t

u

p

d

a

t

i

n

g

and

n

u

m

a

c

c

s

h

u

t

d

o

w

n

-LRB-

-RRB-

shutdown

n

o

d

e

f

u

n

c

t

i

o

n

s

e

t

u

p

u

p

d

a

t

e

r

e

q

u

e

s

t

-LRB-

-RRB-

k

R

-LCB-

-RCB-

f

o

r

i

n

i

m

w

-LRB-

s

t

a

r

t

n

o

d

e

i

-RRB-

mod

n

Q

-LCB-

-RCB-

f

o

r

j

F

n

i

f

n

o

t

f

r

a

m

e

u

p

d

a

t

e

d

-LSB-

k

-RSB-

Q

Q

-LCB-

k

-RCB-

k

k

i

f

Q

-LCB-

-RCB-

R

R

-LCB-

-LRB-

w

Q

-RRB-

-RCB-

s

t

a

r

t

n

o

d

e

s

t

a

r

t

n

o

d

e

r

e

t

u

r

n

R

upon

r

e

c

e

p

t

i

o

n

o

f

a

c

c

u

s

a

t

i

o

n

v

m

acc

update

-LCB-

h

h

F

-RCB-

f

r

o

m

v

s

e

n

d

a

c

c

u

s

a

t

i

o

n

a

c

k

m

acc

update

t

o

v

i

f

n

o

t

u

p

d

a

t

i

n

g

and

n

o

t

a

c

c

u

p

d

a

t

e

r

e

c

v

d

-LSB-

v

-RSB-

a

c

c

u

p

d

a

t

e

r

e

c

v

d

-LSB-

v

-RSB-

true

n

u

m

a

c

c

u

p

d

a

t

e

n

u

m

a

c

c

u

p

d

a

t

e

f

o

r

i

F

i

f

f

r

a

m

e

d

i

g

e

s

t

-LSB-

i

-RSB-

h

i

f

r

a

m

e

c

o

u

n

t

-LSB-

i

-RSB-

f

r

a

m

e

c

o

u

n

t

-LSB-

i

-RSB-

e

l

s

e

i

f

f

r

a

m

e

c

o

u

n

t

-LSB-

i

-RSB-

f

r

a

m

e

c

o

u

n

t

-LSB-

i

-RSB-

f

r

a

m

e

c

o

u

n

t

-LSB-

i

-RSB-

e

l

s

e

f

r

a

m

e

d

i

g

e

s

t

-LSB-

i

-RSB-

h

i

i

f

n

o

t

u

p

d

a

t

i

n

g

and

n

u

m

a

c

c

u

p

d

a

t

e

-LRB-

-RRB-

R

s

e

t

u

p

u

p

d

a

t

e

r

e

q

u

e

s

t

-LRB-

-RRB-

b

r

o

a

d

c

a

s

t

u

p

d

a

t

e

r

e

q

u

e

s

t

m

R

s

t

a

r

t

t

i

m

e

r

U

u

p

d

a

t

i

n

g

true

upon

t

i

m

e

o

u

t

o

f

t

i

m

e

r

U

R

s

e

t

u

p

u

p

d

a

t

e

r

e

q

u

e

s

t

-LRB-

-RRB-

b

r

o

a

d

c

a

s

t

u

p

d

a

t

e

r

e

q

u

e

s

t

m

R

s

t

a

r

t

t

i

m

e

r

U

upon

r

e

c

e

p

t

i

o

n

o

f

u

p

d

a

t

e

v

m

i

f

f

r

o

m

v

r

e

s

e

t

t

i

m

e

r

U

i

f

h

-LRB-

f

-RRB-

f

r

a

m

e

d

i

g

e

s

t

-LSB-

i

-RSB-

and

n

o

t

f

r

a

m

e

u

p

d

a

t

e

d

-LSB-

i

-RSB-

u

p

d

a

t

e

memory

f

r

a

m

e

i

f

r

a

m

e

u

p

d

a

t

e

d

-LSB-

i

-RSB-

true

i

f

i

i

F

f

r

a

m

e

u

p

d

a

t

e

d

-LSB-

i

-RSB-

r

e

s

e

t

n

o

d

e

In

this

section

we

proof

the

correctness

of

the

proposed

algorithm

with

respect

to

the

specifications

of

section

Theorem

Given

the

network

and

adversary

model

specified

in

section

the

proposed

recovery

algorithm

is

correct

and

fulfils

the

properties

and

if

the

recovery

module

of

the

accused

node

m

is

intact

if

h

-LRB-

-RRB-

is

a

secure

hashfunction

and

if

less

than

one

third

of

the

participating

nodes

are

malicious

-LRB-

i.e.

n

-RRB-

In

order

to

prove

Theorem

we

have

to

show

that

the

properties

and

hold

We

therefore

first

prove

some

helper

Lemmas

Lemma

If

all

correct

nodes

accuse

a

node

m

its

recovery

process

will

be

initiated

with

high

probability

Proof

The

probability

that

less

than

-LRB-

-RRB-

accusations

are

received

is

equal

to

the

probability

that

more

than

messages

are

either

not

sent

or

lost

Assuming

that

the

t

malicious

nodes

do

not

participate

in

the

distributed

update

at

least

t

accusations

must

get

lost

Given

p

l

the

probability

for

this

is

-LRB-

p

r

l

-RRB-

t

-LRB-

p

r

l

-RRB-

t

-LRB-

p

r

l

-RRB-

-LRB-

-RRB-

t

-LRB-

p

r

l

-RRB-

t

It

holds

that

c

r

such

that

-LRB-

-RRB-

t

p

t

l

r

n

c

Thus

the

node

m

gets

-LRB-

-RRB-

accusations

w.h.p.

and

the

recovery

process

is

initiated

Lemma

Once

the

recovery

process

for

a

node

m

has

been

initiated

it

will

eventually

terminate

as

long

as

there

remain

at

least

k

correct

nodes

Proof

In

order

that

a

frame

is

updated

in

a

specific

round

the

dedicated

request

as

well

as

its

actual

transmission

must

succeed

The

probability

that

this

is

the

case

is

-LRB-

p

l

-RRB-

With

only

one

correct

node

-LRB-

k

-RRB-

the

expected

number

of

update

rounds

per

frame

a

can

be

described

as

a

Markov

chain

described

by

the

expression

a

-LRB-

a

-RRB-

-LRB-

-LRB-

p

l

-RRB-

-RRB-

-LRB-

p

l

-RRB-

with

the

solution

a

-LRB-

l

-RRB-

The

overall

expected

number

of

rounds

is

thus

aF

F

-LRB-

l

-RRB-

O

-LRB-

-RRB-

In

each

round

at

most

one

request

and

F

updates

are

transmitted

leading

to

an

upper

bound

for

its

duration

of

-LRB-

F

-RRB-

p

O

-LRB-

-RRB-

Altogether

the

expected

worst

case

duration

is

F

-LRB-

F

-RRB-

-LRB-

l

-RRB-

p

O

-LRB-

-RRB-

Lemma

If

no

more

than

correct

nodes

accuse

another

correct

node

v

M

the

recovery

process

will

not

be

initiated

Proof

From

each

node

only

one

accusation

is

accepted

and

thus

the

number

of

valid

accusations

is

at

most

t

-LRB-

-RRB-

Lemma

At

the

start

of

a

program

code

update

the

target

node

m

has

stored

the

correct

hash

value

h

i

for

all

frames

given

that

all

correct

nodes

have

loaded

the

same

program

code

Proof

Let

's

assume

that

there

is

a

hash

value

h

i

which

is

not

correct

when

the

program

code

update

starts

As

a

stored

hash

value

is

only

substituted

if

the

dedicated

counter

c

i

is

zero

the

node

must

have

received

at

least

as

many

wrong

values

as

correct

ones

From

each

node

only

one

accusation

is

accepted

thus

of

the

a

-LRB-

-RRB-

received

values

at

most

t

are

false

It

follows

that

at

least

a

t

-LRB-

-RRB-

n

t

values

must

be

correct

which

contradicts

the

assumption

that

at

least

as

many

false

as

correct

hash

values

were

received

The

properties

and

are

proven

by

Lemma

and

respectively

If

the

accused

node

is

turned

off

or

restarted

property

holds

by

definition

Otherwise

if

the

program

code

is

updated

Lemma

and

property

guarantee

that

only

correct

code

frames

are

installed

and

that

the

procedure

finally

terminates

In

this

section

we

provide

an

analytical

evaluation

of

the

proposed

algorithm

and

present

the

findings

of

extensive

simulations

as

well

as

of

a

real

implementation

for

the

BTnodes

The

evaluated

metrics

are

-LRB-

a

-RRB-

number

of

update

rounds

-LRB-

b

-RRB-

update

load

for

the

accused

nodes

-LRB-

c

-RRB-

update

load

for

the

other

participating

nodes

and

-LRB-

d

-RRB-

update

duration

Analytical

Evaluation

Number

of

Update

Rounds

In

order

to

update

a

frame

it

is

required

that

the

dedicated

request

as

well

as

the

actual

frame

itself

are

successfully

transmitted

The

expected

fraction

of

erroneous

updates

is

therefore

-LRB-

p

l

-RRB-

If

one

further

assumes

that

the

t

malicious

nodes

do

not

participate

in

the

program

update

the

fraction

increases

to

n

-LRB-

p

l

-RRB-

Thus

the

expected

number

of

outstanding

frames

after

the

first

and

second

update

round

are

E

f

-LRB-

-RRB-

F

-LRB-

n

-LRB-

p

l

-RRB-

-RRB-

and

E

f

-LRB-

-RRB-

E

f

-LRB-

-RRB-

-LRB-

-LRB-

l

-RRB-

-RRB-

F

-LRB-

-LRB-

l

-RRB-

-RRB-

respectively

In

general

the

expected

number

of

outstanding

frames

after

i

rounds

is

E

f

-LRB-

i

-RRB-

E

f

-LRB-

i

-RRB-

n

t

n

-LRB-

p

l

-RRB-

F

n

t

n

-LRB-

p

l

-RRB-

i

Consequently

the

expected

number

of

update

rounds

is

E

r

log

-LRB-

-RRB-

log

-LRB-

F

-RRB-

log

-LRB-

n

-LRB-

p

l

-RRB-

-RRB-

For

reliable

connections

-LRB-

p

l

-RRB-

we

get

E

r

log

-LRB-

-RRB-

log

-LRB-

F

-RRB-

log

-LRB-

-RRB-

O

-LRB-

-RRB-

In

a

worst

case

scenario

a

continuous

sequence

of

frames

is

assigned

to

the

t

malicious

nodes

and

thus

at

least

t

that

is

O

-LRB-

t

-RRB-

O

-LRB-

n

-RRB-

rounds

are

required

Moreover

for

a

fixed

p

l

the

expected

number

of

rounds

is

-LRB-

almost

-RRB-

independent

of

the

cluster

size

n

as

for

a

fixed

t

t

Update

Load

The

expected

amount

of

data

-LRB-

in

bytes

-RRB-

to

transfer

for

the

accused

node

is

E

tm

E

r

X

i

C

req

C

mac

-LRB-

n

-RRB-

C

sel

-LRB-

n

-RRB-

E

f

-LRB-

i

-RRB-

F

-LRB-

C

req

C

mac

-LRB-

n

-RRB-

C

sel

-LRB-

n

-RRB-

-RRB-

E

r

and

E

tv

E

r

X

i

E

f

-LRB-

i

-RRB-

n

-LRB-

p

l

-RRB-

-LRB-

C

update

f

s

-RRB-

F

n

-LRB-

p

l

-RRB-

-LRB-

C

update

f

s

-RRB-

E

r

for

the

other

participating

nodes

In

the

above

expressions

-LRB-

n

-RRB-

E

f

-LRB-

i

-RRB-

F

is

the

expected

number

of

addressed

nodes

and

E

f

-LRB-

i

-RRB-

-LRB-

p

l

-RRB-

expresses

the

expected

number

of

successfully

requested

frames

per

node

Update

Duration

The

total

number

of

sent

messages

E

m

is

bound

by

-LRB-

F

-RRB-

E

r

O

-LRB-

E

r

-RRB-

as

there

are

only

one

request

and

no

more

that

F

update

messages

per

round

Thus

the

expected

value

of

E

m

is

in

O

-LRB-

-RRB-

for

reliable

connections

and

in

O

-LRB-

n

-RRB-

in

the

worst

case

More

precisely

the

expected

number

of

messages

is

given

by

E

m

E

r

X

i

E

f

-LRB-

i

-RRB-

n

-LRB-

p

l

-RRB-

E

r

-LRB-

n

t

-RRB-

E

tv

C

update

f

s

Neglecting

the

delays

caused

by

the

involved

software

routines

a

good

approximation

for

the

update

duration

can

be

achieved

by

considering

the

overall

transfer

time

and

the

delays

caused

by

the

round

timeouts

The

expected

time

to

transfer

all

messages

is

E

tm

-LRB-

-RRB-

E

tv

B

E

m

mac

whereas

the

overhead

of

the

round

timer

is

given

by

-LRB-

E

r

-RRB-

round

resulting

in

a

total

update

duration

of

E

d

E

tm

-LRB-

n

t

-RRB-

E

tv

B

E

m

mac

-LRB-

E

r

-RRB-

round

Parametrisation

For

the

comparison

of

the

analytical

results

with

the

simulation

and

implementation

of

the

algorithm

the

following

parameters

were

used

Baudrate

B

kBit/s

B-MAC

preamble

mac

ms

Round

timeout

round

s

Request

header

size

C

req

Bytes

Update

header

size

C

update

Bytes

Frame

selector

site

C

sel

Bytes

MAC

size

C

mac

Bytes

Frame

size

f

s

Bytes

Simulation

The

simulation

of

the

algorithm

was

carried

out

with

the

Java

based

JiST/SWANS

simulator

-LSB-

-RSB-

In

order

to

make

the

results

comparable

to

the

real

BTnode

implementation

the

radio

module

was

set

up

according

to

the

characteristics

of

packet

loss

ratio

Expected

number

of

update

rounds

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

simulated

-RRB-

F

n

t

-LRB-

simulated

-RRB-

Figure

Expected

number

of

rounds

to

update

a

node

packet

loss

ratio

Expected

duration

-LRB-

in

sec

-RRB-

to

update

a

node

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

simulated

-RRB-

F

n

t

-LRB-

simulated

-RRB-

Figure

Expected

duration

to

update

a

node

the

Chipcon

transceiver

-LSB-

-RSB-

and

B-MAC

was

chosen

as

the

data

link

layer

protocol

The

complete

parametrisa-tion

of

the

simulation

is

given

in

the

table

below

Transmission

frequency

MHz

Transmission

power

dBm

Receiver

sensitivity

dBm

Memory

size

kByte

Number

of

nodes

Deployment

area

x

m

-LRB-

u.r.d.

-RRB-

Implementation

In

addition

to

the

above

mentioned

simulations

the

algorithm

was

also

implemented

for

the

BTnodes

a

wireless

sensor

platform

running

NutOS

-LSB-

-RSB-

A

detailed

description

of

the

created

software

is

omitted

due

to

space

reasons

but

can

be

found

in

-LSB-

-RSB-

The

implementation

was

evaluated

by

randomly

distributing

a

cluster

of

nodes

in

a

field

of

x

m

whereupon

each

node

in

turn

initiated

a

complete

program

update

Altogether

over

recovery

procedures

where

measured

Results

The

packet

loss

ratio

has

as

expected

a

significant

effect

on

all

evaluated

metrics

and

each

of

them

increases

exponen-tially

if

the

ratio

worsens

The

number

of

nodes

in

contrast

packet

loss

ratio

Expected

amount

of

data

to

transfer

-LRB-

in

kBytes

-RRB-

for

the

updating

node

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

simulated

-RRB-

F

n

t

-LRB-

simulated

-RRB-

Figure

Expected

update

load

for

the

accused

node

packet

loss

ratio

Expected

amount

of

data

to

transfer

-LRB-

in

kBytes

-RRB-

for

the

participating

nodes

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

analytic

-RRB-

F

n

t

-LRB-

simulated

-RRB-

F

n

t

-LRB-

simulated

-RRB-

Figure

Expected

update

load

for

the

participating

nodes

has

for

a

fixed

packet

loss

ratio

almost

no

negative

impact

on

the

evaluated

metrics

showing

that

the

algorithm

itself

scales

well

Furthermore

the

results

show

that

the

update

algorithm

is

fair

and

equally

distributes

the

update

load

over

all

participating

nodes

Update

Rounds

and

Update

Duration

Whilst

the

expected

number

of

update

rounds

-LRB-

see

Figure

-RRB-

is

only

of

secondary

importance

the

update

duration

-LRB-

see

Figure

-RRB-

is

of

major

interest

for

the

feasibility

of

the

algorithm

The

faster

a

node

recovery

is

completed

the

sooner

the

network

is

operable

again

Even

though

the

update

duration

almost

triples

from

to

s

if

the

packet

loss

ratio

increases

from

to

percent

it

is

still

in

a

range

which

most

WSN

application

should

be

able

to

cope

with

Update

Load

In

a

cluster

of

nodes

the

update

load

for

the

accused

node

-LRB-

see

Figure

-RRB-

is

to

kByte

for

p

l

and

thus

considerably

smaller

than

for

the

other

participating

nodes

-LRB-

see

Figure

-RRB-

with

a

load

of

to

kByte

However

the

latter

is

as

expected

inverse

proportional

to

the

cluster

size

-LRB-

see

Figure

-RRB-

the

larger

a

cluster

and

the

lower

the

number

of

malicious

nodes

the

smaller

the

expected

update

load

per

participating

node

number

of

nodes

Expected

duration

-LRB-

in

sec

-RRB-

to

update

a

node

F

plr

t

F

plr

t

F

plr

t

F

plr

t

Figure

Influence

of

the

cluster

size

on

the

update

duration

number

of

nodes

Expected

amount

of

data

to

transfer

-LRB-

in

kBytes

-RRB-

for

the

participating

nodes

F

plr

t

F

plr

t

F

plr

t

F

plr

t

Figure

Influence

of

the

cluster

size

on

the

expected

update

load

for

the

participating

nodes

Implementation

In

the

experiments

conducted

with

the

BTnode

implementation

the

average

number

of

update

rounds

was

five

the

update

duration

about

s

-LRB-

s

-RRB-

and

the

load

for

the

participating

nodes

about

kByte

-LRB-

kByte

-RRB-

Applied

to

the

analytical

model

this

would

mean

that

the

gross

packet

loss

ratio

was

roughly

In

this

paper

we

presented

an

autonomous

and

distributed

recovery

algorithm

for

sensor

networks

The

algorithm

allows

for

bringing

malicious

or

failed

nodes

back

into

normal

operation

or

at

least

for

securely

shutting

them

down

Particularly

in

remote

or

unwieldy

areas

such

as

deserts

the

bottom

of

the

sea

mountains

or

even

on

planets

in

outer

space

where

redeployment

is

expensive

and

sensor

nodes

can

not

easily

be

exchanged

or

maintained

the

application

of

a

node

recovery

system

is

most

likely

to

extend

the

lifetime

of

the

whole

network

The

results

of

the

simulation

and

analytical

analysis

were

confirmed

by

the

real

BTnode

implementation

They

show

that

recovering

sensor

nodes

is

as

any

form

of

reprogramming

an

expensive

though

feasible

task

Moreover

the

proposed

program

code

update

algorithm

is

not

only

ably

secure

but

also

fair

and

robust

It

distributes

the

update

load

equally

over

all

participating

nodes

and

terminates

as

long

as

at

least

one

of

the

nodes

remains

correct

To

all

presented

recovery

measures

applies

that

they

are

only

applicable

if

at

least

the

therefore

needed

systems

of

the

corrupt

node

still

work

correctly

However

although

it

is

generally

agreed

that

entirely

tamper-proof

sensor

nodes

are

too

expensive

current

trends

in

the

hardware

development

of

embedded

devices

indicate

that

at

least

some

logical

and

physical

protection

-LRB-

e.g.

CPUs

which

support

isolated

memory

areas

or

automatic

memory

erason

if

a

node

is

tempered

with

-RRB-

will

be

available

in

the

near

future

We

discussed

how

these

upcoming

technologies

can

be

exploited

to

protect

the

recovery

mechanisms

of

a

sensor

node

and

what

is

already

feasible

with

existing

systems

-LSB-

-RSB-

T.

Alves

and

D.

Felton

TrustZone

Integrated

Hardware

and

Software

Security

ARM

Ltd

July

-LSB-

-RSB-

R.

Barr

Z.

J.

Haas

and

R.

van

Renesse

Jist

an

efficient

approach

to

simulation

using

virtual

machines

Research

articles

Softw

Pract

Exper.

-LRB-

-RRB-

-LSB-

-RSB-

J.

Beutel

O.

Kasten

and

M.

Ringwald

Poster

abstract

Btnodes

a

distributed

platform

for

sensor

nodes

In

Proceedings

of

the

International

Conference

on

Embedded

Networked

Sensor

Systems

pages

Los

Angeles

California

USA

Jan.

ACM

Press

-LSB-

-RSB-

Chipcon

AS

Oslo

Norway

Single

Chip

Very

Low

Power

RF

Transceiver

Rev.

Apr.

-LSB-

-RSB-

A.

P.

R.

da

Silva

M.

H.

T.

Martins

B.

P.

S.

Rocha

A.

A.

F.

Loureiro

L.

B.

Ruiz

and

H.

C.

Wong

Decentralized

intrusion

detection

in

wireless

sensor

networks

In

Proceedings

of

the

ACM

international

workshop

on

Quality

of

service

security

in

wireless

and

mobile

networks

pages

New

York

NY

USA

ACM

Press

-LSB-

-RSB-

S.

B.

Douglas

Herbert

Yung-Hsiang

Lu

and

Z.

Li

Detection

and

repair

of

software

errors

in

hierarchical

sensor

networks

To

appear

in

IEEE

conference

on

Sensor

Networks

and

Ubiquitous

Trustworthy

Computing

-LRB-

SUTC

-RRB-

June

-LSB-

-RSB-

P.

K.

Dutta

J.

W.

Hui

D.

C.

Chu

and

D.

E.

Culler

Towards

secure

network

programming

and

recovery

in

wireless

sensor

networks

Technical

Report

UCB/EECS

Electrical

Engineering

and

Computer

Sciences

University

of

California

at

Berkeley

Oct.

-LSB-

-RSB-

S.

Ganeriwal

and

M.

B.

Srivastava

Reputation-based

framework

for

high

integrity

sensor

networks

In

SASN

Proceedings

of

the

ACM

workshop

on

Security

of

ad

hoc

and

sensor

networks

pages

New

York

NY

USA

ACM

Press

-LSB-

-RSB-

C.

Hartung

J.

Balasalle

and

R.

Han

Node

compromise

in

sensor

networks

The

need

for

secure

systems

Technical

Report

Department

of

Computer

Science

University

of

Colorado

Jan.

-LSB-

-RSB-

C.

Hsin

and

M.

Liu

A

distributed

monitoring

mechanism

for

wireless

sensor

networks

In

WiSE

Proceedings

of

the

ACM

workshop

on

Wireless

security

pages

New

York

NY

USA

ACM

Press

-LSB-

-RSB-

Intel

Corporation

LaGrande

Technology

Architectural

Overview

Sept.

-LSB-

-RSB-

P.

Inverardi

L.

Mostarda

and

A.

Navarra

Distributed

IDSs

for

enhancing

security

in

mobile

wireless

sensor

networks

AINA

-LSB-

-RSB-

J.

Jeong

and

D.

Culler

Incremental

network

programming

for

wireless

sensors

In

Proceedings

of

the

First

IEEE

Communications

Society

Conference

on

Sensor

and

Ad-Hoc

Communications

and

Networks

-LRB-

SECON

-RRB-

-LSB-

-RSB-

I.

Khalil

S.

Bagchi

and

C.

Nina-Rotaru

Dicas

Detection

diagnosis

and

isolation

of

control

attacks

in

sensor

networks

securecomm

-LSB-

-RSB-

P.

Kocher

R.

Lee

G.

McGraw

and

A.

Raghunathan

Security

as

a

new

dimension

in

embedded

system

design

In

DAC

Proceedings

of

the

annual

conference

on

Design

automation

pages

New

York

NY

USA

ACM

Press

Moderator-Srivaths

Ravi

-LSB-

-RSB-

S.

S.

Kulkarni

and

L.

Wang

Mnp

Multihop

network

reprogramming

service

for

sensor

networks

icdcs

-LSB-

-RSB-

T.

Liu

and

M.

Martonosi

Impala

a

middleware

system

for

managing

autonomic

parallel

sensor

systems

SIGPLAN

Not.

-LRB-

-RRB-

-LSB-

-RSB-

S.

Northcutt

and

J.

Novak

IDS

Intrusion

Detection-Systeme

mitp

Verlag

Bonn

-LSB-

-RSB-

N.

B.

of

Standards

Security

Requirements

for

Cryptographic

Modules

National

Bureau

of

Standards

Dec.

-LSB-

-RSB-

S.

Ravi

A.

Raghunathan

P.

Kocher

and

S.

Hattangady

Security

in

embedded

systems

Design

challenges

Trans

on

Embedded

Computing

Sys.

-LRB-

-RRB-

-LSB-

-RSB-

E.

Shi

and

A.

Perrig

Designing

secure

sensor

networks

IEEE

Wireless

Communication

Magazine

-LRB-

-RRB-

Dec.

-LSB-

-RSB-

M.

Strasser

Intrusion

detection

and

failure

recovery

in

sensor

networks

Master

's

thesis

Department

of

Computer

Science

ETH

Zurich

-LSB-

-RSB-

H.

Vogt

M.

Ringwald

and

M.

Strasser

Intrusion

detection

and

failure

recovery

in

sensor

nodes

In

Tagungsband

INFORMATIK

Workshop

Proceedings

LNCS

Heidelberg

Germany

Sept.

Springer-Verlag

intrusion detection
ids
distributed algorithm
security
countermeasures
sensor networks
sensor nodes
node recovery


An

empirical

comparison

of

supervised

machine

learning

techniques

in

bioinformatics

Research

in

bioinformatics

is

driven

by

the

experimental

data

Current

biological

databases

are

populated

by

vast

amounts

of

experimental

data

Machine

learning

has

been

widely

applied

to

bioinformatics

and

has

gained

a

lot

of

success

in

this

research

area

At

present

with

various

learning

algorithms

available

in

the

literature

researchers

are

facing

difficulties

in

choosing

the

best

method

that

can

apply

to

their

data

We

performed

an

empirical

study

on

individual

learning

systems

and

different

combined

methods

on

different

biological

data

sets

and

provide

some

suggested

issues

to

be

considered

when

answering

the

following

questions

-LRB-

i

-RRB-

How

does

one

choose

which

algorithm

is

best

suitable

for

their

data

set

-LRB-

ii

-RRB-

Are

combined

methods

better

than

a

single

approach

-LRB-

iii

-RRB-

How

does

one

compare

the

effectiveness

of

a

particular

algorithm

to

the

others

In

the

post-genome

era

research

in

bioinformatics

has

been

overwhelmed

by

the

experimental

data

The

complexity

of

biological

data

ranges

from

simple

strings

-LRB-

nucleotides

and

amino

acids

sequences

-RRB-

to

complex

graphs

-LRB-

biochemical

networks

-RRB-

from

-LRB-

sequence

data

-RRB-

to

-LRB-

protein

and

RNA

structures

-RRB-

Considering

the

amount

and

complexity

of

the

data

it

is

becoming

impossible

for

an

expert

to

compute

and

compare

the

entries

within

the

current

databases

Thus

machine

learning

and

artificial

intelligence

techniques

have

been

widely

applied

in

this

domain

to

discover

and

mine

the

knowledge

in

the

databases

Quoting

from

Baldi

and

Brunak

-LRB-

Baldi

and

Brunak

-RRB-

As

a

result

the

need

for

computer

statistical

machine

learning

techniques

is

today

stronger

rather

than

weaker

Shavlik

et

al.

-LRB-

Shavlik

et

al.

-RRB-

described

the

field

of

molecular

biology

as

tailor-made

for

machine

learning

approaches

This

is

due

to

the

nature

of

machine

learning

approaches

that

performs

well

in

domains

where

there

is

a

vast

amount

of

data

but

little

theory

this

is

exactly

the

situation

in

bioinformatics

Since

the

introduction

of

machine

learning

to

this

field

various

algorithms

and

methods

have

been

produced

and

applied

to

study

different

data

sets

Most

of

these

studies

compare

a

new

algorithm

with

the

conventional

ones

asserting

the

effectiveness

and

efficiencies

of

their

methods

in

particular

data

sets

The

variety

of

learning

algorithms

currently

available

for

the

researchers

are

enormous

and

the

main

problems

faced

by

researchers

are

-LRB-

i

-RRB-

How

does

one

choose

which

algorithm

is

best

suitable

for

their

data

set

-LRB-

ii

-RRB-

Are

combined

methods

better

than

a

single

approach

-LRB-

iii

-RRB-

How

does

one

compare

the

effectiveness

of

a

particular

algorithm

to

the

others

Copyright

Australian

Computer

Society

Inc.

This

paper

appeared

at

First

Asia-Pacific

Bioinformatics

Conference

Adelaide

Australia

Conferences

in

Research

and

Practice

in

Information

Technology

Vol

Yi-Ping

Phoebe

Chen

Ed

Reproduction

for

academic

not-for

profit

purposes

permitted

provided

this

text

is

included

The

objective

of

this

study

is

to

provide

some

suggestions

for

the

community

by

answering

the

above

questions

This

paper

is

organised

as

follows

Section

presents

a

brief

summary

of

machine

learning

Section

outlines

the

materials

and

methods

used

in

this

study

Section

presents

the

results

and

discussion

and

the

final

section

summarises

this

work

A

machine

learning

algorithm

is

one

that

can

learn

from

experience

-LRB-

observed

examples

-RRB-

with

respect

to

some

class

of

tasks

and

a

performance

measure

-LRB-

Mitchell

-RRB-

Machine

learning

methods

are

suitable

for

molecular

biology

data

due

to

the

learning

algorithm

's

ability

to

construct

classifiers/hypotheses

that

can

explain

complex

relationships

in

the

data

The

classifiers

or

hypotheses

can

then

be

interpreted

by

a

domain

expert

who

suggests

some

wet-lab

experiments

to

validate

or

refute

the

hypotheses

This

feedback

loop

between

in

silico

and

in

vivo

in

vitro

experiments

accelerates

the

knowledge

discovery

process

over

the

biological

data

This

feedback

is

an

important

characteristic

of

machine

learning

in

bioinformatics

Generally

there

are

two

types

of

learning

schemes

in

machine

learning

supervised

learning

where

the

output

has

been

given

a

priori

labelled

or

the

learner

has

some

prior

knowledge

of

the

data

and

unsupervised

learning

where

no

prior

information

is

given

to

the

learner

regarding

the

data

or

the

output

The

overall

tasks

for

the

learner

are

to

classify

characterise

and

cluster

the

input

data

Classification

is

the

most

common

task

in

biological

problem

where

given

two

different

sets

of

examples

namely

positive

E

and

negative

E

examples

-LRB-

E

E

-RRB-

the

learner

needs

to

construct

a

classifier

to

distinguish

between

the

positive

examples

and

the

negative

set

This

classifier

can

then

be

used

as

the

basis

for

classifying

as

yet

unseen

data

in

the

future

Usually

for

a

supervised

classification

problem

the

training

examples

are

in

the

form

of

a

set

of

tuples

-LCB-

-LRB-

where

x

-RRB-

-RCB-

-LRB-

-RRB-

j

n

n

j

y

x

y

x

i

is

the

class

label

and

y

ij

is

the

set

of

attributes

for

the

instances

The

task

of

the

learning

algorithm

is

to

produce

a

classifier

-LRB-

hypothesis

function

-RRB-

to

classify

the

instances

into

the

correct

class

In

this

study

we

only

consider

supervised

machine

learning

applied

to

classification

We

performed

an

empirical

comparison

of

rule-based

learning

systems

-LRB-

Decision

trees

One

Rule

Decision

rules

-RRB-

statistical

learning

system

-LRB-

ve

Bayes

Instance

Based

SVM

and

neural

networks

-RRB-

and

ensemble

methods

-LRB-

Stacking

Bagging

and

Boosting

-RRB-

on

the

data

listed

in

Table

based

on

the

accuracy

positive

predicted

value

specificity

and

sensitivity

of

the

learning

algorithms

All

the

learning

methods

used

in

this

study

were

obtained

from

the

WEKA

machine

learning

package

-LRB-

-RRB-

Data

set

In

this

study

we

used

the

following

data

sets

obtained

from

UCI

machine

learning

repository

-LRB-

Blake

and

Merz

-RRB-

We

briefly

describe

the

biological

motivation

for

the

data

sets

interested

readers

should

refer

to

the

cited

papers

for

details

E.coli

data

set

The

objective

of

this

data

set

is

to

predict

the

cellular

localisation

sites

of

E.coli

proteins

-LRB-

Horton

and

Nakai

-RRB-

There

are

different

cellular

sites

which

are

cytoplasm

-LRB-

cp

-RRB-

inner

membrane

without

signal

sequence

-LRB-

im

-RRB-

periplasm

-LRB-

pp

-RRB-

inner

membrane

with

uncleavable

signal

sequence

-LRB-

imU

-RRB-

outer

membrane

-LRB-

om

-RRB-

outer

membrane

lipoprotein

-LRB-

omL

-RRB-

inner

membrane

lipoprotein

-LRB-

imL

-RRB-

and

inner

membrane

with

cleavable

signal

sequence

-LRB-

imS

-RRB-

The

attributes

are

signal

sequence

recognition

methods

-LRB-

specifically

those

of

McGeoch

and

von

Heijne

-RRB-

the

presence

of

charge

on

N-terminus

of

predicted

lipoproteins

and

different

scoring

functions

on

the

amino

acid

contents

whether

predicted

as

a

outer

membrane

or

inner

membrane

cleavable

or

uncleavable

sequence

signal

Yeast

data

set

The

objective

is

similar

to

the

E.coli

data

which

is

to

determine

the

cellular

localisation

of

the

yeast

proteins

-LRB-

Horton

and

Nakai

-RRB-

There

are

different

sites

which

include

CYT

-LRB-

cytosolic

or

cytoskeletal

-RRB-

NUC

-LRB-

nuclear

-RRB-

MIT

-LRB-

mitochondrial

-RRB-

-LRB-

membrane

protein

no

N-terminal

signal

-RRB-

-LRB-

membrane

protein

uncleaved

signal

-RRB-

-LRB-

membrane

protein

cleaved

signal

-RRB-

EXC

-LRB-

extracellular

-RRB-

VAC

-LRB-

vacuolar

-RRB-

POX

-LRB-

peroxisomal

-RRB-

and

ERL

-LRB-

endoplasmic

reticulum

lumen

-RRB-

The

attributes

are

similar

to

the

E.coli

data

set

with

the

addition

of

nuclear

localisation

information

Promoter

data

set

The

task

of

the

classifier

is

to

predict

whether

a

DNA

sequence

from

E.coli

is

either

a

promoter

or

not

-LRB-

Towell

et

al.

-RRB-

The

input

data

is

a

sequence

-LRB-

A

C

T

or

G

-RRB-

HIV

data

set

The

data

set

contains

octamer

protein

sequences

each

of

which

needs

to

be

classified

as

an

HIV

protease

cleavable

site

or

uncleavable

site

-LRB-

Cai

and

Chou

-RRB-

Data

set

E.coli

Yeast

Promoters

HIV

Continuous

Attribute

Discrete

Attribute

Classes

Data

Size

336 1484 106

Table

Data

sets

used

in

this

study

Evaluation

We

constructed

a

confusion

matrix

-LRB-

contingency

table

-RRB-

to

evaluate

the

classifier

's

performance

Table

shows

a

generic

contingency

table

for

a

binary

class

problem

True

positives

-LRB-

TP

-RRB-

denote

the

correct

classifications

of

positive

examples

True

negatives

-LRB-

TN

-RRB-

are

the

correct

classifications

of

negative

examples

False

positives

-LRB-

FP

-RRB-

represent

the

incorrect

classifications

of

negative

examples

into

class

positive

and

False

negatives

-LRB-

FN

-RRB-

are

the

positive

examples

incorrectly

classified

into

class

negative

Predicted

Positive

Negative

Positive

TP

FN

Actual

Negative

FP

TN

Table

A

contingency

table

for

a

binary

class

problem

Based

on

the

contingency

table

several

measurements

can

be

carried

out

to

evaluate

the

performance

of

the

induced

classifier

The

most

popular

performance

evaluation

measure

used

in

prediction

or

classification

learning

is

classifier

accuracy

which

measures

the

proportion

of

correctly

classified

instances

FN

FP

TN

TP

TN

TP

Acc

Positive

Predictive

Accuracy

-LRB-

PPV

or

the

reliability

of

positive

predictions

of

the

induced

classifier

-RRB-

is

computed

by

FP

TP

TP

PPV

Sensitivity

-LRB-

S

n

-RRB-

measures

the

fraction

of

actual

positive

examples

that

are

correctly

classified

FN

TP

TP

S

n

while

specificity

-LRB-

S

p

-RRB-

measures

the

fraction

of

actual

negative

examples

that

are

correctly

classified

FP

TN

TN

S

p

Cross-validation

To

evaluate

the

robustness

of

the

classifier

the

normal

methodology

is

to

perform

cross

validation

on

the

classifier

Ten

fold

cross

validation

has

been

proved

to

be

statistically

good

enough

in

evaluating

the

performance

of

the

classifier

-LRB-

Witten

and

Frank

-RRB-

In

ten

fold

cross

validation

the

training

set

is

equally

divided

into

different

subsets

Nine

out

of

ten

of

the

training

subsets

are

used

to

train

the

learner

and

the

tenth

subset

is

used

as

the

test

set

The

procedure

is

repeated

ten

times

with

a

different

subset

being

used

as

the

test

set

We

summarise

our

experimental

results

in

Figure

and

The

full

analysis

of

this

study

is

available

in

Figure

Accuracy

vs

Positive

Predictive

Value

Figure

Specificity

vs

Sensitivity

From

the

results

we

observed

that

most

of

the

individual

learners

tend

to

perform

well

either

in

accuracy

or

specificity

Probably

this

is

due

to

the

induced

classifier

being

able

to

characterise

the

negative

examples

-LRB-

most

of

the

training

sets

have

large

ratio

of

negative

examples

compared

to

positive

examples

-RRB-

Furthermore

the

results

suggest

that

combination

approaches

are

in

general

better

at

minimising

overfitting

of

the

training

data

We

also

observed

from

this

experiment

that

boosting

performs

better

than

bagging

This

is

because

attributes

which

are

highly

important

in

discriminating

between

classes

are

randomly

removed

by

bagging

however

they

are

preserved

in

boosting

and

thus

contribute

to

the

final

voting

scheme

The

only

individual

learning

system

that

perform

better

than

the

combined

methods

is

ve

Bayes

learning

This

may

suggest

that

ve

Bayes

is

capable

of

classifying

instances

based

on

simple

prior

probabilistic

knowledge

In

this

study

SVM

does

not

perform

well

compared

to

other

methods

probably

due

to

the

fact

that

training

data

are

not

separable

in

the

vector

space

Rules-of-thumb

In

this

section

we

address

the

following

questions

by

providing

some

suggested

issues

-LRB-

rules-of-thumb

-RRB-

to

be

considered

when

answering

them

-LRB-

i

-RRB-

How

does

one

choose

which

algorithm

is

best

suitable

for

their

data

set

Ratio

of

the

training

data

From

these

experiments

we

observed

that

the

division

of

the

training

data

plays

a

crucial

role

in

determining

the

performance

of

the

algorithms

If

the

training

TPs

and

TNs

are

almost

equal

in

size

the

algorithms

tend

to

construct

much

better

classifiers

This

observation

suggested

that

the

classifier

induced

from

equal

size

of

TP

and

TN

tend

to

be

more

robust

in

classifying

the

instances

Furthermore

the

classifiers

generated

consider

all

the

discriminative

attributes

that

distinguish

between

two

different

classes

If

the

size

of

the

TP

set

is

small

compared

to

that

of

TN

most

probably

the

classifier

will

overfit

the

positive

examples

and

thus

perform

poorly

in

the

cross

validation

stages

Attributes

Another

factor

that

must

be

taken

into

consideration

when

choosing

a

learning

method

is

the

nature

of

the

attributes

Generally

statistical

methods

-LRB-

e.g.

SVM

neural

networks

-RRB-

tend

to

perform

much

better

over

multi-dimensions

and

continuous

attributes

This

is

because

the

learning

strategy

embedded

in

these

algorithms

enables

the

learners

to

find

a

maximal

margin

that

can

distinguish

different

classes

in

the

vector

space

By

contrast

rule-based

systems

-LRB-

e.g.

Decision

trees

PART

-RRB-

tend

to

perform

better

in

discrete

categorical

attributes

The

algorithms

of

these

methods

operate

in

a

top-down

manner

where

the

first

step

is

to

find

the

most

discriminative

attribute

that

classifies

different

classes

The

process

is

iterated

until

most

of

the

instances

are

classified

into

their

class

Credibility

vs.

Comprehensibility

When

choosing

a

machine

learning

technique

users

need

to

ask

themselves

what

they

really

want

to

discover

from

the

data

If

they

are

interested

in

generating

understandable

hypotheses

then

a

rule-base

learning

algorithm

should

be

used

instead

of

statistical

ones

Most

machine

learning

algorithms

follow

Occam

's

principle

when

constructing

the

final

hypothesis

According

to

this

principle

the

algorithm

tends

to

find

the

simplest

hypotheses

by

avoiding

overfitting

the

training

data

But

does

this

principle

still

hold

in

bioinformatics

In

bioinformatics

we

often

wish

to

explore

data

and

explain

results

and

hence

we

are

interested

in

applying

intelligent

systems

to

provide

an

insight

to

understand

the

relations

between

complex

data

The

question

then

arises

as

to

whether

we

prefer

a

simple

classifier

or

a

highly

comprehensible

model

In

general

there

is

a

trade

off

between

the

credibility

and

comprehensibility

of

a

model

Domingos

-LRB-

-RRB-

suggested

applying

domain

constraints

as

an

alternative

for

avoiding

overfitting

the

data

We

agree

with

Muggleton

et

al.

-LRB-

-RRB-

that

when

comparing

the

performance

of

learning

systems

in

a

bioinformatics

context

the

hypothesis

with

better

explanatory

power

is

preferable

when

there

exist

more

than

one

hypotheses

with

statistical

equivalent

predictive

accuracy

-LRB-

ii

-RRB-

Are

combined

methods

better

than

a

single

approach

From

the

experiments

most

of

the

combined

methods

perform

better

than

the

individual

learner

This

is

because

of

the

individual

methods

can

claim

that

they

are

superior

to

the

others

due

to

statistical

computational

and

representational

reasons

-LRB-

Dietterich

-RRB-

Every

learning

algorithm

uses

a

different

search

strategy

If

the

training

data

is

too

small

the

individual

learner

can

induce

different

hypotheses

with

similar

performances

from

the

search

space

Thus

by

averaging

the

different

hypotheses

the

combined

classifier

may

produce

a

good

approximation

to

the

true

hypotheses

The

computational

reason

is

to

avoid

local

optima

of

individual

search

strategy

By

performing

different

initial

searches

and

combining

the

outputs

the

final

classifier

may

provide

a

better

approximation

to

the

true

hypotheses

Lastly

due

to

the

limited

amount

of

training

data

the

individual

classifier

may

not

represent

the

true

hypotheses

Thus

through

considering

different

classifiers

it

may

be

possible

to

expand

the

final

classifier

to

an

approximate

representation

of

the

true

hypotheses

Ensemble

learning

has

been

an

active

research

topic

in

machine

learning

but

not

in

the

bioinformatics

community

Since

most

of

the

hypotheses

induced

are

from

incomplete

biological

data

it

is

essential

to

generate

a

good

approximation

by

combining

individual

learners

-LRB-

iii

-RRB-

How

does

one

compare

the

effectiveness

of

a

particular

algorithm

to

the

others

Predictive

accuracy

Most

of

the

time

we

can

find

in

the

literature

reports

that

a

learning

scheme

performs

better

than

another

in

term

of

one

model

's

accuracy

when

applied

to

a

particular

data

set

From

this

study

we

found

that

accuracy

is

not

the

ultimate

measurement

when

comparing

the

learner

's

credibility

Accuracy

is

just

the

measurement

of

the

total

correctly

classified

instances

This

measurement

is

the

overall

error

rate

but

there

can

be

other

measures

of

the

accuracy

of

a

classifier

rule

If

the

training

data

set

has

TNs

and

TPs

by

classifying

all

the

instances

into

a

negative

class

the

classifier

still

can

achieve

a

accuracy

But

the

sensitivity

and

the

positive

predicted

value

is

-LRB-

both

measurements

evaluate

the

performance

in

classifying

TPs

-RRB-

This

means

that

although

the

accuracy

of

the

classifier

is

it

still

can

not

discriminate

between

the

positive

examples

and

the

negatives

Thus

when

comparing

the

performance

of

different

classifiers

accuracy

as

a

measure

is

not

enough

Different

measures

should

be

evaluated

depending

on

what

type

of

question

that

the

user

seeks

to

answer

See

Salzberg

-LRB-

Salzberg

-RRB-

for

a

tutorial

on

comparing

classifiers

Machine

learning

has

increasingly

gained

attention

in

bioinformatics

research

With

the

availability

of

different

types

of

learning

methods

it

has

become

common

for

the

researchers

to

apply

the

off-shelf

systems

to

classify

and

mine

their

databases

In

the

research

reported

in

this

paper

we

have

performed

a

comparison

of

different

supervised

machine

learning

techniques

in

classifying

biological

data

We

have

shown

that

of

the

single

methods

could

consistently

perform

well

over

all

the

data

sets

The

performance

of

the

learning

techniques

is

highly

dependant

on

the

nature

of

the

training

data

This

study

also

shows

that

combined

methods

perform

better

than

the

individual

ones

in

terms

of

their

specificity

sensitivity

positive

predicted

value

and

accuracy

We

have

suggested

some

rules-of-thumb

for

the

reader

on

choosing

the

best

suitable

learning

method

for

their

dataset

We

would

like

to

thank

colleagues

in

the

Bioinformatics

Research

Centre

for

constructive

discussions

We

would

also

like

to

thank

the

anonymous

reviewers

for

their

useful

comments

The

University

of

Glasgow

funded

AC

Tan

's

studentship

BALDI

P.

AND

BRUNAK

S.

-LRB-

-RRB-

Bioinformatics

The

Machine

Learning

Approach

nd

Ed.

MIT

Press

Blake

C.L.

AND

Merz

C.J.

-LRB-

-RRB-

UCI

Repository

of

machine

learning

databases

-LSB-

-RSB-

CAI

Y.-D

AND

CHOU

K.-C

-LRB-

-RRB-

Artificial

neural

network

model

for

predicting

HIV

protease

cleavage

sites

in

protein

Advances

in

Engineering

Software

DIETTERICH

T.G.

-LRB-

-RRB-

Ensemble

methods

in

machine

learning

In

Proceedings

of

the

First

International

Workshop

on

MCS

LNCS

DOMINGOS

P.

-LRB-

-RRB-

The

role

of

Occam

's

razor

in

knowledge

discovery

Data

Mining

and

Knowledge

Discovery

HORTON

P.

AND

NAKAI

K.

-LRB-

-RRB-

A

probabilistic

classification

system

for

predicting

the

cellular

localization

sites

of

proteins

In

Proceedings

of

Fourth

International

Conference

on

ISMB

AAAI

MIT

Press

MITCHELL

T.

-LRB-

-RRB-

Machine

Learning

McGraw-Hill

MUGGLETON

S.

SRINIVASAN

A.

KING

R.D.

AND

STERNBERG

M.J.E.

-LRB-

-RRB-

Biochemical

knowledge

discovery

using

inductive

logic

programming

In

H.

Motoda

-LRB-

Ed

-RRB-

Proceedings

of

the

First

Conference

on

Discovery

Science

Springer-Verlag

SALZBERG

S.

-LRB-

-RRB-

On

comparing

classifiers

a

critique

of

current

research

and

methods

Data

mining

and

knowledge

discovery

SHAVLIK

J.

HUNTER

L.

SEARLS

D.

-LRB-

-RRB-

Introduction

Machine

Learning

TOWELL

G.G.

SHAVLIK

J.W.

AND

NOORDEWIER

M.O.

-LRB-

-RRB-

Refinement

of

approximate

domain

theories

by

knowledge-based

neural

networks

In

Proceedings

of

the

Eighth

National

Conference

on

Artificial

Intelligence

p.

AAAI

Press

WITTEN

I.H.

AND

FRANK

E.

-LRB-

-RRB-

Data

Mining

Practical

machine

learning

tools

and

techniques

with

java

implementations

Morgan

Kaufmann

O O O O O B I O O B O O B O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O

performance evaluation
classification
biological data
cross validation
machine learning
training data
bioinformatics
ensemble methods