An

Intensional

Approach

to

the

Specification

of

Test

Cases

for

Database

Applications

When

testing

database

applications

in

addition

to

creating

in-memory

fixtures

it

is

also

necessary

to

create

an

initial

database

state

that

is

appropriate

for

each

test

case

Current

approaches

either

require

exact

database

states

to

be

specified

in

advance

or

else

generate

a

single

initial

state

-LRB-

under

guidance

from

the

user

-RRB-

that

is

intended

to

be

suitable

for

execution

of

all

test

cases

The

first

method

allows

large

test

suites

to

be

executed

in

batch

but

requires

considerable

programmer

effort

to

create

the

test

cases

-LRB-

and

to

maintain

them

-RRB-

The

second

method

requires

less

programmer

effort

but

increases

the

likelihood

that

test

cases

will

fail

in

non-fault

situations

due

to

unexpected

changes

to

the

content

of

the

database

In

this

paper

we

propose

a

new

approach

in

which

the

database

states

required

for

testing

are

specified

intensionally

as

constrained

queries

that

can

be

used

to

prepare

the

database

for

testing

automatically

This

technique

overcomes

the

limitations

of

the

other

approaches

and

does

not

appear

to

impose

significant

performance

overheads

D.

-LSB-

Software

Engineering

-RSB-

Testing

and

Debugging-Testing

tools

Experimentation

Verification

Modern

information

systems

are

typically

organised

as

collections

of

independent

application

programs

that

communicate

with

one

another

by

means

of

a

central

database

The

database

records

the

state

of

the

organisation

that

the

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

to

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

ICSE

May

Shanghai

China

Copyright

ACM

information

system

supports

while

the

application

programs

implement

the

business

processes

that

manipulate

the

state

To

take

a

simple

but

ubiquitous

example

a

database

system

might

record

details

of

customers

products

and

sales

while

the

application

programs

associated

with

it

handle

operations

such

as

new

product

purchases

and

update

of

the

product

catalogue

as

well

as

supporting

decision

making

by

generating

reports

regarding

the

most

profitable

product

lines

names

and

addresses

of

loss-making

customers

etc.

In

order

to

test

such

application

programs

it

is

necessary

to

create

test

fixtures

that

simulate

the

presence

of

the

rest

of

the

information

system

Fixtures

for

traditional

test

cases

typically

consist

of

in-memory

objects

and

data

structures

that

provide

the

inputs

to

the

program

being

tested

This

kind

of

fixture

is

also

needed

when

testing

database

applications

-LRB-

especially

when

performing

unit

testing

-RRB-

however

since

it

is

unrealistic

-LRB-

and

often

incorrect

-RRB-

to

execute

test

cases

against

an

empty

database

we

need

to

create

additional

fixture

elements

within

the

database

itself

Current

practice

in

the

software

industry

is

to

maintain

one

or

more

test

databases

that

can

be

used

for

testing

individual

programs

These

databases

can

be

artificially

generated

-LRB-

e.g.

using

tools

such

as

DBMonster

and

DataFac-tory

-RRB-

or

they

may

be

subsets

of

the

live

database

taken

as

a

snapshot

at

some

recent

point

in

time

Copies

of

the

live

data

sets

have

the

advantage

that

they

are

more

likely

to

be

representative

of

the

patterns

of

data

encountered

in

practice

while

artificial

data

sets

have

the

advantage

that

they

can

be

made

to

embody

specific

characteristics

-LRB-

such

as

particular

data

skew

patterns

or

volumes

-RRB-

which

may

be

useful

for

load

and

stress

testing

Both

approaches

however

suffer

from

several

disadvantages

The

most

significant

problem

occurs

when

of

the

available

test

databases

are

suitable

starting

points

for

a

particular

test

case

For

example

suppose

a

particular

test

case

executes

a

program

which

purges

inactive

customers

with

the

aim

of

verifying

that

the

business

rule

forbidding

deletion

of

customers

with

negative

balances

is

correctly

en-forced

If

of

the

test

databases

contains

any

inactive

customers

with

negative

balances

then

the

test

case

can

not

be

executed

successfully

For

a

one-off

test

run

testing

personnel

can

choose

a

database

that

is

close

to

what

is

required

and

manually

update

it

so

that

it

is

suitable

for

use

with

the

test

case

But

if

a

complete

test

suite

is

to

be

executed

-LRB-

possibly

including

test

cases

which

themselves

make

modifications

to

the

database

state

-RRB-

then

in

the

worst

case

this

manual

intervention

will

be

required

in

between

every

test

case

execution

This

is

clearly

undesirable

if

test

suites

are

large

or

time-consuming

to

execute

or

if

the

test

suite

is

to

be

run

in

batch

-LRB-

as

in

the

case

of

overnight

regression

testing

for

example

-RRB-

Current

research

in

testing

for

database

systems

proposes

two

approaches

to

this

problem

One

of

these

is

to

include

within

the

test

case

description

a

full

-LRB-

extensional

-RRB-

specification

of

the

database

state

against

which

it

is

to

be

run

-LRB-

and

of

the

database

state

that

should

be

produced

if

the

test

has

executed

successfully

-RRB-

-LSB-

-RSB-

This

solution

is

exemplified

by

DBUnit

an

extension

of

the

JUnit

testing

framework

that

is

designed

for

testing

database

applications

written

in

Java

Each

DBUnit

test

case

is

accompanied

by

an

XML

file

describing

the

data

set

required

for

the

test

Before

each

test

run

DBUnit

clears

the

database

state

and

inserts

the

data

described

by

the

XML

file

This

approach

has

the

advantage

of

simplicity

but

it

places

a

considerable

burden

on

testing

personnel

especially

when

complex

database

states

are

required

It

is

also

inefficient

since

the

database

must

be

continually

destroyed

and

recre-ated

between

tests

even

when

significant

parts

of

the

database

might

have

been

reused

by

the

succeeding

tests

Moreover

maintenance

of

a

large

suite

of

such

tests

is

extremely

challenging

since

any

small

change

to

the

database

schema

may

require

corresponding

changes

to

many

test

cases

The

second

approach

that

has

been

explored

in

the

literature

is

more

efficient

in

that

it

requires

the

creation

of

only

one

database

state

per

test

suite

-LRB-

rather

than

one

per

test

case

-RRB-

It

is

exemplified

by

the

AGENDA

database

testing

toolkit

-LSB-

-RSB-

which

can

automatically

generate

a

database

state

given

information

about

the

schema

some

data

generation

functions

for

individual

attributes

and

some

user-selected

heuristics

describing

the

kind

of

database

state

required

The

AGENDA

tool

also

generates

test

cases

from

a

simple

analysis

of

the

program

being

verified

The

user

must

then

add

preconditions

to

each

test

case

that

are

checked

just

before

it

is

executed

and

that

will

prevent

a

case

from

being

executed

against

an

inappropriate

database

state

This

approach

successfully

relieves

the

user

of

the

need

to

specify

complete

database

states

in

full

detail

but

at

a

cost

The

user

must

accept

that

some

of

the

test

cases

may

not

be

executed

because

the

database

state

fails

the

precondition

even

when

it

would

require

only

a

small

change

to

bring

the

database

into

a

suitable

state

for

the

test

Since

only

one

database

state

is

created

per

test

suite

this

problem

of

failed

tests

is

likely

to

become

more

severe

as

the

size

of

the

test

suite

grows

There

is

also

a

potential

inefficiency

involved

in

generating

test

descriptions

and

inputs

and

in

creating

the

additional

log

tables

and

constraints/triggers

needed

by

the

AGENDA

tool

for

test

cases

that

are

not

in

fact

going

to

be

executed

Ideally

we

would

prefer

to

be

able

to

combine

the

advantages

of

both

these

approaches

to

give

a

form

of

database

test

case

that

is

quick

and

natural

to

specify

and

which

maximises

the

number

of

cases

within

the

suite

that

can

be

executed

while

minimising

the

number

of

full

test

databases

that

need

to

be

maintained

Our

thesis

is

that

this

can

be

achieved

by

allowing

testing

personnel

to

describe

the

database

states

involved

in

their

test

cases

intensionally

in

the

form

of

declarative

conditions

that

the

input

database

must

satisfy

and

by

providing

a

testing

harness

that

can

automatically

adjust

the

input

database

so

that

the

test

conditions

are

satisfied

-LSB-

-RSB-

In

this

paper

we

present

a

language

for

specifying

such

intensional

database

tests

and

describe

its

semantics

and

operational

behaviour

-LRB-

Section

-RRB-

We

present

an

algorithm

for

automatically

modifying

database

states

so

that

test

preconditions

are

satisfied

-LRB-

Section

-RRB-

thus

ensuring

that

all

test

cases

can

be

executed

without

requiring

any

human

intervention

We

further

describe

how

we

have

extended

the

JUnit

testing

framework

to

allow

intensional

database

tests

to

be

specified

and

executed

in

practice

-LRB-

Section

-RRB-

Finally

we

present

the

results

of

an

evaluation

of

the

performance

of

the

techniques

-LRB-

Section

-RRB-

and

conclude

-LRB-

Section

-RRB-

A

conventional

test

case

is

typically

modelled

as

a

triple

p

i

o

which

denotes

a

test

that

executes

program

p

with

inputs

-LRB-

e.g.

parameters

-RRB-

denoted

by

i

If

no

faults

are

encountered

during

the

test

execution

the

output

that

will

be

produced

is

o

In

the

case

of

test

cases

for

database

applications

we

must

add

two

further

elements

the

specification

of

the

database

state

against

which

p

is

to

be

executed

and

some

statement

of

the

database

state

that

should

result

from

the

execution

of

p

if

it

is

operating

correctly

according

to

its

specification

For

example

consider

the

example

program

mentioned

in

Section

that

prunes

inactive

customer

details

from

the

database

For

this

test

case

we

require

a

database

state

that

contains

at

least

one

inactive

customer

This

could

easily

be

stated

as

a

predicate

logic

condition

over

the

database

assuming

the

obvious

mapping

between

stored

relations

and

predicates

e.g.

-LRB-

custNo

lastOrderOn

a

b

c

-RRB-

customer

-LRB-

custNo

a

b

c

lastOrderOn

-RRB-

lastOrderOn

today

The

program

in

question

does

not

access

any

parts

of

the

database

other

than

the

customer

table

Therefore

we

do

not

care

what

values

the

other

tables

contain

and

need

not

mention

them

in

the

intensional

specification

of

the

test

This

approach

works

equally

well

for

observing

the

results

of

the

test

For

example

when

testing

the

customer

pruning

behaviour

we

might

require

that

no

inactive

customer

with

a

non-negative

balance

should

exist

in

the

database

after

the

test

-LRB-

-LRB-

custNum

lastOrderDate

a

b

c

-RRB-

customer

-LRB-

custNum

a

bal

c

lastOrderDate

-RRB-

lastOrderDate

today

bal

-RRB-

Effectively

the

test

case

describes

a

set

of

valid

-LRB-

i.e.

fault-free

-RRB-

state

transition

for

the

database

as

a

classic

pre/post-condition

pair

This

first-order-logic

style

of

database

specification

does

not

work

so

well

when

we

consider

the

testing

problem

in

more

depth

however

The

problem

is

that

we

need

to

do

more

than

test

the

input

database

for

compliance

with

the

requirements

of

the

test

case

we

also

need

to

extract

information

from

it

to

be

used

to

instantiate

other

elements

of

the

test

case

For

example

suppose

we

wish

to

test

a

program

that

deletes

details

of

individual

customers

Such

programs

typically

require

some

input

from

the

user

identifying

the

specific

customer

record

that

is

to

be

deleted

-LRB-

e.g.

by

supplying

the

relevant

customer

code

as

a

parameter

-RRB-

This

could

be

achieved

by

requiring

the

tester

to

embed

the

customer

code

into

the

test

case

elements

as

literal

values

Alternatively

we

could

search

for

a

suitable

customer

that

already

exists

in

the

database

using

a

standard

database

query

and

use

the

values

from

that

in

specifying

the

inputs

for

the

test

case

This

would

minimise

the

amount

of

work

required

to

prepare

the

database

for

test

execution

-LRB-

since

we

would

be

using

data

already

present

in

the

database

-RRB-

and

it

would

also

mean

that

test

cases

can

be

written

very

quickly

since

the

user

does

not

need

to

specify

every

last

detail

of

the

data

to

be

used

Under

this

approach

the

specification

of

the

input

database

state

now

has

a

dual

role

it

must

state

the

condition

that

determines

whether

the

database

state

is

suitable

for

execution

of

the

test

case

and

it

must

also

return

bindings

for

the

free

variables

that

appear

in

the

remaining

components

of

the

test

case

For

the

latter

purpose

we

would

prefer

to

use

a

straightforward

query

language

while

for

the

former

we

require

the

ability

to

place

conditions

on

the

data

With

a

simple

extension

of

a

standard

query

language

such

as

SQL

we

can

combine

both

these

purposes

in

a

single

statement

For

example

the

following

statement

ANY

cn

GENERATED

BY

SELECT

custNo

FROM

customer

WHERE

lastOrderDate

today

-LRB-

-RRB-

AND

balance

retrieves

the

customer

code

of

some

record

that

meets

the

given

conditions

-LRB-

an

inactive

customer

with

negative

balance

-RRB-

from

the

database

and

binds

it

to

the

variable

cn

It

also

places

a

cardinality

constraint

on

the

result

of

the

query

that

at

least

one

such

binding

must

exist

-LRB-

implied

by

the

use

of

the

keyword

ANY

-RRB-

The

variable

cn

can

then

be

used

to

specify

other

elements

of

the

test

case

The

obvious

usage

in

this

example

is

in

specifying

the

inputs

to

the

program

being

tested

but

it

can

also

be

used

in

describing

the

expected

outputs

of

the

program

In

this

example

test

case

the

correct

behaviour

of

the

DeleteCustomer

program

is

to

reject

the

deletion

of

cn

since

customers

with

a

negative

balance

can

not

be

purged

from

the

database

We

might

therefore

give

the

following

specification

of

the

desired

output

database

state

AT

LEAST

GENERATED

BY

SELECT

custNo

FROM

customer

WHERE

custNo

cn

Of

course

not

all

test

cases

are

best

specified

in

terms

of

values

retrieved

from

the

database

For

example

suppose

that

we

wish

to

write

test

cases

for

a

program

that

adds

new

customers

to

the

database

The

inputs

to

this

program

are

the

details

of

the

new

customer

and

the

precondition

for

one

particular

test

case

states

that

no

customer

should

exist

that

has

the

same

customer

code

as

that

of

the

customer

being

created

We

can

not

retrieve

the

customer

details

from

the

database

in

this

case

as

they

have

not

yet

been

stored

in

it

Again

we

could

force

the

user

to

include

the

required

values

as

literals

in

the

test

case

but

ideally

we

would

like

to

give

CONDITION

TYPE

BINDINGLIST

GENERATED

BY

SELECT

TYPE

ANY

NO

AT

LEAST

i

AT

MOST

i

EXACTLY

i

ALL

FIRST

i

-LCB-

-RCB-

BINDINGLIST

BINDING

-LCB-

BINDINGLIST

-RCB-

BINDING

-LCB-

A-Z

a-z

-RCB-

SELECT

Figure

Simplified

BNF

Grammar

for

SQL

Extensions

more

support

to

the

process

of

test

case

generation

One

way

to

achieve

this

is

to

allow

user-defined

data

generator

functions

to

be

incorporated

within

queries

as

though

they

were

relations

For

example

the

following

expression

states

our

requirements

for

this

test

case

while

also

binding

the

variables

needed

for

input

to

the

program

ANY

cn

name

addr

bal

GENERATED

BY

SELECT

gc.custno

gc.name

gc.addr

FROM

genCustomerDetails

-LRB-

-RRB-

AS

gc

WHERE

gc.custno

NOT

IN

-LRB-

SELECT

custno

FROM

customer

WHERE

balance

-RRB-

Here

the

data

generator

function

getCustomerDetails

-LRB-

-RRB-

is

used

as

if

it

were

a

normal

relation

whereas

in

fact

the

results

it

returns

are

computed

on

the

fly

In

fact

several

of

the

main

commercial

database

management

systems

already

allow

user-defined

functions

to

be

embedded

in

queries

in

this

way

so

this

does

not

require

a

further

extension

of

SQL

Figure

shows

the

minimal

extensions

that

are

needed

to

support

all

the

kinds

of

constrained

query

shown

above

using

the

standard

-LSB-

-RSB-

Test

Case

Semantics

Clearly

the

semantics

of

these

intensional

database

test

cases

is

more

complex

than

for

traditional

extensional

tests

However

we

can

define

their

semantics

formally

in

terms

of

a

mapping

from

intensional

tests

to

sets

of

equivalent

extensional

database

test

cases

We

first

present

a

formal

definition

of

the

structure

of

our

intensional

test

cases

Definition

An

intensional

database

test

case

is

a

quintuple

p

i

DB

i

o

DB

o

where

p

is

the

program

to

be

executed

in

the

test

i

is

a

tuple

of

n

variables

and

literals

that

describes

the

inputs

to

be

given

to

program

p

where

n

is

the

number

of

parameters

expected

by

p

DB

i

is

a

set

of

constrained

queries

that

together

specify

the

initial

database

state

o

is

a

tuple

of

m

variables

and

literal

that

describes

the

expected

outputs

from

the

program

p.

DB

o

is

a

set

of

constrained

queries

that

together

specify

the

conditions

that

must

hold

in

the

database

state

after

execution

of

p

if

no

fault

has

been

encountered

A

constrained

query

has

the

form

Q

min

max

vars

where

Q

is

a

standard

relational

algebra

query

min

and

max

describe

the

constraints

on

the

cardinality

of

the

query

result

set

and

vars

is

the

list

of

variables

bound

by

the

query

result

A

database

test

case

is

well-formed

for

use

with

a

particular

database

schema

iff

for

every

variable

v

that

occurs

free

in

i

DB

i

o

and

DB

o

there

exists

a

query

in

DB

i

that

provides

a

binding

for

v

for

every

query

q

n

m

vs

in

DB

i

DB

o

q

is

a

well-formed

query

over

that

returns

k-tuples

where

vs

k

and

there

are

no

circular

variable

dependencies

amongst

the

queries

in

DB

i

We

can

now

define

a

semantics

for

the

intensional

database

test

cases

as

follows

Every

intensional

test

case

is

equivalent

to

a

set

of

extensional

test

cases

An

extensional

test

case

defines

a

specific

test

run

in

terms

of

actual

inputs

and

outputs

rather

than

expressions

denoting

sets

of

inputs

and

outputs

The

set

of

all

possible

extensional

test

cases

is

given

by

P

L

n

DB

L

DB

where

P

is

the

set

of

all

programs

L

is

the

set

of

all

literals

L

n

is

the

set

of

all

n-tuples

formed

from

L

and

DB

is

the

set

of

all

database

states

-LRB-

relative

to

all

schemas

-RRB-

The

components

of

each

extensional

test

are

the

program

to

be

tested

the

input

values

the

initial

database

state

the

expected

output

and

the

expected

final

database

state

respectively

An

intensional

test

case

is

effectively

a

shorthand

expression

for

a

set

of

extensional

test

cases

that

are

all

derived

from

the

same

equivalence

partition

of

the

test

case

inputs

An

intensional

database

test

p

i

DB

i

o

DB

o

where

DB

i

-LCB-

q

i

n

i

m

i

v

i

-RCB-

and

DB

o

-LCB-

q

o

n

o

m

o

v

o

-RCB-

is

equivalent

to

the

following

set

of

extensional

tests

-LCB-

p

i

-LSB-

v

i

v

-RSB-

db

i

o

-LSB-

v

i

v

-RSB-

db

o

db

i

DB

-LRB-

n

i

q

i

-LRB-

db

i

-RRB-

m

i

-RRB-

v

q

i

-LRB-

db

i

-RRB-

db

o

DB

-LRB-

n

o

-LRB-

q

o

-LSB-

v

i

v

-RSB-

-RRB-

-LRB-

db

o

-RRB-

m

o

-RRB-

-RCB-

We

use

the

notation

exp

-LSB-

-RSB-

to

express

the

substitution

of

the

values

in

by

the

corresponding

values

in

whereever

they

occur

in

exp

Therefore

this

expression

denotes

the

set

of

extensional

tests

where

the

input

database

satisfies

the

constraints

imposed

by

the

initial

constrained

query

and

where

the

bindings

from

execution

of

that

query

-LRB-

here

expressed

as

the

tuple

of

variables

v

-RRB-

are

substituted

into

the

For

simplicity

of

presentation

we

assume

that

all

programs

require

the

same

number

of

inputs

-LRB-

n

-RRB-

In

practice

n

can

be

the

largest

number

of

inputs

required

by

any

program

and

the

unused

values

can

be

filled

with

nulls

expressions

defining

the

inputs

expected

output

and

expected

final

database

state

before

they

too

are

evaluated

The

idea

underlying

this

notion

of

an

intensional

test

is

that

when

any

of

its

corresponding

extensional

sets

are

executed

the

intensional

test

is

itself

deemed

to

have

been

executed

Thus

the

use

of

intensional

tests

allows

much

greater

freedom

at

test

execution

time

since

we

may

choose

any

of

the

possible

extensional

tests

depending

on

which

is

closest

to

our

starting

environment

In

the

next

section

we

will

consider

the

practical

ramifications

of

this

approach

to

testing

and

describe

how

the

semantics

just

described

can

be

implemented

in

practice

The

execution

of

an

intensional

database

test

case

consists

of

three

distinct

phases

-RRB-

preparation

of

the

environment

for

test

execution

-RRB-

execution

of

the

test

with

the

prepared

inputs

and

-RRB-

capture

and

storage

of

the

results

for

later

analysis

Since

all

the

work

of

finding

bindings

for

the

variables

in

the

test

case

specification

is

done

in

the

preparation

phase

the

final

two

phases

are

straightforward

and

differ

little

from

standard

testing

procedures

When

program

execution

is

complete

the

constrained

query

that

determines

whether

the

test

has

been

successful

or

not

is

evaluated

against

the

database

and

the

output

from

the

program

is

checked

against

what

is

expected

In

the

case

of

test

failure

the

details

of

the

actual

extensional

test

that

was

executed

are

recorded

for

diagnosis

purposes

The

first

phase

however

is

more

complex

If

we

were

content

to

execute

only

those

test

cases

which

happen

to

be

suitable

for

use

with

the

initial

database

state

then

the

preparation

phase

would

simply

be

a

matter

of

executing

the

input

constrained

queries

against

the

database

and

if

they

are

all

successful

using

the

bindings

thus

produced

to

instantiate

the

remaining

components

of

the

test

case

However

thanks

to

the

declarative

nature

of

our

test

case

specifications

the

testing

framework

can

be

pro-active

in

cases

where

the

given

database

is

not

suitable

for

use

by

the

test

case

and

can

automatically

generate

a

sequence

of

updates

that

will

cause

the

constrained

queries

to

produce

the

required

number

of

bindings

In

fact

this

problem

is

similar

-LRB-

though

not

identical

-RRB-

to

one

that

has

been

studied

by

the

database

and

artificial

intelligence

communities

for

many

years

It

is

known

variously

as

the

view

update

problem

-LSB-

-RSB-

the

knowledge

base

update

problem

-LSB-

-RSB-

and

the

transaction

repair

problem

-LSB-

-RSB-

Many

database

systems

have

the

capability

to

define

views

on

top

of

the

basic

database

A

view

is

a

kind

of

virtual

relation

To

the

user

it

appears

to

be

a

normal

relation

but

it

contains

no

stored

data

Instead

the

contents

of

the

view

are

defined

by

a

expression

over

other

relations

and

attempts

to

retrieve

data

from

the

view

are

converted

into

queries

over

these

relations

To

take

a

simple

example

for

illustration

we

might

create

a

view

called

Debtors

which

appears

to

be

a

relation

of

the

same

name

containing

all

customers

with

a

negative

balance

Attempts

to

retrieve

Debtors

is

For

simplicity

of

presentation

we

assume

here

that

there

is

only

one

query

in

each

of

DB

i

and

DB

o

In

practice

it

may

be

necessary

to

include

several

queries

each

producing

different

bindings

and

imposing

different

cardinality

constraints

In

this

case

the

constraints

must

be

conjoined

and

the

full

set

of

bindings

can

be

retrieved

by

performing

a

natural

join

of

all

the

queries

with

join

condition

true

converted

into

a

query

against

the

customer

table

with

an

added

constraint

on

the

balance

If

views

are

truly

to

act

as

normal

relations

then

it

should

be

possible

to

update

them

as

well

query

them

But

what

does

it

mean

to

update

a

virtual

relation

In

this

case

the

view

update

must

be

converted

into

a

sequence

of

updates

on

the

stored

relations

that

will

cause

the

desired

change

in

the

contents

of

the

view

itself

This

is

a

non-trivial

problem

for

realistic

view

languages

and

becomes

even

more

difficult

when

we

move

into

the

context

of

knowledge

bases

where

virtual

relations

can

be

defined

using

rules

over

other

relations

and

when

we

add

integrity

constraints

that

must

be

maintained

by

all

updates

-LSB-

-RSB-

Only

in

very

narrow

circumstances

does

a

view

update

have

a

single

translation

into

real

updates

-LSB-

-RSB-

Various

heuristics

for

selecting

from

amongst

the

possible

translations

have

been

proposed

-LRB-

of

which

the

most

common

is

to

choose

the

update

that

results

in

the

smallest

change

to

the

existing

data

set

-LSB-

-RSB-

-RRB-

but

in

real

applications

user

input

is

needed

in

order

to

identify

the

translation

that

corresponds

most

closely

to

the

real

world

state

that

the

database

should

reflect

-LSB-

-RSB-

In

the

case

of

intensional

database

tests

we

have

a

query

-LRB-

the

constrained

query

that

describes

our

requirements

for

the

test

-RRB-

that

does

not

produce

the

correct

number

of

answers

when

executed

against

the

test

database

We

need

to

find

a

sequence

of

updates

to

the

base

data

that

will

cause

our

query

to

produce

the

number

of

answers

we

need

However

in

this

case

there

is

no

requirement

to

find

the

set

of

updates

that

matches

the

state

of

reality

any

sensible

update

that

satisfies

the

query

conditions

will

be

acceptable

This

simplifies

the

problem

considerably

removing

the

need

for

complex

search

procedures

and

for

any

user

input

The

Preparation

Algorithm

One

of

the

advantages

of

using

a

query-based

language

for

test

specification

-LRB-

as

opposed

to

a

predicate

calculus-based

language

-RRB-

is

that

we

can

make

use

of

a

very

common

and

easy-to-analyse

internal

form

for

-LRB-

relational

-RRB-

database

queries

called

relational

algebra

This

form

provides

a

small

number

of

operations

on

relations

that

can

be

combined

to

form

complex

queries

For

example

the

three

most

basic

-LRB-

and

useful

-RRB-

relational

algebra

operators

are

The

projection

operator

Atts

R

which

creates

a

relation

from

R

by

deleting

all

attributes

not

in

Atts

For

example

-LSB-

Country

-RSB-

Customer

produces

a

relation

that

contains

just

the

countries

that

appear

in

the

Customer

relation

The

selection

operator

c

R

which

creates

a

relation

that

contains

all

the

rows

from

relation

R

that

satisfy

the

condition

c.

For

example

bal

Customer

returns

a

relation

containing

details

of

all

customers

with

negative

balances

The

join

operator

R

c

S

which

creates

a

relation

containing

rows

from

the

cross

product

of

R

and

S

that

satisfy

the

join

condition

c

The

query

Debtor

dNo

iNo

Inactive

returns

details

of

all

debtors

who

are

also

inactive

Since

the

result

of

each

relational

algebra

operator

is

itself

a

relation

together

they

form

a

closed

algebra

This

means

that

we

can

form

arbitrarily

complex

queries

by

applying

operators

to

the

results

of

other

operators

For

example

a

query

which

retrieves

the

customer

number

of

all

customers

with

a

negative

balance

would

be

written

as

-LSB-

custNo

-RSB-

-LRB-

balance

Customer

-RRB-

A

common

way

to

visualise

such

expressions

is

as

a

tree

of

operators

The

tree

for

the

above

query

is

shown

in

Figure

Figure

Relational

Algebra

Tree

for

Negative

Balance

Query

Our

algorithm

for

preparing

a

database

for

testing

is

based

around

this

notion

of

a

relational

algebra

tree

We

take

the

cardinality

constraints

from

the

test

specification

and

push

them

down

through

the

nodes

of

the

input

database

query

tree

collecting

up

additional

conditions

as

we

go

When

we

reach

a

leaf

node

-LRB-

i.e.

a

base

relation

-RRB-

we

make

updates

to

the

database

so

that

the

pushed-down

constraints

are

satisfied

for

that

relation

At

each

stage

we

collect

up

the

different

kinds

of

constraint

and

push

them

further

down

into

the

tree

These

constraint

types

are

Min

and

Max

the

upper

and

lower

bounds

on

the

desired

cardinality

of

the

result

set

SelC

the

selection

conditions

on

the

relations

that

we

are

interested

in

UAtts

the

collection

of

attributes

that

are

used

in

the

constrained

query

and

that

must

be

populated

in

any

new

data

that

we

insert

We

also

build

up

a

collection

of

queries

that

describe

the

data

that

has

been

prepared

for

testing

so

far

as

we

progress

through

the

tree

We

call

these

queries

bindings

-LRB-

Bgs

-RRB-

since

they

give

us

values

for

the

variables

that

occur

within

the

selection

and

join

conditions

At

each

stage

the

bindings

should

contain

one

query

for

each

leaf

node

that

has

so

far

been

prepared

It

is

easiest

to

see

how

this

works

by

considering

a

simple

example

such

as

that

shown

in

Figure

Let

us

assume

we

have

a

constrained

query

that

requires

at

least

one

customer

with

negative

balance

to

exist

and

that

our

database

does

not

currently

contain

any

such

customers

We

begin

at

the

root

node

of

the

tree

with

only

the

cardinality

constraints

extracted

from

the

test

specification

Min

Max

SelC

true

UAtts

Bgs

The

top

node

is

a

projection

operator

Projection

does

not

affect

the

cardinality

of

the

result

set

nor

impose

any

conditions

but

it

does

tell

us

something

about

the

attributes

used

Figure

Relational

Algebra

Tree

Showing

Multiple

Joins

by

the

query

We

therefore

add

the

projection

attributes

to

UAtts

and

push

the

constraints

down

to

the

next

node

Min

Max

SelC

true

UAtts

-LCB-

custNo

-RCB-

Bgs

Next

we

must

deal

with

the

selection

node

Selection

nodes

reduce

the

cardinality

of

their

input

so

we

need

to

push

down

the

selection

conditions

to

ensure

that

any

updates

we

may

make

affect

the

correct

tuples

We

also

need

to

add

any

attributes

appearing

in

the

selection

condition

to

UAtts

Min

Max

SelC

balance

UAtts

-LCB-

custNo

balance

-RCB-

Bgs

The

final

node

is

the

leaf

node

representing

the

Customer

relation

We

construct

a

query

from

the

conditions

on

that

relation

and

execute

it

to

find

out

how

many

answers

are

currently

in

the

database

In

this

case

there

are

so

we

need

to

insert

a

new

Customer

record

with

at

least

the

custNo

and

balance

attributes

populated

and

with

a

negative

balance

If

there

are

any

integrity

constraints

on

this

relation

then

we

need

to

make

sure

they

are

also

satisfied

by

the

new

data

We

use

the

DBMonster

data

generator

mentioned

earlier

to

create

the

new

data

It

allows

generation

functions

to

be

specified

for

attributes

and

additional

constraints

to

be

placed

on

them

It

will

also

maintain

primary

key

foreign

key

non-null

and

domain

constraints

if

configured

appro-priately

using

the

information

present

in

the

pushed-down

constraints

Of

course

this

is

a

very

simple

example

In

general

we

can

expect

to

have

to

deal

with

more

complicated

queries

involving

several

joins

such

as

that

shown

in

Figure

This

relational

algebra

tree

is

equivalent

to

the

following

constrained

query

ANY

orderNo

productNo

GENERATED

BY

SELECT

o.orderno

p.productno

FROM

Order

o

Orderdetail

d

Product

p

WHERE

o.orderno

d.orderno

AND

d.productno

p.productno

AND

p.price

which

requires

that

at

least

one

order

must

exist

that

involves

the

purchase

of

at

least

one

product

that

costs

more

than

Joins

complicate

the

process

of

preparing

the

database

because

they

introduce

dependencies

between

the

updates

that

take

place

at

different

leaf

nodes

For

example

imagine

that

we

have

processed

the

tree

shown

in

Figure

as

far

as

the

leaf

node

representing

the

OrderDetail

relation

Join

operators

further

constrain

the

selection

condition

-LRB-

by

conjoining

in

their

join

condition

-RRB-

but

add

no

other

constraints

So

by

the

time

we

reach

this

leaf

node

SelC

will

have

been

set

to

o.orderno

d.orderno

d.productno

p.productno

We

need

to

find

out

whether

a

suitable

OrderDetail

record

exists

within

the

database

However

in

order

to

do

this

we

need

to

know

something

about

what

preparation

actions

were

performed

when

the

Product

leaf

node

was

processed

Maybe

there

were

already

plenty

of

products

in

the

catalogue

or

maybe

there

were

and

one

had

to

be

created

How

is

this

information

passed

through

to

the

OrderDetail

node

so

that

the

correct

tuple

can

be

identified

or

created

In

the

current

version

of

our

algorithm

we

have

chosen

to

use

the

database

itself

to

communicate

these

values

If

there

are

many

suitable

Product

records

then

we

can

find

one

by

querying

the

database

directly

once

again

If

a

new

product

had

to

be

created

then

it

will

now

be

present

in

the

database

so

we

can

still

retrieve

it

by

querying

The

information

needed

to

construct

these

queries

is

present

in

the

selection

conditions

that

have

been

considered

during

the

processing

of

the

relational

algebra

tree

up

to

this

point

For

example

in

order

to

search

for

an

OrderDetail

tuple

that

is

connected

to

a

suitable

Product

we

need

to

issue

the

following

query

SELECT

d.

FROM

OrderDetail

d

Product

p

WHERE

d.productno

p.productno

AND

p.price

This

query

can

not

be

constructed

from

only

the

constraints

pushed-down

from

the

parent

nodes

of

the

leaf

node

instead

we

need

to

collect

up

the

constraints

imposed

by

all

nodes

visited

before

the

current

node

so

that

they

are

available

for

query

formation

This

is

done

using

the

Bgs

data

structure

mentioned

earlier

Figure

presents

the

complete

algorithm

showing

the

behaviour

required

for

each

different

type

of

operator

The

algorithm

is

presented

as

a

side-effecting

function

which

takes

the

constrained

query

that

is

to

be

satisfied

by

the

database

and

a

set

of

initial

conditions

that

state

the

required

cardinality

bounds

and

initialise

SelC

to

true

UAtts

to

and

Bgs

to

The

function

returns

a

set

of

bindings

but

these

are

discarded

The

main

task

of

the

algorithm

is

carried

out

by

the

side-effecting

updates

that

occur

when

leaf

nodes

are

processed

The

intensional

database

test

language

and

accompanying

preparation

algorithm

have

been

implemented

within

a

testing

tool

called

DOT-Unit

This

tool

is

part

of

a

larger

Data-Oriented

Testing

framework

that

is

under

development

at

the

University

of

Manchester

-LSB-

-RSB-

DOT-Unit

has

been

implemented

as

an

extension

to

the

JUnit

testing

framework

willmord/dot

Projection

operator

prepare

-LRB-

Atts

Q

Min

Max

UAtts

SelC

Bgs

-RRB-

prepare

-LRB-

Q

Min

Max

UAtts

Atts

SelC

Bgs

-RRB-

Selection

operator

prepare

-LRB-

c

Q

Min

Max

UAtts

SelC

Bgs

-RRB-

prepare

-LRB-

Q

Min

Max

UAtts

SelC

c

Bgs

-RRB-

Join

operator

prepare

-LRB-

Q

jc

Q

Min

Max

UAtts

SelC

Bgs

-RRB-

prepare

-LRB-

Q

Min

Max

UAtts

SelC

jc

prepare

-LRB-

Q

Min

Max

UAtts

SelC

Bgs

-RRB-

-RRB-

Relation

-LRB-

leaf

node

-RRB-

prepare

-LRB-

Rasv

Min

Max

UAtts

SelC

Bgs

-RRB-

Q

bindingQuery

-LRB-

v

SelC

Bgs

-RRB-

Execute

Q

to

produce

result

set

RS

if

RS

Min

then

Invoke

DBMonster

to

create

-LRB-

Min

RS

-RRB-

more

instances

of

R

that

satisfy

the

conditions

in

Q

else

if

RS

Max

then

Delete

the

first

-LRB-

RS

Max

-RRB-

tuples

in

RS

else

No

preparation

updates

needed

return

-LRB-

Bgs

binding

-LRB-

v

Q

-RRB-

-RRB-

Figure

The

Database

Preparation

Algorithm

for

the

unit

testing

of

Java

applications

-LSB-

-RSB-

We

have

subclassed

the

standard

JUnit

TestCase

class

to

create

a

dedicated

DatabaseTestCase

class

for

specifying

and

managing

intensional

database

tests

DatabaseTestCase

provides

facilities

for

specifying

pre-conditions

on

database

state

generating

and

manipulating

the

bindings

that

are

produced

by

such

pre-conditions

and

evaluating

post-conditions

on

the

database

state

after

the

test

has

been

completed

The

standard

JUnit

methods

for

determining

the

results

of

test

execution

on

the

in-memory

fixture

can

also

be

used

Figure

shows

an

example

DatabaseTestCase

that

includes

two

individual

tests

The

first

verifies

that

when

a

customer

with

a

non-negative

balance

is

deleted

all

customers

with

that

customer

number

really

do

disappear

from

the

database

The

second

uses

a

data

generation

function

to

propose

attribute

values

for

a

new

customer

record

-LRB-

including

a

unique

customer

number

-RRB-

and

checks

that

after

the

program

has

executed

only

one

customer

with

the

generated

customer

number

exists

We

use

a

prefixed

colon

to

indicate

variables

that

are

shared

amongst

the

test

components

a

notation

that

will

be

familiar

to

many

database

programmers

since

it

is

commonly

used

in

various

forms

of

embedded

SQL

The

shared

variables

acquire

their

values

when

the

test

harness

evaluates

the

precondition

-LRB-

and

performs

any

necessary

database

preparation

steps

-RRB-

These

values

can

then

be

accessed

using

the

binding

method

and

can

be

used

in

arbitrarily

complex

assert

conditions

as

well

as

in

instantiating

the

post-condition

query

One

of

the

main

advantages

of

using

the

JUnit

framework

as

the

basis

for

the

implementation

of

DOT-Unit

is

that

it

allows

us

to

integrate

our

tool

seamlessly

into

existing

development

environments

such

as

Eclipse

Thus

DOT-Unit

tests

are

executed

in

exactly

the

same

way

as

a

standard

JUnit

test

case

and

the

results

are

displayed

using

the

same

interface

components

This

allows

testing

of

database

and

non-database

components

to

be

interleaved

in

a

convenient

and

natural

manner

The

practicality

of

this

intensional

test

case

approach

depends

largely

on

the

performance

overhead

imposed

by

the

database

preparation

algorithm

If

the

time

required

to

execute

each

individual

test

case

is

significantly

higher

using

our

approach

than

with

DBUnit

say

then

fewer

tests

will

be

able

to

be

executed

in

the

time

available

and

the

benefits

of

faster

test

development

and

fewer

spurious

test

failures

will

be

negated

To

gain

a

handle

on

the

degree

of

performance

overhead

to

be

expected

from

DOT-Unit

we

made

use

of

an

existing

extensional

DB

test

suite

that

we

created

for

earlier

work

-LSB-

-RSB-

This

suite

was

designed

for

browser

an

open-source

Java/JDBC

program

that

stories

information

about

files

in

a

MySQL

database

The

schema

of

the

database

consists

of

relations

with

attributes

primary

key

constraints

and

foreign

key

constraints

We

created

an

equivalent

intensional

test

suite

consisting

of

test

cases

from

the

extensional

suite

by

converting

each

test

case

into

DOT-Unit

pre

and

post-conditions

We

also

re-placed

each

hard-coded

test

parameter

in

the

original

tests

into

constrained

query

bindings

We

wanted

to

investigate

two

specific

aspects

of

the

performance

of

DOT-Unit

First

we

wanted

to

compare

its

performance

with

that

of

DBUnit

over

the

equivalent

test

cases

as

the

database

size

grows

Second

we

wanted

to

gain

some

idea

of

what

aspects

of

DB

preparation

and

testing

were

dominating

the

performance

of

DOT-Unit

The

results

of

the

experiments

we

performed

are

presented

below

All

experiments

were

run

on

a

Pentium-M

GHz

machine

with

RAM

running

Ubuntu

Linux

Comparison

with

DBUnit

At

first

sight

the

extensional

approach

as

exemplified

by

DBUnit

would

seem

to

be

the

more

efficient

method

of

the

two

as

the

testing

harness

does

not

need

to

spend

any

time

figuring

out

what

updates

need

to

be

made

prior

to

each

test

it

only

needs

to

execute

them

This

does

public

class

ProgramTest

extends

DatabaseTestCase

-LCB-

public

void

testDeleteCustomer

-LRB-

-RRB-

-LCB-

preCondition

-LRB-

ANY

cn

GENERATED

BY

SELECT

custNo

FROM

customer

WHERE

balance

-RRB-

Program

p

new

Program

-LRB-

-RRB-

p.deleteCustomer

-LRB-

binding

-LRB-

cn

-RRB-

-RRB-

postCondition

-LRB-

NO

GENERATED

BY

SELECT

custno

FROM

customer

WHERE

custNo

cn

-RRB-

-RCB-

public

void

testNewCustomer

-LRB-

-RRB-

-LCB-

preCondition

-LRB-

ANY

cn

name

addr

GENERATED

BY

SELECT

gc.custNo

gc.name

gc.addr

FROM

genCustomerDetails

-LRB-

-RRB-

AS

gc

WHERE

gc.custNo

NOT

IN

-LRB-

SELECT

custNo

FROM

customer

-RRB-

-RRB-

Program

p

new

Program

-LRB-

-RRB-

boolean

b

p.newCustomer

-LRB-

binding

-LRB-

cn

-RRB-

binding

-LRB-

name

-RRB-

binding

-LRB-

addr

-RRB-

-RRB-

assertTrue

-LRB-

b

-RRB-

postCondition

-LRB-

EXACTLY

cn

name

addr

GENERATED

BY

SELECT

custno

name

addr

FROM

customer

-RRB-

-RCB-

-RCB-

Figure

Example

DOT-Unit

Test

Case

not

happen

by

accident

but

because

a

human

programmer

has

spent

time

earlier

deciding

exactly

what

the

database

should

look

like

for

each

test

case

However

when

writing

DBUnit

tests

it

is

common

to

try

to

reuse

database

descriptions

for

multiple

test

cases

where

possible

to

reduce

the

amount

of

programming

and

maintenance

time

In

this

case

some

redundant

updates

will

be

made

before

each

test

case

updates

that

our

extensional

approach

will

not

bother

to

make

It

is

also

the

case

that

DBUnit

makes

its

updates

blindly

whether

they

are

needed

or

not

whereas

the

intensional

approach

will

be

able

to

reuse

much

of

the

existing

database

state

for

each

new

test

case

Given

this

it

seems

likely

that

the

performance

of

DBUnit

will

be

better

when

the

database

state

required

for

each

test

case

is

relatively

small

but

that

the

situation

will

be

reversed

when

the

database

state

grows

much

larger

In

order

to

gauge

the

point

at

which

this

change

occurs

we

ran

our

two

test

suites

-LRB-

extensional

and

intensional

-RRB-

with

databases

of

varying

sizes

and

measured

the

execution

time

taken

to

execute

the

whole

test

suite

In

each

case

we

generated

initial

database

states

of

varying

sizes

at

random

either

populating

the

database

directly

-LRB-

for

the

intensional

test

cases

-RRB-

or

generating

XML

descriptions

of

the

required

state

-LRB-

for

the

extensional

test

cases

-RRB-

The

results

are

shown

in

Figure

Figure

Comparison

of

Approaches

as

DB

Size

Increases

To

our

surprise

although

the

performance

of

DOT-Unit

was

initially

worse

than

that

of

DBUnit

it

overtook

its

competitor

at

a

comparatively

small

database

size

of

around

tuples

per

relation

Obviously

this

experiment

is

a

little

unfair

to

DBUnit

since

programmers

are

unlikely

to

create

database

descriptions

consisting

of

of

tuples

per

relation

However

tests

of

this

scale

will

be

needed

at

some

point

in

the

development

cycle

in

order

to

verify

the

behaviour

of

the

system

on

more

realistic

data

sets

In

order

to

assess

the

behaviour

of

DOT-Unit

more

pre-cisely

consider

the

graph

in

Figure

which

shows

the

results

at

small

databases

sizes

in

more

detail

It

can

be

ob-served

that

the

performance

of

DOT-Unit

first

improves

and

then

begins

to

degrade

again

at

a

database

size

of

around

tuples

per

relation

Figure

Detailed

Comparison

of

Approaches

One

possible

explanation

for

this

initial

improvement

in

performance

is

that

as

the

database

size

rises

so

does

the

probability

that

the

data

needed

for

the

test

case

is

already

present

in

the

database

For

the

very

small

states

a

lot

of

preparation

work

is

required

to

create

the

needed

data

whereas

less

work

is

needed

for

a

more

fully

populated

database

As

the

database

size

increases

further

however

the

costs

of

making

the

queries

needed

to

test

the

preconditions

and

formulate

the

preparation

updates

rises

pushing

up

the

time

required

for

the

entire

preparation

step

This

behaviour

may

be

a

peculiarity

of

the

particular

test

suite

used

of

course

and

further

more

extensive

studies

will

be

required

in

order

to

completely

characterise

the

performance

of

the

DOT-Unit

test

harness

From

these

initial

results

however

DOT-Unit

appears

to

scale

well

relative

to

database

size

and

the

execution

times

are

of

the

same

order

of

magnitude

as

those

resulting

from

DBUnit

This

suggests

that

the

intensional

approach

may

provide

a

good

compromise

between

saving

expensive

programmer

time

in

developing

new

test

cases

and

expenditure

of

cheaper

processing

time

in

executing

the

test

cases

Effect

of

Constraint

Complexity

A

further

concern

was

the

effect

of

increasing

constraint

complexity

on

the

performance

of

DOT-Unit

test

cases

How

much

additional

overhead

is

added

for

conditions

involving

a

higher

number

of

selection

conditions

and

-LRB-

most

impor-tantly

-RRB-

joins

In

order

to

assess

this

we

grouped

the

test

cases

into

three

groups

according

to

their

complexity

A

queries

with

one

or

more

selections

and

no

joins

B

queries

with

one

or

more

selections

and

a

join

between

two

relations

C

queries

with

one

or

more

selections

and

joins

between

three

relations

This

gave

a

test

suite

with

test

cases

in

each

of

these

categories

which

we

executed

against

a

randomly

generated

database

state

with

tuples

per

relation

that

does

not

satisfy

any

of

the

test

case

pre-conditions

Figure

shows

the

results

obtained

for

the

three

complexity

categories

We

measured

the

average

time

taken

to

execute

the

test

cases

in

each

category

including

a

breakdown

of

where

the

time

is

spent

in

each

case

Test

the

time

required

to

execute

the

procedural

aspects

of

the

test

case

Query

the

time

required

to

execute

the

query

aspect

of

the

test

case

condition

Prepare

the

time

required

to

execute

the

preparation

aspect

of

the

test

case

condition

While

the

overall

time

required

to

execute

the

test

cases

rises

as

the

complexity

rises

-LRB-

unsurprisingly

-RRB-

the

relative

proportions

of

time

spent

in

the

various

phases

remains

roughly

the

same

The

preparation

phase

seems

to

account

for

slightly

more

than

half

of

the

time

in

each

case

indicating

that

significant

improvements

could

be

achieved

with

a

less-naive

preparation

algorithm

We

have

presented

a

new

approach

to

the

specification

of

test

cases

for

database

systems

that

attempts

to

reduce

the

amount

of

manual

intervention

required

in

between

test

case

runs

while

also

minimising

the

number

of

spurious

test

failures

due

to

inappropriate

input

database

states

The

approach

has

the

further

advantage

that

it

sits

naturally

on

top

of

test

data

sets

taken

from

live

databases

and

this

allows

testing

to

be

carried

out

using

realistic

data

sets

without

requiring

significant

programmer

effort

to

tailor

the

data

set

to

the

test

cases

In

effect

the

intensional

approach

we

have

Figure

The

Affect

of

Changing

Constraint

Complexity

described

allows

software

developers

to

trade

programmer

time

for

test

execution

time

Our

experience

has

indicated

that

intensional

test

cases

are

quick

and

natural

to

write

for

anyone

who

is

familiar

with

SQL

and

database

programming

although

a

study

with

an

independent

testing

team

would

be

necessary

before

we

can

make

any

strong

claims

in

this

regard

However

compared

with

what

is

involved

in

writing

pure

JDBC

database

test

cases

and

DBUnit

test

cases

we

found

that

the

self-contained

nature

of

the

intensional

test

cases

was

a

definite

advantage

Writing

DBUnit

test

cases

requires

the

programmer

to

continually

check

that

the

test

case

is

compatible

with

the

database

description

Moreover

since

it

is

common

to

try

to

reuse

database

descriptions

for

multiple

test

cases

by

combining

their

requirements

into

one

database

state

it

becomes

very

easy

to

break

one

test

case

by

changing

the

database

description

in

order

to

ready

it

for

another

These

problems

do

not

arise

with

intensional

testing

since

all

the

information

about

the

test

case

is

present

in

a

single

file

-LRB-

the

Java

class

file

-RRB-

We

designed

this

first

version

of

the

preparation

algorithm

for

simplicity

and

correctness

rather

than

efficiency

and

as

such

it

performs

rather

stupidly

in

many

cases

We

are

currently

exploring

options

for

improving

the

algorithm

including

more

intelligent

selection

of

the

order

in

which

the

relational

algebra

tree

is

traversed

alternating

between

passing

query

bindings

and

passing

literal

value

bindings

as

is

most

efficient

and

making

use

of

modifications

to

existing

tuples

as

well

as

simply

adding

and

deleting

tuples

-LRB-

both

of

which

are

comparatively

expensive

operations

-RRB-

The

complexity

of

the

conditions

we

can

handle

is

at

present

limited

by

the

capabilities

of

DBMonster

and

can

be

expanded

by

development

of

a

custom

data

generation

facility

We

also

need

to

expand

the

range

of

queries

that

can

be

handled

beyond

simple

select-project-join

queries

For

example

standard

SQL

also

allows

aggregation

and

ordering

within

queries-both

of

which

offer

challenges

in

terms

of

automatic

preparation

A

further

problem

with

our

current

algorithm

is

that

it

may

sometimes

fail

to

find

a

solution

to

the

database

preparation

problem

even

though

one

exists

This

is

due

to

the

fact

that

updates

are

made

at

leaf

nodes

before

the

full

set

of

constraints

on

those

nodes

has

been

encountered

It

should

be

possible

to

address

the

problem

with

more

sophisticated

querying

techniques

-LRB-

this

is

an

example

of

a

fairly

standard

constrained

search

problem

after

all

-RRB-

although

this

will

add

to

the

performance

overhead

A

thorough

study

of

the

trade-offs

between

spurious

failures

and

more

intelligent

searching

will

need

to

be

carried

out

before

any

concrete

recommendations

can

be

made

Finally

we

note

that

where

it

is

important

to

test

large

numbers

of

frame

constraints

-LRB-

i.e.

aspects

of

the

original

database

state

that

are

not

affected

by

the

execution

of

the

program

under

test

-RRB-

it

may

be

easier

to

express

the

test

case

using

DBUnit

rather

than

cluttering

up

the

intensional

test

with

many

such

constraints

Our

work

presents

a

number

of

possible

avenues

for

future

work

beyond

the

improvements

mentioned

above

of

which

the

most

urgent

is

the

question

of

ordering

of

test

cases

within

suites

This

ordering

can

be

in

terms

of

reducing

the

cost

of

the

modifications

to

database

state

or

to

maximise

fault

coverage

There

is

also

the

question

of

whether

the

modifications

to

database

state

should

always

persist

between

test

cases

or

under

certain

conditions

discarded

For

example

a

test

case

may

specify

that

a

relation

be

empty

and

to

satisfy

the

condition

the

content

is

discarded

However

this

relation

may

be

required

by

later

test

cases

and

so

by

discarding

its

contents

we

increase

the

divide

between

the

test

state

and

the

real

world

This

could

be

accomplished

by

either

embedding

the

modifications

inside

of

a

transaction

which

can

then

be

aborted

or

by

using

a

hypothetical

database

engine

We

thank

Leonardo

Mariani

and

the

anonymous

reviewers

for

comments

on

earlier

drafts

of

this

paper

David

Willmor

is

supported

by

a

research

studentship

from

the

UK

Engineering

and

Physical

Sciences

Research

Council

-LSB-

-RSB-

M.

Arenas

L.

E.

Bertossi

and

J.

Chomicki

Consistent

query

answers

in

inconsistent

databases

In

Proceedings

of

the

ACM

SIGACT-SIGMOD-SIGART

Symposium

on

Principles

of

Database

Systems

-LRB-

PODS

-RRB-

pages

ACM

Press

-LSB-

-RSB-

L.

E.

Bertossi

and

J.

Chomicki

Query

answering

in

inconsistent

databases

In

J.

Chomicki

R.

van

der

Meyden

and

G.

Saake

editors

Logics

for

Emerging

Applications

of

Databases

pages

Springer

-LSB-

-RSB-

P.

Bohannon

M.

Flaster

W.

Fan

and

R.

Rastogi

A

cost-based

model

and

effective

heuristic

for

repairing

constraints

by

value

modification

In

Proceedings

of

the

SIGMOD

Conference

pages

ACM

-LSB-

-RSB-

L.

Bravo

and

L.

E.

Bertossi

Logic

programs

for

consistently

querying

data

integration

systems

In

G.

Gottlob

and

T.

Walsh

editors

Proceedings

of

the

International

Joint

Conference

on

Artificial

Intelligence

-LRB-

IJCAI

-RRB-

pages

Morgan

Kaufmann

August

-LSB-

-RSB-

A.

Cal

i

D.

Lembo

and

R.

Rosati

On

the

decidability

and

complexity

of

query

answering

over

inconsistent

and

incomplete

databases

In

Proceedings

of

the

ACM

SIGACT-SIGMOD-SIGART

Symposium

on

Principles

of

Database

Systems

-LRB-

PODS

-RRB-

pages

ACM

June

-LSB-

-RSB-

D.

Chays

S.

Dan

P.

G.

Frankl

F.

I.

Vokolos

and

E.

J.

Weber

A

framework

for

testing

database

applications

In

Proceedings

of

the

International

Symposium

on

Software

Testing

and

Analysis

-LRB-

ISSTA

-RRB-

pages

August

-LSB-

-RSB-

D.

Chays

Y.

Deng

P.

G.

Frankl

S.

Dan

F.

I.

Vokolos

and

E.

J.

Weyuker

An

AGENDA

for

testing

relational

database

applications

Software

Testing

Verification

and

Reliability

-LRB-

-RRB-

-LSB-

-RSB-

J.

Chomicki

and

J.

Marcinkowski

On

the

computational

complexity

of

minimal-change

integrity

maintenance

in

relational

databases

In

L.

E.

Bertossi

A.

Hunter

and

T.

Schaub

editors

Inconsistency

Tolerance

volume

of

Lecture

Notes

in

Computer

Science

pages

Springer

-LSB-

-RSB-

S.

S.

Cosmadakis

and

C.

H.

Papadimitriou

Updates

of

relational

views

Journal

of

the

ACM

-LRB-

-RRB-

-LSB-

-RSB-

S.

M.

Embury

S.

M.

Brandt

J.

S.

Robinson

I.

Sutherland

F.

A.

Bisby

W.

A.

Gray

A.

C.

Jones

and

R.

J.

White

Adapting

integrity

enforcement

techniques

for

data

reconciliation

Information

Systems

-LRB-

-RRB-

-LSB-

-RSB-

G.

Greco

S.

Greco

and

E.

Zumpano

A

logical

framework

for

querying

and

repairing

inconsistent

databases

IEEE

Transactions

on

Knowledge

and

Data

Engineering

-LRB-

-RRB-

-LSB-

-RSB-

A.

Guessoum

and

J.

W.

Lloyd

Updating

knowledge

bases

New

Generation

Computing

-LRB-

-RRB-

-LSB-

-RSB-

F.

Haftmann

D.

Kossmann

and

A.

Kreutz

Efficient

regression

tests

for

database

applications

In

Proceedings

of

the

Biennial

Conference

on

Innovative

Data

Systems

Research

-LRB-

CIDR

-RRB-

pages

Online

Proceedings

January

-LSB-

-RSB-

G.

M.

Kapfhammer

and

M.

L.

Soffa

A

family

of

test

adequacy

criteria

for

database-driven

applications

In

Proceedings

of

the

ACM

SIGSOFT

Symposium

on

Foundations

of

Software

Engineering

pages

ACM

September

-LSB-

-RSB-

R.

Langerak

View

updates

in

relational

databases

with

an

independent

scheme

ACM

Transactions

on

Database

Systems

-LRB-

TODS

-RRB-

-LRB-

-RRB-

-LSB-

-RSB-

P.

Louridas

Junit

Unit

testing

and

coding

in

tandem

IEEE

Software

-LRB-

-RRB-

July-Aug

-LSB-

-RSB-

J.

Melton

and

A.

R.

Simon

SQL

Understanding

Relational

Language

Components

Morgan

Kaufmann

-LSB-

-RSB-

H.

Shu

Using

constraint

satisfaction

for

view

update

Journal

of

Intelligent

Information

Systems

-LRB-

-RRB-

-LSB-

-RSB-

D.

Willmor

and

S.

M.

Embury

Exploring

test

adequacy

for

database

systems

In

Proceedings

of

the

UK

Software

Testing

Research

Workshop

-LRB-

UKTest

-RRB-

pages

The

University

of

Sheffield

September

-LSB-

-RSB-

D.

Willmor

and

S.

M.

Embury

A

safe

regression

test

selection

technique

for

database

driven

applications

In

Proceedings

of

the

International

Conference

on

Software

Maintenance

-LRB-

ICSM

-RRB-

pages

IEEE

Computer

Society

September

database testing
testing for database systems
software testing
intensional test cases
databases
testing framework

improvement for the intensional test cases


An Integrated Environment to Visually Construct 3D Animations In this paper , we present an expressive 3D animation environment that enables users to rapidly and visually prototype animated worlds with a fully 3D user-interface . A 3D device allows the specification of complex 3D motion , while virtual tools are visible mediators that live in the same 3D space as application objects and supply the interaction metaphors to control them . In our environment , there is no intrinsic difference between user interface and application objects . Multi-way constraints provide the necessary tight coupling among components that makes it possible to seamlessly compose animated and interactive behaviors . By recording the effects of manipulations , all the expressive power of the 3D user interface is exploited to define animations . Effective editing of recorded manipulations is made possible by compacting all continuous parameter evolutions with an incremental data-reduction algorithm , designed to preserve both geometry and timing . The automatic generation of editable representations of interactive performances overcomes one of the major limitations of current performance animation systems . Novel interactive solutions to animation problems are made possible by the tight integration of all system components . In particular , animations can be synchronized by using constrained manipulation during playback . The accompanying video-tape illustrates our approach with interactive sequences showing the visual construction of 3D animated worlds . All the demonstrations in the video were recorded live and were not edited . CRS4 , Scientific Visualization Group , Via Nazario Sauro 10 , 09123 Cagliari , Italy . E-mail : -LCB- gobbetti | balaguer -RCB- @crs4 . it WWW : http://www.crs4.it/~gobbetti http://www.crs4.it/~balaguer Modern 3D graphics systems allow a rapidly growing user community to create and animate increasingly sophisticated worlds . Despite their inherent three-dimensionality , these systems are still largely controlled by 2D WIMP user-interfaces . The lack of correlation between manipulation and effect and the high cognitive distance from users to edited models are the major drawbacks of this solution -LSB- 13 -RSB- . The inadequacy of user-interfaces based on 2D input devices and mindsets becomes particularly evident in the realm of interactive 3D animation . In this case , the low-bandwidth communication between user-interface and application and the restrictions in interactive 3D motion specification capabilities make it extremely difficult to define animations with straight-ahead actions . This inability to interactively specify the animation timing is a major obstacle in all cases where the spontaneity of the animated object 's behavior is important -LSB- 21 ; 35 ; 4 -RSB- . In this paper , we present an expressive 3D animation environment that enables users to rapidly and visually prototype animated worlds with a fully 3D user-interface . A 3D device allows the specification of complex 3D motion , while virtual tools supply the interaction metaphors to control application objects . In our environment , there is no intrinsic difference between user interface and application objects . Multi-way constraints provide the necessary tight coupling among components that makes it possible to compose animated and interactive behaviors . By recording the effects of manipulations , all the expressive power of the 3D user interface is exploited to define animations . Effective editing of recorded manipulations is made possible by compacting all continuous parameter evolutions with our data-reduction algorithm , designed to preserve both geometry and timing . Novel interactive solutions to animation problems are made possible by the tight integration of all system components . In particular , animations can be synchronized using constrained manipulation during playback . In the following sections , we present an overview of the system , we make comparisons with related work , and we conclude with a view of future directions . The accompanying video-tape illustrates our approach with interactive sequences showing the visual construction of 3D animated worlds . All demonstrations in the video were recorded live and were not edited . Our animation environment is built on top of VB2 -LSB- 17 ; 18 -RSB- , a graphics architecture based on objects and constraints . During interaction , the user is the source of a flow of information propagating from input device sensors to manipulated models . Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage , the copyright notice , the title of the publication and its date appear , and notice is given that copying is by permission of ACM , Inc. . To copy otherwise , to republish , to post on servers , or to redistribute to lists , requires prior specific permission and/or a fee . 1995 ACM-0-89791-701-4 / 95/008 ... $ 3.50 395 VB2 applications are represented by a network of interrelated objects , and the maintenance of relationships is delegated to a constraint-based change propagation mechanism . Different primitive elements represent the various aspects of the system 's state and behavior : active variables store the system 's state , domain-independent hierarchical constraints -LSB- 9 -RSB- maintain multi way relations between active variables , daemons provide support for discrete simulation tasks , and indirect expressions allow constraints and daemons to dynamically locate their variables . Constraints are maintained using an efficient local propagation algorithm based on Skyblue -LSB- 27 ; 17 ; 18 -RSB- . The solver is domain independent and can maintain a hierarchy of multi-way , multi output dataflow constraints . The fact that constraint solving consists in performing method selection on the basis of constraint priorities and graph structure , without considering the variables ' values , allows an effective application of a lazy evaluation strategy -LSB- 17 ; 18 -RSB- . The main drawback of such a local propagation algorithm is the limitation to acyclic constraint graphs . However , as noted by Sannella et al. -LSB- 28 -RSB- , cyclic constraint networks are seldom encountered in the construction of user interfaces , and limiting the constraint solver to graphs without cycles gives enough efficiency and flexibility to create highly responsive complex interactive systems . In VB2 , the objects ' internal constraint networks are designed so as to reduce the possibility of creating cyclic constraint graphs . Runtime introduction of a constraint that would create a cyclic graph causes an exception that can be handled to remove the offending constraint 1 . The state manager behavior and the constraint solving techniques are detailed in -LSB- 17 ; 18 -RSB- . 2.2 Interaction The system 's desktop configuration uses keyboard commands to trigger mode changes and animation playback , a Spaceball for continuous specification of spatial transformations , and a mouse for picking . Both hands are thus used simultaneously to input information . LCD shutter glasses provide binocular perception of the synthetic world . Since our main research goal is to explore the potentialities of 3D interaction , we do not provide a two dimensional graphical user interface . A 3D cursor , controlled by the Spaceball , is used to select and manipulate objects of the synthetic world . Direct manipulation and virtual tools are the two techniques used to input information . Both techniques involve using mediator objects that transform the cursor 's movements into modifications of manipulated objects . Virtual tools are visible first class objects that live in the same 3D space as application objects and offer the interaction metaphor to control them . Their visual appearance is determined by a modeling hierarchy , while their behavior is controlled by an internal constraint network -LSB- 18 -RSB- . As in the real world , users configure their workspaces by selecting tools , positioning and orienting them in space , and binding them to application objects . At the moment of binding , the tool decides whether to accept the connection by checking if the application object contains all the needed information and by verifying that the constraint graph obtained by connecting the tool to the model can be handled by the underlying solver -LRB- i.e. it is acyclic -RRB- . The binding mechanism is defined in a declarative way by using indirect constraints -LSB- 18 -RSB- . 1 VB2 's current constraint solver -LSB- 17 ; 28 -RSB- is unable to find acyclic solutions of potentially cyclic constraint graphs . An algorithm that removes this limitation is presented in -LSB- 36 -RSB- . Information control Information display MODEL TOOL v1 v2 c1 c2 bound bound.v1 bound.v2 v1 v2 out_variable in_variable constraint in_out_variable direct reference indirect reference Instance -LRB- a -RRB- -LRB- b -RRB- Figure 1a . Design notation Figure 1b . Model and virtual tool When bound , the tool changes its visual appearance to a shape that provides information about its behavior and offers semantic feedback . During manipulation , the tool 's and the application object 's constraint networks remain continuously connected , so as to ensure information propagation . Multiple tools can be active simultaneously in the same 3D environment in order to control all its aspects . The environment 's consistency is continuously ensured by the underlying constraint solver . The bi-directionality of the relationships between user-interface and application objects makes it possible to use virtual tools to interact with a dynamic environment , opening the door to the integration of animation and interaction techniques . 2.3 Animation By recording the effects of manipulations , animations can be sketched . In order to be able to edit the captured performance , a compact representation of continuous parameter evolution must be obtained . This representation must not only precisely approximate the shape of the initial parameter curves but also their timing . The data reduction algorithm must therefore treat the geometry and time components simultaneously in order to avoid the introduction of errors that would be difficult to control . We have developed an algorithm that incrementally builds , from the input sequence , a parametric B-spline preserving value and time of each input sample within a given tolerance . It is an incremental version of the Lyche and M rken algorithm -LSB- 22 -RSB- that works in parallel with the interactive specification by considering only a small portion of the input curve at any time . Latency time and memory requirements for handling each portion of the curve are constant . Data reduction may therefore be performed concurrently with interactive parameter input , and the responsiveness of the application can be ensured when handling animations defined by any number of samples . The algorithm is presented in detail in -LSB- 2 ; 4 -RSB- . This performance-based approach complements key-framing by providing the ability to create animations with straight-ahead actions . It provides complete control over the animation shape and timing , while key-framing offers control only at a limited number of points . The mediation of virtual tools makes it possible to sketch the evolution of non-geometric attributes , while constrained or free motion can be specified with 3D devices . Since these devices offer continuous control of spatial transformations , subtle synchronizations between position and orientation components can be directly specified . In our environment , straight-ahead animations are defined by expressing the desire to record parameter evolution during interaction . This is done simply by pressing a different mouse button when starting an interaction task . A controller object is connected to each animatable model and is responsible for monitoring model state changes . While recording , all changes are handled by the controller to feed the animation tracks . Continuous tracks apply the data reduction 396 algorithm to the incoming information , while discrete tracks simply store a change value event . During playback , information propagates from the animation tracks through the controllers and down to the models . All connections are realized by bi-directional constraints . Since playback constraints are weaker than interaction constraints , the user can take control over animated models during playback . Animations involving synchronizations with the environment 's evolution can thus be specified by interacting during playback -LSB- 5 -RSB- . Discrete Track Interaction Mediator Data reduction Data sampling Editing Animation recording Animation playback Continuous Track Application Object Animation Controller Figure 2 . Interactive animation and playback Constraint-based architectures have long been used for 2D graphics systems -LRB- see -LSB- 28 -RSB- for a survey -RRB- . In the 3D graphics world , one-way constraints are commonly employed to maintain dependencies between components -LSB- 20 ; 34 ; 37 ; 38 -RSB- . This type of constraint can not easily model mutual relations between objects , thus hindering the tight coupling between user-interface and application objects -LSB- 28 -RSB- . Our system uses instead multi-way local propagation constraints , which offer support for two-way communication between objects while remaining efficient enough to ensure the responsiveness of the system -LSB- 17 ; 18 ; 27 -RSB- . TBAG -LSB- 14 -RSB- also uses multi-way constraints maintained by Skyblue -LSB- 27 -RSB- , but its functional approach concentrates more on modeling time varying behaviors than on creating interactive systems . Much effort has been spent in developing powerful numerical solvers for computer graphics -LRB- e.g. -LSB- 7 ; 15 ; 16 -RSB- -RRB- . This work is complementary to ours , which focuses more on providing ways to interact with constrained environments . Such advanced solvers could replace local propagation in our system for the maintenance of numerical relationships . 3.2 Three-dimensional User Interfaces Much recent research has focused on obtaining rich interaction with 3D environments by means of advanced devices and 3D interaction metaphors -LSB- 8 ; 10 ; 11 ; 13 ; 16 ; 19 ; 26 ; 30 ; 32 -RSB- . 3D widgets or manipulators , similar to our virtual tools , are presented in -LSB- 13 ; 32 -RSB- . These works focused on providing support for 3D widget construction , while we concentrate more on the integration of multiple tools in a single dynamic environment . We are not aware of any attempts to apply the results of 3D interaction research to enhance animation capabilities . 3.3 Performance Animation A number of authors have proposed using live performances to drive computer animations -LRB- e.g. -LSB- 1 ; 23 ; 33 ; 35 -RSB- -RRB- . We strive to bring the expressiveness of these approaches to general purpose animation systems running on graphics workstations . Instead of relying on advanced motion capture devices , we exploit our fully 3D user-interface to control the animated environment at a higher level of abstraction . The guiding approach proposed in -LSB- 23 -RSB- also seeks to provide better control of synthetic objects by raising the abstraction level of user interaction . That work concentrates on modeling complex behaviors in a discrete simulation framework , while we focus on providing intuitive user interfaces . A major limitation of current performance animation systems is the inability to build editable representations out of captured performances -LSB- 35 -RSB- . 3.4 Data Reduction Data reduction or curve fitting techniques have been successfully applied for the interactive specification of 2D or 3D curves or surfaces -LRB- e.g. -LSB- 12 ; 24 ; 25 ; 29 -RSB- -RRB- . These techniques can not be easily adapted to sketching animations of multi-dimensional parameters because they all exhibit one or more of the following problems : -LRB- i -RRB- restriction to 2D or 3D geometric constructions , -LRB- ii -RRB- lack of control on parameterization errors , and -LRB- iii -RRB- need to consider the entire input curve before reduction . An early attempt to use data reduction for animation is described in -LSB- 29 -RSB- . In that system , path geometry and path timing specifications were decoupled , loosing thus the advantages of performance approaches . Banks and Cohen -LSB- 6 -RSB- proposed for their drafting tool an incremental version of the Lyche and M rken algorithm -LSB- 22 -RSB- that does not have the aforementioned drawbacks and could be used in a performance animation context . Their method shares with ours the idea of processing successive portions of the input curve which are then spliced together , but is unable to ensure constant latency times and memory needs -LSB- 4 -RSB- . In this video-paper , we have presented an integrated environment for the rapid and visual prototyping of 3D animated worlds . Using our fully 3D user-interface , non-professional users can swiftly create complex animations with pose-to-pose and straight-ahead techniques . Thanks to automatic data-reduction , animations created by interactive performances can then be effectively edited . In our future work , we intend to develop new virtual tools and visualizations that will improve our 3D user interface for discrete and continuous track manipulation . To allow the system to adhere to timing requirements , we are developing time-critical techniques for controlling rendering complexity and constraint evaluation . The authors would like to thank Ronan Boulic for providing the walking engine used in the interactive sequences , Sally Kleinfeldt as well as Dean Allaman for helpful comments and suggestions , Angelo Mangili for technical help , and Michele M ller for doing the voice on the video . This research was conducted by the authors while at the Swiss Federal Institute of Technology in Lausanne . 397 -LSB- 1 -RSB- Baecker RM -LRB- 1969 -RRB- Picture-driven Animation . Proc . Spring Joint Computer Conference 34 : 273-288 . -LSB- 2 -RSB- Balaguer JF -LRB- 1993 -RRB- Virtual Studio : Un syst me d'animation en environnement virtuel . PhD Thesis , Swiss Federal Institute of Technology in Lausanne . -LSB- 3 -RSB- Balaguer JF , Gobbetti E -LRB- 1995 -RRB- Animating Spaceland . To appear in IEEE Computer Special Isssue on Real-world Virtual Environments 28 -LRB- 7 -RRB- . -LSB- 4 -RSB- Balaguer JF , Gobbetti E -LRB- 1995 -RRB- Sketching 3D Animations . To appear in Proc . EUROGRAPHICS . -LSB- 5 -RSB- Balaguer JF , Gobbetti E -LRB- 1995 -RRB- Supporting Interactive Animation using Multi-way Constraints . Submitted for publication . -LSB- 6 -RSB- Banks M , Cohen E -LRB- 1990 -RRB- Real-time Spline Curves from Interactively Sketched Data . Proc . SIGGRAPH Symposium on Interactive 3D Graphics : 99-107 -LSB- 7 -RSB- Barzel R , Barr A -LRB- 1988 -RRB- A Modeling System Based on Dynamic Constraints . Proc . SIGGRAPH : 179-188 . -LSB- 8 -RSB- Bier EA -LRB- 1990 -RRB- Snap-Dragging in Three Dimensions . Proc . SIGGRAPH Symposium on Interactive 3D Graphics : 193 204 . -LSB- 9 -RSB- Borning A , Freeman-Benson B , Wilson M -LRB- 1992 -RRB- Constraint Hierarchies . Lisp and Symbolic Computation 5 -LRB- 3 -RRB- : 221-268 . -LSB- 10 -RSB- Butterworth J , Davidson A , Hench S , Olano TM -LRB- 1992 -RRB- 3DM : A Three Dimensional Modeler Using a Head-Mounted Display . Proc . SIGGRAPH Symposium on Interactive 3D Graphics : 135-138 . -LSB- 11 -RSB- Card SK , Robertson GG , Mackinlay JD -LRB- 1991 -RRB- The Information Visualizer : An Information Workspace . Proc . SIGCHI : 181-188 . -LSB- 12 -RSB- Chou JJ , Piegl LA -LRB- 1992 -RRB- Data Reduction Using Cubic Rational Splines . IEEE Computer Graphics and Applications 12 -LRB- 3 -RRB- : 60-68 . -LSB- 13 -RSB- Conner DB , Snibbe SS , Herndon KP , Robbins DC , Zeleznik RC , Van Dam A -LRB- 1992 -RRB- Three-Dimensional Widgets . SIGGRAPH Symposium on Interactive 3D Graphics : 183 188 . -LSB- 14 -RSB- Elliott C , Schechter G , Yeung R , Abi-Ezzi S -LRB- 1994 -RRB- TBAG : A High Level Framework for Interactive , Animated 3D Graphics Applications . Proc . SIGGRAPH : 421-434 . -LSB- 15 -RSB- Gleicher M -LRB- 1993 -RRB- A Graphics Toolkit Based on Differential Constraints . Proc . UIST : 109-120 . -LSB- 16 -RSB- Gleicher M , Witkin A -LRB- 1992 -RRB- Through-the-Lens Camera Control . Proc . SIGGRAPH : 331-340 . -LSB- 17 -RSB- Gobbetti E -LRB- 1993 -RRB- Virtuality Builder II : Vers une architecture pour l'interaction avec des modes sysnth tiques . PhD Thesis , Swiss Federal Institute of Technology in Lausanne . -LSB- 18 -RSB- Gobbetti E , Balaguer JF -LRB- 1993 -RRB- VB2 : A Framework for Interaction in Synthetic Worlds . Proc . UIST : 167-178 . -LSB- 19 -RSB- Herndon KP , van Dam A , Gleicher M -LRB- 1994 -RRB- Report : Workshop on the Challenges of 3D Interaction , CHI Bulletin , October . -LSB- 20 -RSB- Kass M -LRB- 1992 -RRB- CONDOR : Constraint-based Dataflow . Proc . SIGGRAPH : 321-330 . -LSB- 21 -RSB- Lasseter J -LRB- 1987 -RRB- Principles of Traditional Animation Applied to 3D Computer Animation . Proc . SIGGRAPH : 35 44 . -LSB- 22 -RSB- Lyche T , M rken K -LRB- 1987 -RRB- Knot Removal for Parametric B spline Curves and Surfaces . Computer Aided Geometric Design 4 : 217-230 . -LSB- 23 -RSB- McKenna M , Pieper S , Zeltzer D -LRB- 1990 -RRB- Control of a Virtual Actor : The Roach . Proc . SIGGRAPH Symposium on Interactive 3D Graphics : 165-174 . -LSB- 24 -RSB- Plass M , Stone M -LRB- 1983 -RRB- Curve Fitting with Piecewise Parametric Cubics . Proc . SIGGRAPH : 229-239 . -LSB- 25 -RSB- Pudet T -LRB- 1994 -RRB- Real Time Fitting of Hand Sketched Pressure Brushstrokes . Proc . EUROGRAPHICS : 205-220 . -LSB- 26 -RSB- Sachs E , Roberts A , Stoops D -LRB- 1990 -RRB- 3-Draw : A Tool for Designing 3D Shapes . IEEE Computer Graphics and Applications 11 -LRB- 6 -RRB- : 18-26 . -LSB- 27 -RSB- Sannella M -LRB- 1994 -RRB- Skyblue : A Multi-Way Local Propagation Constraint Solver for User Interface Construction . Proc . UIST : 137-146 . -LSB- 28 -RSB- Sannella M , Maloney J , Freeman-Benson B , Borning A -LRB- 1992 -RRB- Multi-way versus One-way Constraints in User Interfaces . Software Practice and Experience 23 -LRB- 5 -RRB- : 529-566 . -LSB- 29 -RSB- Schneider PJ -LRB- 1988 -RRB- Phoenix : An Interactive Curve Design System Based on the Automatic Fitting of Hand-Sketched Curves . Master 's Thesis , University of Washington . -LSB- 30 -RSB- Shaw C , Green M -LRB- 1994 -RRB- Two-Handed Polygonal Surface Design . Proc . UIST : 212-215 . -LSB- 31 -RSB- Shelley KL , Greenberg DP -LRB- 1982 -RRB- Path Specification and Path Coherence . Proc . SIGGRAPH : 157-166 . -LSB- 32 -RSB- Strauss PS , Carey R -LRB- 1992 -RRB- An Object-Oriented 3D Graphics Toolkit . Proc . SIGGRAPH : 341-347 . -LSB- 33 -RSB- Tice S -LRB- 1993 -RRB- VActor Animation Creation System . SIGGRAPH Tutorial 1 . -LSB- 34 -RSB- Upson C , Fulhauber T , Kamins D , Laidlaw D , Schlegel D , Vroom J , Gurwitz R , van Dam A -LRB- 1989 -RRB- The Application Visualization System : A Computational Environment for Scientific Visualization . IEEE CG&A 9 -LRB- 4 -RRB- : 30-42 . -LSB- 35 -RSB- Walters G -LRB- 1993 -RRB- Performance Animation at PDI . SIGGRAPH Tutorial 1 . -LSB- 36 -RSB- Vander Zanden B -LRB- 1995 -RRB- An Incremental Algorithm for Satisfying Hierarchies of Multi-way , Dataflow Constraints . Technical Report , University of Tennessee , Knoxville . -LSB- 37 -RSB- Zeleznik RC , Conner DB , Wlocka MM , Aliaga DG , Wang NT , Hubbard PM , Knepp B , Kaufman H , Hughes JF , van Dam A -LRB- 1991 -RRB- An Object-Oriented Framework for the Integration of Interactive Animation Techniques . Proc . SIGGRAPH : 105-112 . -LSB- 38 -RSB- Zeltzer D , Pieper S , Sturman DJ -LRB- 1989 -RRB- An Integrated Graphical Simulation Platform . Proc . Graphics Interface : 266-274 . 398

O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

data reduction
computer graphics
3d user interface
3d widgets
local propagation constraints
3d animation environment
virtual tools
3d animation
3d interaction
visualization

object-oriented graphics
recording 3d manipulation
human interaction
multi-way constrained architecture
animation synchronization
dynamic model