Computing

Consistent

Query

Answers

using

Conflict

Hypergraphs

A

consistent

query

answer

in

a

possibly

inconsistent

database

is

an

answer

which

is

true

in

every

-LRB-

minimal

-RRB-

repair

of

the

database

We

present

here

a

practical

framework

for

computing

consistent

query

answers

for

large

possibly

inconsistent

relational

databases

We

consider

relational

algebra

queries

without

projection

and

denial

constraints

Because

our

framework

handles

union

queries

we

can

effectively

-LRB-

and

efficiently

-RRB-

extract

indefinite

disjunctive

information

from

an

inconsistent

database

We

describe

a

number

of

novel

optimization

techniques

applicable

in

this

context

and

summarize

experimental

results

that

validate

our

approach

Categories

H.

H.

F.

General

Terms

Algorithms

Languages

Theory

Traditionally

the

main

role

of

integrity

constraints

in

databases

was

to

enforce

consistency

The

occurrence

of

integrity

violations

was

prevented

by

DBMS

software

However

while

integrity

constraints

continue

to

express

important

semantic

properties

of

data

enforcing

the

constraints

has

become

problematic

in

current

database

applications

For

example

in

data

integration

systems

integrity

violations

may

be

due

to

the

presence

of

multiple

autonomous

data

sources

The

sources

may

separately

satisfy

the

constraints

but

when

they

are

integrated

the

constraints

may

not

hold

Moreover

because

the

sources

are

autonomous

the

violations

can

not

be

simply

fixed

by

removing

the

data

involved

in

the

violations

Example

Let

Student

be

a

relation

schema

with

the

attributes

Name

and

Address

and

the

key

functional

dependency

N

ame

Address

Consider

the

following

instance

of

Student

Research

supported

by

NSF

Grant

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page

To

copy

otherwise

to

republish

to

post

on

servers

or

to

redistribute

to

lists

requires

prior

specific

permission

and/or

a

fee

CIKM

November

Washington

DC

USA

Copyright

ACM

Name

Address

Jeremy

Burford

Los

Angeles

Jeremy

Burford

New

York

Linda

Kenner

Chicago

The

first

two

tuples

may

come

from

different

data

sources

so

it

may

be

impossible

or

impractical

to

resolve

the

inconsistency

between

them

However

there

is

clearly

a

difference

between

the

first

two

tuples

and

the

third

one

We

do

n't

know

whether

Jeremy

Burford

lives

in

Los

Angeles

or

New

York

but

we

do

know

that

Linda

Kenner

lives

in

Chicago

An

approach

to

query

answering

that

ignores

inconsistencies

will

be

unable

to

make

this

distinction

the

distinction

between

reliable

and

unreliable

data

On

the

other

hand

any

approach

that

simply

masks

out

inconsistent

data

-LRB-

the

first

two

tuples

in

this

example

-RRB-

will

lose

indefinite

information

present

in

inconsistent

databases

In

this

example

we

know

that

there

is

a

student

named

Jeremy

Burford

-LRB-

existential

information

-RRB-

and

that

Jeremy

Burford

lives

in

Los

Angeles

or

New

York

-LRB-

disjunctive

information

-RRB-

The

above

example

illustrates

the

need

to

modify

the

standard

notion

of

query

answer

in

the

context

of

inconsistent

databases

We

need

to

be

able

to

talk

about

query

answers

that

are

unaffected

by

integrity

violations

In

-LSB-

-RSB-

the

notion

of

consistent

query

answer

was

proposed

to

achieve

that

ob

jective

-LSB-

-RSB-

introduced

the

notion

of

repair

a

database

that

satisfies

the

integrity

constraints

and

is

minimally

different

from

the

original

database

A

consistent

answer

to

a

query

in

this

framework

is

an

answer

present

in

the

result

of

the

query

in

every

repair

Example

In

Example

there

are

two

repairs

corresponding

to

two

different

ways

of

restoring

the

consistency

either

the

first

or

the

second

tuple

is

deleted

If

a

query

asks

for

all

the

information

about

students

only

the

tuple

-LRB-

Linda

Kenner

Chicago

-RRB-

is

returned

as

a

consistent

answer

because

it

is

the

only

tuple

that

is

present

in

both

repairs

On

the

other

hand

if

a

query

asks

for

the

names

of

students

living

in

Los

Angeles

or

New

York

then

Jeremy

Burford

is

a

consistent

answer

The

framework

of

-LSB-

-RSB-

has

served

as

a

foundation

for

most

of

the

subsequent

work

in

the

area

of

querying

inconsistent

databases

-LSB-

-RSB-

-LRB-

see

-LSB-

-RSB-

for

a

survey

and

an

in-depth

discussion

-RRB-

The

work

presented

here

addresses

the

issue

of

computing

consistent

query

answers

for

projection-free

queries

and

denial

integrity

constraints

It

is

shown

in

-LSB-

-RSB-

that

this

task

can

be

done

in

polynomial

time

using

the

notion

of

conflict

hypergraph

that

succinctly

represents

all

the

integrity

violations

in

a

given

database

This

line

research

is

pursued

further

in

the

present

paper

The

main

contributions

of

this

paper

are

as

follows

A

complete

scalable

framework

for

computing

consistent

answers

to

projection-free

relational

algebra

queries

in

the

presence

of

denial

constraints

Our

approach

uses

a

relational

DBMS

as

a

backend

and

scales

up

to

large

databases

Novel

optimization

techniques

to

eliminate

redundant

DBMS

queries

Encouraging

experimental

results

that

compare

our

approach

with

an

approach

based

on

query

rewriting

and

estimate

the

overhead

of

computing

consistent

query

answers

No

comprehensive

results

of

this

kind

exist

in

the

literature

Because

our

query

language

includes

union

our

approach

can

extract

indefinite

disjunctive

information

present

in

an

inconsistent

database

-LRB-

see

Example

-RRB-

Moreover

consistent

query

answers

are

computed

in

polynomial

time

Other

existing

approaches

are

either

unable

to

handle

disjunction

in

queries

-LSB-

-RSB-

or

can

not

guarantee

polynomial

time

com-putability

of

consistent

query

answers

-LSB-

-RSB-

The

latter

is

due

to

the

fact

that

those

approaches

rely

on

the

computation

of

answers

sets

of

logic

programs

with

disjunction

and

negation

a

p

complete

problem

Only

the

approach

of

-LSB-

-RSB-

-LRB-

which

uses

query

rewriting

-RRB-

and

the

approach

presented

here

scale

up

to

large

databases

Related

research

is

further

discussed

in

Section

The

plan

of

the

paper

is

as

follows

In

Section

we

introduce

basic

concepts

In

Section

we

present

our

approach

to

computing

consistent

answers

to

projection-free

queries

and

describe

its

implementation

in

a

system

called

Hippo

In

Section

we

describe

several

techniques

for

eliminating

redundant

DBMS

queries

that

we

have

implemented

in

Hippo

In

Section

we

discuss

a

number

of

experiments

we

have

conducted

with

Hippo

and

query

rewriting

In

Section

we

briefly

discuss

related

work

Section

contains

conclusions

and

a

discussion

of

possible

future

research

directions

In

this

paper

we

work

in

the

relational

model

of

data

We

recall

that

a

database

schema

S

is

a

set

of

relation

names

with

attribute

names

and

types

An

instance

of

a

database

is

a

function

that

assigns

a

finite

set

of

tuples

to

each

relation

name

For

the

purposes

of

this

paper

we

consider

only

two

fixed

database

domains

N

-LRB-

natural

numbers

-RRB-

and

D

-LRB-

unin-terpreted

constants

-RRB-

We

also

use

the

natural

interpretation

over

N

of

binary

relational

symbols

and

we

assume

that

two

constants

are

equal

only

if

they

have

the

same

name

We

also

view

I

as

a

structure

for

the

first-order

language

over

the

vocabulary

consisting

of

symbols

of

S

and

standard

built-in

predicates

over

N

-LRB-

-RRB-

In

this

article

we

use

projection-free

-LRB-

free

-RRB-

relational

algebra

expressions

defined

using

the

following

grammar

E

R

-LRB-

E

-RRB-

E

E

E

E

E

E.

R

is

the

arity

of

the

relation

symbol

R

and

-LRB-

unless

specified

otherwise

-RRB-

for

the

sake

of

simplicity

we

assume

that

attribute

names

are

consecutive

natural

numbers

We

extend

this

to

expressions

i.e.

E

is

the

arity

of

the

expression

and

E.i

is

the

reference

to

the

i-th

column

resulting

from

the

expression

E

-LRB-

used

in

conditions

for

subexpressions

-RRB-

Morover

t

-LSB-

i

-RSB-

is

the

value

on

the

i-th

position

of

t

t

-LSB-

i

j

-RSB-

is

an

abbreviation

for

a

tuple

-LRB-

t

-LSB-

i

-RSB-

t

-LSB-

j

-RSB-

-RRB-

and

with

t

we

denote

the

length

of

the

tuple

t

We

say

that

a

tuple

t

is

compatible

with

an

expression

E

if

the

length

of

the

tuple

is

equal

to

the

arity

of

the

expression

i.e.

t

E

For

a

given

expression

E

QA

E

-LRB-

I

-RRB-

is

the

result

of

evaluating

E

in

the

database

instance

I

In

this

paper

we

use

only

the

set

semantics

of

relational

algebra

expressions

We

also

use

relational

calculus

queries

consisting

of

quantifier-free

first-order

formulas

which

may

be

open

-LRB-

having

free

variables

-RRB-

or

ground

In

fact

our

approach

can

handle

relational

algebra

queries

that

require

projection

as

long

as

they

can

be

translated

to

quantifier-free

relational

calculus

queries

That

's

why

we

can

deal

with

the

relational

algebra

query

corresponding

to

the

query

Student

-LRB-

X

LosAngeles

-RRB-

Student

-LRB-

X

NewYork

-RRB-

in

Example

We

also

occasionally

use

SQL

Repairs

and

consistent

query

answers

An

integrity

constraint

is

a

consistent

closed

first-order

formula

In

this

paper

we

consider

only

the

class

of

denial

integrity

constraints

of

the

form

x

x

k

-LSB-

R

i

-LRB-

x

-RRB-

R

i

k

-LRB-

x

k

-RRB-

-LRB-

x

x

k

-RRB-

-RSB-

-LRB-

-RRB-

where

is

a

boolean

expression

consisting

of

atomic

formulas

referring

to

built-in

predicates

The

number

k

is

called

the

arity

of

a

constraint

Note

that

for

example

functional

dependencies

and

exclusion

constraints

are

of

the

above

form

Below

we

give

another

example

Example

Consider

the

relation

Emp

with

attributes

Name

Salary

and

Manager

with

Name

being

the

primary

key

The

constraint

that

no

employee

can

have

a

salary

greater

that

that

of

her

manager

is

a

denial

constraint

n

s

m

s

m

-LSB-

Emp

-LRB-

n

s

m

-RRB-

Emp

-LRB-

m

s

m

-RRB-

s

s

-RSB-

Definition

-LRB-

Consistent

database

-RRB-

A

database

instance

I

is

consistent

with

a

set

of

integrity

constraints

C

if

I

C

-LRB-

i.e.

C

is

true

in

I

-RRB-

inconsistent

otherwise

Definition

For

a

given

database

instance

I

of

schema

S

its

set

of

facts

-LRB-

I

-RRB-

is

the

set

of

all

positive

facts

that

hold

in

this

database

-LRB-

I

-RRB-

-LCB-

R

-LRB-

t

-RRB-

R

S

t

I

-LRB-

R

-RRB-

-RCB-

Definition

-LRB-

Database

distance

-RRB-

Given

two

instances

I

and

I

of

the

same

database

the

distance

between

those

instances

-LRB-

I

I

-RRB-

is

the

symmetric

difference

between

sets

of

facts

of

those

instances

-LRB-

I

I

-RRB-

-LRB-

-LRB-

I

-RRB-

-LRB-

I

-RRB-

-RRB-

-LRB-

-LRB-

I

-RRB-

-LRB-

I

-RRB-

-RRB-

Definition

-LRB-

Proximity

relation

-RRB-

Given

three

instances

I

I

I

the

instance

I

is

closer

to

I

than

the

instance

I

if

the

distance

between

I

and

I

is

contained

in

the

distance

between

I

and

I

i.e.

I

I

I

-LRB-

I

I

-RRB-

-LRB-

I

I

-RRB-

Definition

-LRB-

Database

repair

-RRB-

For

a

given

instance

I

and

set

of

integrity

constraints

C

I

is

a

repair

of

I

w.r.t.

C

if

I

is

the

closest

instance

to

I

which

is

consistent

with

C

i.e.

I

C

and

I

is

I

minimal

among

the

instances

that

satisfy

C.

By

Rep

C

-LRB-

I

-RRB-

we

denote

the

set

of

all

repairs

of

I

with

respect

to

C

The

following

fact

captures

an

important

property

of

repairs

of

denial

constraints

each

repair

is

a

maximal

consistent

subset

of

the

database

Fact

If

C

consists

only

of

denial

constraints

then

I

Rep

C

-LRB-

I

-RRB-

-LRB-

I

-RRB-

-LRB-

I

-RRB-

Definition

-LRB-

Core

instance

-RRB-

For

a

given

instance

I

its

core

w.r.t

a

set

of

integrity

constraints

C

is

an

instance

Core

I

C

such

that

Core

I

C

-LRB-

R

-RRB-

I

Rep

C

-LRB-

I

-RRB-

I

-LRB-

R

-RRB-

For

any

relation

R

and

set

of

integrity

constraints

C

if

there

exists

a

relational

algebra

expression

R

C

such

that

that

for

any

instance

I

QA

R

C

-LRB-

I

-RRB-

Core

I

C

-LRB-

R

-RRB-

we

call

R

C

a

core

expression

of

the

relation

R

w.r.t

the

set

of

integrity

constraints

C.

Fact

If

C

is

a

set

of

denial

integrity

constraints

then

for

any

R

S

there

exists

a

core

expression

R

C

of

R

w.r.t

C.

Example

Suppose

we

have

a

table

P

-LRB-

A

B

-RRB-

with

a

functional

dependency

A

B

The

core

expression

for

P

in

SQL

is

SELECT

FROM

P

WHERE

NOT

EXISTS

-LRB-

SELECT

FROM

P

WHERE

AND

-RRB-

Having

defined

repairs

we

can

define

consistent

answers

to

queries

In

general

the

intuition

is

that

the

consistent

query

answer

is

an

answer

to

the

query

in

every

repair

In

this

paper

we

consider

consistent

answers

for

two

classes

of

queries

Definition

-LRB-

CQA

for

ground

queries

-RRB-

Given

a

database

instance

I

and

a

set

of

denial

integrity

constraints

C

we

say

that

true

-LRB-

resp

false

-RRB-

is

the

consistent

answer

to

a

ground

query

w.r.t.

C

in

I

and

we

write

I

C

if

in

every

repair

I

Rep

C

-LRB-

I

-RRB-

I

-LRB-

resp

I

-RRB-

Definition

-LRB-

CQA

for

relational

algebra

-RRB-

Given

a

database

instance

I

and

a

set

of

denial

integrity

constraints

C

the

set

of

consistent

answers

to

a

query

E

w.r.t.

C

in

I

is

defined

as

follows

CQA

E

C

-LRB-

I

-RRB-

I

Rep

C

-LRB-

I

-RRB-

QA

E

-LRB-

I

-RRB-

Conflict

hypergraphs

The

conflict

hypergraph

-LSB-

-RSB-

constitutes

a

compact

space-efficient

representation

of

all

repairs

of

a

given

database

instance

Note

that

this

representation

is

specifically

geared

toward

denial

constraints

Definition

-LRB-

Conflict

-RRB-

For

a

given

integrity

constraint

c

of

form

-LRB-

-RRB-

a

set

of

facts

-LCB-

R

i

-LRB-

t

-RRB-

R

i

k

-LRB-

t

k

-RRB-

-RCB-

where

t

j

I

-LRB-

R

i

j

-RRB-

is

a

conflict

in

a

database

instance

I

if

-LRB-

t

t

k

-RRB-

By

E

c

I

we

denote

the

set

of

all

conflicts

generated

by

the

integrity

constraint

c

in

I.

Definition

-LRB-

Conflict

hypergraph

-RRB-

For

a

given

set

of

integrity

constraints

C

and

a

database

instance

I

a

conflict

hypergraph

G

C

I

is

a

hypergraph

with

the

set

of

vertices

being

the

set

of

facts

from

the

instance

I

and

the

set

of

hyperedges

consisting

of

all

conflicts

generated

by

constraints

from

C

in

I

i.e.

G

C

I

-LRB-

V

I

E

C

I

-RRB-

where

V

I

-LRB-

I

-RRB-

and

E

C

I

cC

E

c

I

Definition

-LRB-

Maximal

independent

set

-RRB-

For

a

hypergraph

G

-LRB-

V

E

-RRB-

the

set

of

vertices

is

a

maximal

independent

set

if

it

is

a

maximal

set

that

contains

no

hyperedge

from

E.

Fact

Let

I

be

a

database

instance

and

C

a

set

of

denial

constraints

then

for

any

repair

I

Rep

C

-LRB-

I

-RRB-

-LRB-

I

-RRB-

is

a

maximal

independent

set

M

in

G

C

I

and

vice

versa

As

shown

in

the

following

example

in

case

of

denial

constraints

the

set

of

conflicts

can

be

defined

using

a

simple

query

Example

Suppose

we

have

a

table

P

-LRB-

A

B

-RRB-

with

a

functional

dependency

A

B

The

SQL

expression

for

selecting

all

conflicts

from

P

generated

by

the

functional

constraint

is

SELECT

FROM

P

P

WHERE

AND

Definition

-LRB-

Data

complexity

-RRB-

The

data

complexity

of

consistent

answers

to

ground

first-order

queries

is

the

complexity

of

determining

the

membership

in

the

set

D

C

-LCB-

I

I

C

-RCB-

where

is

a

fixed

ground

first-order

query

and

C

is

a

fixed

finite

set

of

integrity

constraints

We

note

that

for

a

fixed

set

of

integrity

constraints

the

conflict

hypergraph

is

of

polynomial

size

-LRB-

in

the

number

of

tuples

in

the

database

instance

-RRB-

We

review

here

the

algorithm

-LSB-

-RSB-

for

checking

the

consistency

of

ground

queries

in

the

presence

of

denial

constraints

and

then

show

how

to

use

it

to

answer

free

queries

relational

algebra

queries

which

which

correspond

to

open

quantifier-free

relational

calculus

queries

We

assume

here

that

we

work

with

a

set

of

integrity

constraints

consisting

only

of

denial

constraints

The

input

to

the

algorithm

consists

of

a

ground

quantifier-free

formula

a

set

of

integrity

constraints

C

and

a

database

instance

I

We

want

the

algorithm

to

answer

the

question

whether

I

C

Theorem

-LSB-

-RSB-

The

data

complexity

of

consistent

answers

to

quantifier-free

ground

queries

w.r.t

a

set

of

denial

constraints

is

in

P

The

proof

of

this

theorem

can

be

found

in

-LSB-

-RSB-

together

with

the

corresponding

algorithm

that

we

call

HProver

This

algorithm

takes

the

query

in

CNF

and

a

conflict

hypergraph

G

C

I

that

corresponds

to

the

database

instance

I

in

the

presence

of

integrity

constraints

C

The

first

step

of

Input

k

ground

input

formula

in

CNF

G

C

I

-LRB-

V

I

E

C

I

-RRB-

conflict

hypergraph

of

I

w.r.t.

C.

for

i

-LCB-

k

-RCB-

do

let

i

R

i

-LRB-

t

-RRB-

R

i

p

-LRB-

t

p

-RRB-

R

i

p

-LRB-

t

p

-RRB-

R

i

m

-LRB-

t

m

-RRB-

for

j

-LCB-

p

m

-RCB-

do

if

t

j

I

-LRB-

R

i

j

-RRB-

then

next

i

B

-LCB-

R

i

p

-LRB-

t

p

-RRB-

R

i

m

-LRB-

t

m

-RRB-

-RCB-

for

j

-LCB-

p

-RCB-

do

if

t

j

I

-LRB-

R

i

j

-RRB-

then

choose

e

j

-LCB-

e

E

C

I

R

i

j

-LRB-

t

j

-RRB-

e

-RCB-

nondeterm

B

B

-LRB-

e

j

-LCB-

R

i

j

-LRB-

t

j

-RRB-

-RCB-

-RRB-

if

B

is

independent

in

G

C

I

then

return

false

return

true

Figure

Algorithm

HProver

the

algorithm

reduces

the

task

of

determining

whether

true

is

the

consistent

answer

to

the

query

to

answering

the

same

question

for

every

conjunct

i

Then

each

formula

i

is

negated

and

the

rest

of

the

algorithm

attempts

to

find

a

repair

I

in

which

i

is

true

i.e.

in

which

t

j

I

-LRB-

R

i

j

-RRB-

for

-LRB-

j

p

m

-RRB-

t

j

I

-LRB-

R

i

j

-RRB-

for

-LRB-

j

p

-RRB-

Such

a

repair

corresponds

to

a

maximal

independent

set

M

in

the

conflict

hypergraph

such

that

every

of

R

i

p

-LRB-

t

p

-RRB-

R

i

m

-LRB-

t

m

-RRB-

is

an

element

of

M

of

R

i

-LRB-

t

-RRB-

R

i

p

-LRB-

t

p

-RRB-

is

an

element

of

M

If

the

algorithm

succeeds

in

building

an

independent

set

satisfying

the

properties

and

such

a

set

can

be

extended

to

a

maximal

one

which

also

satisfies

those

properties

That

means

that

there

is

a

repair

in

which

i

and

thus

also

is

true

If

the

algorithm

does

not

succeed

for

any

i

i

k

then

true

is

the

consistent

answer

to

The

condition

is

satisfied

by

simply

including

the

appropriate

facts

in

M

The

condition

is

satisfied

by

excluding

the

appropriate

facts

from

M

A

fact

can

be

excluded

if

it

is

not

in

-LRB-

I

-RRB-

or

if

it

belongs

to

a

hyperedge

whose

remaining

elements

are

already

in

M

Finding

an

envelope

Any

relational

algebra

expression

E

can

be

translated

to

a

corresponding

first-order

formula

E

-LRB-

x

-RRB-

in

a

standard

way

Since

we

consider

only

free

algebra

expressions

the

formula

E

-LRB-

x

-RRB-

is

quantifier-free

To

be

able

to

use

HProver

we

have

to

ground

this

formula

i.e.

find

an

appropriate

set

of

bindings

for

the

variables

in

the

formula

This

will

be

done

by

evaluating

an

envelope

query

over

the

database

An

envelope

query

should

satisfy

two

properties

-LRB-

-RRB-

it

should

return

a

superset

of

the

set

of

consistent

query

answers

for

every

database

instance

and

-LRB-

-RRB-

it

should

be

easily

constructible

from

the

original

query

The

result

of

evaluating

an

envelope

query

over

a

given

database

will

be

called

an

envelope

Suppose

K

E

is

an

envelope

query

for

a

query

E

We

have

that

CQA

E

C

-LRB-

I

-RRB-

-LCB-

t

QA

K

E

-LRB-

I

-RRB-

I

C

E

-LRB-

t

-RRB-

-RCB-

If

an

expression

E

does

not

use

the

difference

operator

-LRB-

and

thus

is

a

monotonic

expression

-RRB-

E

itself

is

an

envelope

query

as

stated

by

the

following

lemma

Lemma

For

any

monotonic

relational

expression

E

the

following

holds

CQA

E

C

-LRB-

I

-RRB-

QA

E

-LRB-

I

-RRB-

However

when

E

is

not

monotonic

then

the

set

of

consistent

query

answers

may

contain

tuples

not

contained

in

QA

E

-LRB-

I

-RRB-

That

kind

of

a

situation

is

shown

in

the

example

below

Example

Suppose

we

have

two

relations

R

-LRB-

A

B

-RRB-

and

S

-LRB-

A

B

C

D

-RRB-

and

we

have

functional

dependency

over

R

A

B

In

case

when

I

-LRB-

R

-RRB-

-LCB-

-LRB-

-RRB-

-LRB-

-RRB-

-RCB-

and

I

-LRB-

S

-RRB-

-LCB-

-LRB-

-RRB-

-RCB-

the

set

of

answers

to

the

query

E

S

-LRB-

R

-LRB-

A

B

-RRB-

B

B

R

-LRB-

A

B

-RRB-

-RRB-

is

while

the

set

of

consistent

query

answers

is

-LCB-

-LRB-

-RRB-

-RCB-

To

obtain

the

expression

for

an

envelope

we

define

two

operators

F

and

G

by

mutual

recursion

The

operator

F

defines

the

envelope

by

overestimating

the

set

of

consistent

answers

The

auxiliary

operator

G

underestimates

the

set

of

consistent

answers

Definition

We

define

the

operators

F

and

G

recursively

F

-LRB-

R

-RRB-

R

F

-LRB-

E

E

-RRB-

F

-LRB-

E

-RRB-

F

-LRB-

E

-RRB-

F

-LRB-

E

E

-RRB-

F

-LRB-

E

-RRB-

G

-LRB-

E

-RRB-

F

-LRB-

E

E

-RRB-

F

-LRB-

E

-RRB-

F

-LRB-

E

-RRB-

F

-LRB-

-LRB-

E

-RRB-

-RRB-

-LRB-

F

-LRB-

E

-RRB-

-RRB-

G

-LRB-

R

-RRB-

R

C

G

-LRB-

E

E

-RRB-

G

-LRB-

E

-RRB-

G

-LRB-

E

-RRB-

G

-LRB-

E

E

-RRB-

G

-LRB-

E

-RRB-

F

-LRB-

E

-RRB-

G

-LRB-

E

E

-RRB-

G

-LRB-

E

-RRB-

G

-LRB-

E

-RRB-

G

-LRB-

-LRB-

E

-RRB-

-RRB-

-LRB-

G

-LRB-

E

-RRB-

-RRB-

Because

C

consist

only

of

denial

constraints

Fact

guarantees

that

the

expression

R

C

exists

and

therefore

the

operators

are

well

defined

The

pair

of

operators

-LRB-

F

G

-RRB-

has

the

following

properties

Lemma

For

any

free

relational

algebra

expression

E

QA

G

-LRB-

E

-RRB-

-LRB-

I

-RRB-

QA

E

-LRB-

I

-RRB-

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

and

CQA

G

-LRB-

E

-RRB-

C

-LRB-

I

-RRB-

CQA

E

C

-LRB-

I

-RRB-

CQA

F

-LRB-

E

-RRB-

C

-LRB-

I

-RRB-

Lemma

For

any

free

relational

algebra

expression

E

I

Rep

C

-LRB-

I

-RRB-

QA

G

-LRB-

E

-RRB-

-LRB-

I

-RRB-

QA

E

-LRB-

I

-RRB-

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

With

those

two

lemmas

we

can

prove

the

following

theorem

Theorem

If

C

contains

only

denial

constraints

then

for

any

free

relational

algebra

expression

E

the

following

holds

for

every

database

instance

I

QA

G

-LRB-

E

-RRB-

-LRB-

I

-RRB-

CQA

E

C

-LRB-

I

-RRB-

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

The

system

Hippo

We

have

implemented

a

system

called

Hippo

for

finding

consistent

answers

to

free

relational

algebra

queries

The

data

is

stored

in

an

RDBMS

-LRB-

in

our

case

PostgreSQL

-RRB-

The

flow

of

data

in

Hippo

is

shown

in

Figure

The

only

E

Estimating

F

-LRB-

E

-RRB-

Evaluation

Conflict

Detection

DB

Translation

E

Envelope

Conflict

Hypergraph

Grounding

HProver

Answer

Set

IC

Figure

Data

flow

in

Hippo

output

of

this

system

is

the

Answer

Set

consisting

of

the

consistent

answers

to

the

input

query

E

with

respect

to

a

set

of

integrity

constraints

IC

in

the

database

instance

DB

Before

processing

any

input

query

the

system

performs

Conflict

Detection

and

creates

the

Conflict

Hypergraph

We

assume

that

the

number

of

conflicts

is

small

enough

to

allow

us

to

store

the

hypergraph

in

main

memory

We

keep

in

main

memory

only

the

set

of

hyperedges

corresponding

to

conflicts

in

database

The

set

of

all

the

vertices

represents

the

entire

contents

of

the

database

and

thus

may

be

too

big

to

fit

in

main

memory

In

this

way

we

guarantee

that

our

approach

is

scalable

The

processing

of

a

query

E

consists

of

Estimating

it

to

an

envelope

query

F

-LRB-

E

-RRB-

that

after

Evaluation

b

y

an

RDBMS

gives

us

the

Envelope

Also

the

system

performs

Translation

of

the

input

query

E

to

a

corresponding

first-order

logic

formula

E

Now

for

every

tuple

from

the

Envelope

we

perform

Grounding

of

E

Having

now

a

first-order

ground

query

we

can

check

if

true

is

the

consistent

answer

to

this

query

using

HProver

Depending

on

the

result

of

this

check

we

return

the

tuple

or

not

It

's

important

to

notice

here

that

because

the

hypergraph

is

stored

in

main

memory

HProver

does

n't

need

any

immediate

knowledge

of

the

integrity

constraints

-LRB-

no

arrow

from

IC

to

HProver

-RRB-

This

is

because

in

HProver

the

independence

of

constructed

sets

B

is

being

checked

only

for

sets

of

vertices

that

are

contained

in

the

database

and

if

such

vertices

are

in

any

conflict

it

is

registered

in

the

hypergraph

HProver

makes

however

database

accesses

to

check

tuple

membership

in

database

relations

The

previous

section

showed

how

to

build

a

system

for

computing

consistent

query

answers

But

even

though

we

have

decided

to

store

the

conflict

hypergraph

in

main

memory

we

still

have

to

perform

tuple

membership

checks

-LRB-

steps

and

in

the

HProver

algorithm

-RRB-

To

check

if

a

tuple

is

present

in

a

given

table

we

execute

a

simple

membership

query

For

every

tuple

from

the

envelope

we

have

to

perform

several

tuple

checks

-LRB-

depending

on

the

complexity

of

the

query

-RRB-

Executing

any

query

is

usually

a

costly

operation

in

the

database

context

Therefore

tuple

membership

checks

are

a

significant

factor

in

the

algorithm

execution

time

In

this

section

we

address

the

problem

of

eliminating

tuple

membership

checks

We

propose

two

improvements

The

first

infers

information

about

the

tuples

present

in

the

database

from

the

current

envelope

tuple

That

makes

it

possible

to

answer

some

tuple

checks

without

interrogating

the

database

The

second

supplements

the

first

by

extending

the

envelope

expression

so

that

we

can

find

the

results

of

all

relevant

tuple

checks

without

executing

any

membership

query

Knowledge

gathering

In

this

section

we

address

the

problem

of

answering

tuple

checks

Definition

-LRB-

Relevant

facts

-RRB-

For

a

given

free

expression

E

and

a

tuple

t

compatible

with

E

the

set

TC

-LRB-

E

t

-RRB-

of

relevant

facts

is

defined

recursively

TC

-LRB-

R

t

-RRB-

-LCB-

R

-LRB-

t

-RRB-

-RCB-

TC

-LRB-

E

E

t

-RRB-

TC

-LRB-

E

t

-RRB-

TC

-LRB-

E

t

-RRB-

TC

-LRB-

E

E

t

-RRB-

TC

-LRB-

E

t

-RRB-

TC

-LRB-

E

t

-RRB-

TC

-LRB-

E

E

-LRB-

t

t

-RRB-

-RRB-

TC

-LRB-

E

t

-RRB-

TC

-LRB-

E

t

-RRB-

TC

-LRB-

-LRB-

E

-RRB-

t

-RRB-

TC

-LRB-

E

t

-RRB-

The

set

of

facts

TC

-LRB-

E

t

-RRB-

consists

of

all

facts

that

HProver

may

need

when

working

with

the

query

E

-LRB-

t

-RRB-

-LRB-

we

conjecture

that

the

same

set

of

facts

will

be

needed

by

any

practical

checker

of

consistent

query

answers

for

quantifier-free

queries

-RRB-

In

the

following

example

we

show

that

the

tuple

t

itself

may

carry

information

that

can

be

used

to

derive

some

relevant

facts

Example

Recall

that

relation

attributes

are

named

by

natural

numbers

Assume

that

we

have

two

tables

R

-LRB-

-RRB-

P

-LRB-

-RRB-

and

a

query

E

F

-LRB-

E

-RRB-

a

-LRB-

R

-LRB-

RP

-RRB-

-RRB-

Suppose

that

a

tuple

t

-LRB-

a

b

c

d

-RRB-

is

the

only

result

of

the

evaluation

of

F

-LRB-

E

-RRB-

in

a

database

instance

I

The

set

of

relevant

facts

is

TC

-LRB-

E

t

-RRB-

-LCB-

R

-LRB-

a

b

-RRB-

R

-LRB-

c

d

-RRB-

P

-LRB-

c

d

-RRB-

-RCB-

A

natural

consequence

of

the

semantics

of

relational

algebra

expressions

is

that

t

QA

a

-LRB-

R

-LRB-

RP

-RRB-

-RRB-

-LRB-

I

-RRB-

implies

-LRB-

a

b

-RRB-

I

-LRB-

R

-RRB-

We

can

use

this

information

to

avoid

performing

some

membership

queries

At

the

same

time

the

tuple

t

itself

does

n't

carry

enough

information

to

decide

whether

-LRB-

c

d

-RRB-

belongs

to

either

I

-LRB-

R

-RRB-

I

-LRB-

P

-RRB-

or

both

of

them

We

call

the

process

of

inferring

the

information

from

result

of

the

evaluation

of

a

query

knowledge

gathering

Formally

we

define

the

set

of

derived

facts

in

the

following

way

Definition

-LRB-

Knowledge

gathering

-RRB-

For

a

given

free

expression

E

and

a

tuple

t

compatible

with

E

we

define

the

set

KG

recursively

KG

-LRB-

R

t

-RRB-

-LCB-

R

-LRB-

t

-RRB-

-RCB-

KG

-LRB-

E

E

t

-RRB-

KG

-LRB-

E

t

-RRB-

KG

-LRB-

E

t

-RRB-

KG

-LRB-

E

E

t

-RRB-

KG

-LRB-

E

t

-RRB-

KG

-LRB-

-LRB-

E

-RRB-

t

-RRB-

KG

-LRB-

E

t

-RRB-

KG

-LRB-

E

E

-LRB-

t

t

-RRB-

-RRB-

KG

-LRB-

E

t

-RRB-

KG

-LRB-

E

t

-RRB-

We

note

here

that

the

cardinality

of

the

set

of

facts

inferred

with

KG

is

linear

in

the

size

of

the

query

and

does

n't

depend

on

the

value

of

the

tuple

t

Now

we

state

the

main

property

of

KG

Theorem

-LRB-

Soundness

of

KG

-RRB-

Given

a

database

instance

I

and

a

free

expression

E

t

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

R

-LRB-

t

-RRB-

TC

-LRB-

E

t

-RRB-

R

-LRB-

t

-RRB-

KG

-LRB-

E

t

-RRB-

I

R

-LRB-

t

-RRB-

Knowledge

gathering

is

also

complete

in

the

case

of

-LCB-

-RCB-

expressions

i.e.

it

derives

all

relevant

facts

that

hold

in

the

database

I.

Theorem

-LRB-

Completeness

of

KG

for

-LCB-

-RCB-

-RRB-

Given

a

database

I

and

any

-LCB-

-RCB-

query

E.

t

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

R

-LRB-

t

-RRB-

TC

-LRB-

E

t

-RRB-

I

R

-LRB-

t

-RRB-

R

-LRB-

t

-RRB-

KG

-LRB-

E

t

-RRB-

Extended

knowledge

gathering

In

general

when

the

expression

translates

to

a

disjunctive

query

we

need

to

extend

the

query

so

that

the

resulting

tuple

carries

some

additional

information

allowing

us

to

derive

all

relevant

facts

The

extended

approach

described

in

detail

below

is

illustrated

first

by

the

following

example

Example

For

the

previously

considered

expression

E

a

-LRB-

R

-LRB-

R

P

-RRB-

-RRB-

the

extended

approach

constructs

the

expression

a

-LRB-

R

-LRB-

R

P

-RRB-

-RRB-

R

P

where

is

the

left

outer

join

operator

Suppose

now

I

-LRB-

R

-RRB-

-LCB-

-LRB-

a

b

-RRB-

-LRB-

e

f

-RRB-

-RCB-

and

I

-LRB-

P

-RRB-

-LCB-

-LRB-

c

d

-RRB-

-LRB-

e

f

-RRB-

-RCB-

Then

the

evaluation

of

the

extended

envelope

expression

yields

the

following

a

-LRB-

R

-LRB-

R

P

-RRB-

-RRB-

R

P

a

b

a

b

a

b

a

b

c

d

c

d

a

b

e

f

e

f

e

f

Now

consider

the

tuple

-LRB-

a

b

c

d

c

d

-RRB-

We

can

decompose

it

into

two

parts

-LRB-

a

b

c

d

-RRB-

and

-LRB-

c

d

-RRB-

The

first

part

is

simply

the

tuple

from

the

envelope

F

-LRB-

E

-RRB-

and

it

can

be

used

to

infer

the

fact

R

-LRB-

a

b

-RRB-

The

second

part

allows

us

to

make

two

other

important

inferences

Namely

-LRB-

c

d

-RRB-

I

-LRB-

R

-RRB-

and

-LRB-

c

d

-RRB-

I

-LRB-

P

-RRB-

Our

goal

is

to

minimally

extend

the

expression

so

that

we

can

derive

all

relevant

facts

In

order

to

find

what

information

is

not

guaranteed

to

be

gathered

from

evaluation

of

the

envelope

expression

we

generalize

the

definitions

of

KG

and

TC

to

non-ground

tuples

consisting

of

distinct

variables

For

clarity

we

simplify

the

notion

of

the

outer

join

condition

When

writing

S

T

we

mean

S

S.

T.

T.

T

and

we

assume

the

left

join

operator

is

left

associative

Definition

-LRB-

Complementary

set

-RRB-

For

a

given

free

expression

E

the

complementary

set

-LRB-

E

-RRB-

is

defined

as

follows

-LRB-

E

-RRB-

TC

-LRB-

E

x

-RRB-

KG

-LRB-

E

x

-RRB-

where

x

-LRB-

x

x

E

-RRB-

Example

Taking

again

under

consideration

the

expression

E

a

-LRB-

R

-LRB-

R

P

-RRB-

-RRB-

and

x

-LRB-

x

x

-RRB-

we

have

TC

-LRB-

E

x

-RRB-

-LCB-

R

-LRB-

x

x

-RRB-

P

-LRB-

x

x

-RRB-

R

-LRB-

x

x

-RRB-

-RCB-

KG

-LRB-

E

x

-RRB-

-LCB-

R

-LRB-

x

x

-RRB-

-RCB-

R

-LRB-

x

x

-RRB-

TC

-LRB-

E

-RRB-

means

that

for

any

tuple

-LRB-

t

t

t

t

-RRB-

from

the

evaluation

of

the

envelope

expression

for

E

HProver

may

perform

the

tuple

check

R

-LRB-

t

t

-RRB-

We

have

also

R

-LRB-

x

x

-RRB-

KG

-LRB-

E

-RRB-

and

therefore

we

are

able

to

answer

this

check

using

knowledge

gathering

On

the

other

hand

R

-LRB-

x

x

-RRB-

TC

-LRB-

E

-RRB-

means

that

for

HProver

may

perform

a

tuple

check

R

-LRB-

t

t

-RRB-

Since

we

do

n't

have

that

R

-LRB-

x

x

-RRB-

KG

-LRB-

E

-RRB-

we

can

not

guarantee

that

we

can

answer

tuple

checks

R

-LRB-

t

t

-RRB-

without

executing

a

membership

query

on

the

database

even

though

we

are

able

to

answer

tuple

checks

R

-LRB-

t

t

-RRB-

The

complementary

set

for

the

discussed

expression

is

-LRB-

E

-RRB-

-LCB-

R

-LRB-

x

x

-RRB-

P

-LRB-

x

x

-RRB-

-RCB-

Analogous

examples

can

be

used

to

show

that

the

simple

knowledge

gathering

is

not

sufficient

to

avoid

membership

checks

when

processing

expressions

with

the

difference

operator

Next

we

extend

the

envelope

expression

so

that

it

evaluation

provides

us

with

all

information

sufficient

to

answer

the

tuple

checks

Definition

-LRB-

Extended

envelope

expression

-RRB-

For

a

given

free

expression

E

the

extended

envelope

expression

is

defined

as

follows

H

-LRB-

E

-RRB-

F

-LRB-

E

-RRB-

R

j

E.

-LRB-

i

-RRB-

R.j

R

-LRB-

x

i

x

i

R

-RRB-

-LRB-

E

-RRB-

R

The

notation

means

that

we

have

as

many

outer

joins

as

there

are

elements

in

-LRB-

E

-RRB-

They

can

appear

in

any

order

We

also

define

the

following

auxiliary

expression

S

-LRB-

E

-RRB-

E

R

-LRB-

x

i

x

i

R

-RRB-

-LRB-

E

-RRB-

R.

For

both

H

-LRB-

E

-RRB-

and

S

-LRB-

E

-RRB-

the

elements

of

-LRB-

E

-RRB-

need

to

be

considered

in

the

same

order

Using

outer

joins

results

in

a

natural

one-to-one

correspondence

between

the

tuples

from

the

evaluation

of

the

extended

envelope

expression

and

the

tuples

from

the

original

envelope

Fact

For

a

given

database

instance

I

and

free

expression

E

the

map

t

t

-LSB-

E

-RSB-

is

a

one-to-one

map

of

QA

H

-LRB-

E

-RRB-

-LRB-

I

-RRB-

onto

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

Extending

knowledge

gathering

to

tuples

KG

-LRB-

R

-LRB-

-RRB-

-RRB-

allows

us

to

state

that

using

the

extended

envelope

expression

we

can

determine

correctly

all

relevant

facts

without

querying

the

database

Theorem

-LRB-

Soundness

completeness

of

ext.

KG

-RRB-

For

any

database

instance

I

and

a

free

expression

E

the

following

holds

t

QA

H

-LRB-

E

-RRB-

-LRB-

I

-RRB-

R

-LRB-

t

-RRB-

TC

-LRB-

E

t

-LSB-

E

-RSB-

-RRB-

R

-LRB-

t

-RRB-

KG

-LRB-

S

-LRB-

E

-RRB-

t

-RRB-

I

R

-LRB-

t

-RRB-

We

note

that

in

the

case

of

-LCB-

-RCB-

expressions

this

approach

does

n't

unnecessarily

extend

the

expression

Other

possibilities

of

optimizations

Negative

knowledge

gathering

Knowledge

gathering

KG

-LRB-

as

defined

in

Section

-RRB-

is

complete

only

for

queries

that

translate

to

a

conjunction

of

positive

literals

However

it

is

possible

to

come

up

with

a

construction

that

will

be

complete

for

queries

that

translate

to

a

conjunction

of

positive

as

well

as

negative

literals

The

following

example

presents

this

idea

Example

Suppose

we

have

tables

R

-LRB-

-RRB-

and

P

-LRB-

-RRB-

and

a

set

of

constraints

C.

For

the

query

E

R

P

we

have

F

-LRB-

E

-RRB-

R

P

C

Take

any

tuple

t

QA

F

-LRB-

E

-RRB-

-LRB-

I

-RRB-

for

some

instance

I

We

can

easily

conclude

that

t

I

-LRB-

R

-RRB-

Also

we

can

say

that

t

QA

P

C

-LRB-

I

-RRB-

Having

this

and

hypergraph

G

C

I

-LRB-

V

I

E

C

I

-RRB-

we

can

easily

find

if

t

I

-LRB-

P

-RRB-

Namely

if

there

exists

an

edge

e

E

C

I

that

P

-LRB-

t

-RRB-

e

then

t

I

-LRB-

P

-RRB-

And

if

the

vertex

P

-LRB-

t

-RRB-

is

not

involved

in

any

conflict

in

E

then

t

I

-LRB-

P

-RRB-

Reasoning

of

that

sort

can

not

be

applied

to

a

query

E

R

R

P

P

Given

a

tuple

t

-LRB-

t

t

-RRB-

from

the

envelope

we

know

that

t

t

R

but

the

fact

-LRB-

t

t

-RRB-

QA

P

C

P

C

-LRB-

I

-RRB-

does

n't

imply

that

t

QA

P

C

-LRB-

I

-RRB-

or

t

QA

P

C

-LRB-

I

-RRB-

And

therefore

we

are

not

able

to

find

if

t

I

-LRB-

P

-RRB-

or

t

I

-LRB-

P

-RRB-

This

mechanism

has

n't

been

included

in

the

tested

implementation

yet

Implementing

only

positive

knowledge

gathering

allows

us

to

better

observe

the

benefits

of

extending

the

envelope

expression

We

notice

here

that

the

query

rewriting

approach

to

computing

consistent

query

answers

described

in

-LSB-

-RSB-

works

also

only

for

queries

that

are

conjunctions

of

literals

However

as

shown

below

our

approach

leads

to

faster

computation

of

consistent

answers

than

query

rewriting

Intersection

Another

possible

venue

of

optimization

comes

from

directly

implementing

derived

operators

of

relational

algebra

For

example

for

intersection

the

appropriate

extensions

of

the

operators

F

and

G

are

very

simple

F

-LRB-

E

E

-RRB-

F

-LRB-

E

-RRB-

F

-LRB-

E

-RRB-

G

-LRB-

E

E

-RRB-

G

-LRB-

E

-RRB-

G

-LRB-

E

-RRB-

Now

R

P

is

equivalent

to

R

-LRB-

R

P

-RRB-

b

ut

F

-LRB-

R

P

-RRB-

R

P

is

not

equivalent

to

F

-LRB-

R

-LRB-

R

P

-RRB-

-RRB-

R

-LRB-

R

C

P

-RRB-

Thus

the

envelope

constructed

by

the

operator

F

becomes

sensitive

to

the

way

the

original

query

is

formulated

Among

available

methods

for

computing

consistent

query

answers

only

the

query

rewriting

technique

-LSB-

-RSB-

seems

to

be

feasible

for

large

databases

This

is

why

in

this

work

we

compare

the

following

engines

SQL

An

engine

that

executes

the

given

query

on

the

underlying

RDBMS

and

returns

the

query

result

This

method

does

n't

return

consistent

query

answers

but

provides

a

baseline

to

observe

the

overhead

of

computing

consistent

query

answers

using

the

proposed

methods

QR

Using

the

SQL

engine

we

execute

the

rewritten

query

constructed

as

decribed

in

-LSB-

-RSB-

More

details

on

this

approach

can

be

found

in

Section

KG

This

method

constructs

the

basic

envelope

expression

and

uses

knowledge

gathering

as

described

in

Section

ExtKG

This

engine

constructs

the

extended

envelope

expression

-LRB-

Section

-RRB-

and

uses

extended

knowledge

gathering

Generating

test

data

Every

test

was

performed

with

the

database

containing

two

tables

P

and

Q

both

having

three

attributes

X

Y

Z.

For

the

constraints

we

took

a

functional

dependency

X

Z

in

each

table

The

test

databases

had

the

following

parameters

n

the

number

of

base

tuples

in

each

table

m

the

number

of

additional

conflicting

tuples

and

had

both

tables

constructed

in

the

following

way

Insert

n

different

base

tuples

with

X

and

Z

being

equal

and

taking

subsequent

values

n

and

Y

being

randomly

drawn

from

the

set

-LCB-

-RCB-

Insert

m

different

conflicting

tuples

with

X

taking

subsequent

values

-LCB-

n/m

n/m

-LRB-

m

-RRB-

n/m

-RCB-

Z

X

and

Y

being

randomly

drawn

from

the

set

-LCB-

-RCB-

In

addition

we

define

auxiliary

tables

-LRB-

P

core

and

Q

core

-RRB-

containing

only

non-conflicting

tuples

from

the

base

tables

-LRB-

resp

P

and

Q

-RRB-

Those

table

were

used

as

materialized

views

of

the

core

expressions

-LRB-

P

C

and

Q

C

-RRB-

Example

We

show

how

a

table

P

with

n

and

m

can

be

generated

First

we

insert

the

base

tuples

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

into

P

Then

we

insert

the

following

conflicting

tuples

-LRB-

-RRB-

-LRB-

-RRB-

into

P

P

core

will

hold

the

following

tuples

-LRB-

-RRB-

-LRB-

-RRB-

In

every

table

constructed

in

such

a

way

the

number

of

tuples

is

n

m

and

the

number

of

conflicts

is

m.

The

environment

The

implementation

is

done

in

using

PostgreSQL

-LRB-

version

-RRB-

as

the

relational

backend

All

test

have

been

performed

on

a

PC

with

a

GHz

AMD

Athlon

processor

under

SuSE

Linux

-LRB-

kernel

ver

-RRB-

using

Sun

JVM

Test

results

Testing

a

query

with

a

given

engine

consisted

of

computing

the

consistent

answers

to

the

query

and

then

iterating

over

the

results

Iteration

over

the

result

is

necessary

as

the

subsequent

elements

of

the

consistent

query

answer

set

are

computed

by

Hippo

in

a

lazy

manner

-LRB-

this

allows

us

to

process

results

bigger

than

the

available

main

memory

-RRB-

Every

test

has

been

repeated

three

times

and

the

median

taken

Finally

we

note

that

the

cost

of

computing

the

conflict

hypergraph

which

is

incurred

only

once

per

session

is

ignored

while

estimating

the

time

of

the

query

evaluation

We

take

a

closer

look

at

the

time

required

for

hypergraph

construction

in

Section

Simple

queries

We

first

compared

performance

of

different

engines

on

simple

queries

join

union

and

difference

Because

we

performed

the

tests

for

large

databases

we

added

a

range

selection

to

the

given

query

to

obtain

small

query

results

factoring

out

the

time

necessary

to

write

the

outputs

As

parameters

in

the

experiments

we

considered

the

database

size

the

conflict

percentage

and

the

estimated

result

size

Figure

shows

the

execution

time

for

join

as

a

function

of

the

size

of

the

database

In

the

case

of

-LCB-

-RCB-

expressions

-LRB-

thus

also

joins

-RRB-

the

execution

times

of

KG

ExtKG

and

SQL

are

essentially

identical

Since

no

membership

queries

have

to

be

performed

it

means

that

for

simple

queries

the

work

done

by

HProver

for

all

tuples

is

practically

negligible

X

-LRB-

P

X

Q

-RRB-

Time

-LRB-

sec

-RRB-

Database

size

-LRB-

increases

in

-RRB-

SQL

QR

KG

ExtKG

Conflicts

Figure

Execution

time

for

join

Figure

contains

the

results

for

union

It

shows

that

basic

knowledge

gathering

KG

is

not

sufficient

to

efficiently

handle

union

The

cost

of

performing

membership

queries

for

all

tuples

is

very

large

Note

that

query

rewriting

is

not

applicable

to

union

queries

Figure

contains

the

results

for

set

difference

-LRB-

the

execution

time

for

KG

was

relatively

much

larger

than

values

of

other

solutions

and

in

order

to

increase

readability

it

has

not

been

included

on

this

figure

-RRB-

Here

the

execution

time

is

a

function

of

the

percentage

of

conflicts

We

note

that

ExtKG

performs

as

well

as

QR

and

both

are

approximately

twice

slower

than

SQL

Except

when

using

SQL

engine

X

-LRB-

P

Q

-RRB-

Time

-LRB-

sec

-RRB-

Database

size

-LRB-

increases

in

-RRB-

SQL

KG

ExtKG

Conflicts

Figure

Execution

time

for

union

X

-LRB-

P

Q

-RRB-

Time

-LRB-

sec

-RRB-

Conflicts

-LRB-

-RRB-

SQL

QR

ExtKG

DB

size

Figure

Execution

time

for

difference

Complex

queries

In

order

to

estimate

the

cost

of

extending

the

envelope

we

considered

a

complex

union

query

X

d

-LRB-

P

X

Q

X

P

X

Q

Q

X

P

X

Q

X

P

-RRB-

with

d

being

a

parameter

that

will

allow

us

to

control

the

number

of

tuples

processed

by

each

engine

To

assure

no

membership

queries

will

be

performed

we

have

to

add

outer

joins

The

main

goal

was

to

compare

two

versions

of

knowledge

gathering

KG

and

ExtKG

We

have

also

included

the

results

for

SQL

-LRB-

It

should

be

noted

here

that

this

query

has

common

subexpressions

and

RDBMS

might

use

this

to

optimize

the

query

evaluation

plan

PostgreSQL

however

does

not

perform

this

optimization

-RRB-

As

we

can

see

in

Figure

KG

outperforms

ExtKG

only

in

the

case

when

the

number

of

processed

tuples

is

small

As

the

result

size

increases

the

execution

time

of

ExtKG

grows

significantly

slower

than

that

of

KG

We

notice

also

that

ExtKG

needs

times

more

time

than

SQL

but

the

execution

times

of

both

grow

in

a

similar

fashion

Hypergraph

computation

The

time

of

constructing

the

hypergraph

is

presented

on

Figure

-RRB-

It

depends

on

the

total

number

of

conflicts

and

the

size

of

the

database

It

should

be

noticed

here

that

the

time

of

hypergraph

construction

consists

mainly

of

the

execution

time

of

conflict

detection

queries

Therefore

the

time

of

hypergraph

computation

depends

also

on

the

number

of

integrity

constraints

and

their

arity

X

d

-LRB-

P

X

Q

X

P

X

Q

Q

X

P

X

Q

X

P

-RRB-

Time

-LRB-

sec

-RRB-

Result

size

estimation

-LRB-

d

tuples

-RRB-

SQL

KG

ExtKG

DB

size

Conflicts

Figure

Impact

of

the

result

size

Time

-LRB-

sec

-RRB-

Database

size

-LRB-

increments

in

-RRB-

Conflicts

Figure

Hypergraph

computation

time

The

discussion

of

related

work

here

is

very

brief

and

focuses

mainly

on

the

most

recent

research

For

a

comprehensive

discussion

please

see

-LSB-

-RSB-

Bry

-LSB-

-RSB-

was

the

first

to

note

that

the

standard

notion

of

query

answer

needs

to

be

modified

in

the

context

of

inconsistent

databases

and

to

propose

the

notion

of

a

consistent

query

answer

Bry

's

definition

of

consistent

query

answer

is

based

on

provability

in

minimal

logic

and

expresses

the

intuition

that

the

part

of

the

database

instance

involved

in

an

integrity

violation

should

not

be

involved

in

the

derivation

of

consistent

query

answers

This

is

not

quite

satisfactory

as

one

would

like

to

have

a

semantic

model-theoretic

notion

of

consistent

query

answer

that

parallels

that

of

the

standard

notion

of

query

answer

in

relational

databases

Moreover

the

data

involved

in

an

integrity

violation

is

not

entirely

useless

and

reliable

indefinite

information

can

often

be

extracted

from

it

as

seen

in

Example

Query

rewriting

-LSB-

-RSB-

rewrites

the

original

query

Q

to

another

query

Q

with

the

property

that

the

set

of

all

the

answers

to

Q

in

the

original

database

is

equal

to

the

set

of

consistent

answers

to

Q

in

that

database

When

applicable

this

approach

provides

an

easy

way

to

compute

consistent

query

answers

as

the

rewritten

query

Q

can

typically

be

evaluated

using

the

same

query

engine

as

the

query

Q.

Because

the

query

Q

is

rewritten

independently

of

the

database

the

existence

of

a

rewriting

shows

that

requesting

consistent

query

answers

instead

of

the

regular

ones

does

not

increase

data

complexity

However

query

rewriting

has

been

found

to

apply

only

to

restricted

classes

of

queries

the

-LCB-

-RCB-

subset

-LSB-

-RSB-

or

the

-LCB-

-RCB-

subset

-LSB-

-RSB-

of

the

relational

algebra

No

method

is

presently

known

to

rewrite

queries

with

projection

considered

together

with

the

binary

operators

or

union

Also

the

class

of

constraints

is

limited

to

binary

universal

constraints

-LSB-

-RSB-

or

single

functional

dependencies

-LSB-

-RSB-

The

line

of

research

from

-LSB-

-RSB-

is

con-tinued

in

-LSB-

-RSB-

where

a

class

of

tractable

conjunctive

queries

based

on

generalized

perfect

matching

is

identified

It

is

proved

that

the

consistent

answers

to

queries

in

this

class

can

not

be

obtained

by

query

rewriting

We

note

here

that

the

nonexistence

of

query

rewriting

for

conjunctive

queries

follows

also

from

the

fact

that

computing

consistent

query

answers

for

such

queries

is

a

co-NP-complete

problem

-LSB-

-RSB-

This

is

because

the

rewritten

query

is

first-order

and

thus

can

be

evaluated

in

AC

while

known

NP-complete

problems

like

SAT

are

not

in

AC

Several

different

approaches

have

been

developed

to

specify

all

repairs

of

a

database

as

a

logic

program

with

disjunction

and

classical

negation

-LSB-

-RSB-

Such

a

program

can

then

be

evaluated

using

an

existing

system

like

dlv

-LSB-

-RSB-

These

approaches

have

the

advantage

of

generality

as

typically

arbitrary

first-order

queries

and

universal

constraints

-LRB-

or

even

some

referential

integrity

constraints

-LSB-

-RSB-

-RRB-

can

be

handled

However

the

generality

comes

at

a

price

The

classes

of

logic

programs

used

are

p

complete

Therefore

the

approaches

based

on

logic

programming

are

unlikely

to

work

for

large

databases

The

paper

-LSB-

-RSB-

proposes

several

optimizations

that

are

applicable

to

logic

programming

approaches

One

is

localization

of

conflict

resolution

another

encoding

tuple

membership

in

individual

repairs

using

bit-vectors

which

makes

possible

efficient

computation

of

consistent

query

answers

using

bitwise

operators

However

it

is

known

that

even

in

the

presence

of

one

functional

dependency

there

may

be

exponentially

many

repairs

-LSB-

-RSB-

With

only

tuples

involved

in

conflicts

the

number

of

repairs

may

exceed

It

is

clearly

impractical

to

efficiently

manipulate

bit-vectors

of

that

size

-LSB-

-RSB-

describes

several

possible

definitions

of

repair

including

Definition

and

analyzes

the

complexity

of

computing

consistent

query

answers

under

those

definitions

Key

and

inclusion

dependencies

are

considered

The

computational

approaches

proposed

are

based

on

combinations

of

repair

enumeration

and

chase

computation

-LSB-

-RSB-

New

tractability

results

are

obtained

for

classes

of

databases

that

satisfy

key

constraints

but

may

violate

inclusion

dependencies

Presently

our

approach

requires

that

the

integrated

database

be

materialized

at

a

single

site

It

remains

to

be

seen

if

it

can

be

generalized

to

a

scenario

where

data

is

pulled

from

different

sites

during

the

evaluation

of

queries

rewritten

using

for

example

the

LAV

approach

-LSB-

-RSB-

This

problem

has

been

considered

in

the

context

of

a

logic-program-based

approach

to

the

computation

of

consistent

query

answers

-LSB-

-RSB-

but

as

explained

earlier

such

an

approach

does

not

scale

up

to

large

databases

A

new

scenario

for

data

integration

data

exchange

has

been

recently

proposed

-LSB-

-RSB-

In

this

scenario

a

target

database

is

materialized

on

the

basis

of

a

source

database

using

source-to-target

dependencies

In

the

presence

of

target

integrity

constraints

a

suitable

consistent

target

database

may

not

exist

This

is

a

natural

context

for

the

application

of

the

concepts

of

repair

and

consistent

query

answer

However

-LSB-

-RSB-

does

not

consider

the

issue

of

the

inconsistency

of

target

databases

-LSB-

-RSB-

addresses

the

problem

of

consistent

query

answering

in

a

restricted

data

exchange

setting

In

this

paper

we

have

presented

a

practical

scalable

framework

for

computing

consistent

query

answers

for

large

databases

We

have

also

described

a

number

of

novel

optimization

techniques

applicable

in

this

context

and

summa-rized

experimental

results

that

validate

our

approach

The

approach

however

has

a

number

of

limitations

Only

projection-free

relational

algebra

queries

and

denial

integrity

constraints

are

currently

supported

Adding

projection

to

the

query

language

is

a

difficult

issue

because

the

complexity

of

computing

consistent

query

answers

becomes

in

that

case

co-NP-complete

-LSB-

-RSB-

So

unless

P

NP

we

can

not

hope

for

computing

consistent

query

answers

efficiently

for

arbitrary

conjunctive

queries

and

arbitrary

database

instances

However

the

evaluation

of

queries

with

projection

can

make

use

of

the

conflict

hypergraph

representation

of

all

repairs

and

of

the

operators

F

and

G

introduced

in

Section

Moreover

we

expect

to

be

able

to

compute

consistent

answers

to

queries

with

projection

in

polynomial

time

if

conflict

hypergraphs

are

suitably

restricted

We

hope

that

such

restrictions

can

be

translated

into

corresponding

restrictions

on

database

instances

and

integrity

constraints

In

-LSB-

-RSB-

we

have

studied

scalar

aggregation

queries

in

the

presence

of

functional

dependencies

also

making

use

of

conflict

graphs

It

remains

to

be

seen

whether

the

techniques

developed

in

-LSB-

-RSB-

can

be

combined

with

those

of

the

present

paper

Going

beyond

denial

constraints

appears

challenging

too

Essentially

integrity

violations

of

denial

constraints

are

due

to

the

presence

of

some

facts

in

the

database

and

thus

can

be

compactly

represented

using

the

conflict

hypergraph

If

arbitrary

universal

constraints

for

example

tuple-generating

dependencies

-LSB-

-RSB-

are

allowed

constraint

violations

may

be

due

to

the

simultaneous

presence

and

absence

of

certain

tuples

in

the

database

It

is

not

clear

how

to

construct

in

this

case

a

compact

representation

of

all

repairs

that

can

be

used

for

the

computation

of

consistent

query

answers

Also

repairs

are

no

longer

guaranteed

to

be

subsets

of

the

original

database

but

can

contain

additional

tuples

If

referential

integrity

is

to

be

captured

constraints

have

to

contain

existentially

quantified

variables

which

leads

to

the

unde-cidability

of

consistent

query

answers

-LSB-

-RSB-

Only

in

very

restricted

cases

this

problem

has

been

shown

to

be

tractable

-LSB-

-RSB-

Another

avenue

of

further

research

involves

using

preferences

to

reduce

the

number

of

repairs

and

consequently

make

the

computation

of

consistent

query

answers

more

efficient

For

example

in

data

integration

we

may

have

a

preference

for

certain

sources

or

for

more

recent

information

The

issue

of

benchmarking

systems

that

compute

consistent

query

answers

requires

more

work

It

would

be

desirable

to

design

mechanisms

that

generate

inconsistent

databases

in

a

systematic

way

and

to

perform

more

extensive

experimental

comparisons

between

implemented

systems

-LSB-

-RSB-

S.

Abiteboul

R.

Hull

and

V.

Vianu

Foundations

of

Databases

Addison-Wesley

-LSB-

-RSB-

M.

Arenas

L.

Bertossi

and

J.

Chomicki

Consistent

Query

Answers

in

Inconsistent

Databases

In

ACM

Symposium

on

Principles

of

Database

Systems

-LRB-

PODS

-RRB-

pages

-LSB-

-RSB-

M.

Arenas

L.

Bertossi

and

J.

Chomicki

Answer

Sets

for

Consistent

Query

Answering

in

Inconsistent

Databases

Theory

and

Practice

of

Logic

Programming

-LRB-

-RRB-

-LSB-

-RSB-

M.

Arenas

L.

Bertossi

J.

Chomicki

X.

He

V.

Raghavan

and

J.

Spinrad

Scalar

Aggregation

in

Inconsistent

Databases

Theoretical

Computer

Science

-LRB-

-RRB-

-LSB-

-RSB-

M.

Arenas

L.

Bertossi

and

M.

Kifer

Applications

of

Annotated

Predicate

Calculus

to

Querying

Inconsistent

Databases

In

International

Conference

on

Computational

Logic

pages

Springer-Verlag

LNCS

-LSB-

-RSB-

P.

Barcelo

and

L.

Bertossi

Logic

Programs

for

Querying

Inconsistent

Databases

In

International

Symposium

on

Practical

Aspects

of

Declarative

Languages

-LRB-

PADL

-RRB-

pages

Springer-Verlag

LNCS

-LSB-

-RSB-

L.

Bertossi

and

J.

Chomicki

Query

Answering

in

Inconsistent

Databases

In

J.

Chomicki

R.

van

der

Meyden

and

G.

Snake

editors

Logics

for

Emerging

Applications

of

Databases

pages

Springer-Verlag

-LSB-

-RSB-

L.

Bertossi

J.

Chomicki

A.

Cortes

and

C.

Gutierrez

Consistent

Answers

from

Integrated

Data

Sources

In

International

Conference

on

Flexible

Query

Answering

Systems

-LRB-

FQAS

-RRB-

pages

Copenhagen

Denmark

October

Springer-Verlag

-LSB-

-RSB-

L.

Bravo

and

L.

Bertossi

Logic

Programs

for

Consistently

Querying

Data

Integration

Systems

In

International

Joint

Conference

on

Artificial

Intelligence

-LRB-

IJCAI

-RRB-

pages

-LSB-

-RSB-

F.

Bry

Query

Answering

in

Information

Systems

with

Integrity

Constraints

In

IFIP

WG

Working

Conference

on

Integrity

and

Control

in

Information

Systems

pages

Chapman

Hall

-LSB-

-RSB-

A.

Cali

D.

Lembo

and

R.

Rosati

On

the

Decidability

and

Complexity

of

Query

Answering

over

Inconsistent

and

Incomplete

Databases

In

ACM

Symposium

on

Principles

of

Database

Systems

-LRB-

PODS

-RRB-

pages

-LSB-

-RSB-

A.

Celle

and

L.

Bertossi

Querying

Inconsistent

Databases

Algorithms

and

Implementation

In

International

Conference

on

Computational

Logic

pages

Springer-Verlag

LNCS

-LSB-

-RSB-

J.

Chomicki

and

J.

Marcinkowski

Minimal-Change

Integrity

Maintenance

Using

Tuple

Deletions

Information

and

Computation

To

appear

Earlier

version

Technical

Report

cs.DB

arXiv.org

e-Print

archive

-LSB-

-RSB-

T.

Eiter

W.

Faber

N.

Leone

and

G.

Pfeifer

Declarative

Problem-Solving

in

DLV

In

J.

Minker

editor

Logic-Based

Artificial

Intelligence

pages

Kluwer

-LSB-

-RSB-

T.

Eiter

M.

Fink

G.

Greco

and

D.

Lembo

Efficient

Evaluation

of

Logic

Programs

for

Querying

Data

Integration

Systems

In

International

Conference

on

Logic

Programming

-LRB-

ICLP

-RRB-

pages

-LSB-

-RSB-

R.

Fagin

P.

G.

Kolaitis

R.

J.

Miller

and

L.

Popa

Data

Exchange

Semantics

and

Query

Answering

In

International

Conference

on

Database

Theory

-LRB-

ICDT

-RRB-

pages

Springer-Verlag

LNCS

-LSB-

-RSB-

A.

Fuxman

and

R.

Miller

Towards

Inconsistency

Management

in

Data

Integration

Systems

In

Workshop

on

Information

Integration

on

the

Web

-LRB-

-RRB-

-LSB-

-RSB-

G.

Greco

S.

Greco

and

E.

Zumpano

A

Logic

Programming

Approach

to

the

Integration

Repairing

and

Querying

of

Inconsistent

Databases

In

International

Conference

on

Logic

Programming

-LRB-

ICLP

-RRB-

pages

Springer-Verlag

LNCS

-LSB-

-RSB-

G.

Greco

S.

Greco

and

E.

Zumpano

A

Logical

Framework

for

Querying

and

Repairing

Inconsistent

Databases

IEEE

Transactions

on

Knowledge

and

Data

Engineering

-LRB-

-RRB-

-LSB-

-RSB-

A.

Y.

Halevy

Answering

Queries

Using

Views

A

Survey

VLDB

Journal

-LRB-

-RRB-

-LSB-

-RSB-

P.

C.

Kanellakis

Elements

of

Relational

Database

Theory

In

Jan

van

Leeuwen

editor

Handbook

of

Theoretical

Computer

Science

volume

B

chapter

pages

Elsevier/MIT

Press

-LSB-

-RSB-

D.

Van

Nieuwenborgh

and

D.

Vermeir

Preferred

Answer

Sets

for

Ordered

Logic

Programs

In

European

Conference

on

Logics

for

Artificial

Intelligence

-LRB-

JELIA

-RRB-

pages

Springer-Verlag

LNAI

-LSB-

-RSB-

J.

Wijsen

Condensed

Representation

of

Database

Repairs

for

Consistent

Query

Answering

In

International

Conference

on

Database

Theory

-LRB-

ICDT

-RRB-

pages

Springer-Verlag

LNCS

conflict hypergraph
integrity constraints
consistent query answer
relational algebra
polynomial time
denial constraints
knowledge gathering
disjunctive query
repair
inconsistent database
optimization
inconsistency