Interactive Machine Learning Perceptual user interfaces -LRB- PUIs -RRB- are an important part of ubiquitous computing . Creating such interfaces is difficult because of the image and signal processing knowledge required for creating classifiers . We propose an interactive machine-learning -LRB- IML -RRB- model that allows users to train , classify/view and correct the classifications . The concept and implementation details of IML are discussed and contrasted with classical machine learning models . Evaluations of two algorithms are also presented . We also briefly describe Image Processing with Crayons -LRB- Crayons -RRB- , which is a tool for creating new camera-based interfaces using a simple painting metaphor . The Crayons tool embodies our notions of interactive machine learning . Categories : H. 5.2 , D. 2.2 General Terms : Design , Experimentation Perceptual user interfaces -LRB- PUIs -RRB- are establishing the need for machine learning in interactive settings . PUIs like VideoPlace -LSB- 8 -RSB- , Light Widgets -LSB- 3 -RSB- , and Light Table -LSB- 15,16 -RSB- all use cameras as their perceptive medium . Other systems use sensors other than cameras such as depth scanners and infrared sensors -LSB- 13,14,15 -RSB- . All of these PUIs require machine learning and computer vision techniques to create some sort of a classifier . This classification component of the UI often demands great effort and expense . Because most developers have little knowledge on how to implement recognition in their UIs this becomes problematic . Even those who do have this knowledge would benefit if the classifier building expense were lessened . We suggest the way to decrease this expense is through the use of a visual image classifier generator , which would allow developers to add intelligence to interfaces without forcing additional programming . Similar to how Visual Basic allows simple and fast development , this tool would allow for fast integration of recognition or perception into a UI . Implementation of such a tool , however , poses many problems . First and foremost is the problem of rapidly creating a satisfactory classifier . The simple solution is to using behind-the-scenes machine learning and image processing . Machine learning allows automatic creation of classifiers , however , the classical models are generally slow to train , and not interactive . The classical machine-learning -LRB- CML -RRB- model is summarized in Figure 1 . Prior to the training of the classifier , features need to be selected . Training is then performed `` off-line '' so that classification can be done quickly and efficiently . In this model classification is optimized at the expense of longer training time . Generally , the classifier will run quickly so it can be done real-time . The assumption is that training will be performed only once and need not be interactive . Many machine-learning algorithms are very sensitive to feature selection and suffer greatly if there are very many features . Feature Selection Train Classify Interactive Use Figure 1 Classical machine learning model With CML , it is infeasible to create an interactive tool to create classifiers . CML requires the user to choose the features and wait an extended amount of time for the algorithm to train . The selection of features is very problematic for most interface designers . If one is designing an interactive technique involving laser spot tracking , most designers understand that the spot is generally red . They are not prepared to deal with how to sort out this spot from red clothing , camera noise or a variety of other problems . There are well-known image processing features for handling these problems , but very few interface designers would know how to carefully select them in a way that the machine learning algorithms could handle . The current approach requires too much technical knowledge on the part of the interface designer . What we would like to do is replace the classical machine-learning model with the interactive model shown in Figure 2 . This interactive training allows the classifier to be coached along until the desired results are met . In this model the designer is correcting and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page . To copy otherwise , or republish , to post on servers or to redistribute to lists , requires prior specific permission and/or a fee . IUI ' 03 , January 12 15 , 2003 , Miami , Florida , USA . Copyright 2003 ACM 1-58113-586-6 / 03/0001 ... $ 5.00 . 39 teaching the classifier and the classifier must perform the appropriate feature selection . Feature Selection Train Classify Interactive Use Feedback To Designer Manual Correction Figure 2 Interactive machine learning -LRB- IML -RRB- model The pre-selection of features can be eliminated and transferred to the learning part of the IML if the learning algorithm used performs feature selection . This means that a large repository of features are initially calculated and fed to the learning algorithm so it can learn the best features for the classification problem at hand . The idea is to feed a very large number of features into the classifier training and let the classifier do the filtering rather than the human . The human designer then is focused on rapidly creating training data that will correct the errors of the classifier . In classical machine learning , algorithms are evaluated on their inductive power . That is , how well the algorithm will perform on new data based on the extrapolations made on the training data . Good inductive power requires careful analysis and a great deal of computing time . This time is frequently exponential in the number of features to be considered . We believe that using the IML model a simple visual tool can be designed to build classifiers quickly . We hypothesize that when using the IML , having a very fast training algorithm is more important than strong induction . In place of careful analysis of many feature combinations we provide much more human input to correct errors as they appear . This allows the interactive cycle to be iterated quickly so it can be done more frequently . The remainder of the paper is as follows . The next section briefly discusses the visual tool we created using the IML model , called Image Processing with Crayons -LRB- Crayons -RRB- . This is done to show one application of the IML model 's power and versatility . Following the explanation of Crayons , we explore the details of the IML model by examining its distinction from CML , the problems it must overcome , and its implementation details . Finally we present some results from some tests between two of the implemented machine learning algorithms . From these results we base some preliminary conclusions of IML as it relates to Crayons . Crayons is a system we created that uses IML to create image classifiers . Crayons is intended to aid UI designers who do not have detailed knowledge of image processing and machine learning . It is also intended to accelerate the efforts of more knowledgeable programmers . There are two primary goals for the Crayons tool : 1 -RRB- to allow the user to create an image/pixel classifier quickly , and 2 -RRB- to allow the user to focus on the classification problem rather than image processing or algorithms . Crayons is successful if it takes minutes rather than weeks or months to create an effective classifier . For simplicity sake , we will refer to this as the UI principle of fast and focused . This principle refers to enabling the designer to quickly accomplish his/her task while remaining focused solely on that task . Figure 3 shows the Crayons design process . Images are input into the Crayons system , which can then export the generated classifier . It is assumed the user has already taken digital pictures and saved them as files to import into the system , or that a camera is set up on the machine running Crayons , so it can capture images from it . Exporting the classifier is equally trivial , since our implementation is written in Java . The classifier object is simply serialized and output to a file using the standard Java mechanisms . Figure 3 Classifier Design Process An overview of the internal architecture of Crayons is shown in Figure 4 . Crayons receives images upon which the user does some manual classification , a classifier is created , then feedback is displayed . The user can then refine the classifier by adding more manual classification or , if the classifier is satisfactory , the user can export the classifier . The internal loop shown in Figure 4 directly correlates to the aforementioned train , feedback , correct cycle of the IML -LRB- see Figure 2 -RRB- . To accomplish the fast and focused UI principle , this loop must be easy and quick to cycle through . To be interactive the training part of the loop must take less than five seconds and generally much faster . The cycle can be broken down into two components : the UI and the Classifier . The UI component needs to be simple so the user can remain focused on the classification problem at hand . The classifier creation needs to be fast and efficient so the user gets feedback as quickly as possible , so they are not distracted from the classification problem . Figure 4 The classification design loop 40 Although the IML and the machine-learning component of Crayons are the primary discussion of this paper it is notable to mention that Crayons has profited from work done by Viola and Jones -LSB- 19 -RSB- and Jaimes and Chang -LSB- 5,6,7 -RSB- . Also a brief example of how Crayons can be used is illustrative . The sequence of images in Figure 5 shows the process of creating a classifier using Crayons . Figure 5 Crayons interaction process Figure 5 illustrates how the user initially paints very little data , views the feedback provided by the resulting classifier , corrects by painting additional class pixels and then iterates through the cycle . As seen in the first image pair in Figure 5 , only a little data can generate a classifier that roughly learns skin and background . The classifier , however , over-generalizes in favor of background ; therefore , in the second image pair you can see skin has been painted where the classifier previously did poorly at classifying skin . The resulting classifier shown on the right of the second image pair shows the new classifier classifying most of the skin on the hand , but also classifying some of the background as skin . The classifier is corrected again , and the resulting classifier is shown as the third image pair in the sequence . Thus , in only a few iterations , a skin classifier is created . The simplicity of the example above shows the power that Crayons has due to the effectiveness of the IML model . The key issue in the creation of such a tool lies in quickly generating effective classifiers so the interactive design loop can be utilized . For the IML model to function , the classifier must be generated quickly and be able to generalize well . As such we will first discuss the distinctions between IML and CML , followed by the problems IML must overcome because of its interactive setting , and lastly its implementation details including specific algorithms . Classical machine learning generally has the following assumptions . There are relatively few carefully chosen features , There is limited training data , The classifier must amplify that limited training data into excellent performance on new training data , Time to train the classifier is relatively unimportant as long as it does not take too many days . None of these assumptions hold in our interactive situation . Our UI designers have no idea what features will be appropriate . In fact , we are trying to insulate them from knowing such things . In our current Crayons prototype there are more than 150 features per pixel . To reach the breadth of application that we desire for Crayons we project over 1,000 features will be necessary . The additional features will handle texture , shape and motion over time . For any given problem somewhere between three and fifteen of those features will actually be used , but the classifier algorithm must automatically make this selection . The classifier we choose must therefore be able to accommodate such a large number of features , and/or select only the best features . In Crayons , when a designer begins to paint classes on an image a very large number of training examples is quickly generated . With 77K pixels per image and 20 images one can rapidly generate over a million training examples . In practice , the number stays in the 100K examples range because designers only paint pixels that they need to correct rather than all pixels in the image . What this means , however , is that designers can generate a huge amount of training data very quickly . CML generally focuses on the ability of a classifier to predict correct behavior on new data . In IML , however , if the classifier 's predictions for new data are wrong , the designer can rapidly make those corrections . By rapid feedback and correction the classifier is quickly -LRB- in a matter of minutes -RRB- focused onto the desired behavior . The goal of the classifier is not to predict the designer 's intent into new situations but rapidly reflect intent as expressed in concrete examples . Because additional training examples can be added so readily , IML 's bias differs greatly from that of CML . Because it extrapolates a little data to create a classifier that will be frequently used in the future , CML is very concerned about overfit . Overfit is where the trained classifier adheres 41 too closely to the training data rather than deducing general principles . Cross-validation and other measures are generally taken to minimize overfit . These measures add substantially to the training time for CML algorithms . IML 's bias is to include the human in the loop by facilitating rapid correction of mistakes . Overfit can easily occur , but it is also readily perceived by the designer and instantly corrected by the addition of new training data in exactly the areas that are most problematic . This is shown clearly in Figure 5 where a designer rapidly provides new data in the edges of the hand where the generalization failed . Our interactive classification loop requires that the classifier training be very fast . To be effective , the classifier must be generated from the training examples in under five seconds . If the classifier takes minutes or hours , the process of ` train-feedback - correct ' is no longer interactive , and much less effective as a design tool . Training on 100,000 examples with 150 features each in less than five seconds is a serious challenge for most CML algorithms . Lastly , for this tool to be viable the final classifier will need to be able to classify 320 x 240 images in less than a fourth of a second . If the resulting classifier is much slower than this it becomes impossible to use it to track interactive behavior in a meaningful way . Throughout our discussion thus far , many requirements for the machine-learning algorithm in IML have been made . The machine-learning algorithm must : learn/train very quickly , accommodate 100s to 1000s of features , perform feature selection , allow for tens to hundreds of thousands of training examples . These requirements put firm bounds on what kind of a learning algorithm can be used in IML . They invoke the fundamental question of which machine-learning algorithm fits all of these criteria . We discuss several options and the reason why they are not viable before we settle on our algorithm of choice : decision trees -LRB- DT -RRB- . Neural Networks -LSB- 12 -RSB- are a powerful and often used machine-learning algorithm . They can provably approximate any function in two layers . Their strength lies in their abilities to intelligently integrate a variety of features . Neural networks also produce relatively small and efficient classifiers , however , there are not feasible in IML . The number of features used in systems like Crayons along with the number of hidden nodes required to produce the kinds of classifications that are necessary completely overpowers this algorithm . Even more debilitating is the training time for neural networks . The time this algorithm takes to converge is far to long for interactive use . For 150 features this can take hours or days . The nearest-neighbor algorithm -LSB- 1 -RSB- is easy to train but not very effective . Besides not being able to discriminate amongst features , nearest-neighbor has serious problems in high dimensional feature spaces of the kind needed in IML and Crayons . Nearest-neighbor generally has a classification time that is linear in the number of training examples which also makes it unacceptably slow . There are yet other algorithms such as boosting that do well with feature selection , which is a desirable characteristic . While boosting has shown itself to be very effective on tasks such as face tracing -LSB- 18 -RSB- , its lengthy training time is prohibitive for interactive use in Crayons . There are many more machine-learning algorithms , however , this discussion is sufficient to preface to our decision of the use of decision trees . All the algorithms discussed above suffer from the curse of dimensionality . When many features are used -LRB- 100s to 1000s -RRB- , their creation and execution times dramatically increase . In addition , the number of training examples required to adequately cover such high dimensional feature spaces would far exceed what designers can produce . With just one decision per feature the size of the example set must approach 2 100 , which is completely unacceptable . We need a classifier that rapidly discards features and focuses on the 1-10 features that characterize a particular problem . Decision trees -LSB- 10 -RSB- have many appealing properties that coincide with the requirements of IML . First and foremost is that the DT algorithm is fundamentally a process of feature selection . The algorithm operates by examining each feature and selecting a decision point for dividing the range of that feature . It then computes the `` impurity '' of the result of dividing the training examples at that decision point . One can think of impurity as measuring the amount of confusion in a given set . A set of examples that all belong to one class would be pure -LRB- zero impurity -RRB- . There are a variety of possible impurity measures -LSB- 2 -RSB- . The feature whose partition yields the least impurity is the one chosen , the set is divided and the algorithm applied recursively to the divided subsets . Features that do not provide discrimination between classes are quickly discarded . The simplicity of DTs also provides many implementation advantages in terms of speed and space of the resulting classifier . Quinlan 's original DT algorithm -LSB- 10 -RSB- worked only on features that were discrete -LRB- a small number of choices -RRB- . Our image features do not have that property . Most of our features are continuous real values . Many extensions of the original DT algorithm , ID3 , have been made to allow use of real valued data -LSB- 4,11 -RSB- . All of these algorithms either discretize the data or by selecting a threshold T for a given feature F divide the training examples into two sets where F < T and F > = T . The trick is for each feature to select a value T that gives the lowest impurity -LRB- best classification improvement -RRB- . The selection of T from a large number of features and a large number of training examples is very slow to do correctly . 42 We have implemented two algorithms , which employ different division techniques . These two algorithms also represent the two approaches of longer training time with better generalization vs. shorter training time with poorer generalization . The first strategy slightly reduces interactivity and relies more on learning performance . The second relies on speed and interactivity . The two strategies are Center Weighted -LRB- CW -RRB- and Mean Split -LRB- MS -RRB- . Our first DT attempt was to order all of the training examples for each feature and step through all of the examples calculating the impurity as if the division was between each of the examples . This yielded a minimum impurity split , however , this generally provided a best split close to the beginning or end of the list of examples , still leaving a large number of examples in one of the divisions . Divisions of this nature yield deeper and more unbalanced trees , which correlate to slower classification times . To improve this algorithm , we developed Center Weighted -LRB- CW -RRB- , which does the same as above , except that it more heavily weights central splits -LRB- more equal divisions -RRB- . By insuring that the split threshold is generally in the middle of the feature range , the resulting tree tends to be more balanced and the sizes of the training sets to be examined at each level of the tree drops exponentially . CW DTs do , however , suffer from an initial sort of all training examples for each feature , resulting in a O -LRB- f * N log N -RRB- cost up front , where f is the number of features and N the number of training examples . Since in IML , we assume that both f and N are large , this can be extremely costly . Because of the extreme initial cost of sorting all N training examples f times , we have extended Center Weighted with CWSS . The ` SS ' stand for sub-sampled . Since the iteration through training examples is purely to find a good split , we can sample the examples to find a statistically sound split . For example , say N is 100,000 , if we sample 1,000 of the original N , sort those and calculate the best split then our initial sort is 100 times faster . It is obvious that a better threshold could be computed using all of the training data , but this is mitigated by the fact that those data items will still be considered in lower levels of the tree . When a split decision is made , all of the training examples are split , not just the sub-sample . The sub-sampling means that each node 's split decision is never greater than O -LRB- f * 1000 * 5 -RRB- , but that eventually all training data will be considered . Quinlan used a sampling technique called `` windowing '' . Windowing initially used a small sample of training examples and increased the number of training examples used to create the DT , until all of the original examples were classified correctly -LSB- 11 -RSB- . Our technique , although similar , differs in that the number of samples is fixed . At each node in the DT a new sample of fixed size is drawn , allowing misclassified examples in a higher level of the DT to be considered at a lower level . The use of sub-sampling in CWSS produced very slight differences in classification accuracy as compared to CW , but reduced training time by a factor of at least two -LRB- for training sets with N 5,000 -RRB- . This factor however will continue to grow as N increases . -LRB- For N = 40,000 CWSS is approximately 5 times faster than CW ; 8 for N = 80,000 . -RRB- The CW and CWSS algorithms spend considerable computing resources in trying to choose a threshold value for each feature . The Mean Split -LRB- MS -RRB- algorithm spends very little time on such decisions and relies on large amounts of training data to correct decisions at lower levels of the tree . The MS algorithm uses T = mean -LRB- F -RRB- as the threshold for dividing each feature F and compares the impurities of the divisions of all features . This is very efficient and produces relatively shallow decision trees by generally dividing the training set in half at each decision point . Mean split , however , does not ensure that the division will necessarily divide the examples at points that are meaningful to correct classification . Successive splits at lower levels of the tree will eventually correctly classify the training data , but may not generalize as well . The resulting MS decision trees are not as good as those produced by more careful means such as CW or CWSS . However , we hypothesized , that the speedup in classification would improve interactivity and thus reduce the time for designers to train a classifier . We believe designers make up for the lower quality of the decision tree with the ability to correct more rapidly . The key is in optimizing designer judgment rather than classifier predictions . MSSS is a sub-sampled version of MS in the same manner as CWSS . In MSSS , since we just evaluate the impurity at the mean , and since the mean is a simple statistical value , the resulting divisions are generally identical to those of straight MS. As a parenthetical note , another important bottleneck that is common to all of the classifiers is the necessity to calculate all features initially to create the classifier . We made the assumption in IML that all features are pre-calculated and that the learning part will find the distinguishing features . Although , this can be optimized so it is faster , all algorithms will suffer from this bottleneck . There are many differences between the performances of each of the algorithms . The most important is that the CW algorithms train slower than the MS algorithms , but tend to create better classifiers . Other differences are of note though . For example , the sub sampled versions , CWSS and MSSS , generally allowed the classifiers to be generated faster . More specifically , CWSS was usually twice as fast as CW , as was MSSS compared to MS. Because of the gains in speed and lack of loss of classification power , only CWSS and MSSS will be used for comparisons . The critical comparison is to see which algorithm allows the user to create a satisfactory classifier the fastest . User tests comparing these algorithms are outlined and presented in the next section . 43 User tests were conducted to evaluate the differences between CWSS and MSSS . When creating a new perceptual interface it is not classification time that is the real issue . The important issue is designer time . As stated before , classification creation time for CWSS is longer than MSSS , but the center-weighted algorithms tend to generalize better than the mean split algorithms . The CWSS generally takes 110 seconds to train on training sets of 10,000-60,000 examples , while MSSS is approximately twice as fast on the same training sets . These differences are important ; as our hypothesis was that faster classifier creation times can overcome poorer inductive strength and thus reduce overall designer time . To test the difference between CWSS and MSSS we used three key measurements : wall clock time to create the classifier , number of classify/correct iterations , and structure of the resulting tree -LRB- depth and number of nodes -RRB- . The latter of these three corresponds to the amount of time the classifier takes to classify an image in actual usage . In order to test the amount of time a designer takes to create a good classifier , we need a standard to define `` good classifier '' . A `` gold standard '' was created for four different classification problems : skin-detection , paper card tracking , robot car tracking and laser tracking . These gold standards were created by carefully classifying pixels until , in human judgment , the best possible classification was being performed on the test images for each problem . The resulting classifier was then saved as a standard . Ten total test subjects were used and divided into two groups . The first five did each task using the CWSS followed by the MSSS and the remaining five MSSS followed by CWSS . The users were given each of the problems in turn and asked to build a classifier . Each time the subject requested a classifier to be built that classifier 's performance was measured against the performance of the standard classifier for that task . When the subject 's classifier agreed with the standard on more than 97.5 % of the pixels , the test was declared complete . Table 1 , shows the average times and iterations for the first group , Table 2 , the second group . CWSS MSSS Problem Time Iterations Time Iterations Skin 03:06 4.4 10:35 12.6 Paper Cards 02:29 4.2 02:23 5.0 Robot Car 00:50 1.6 01:00 1.6 Laser 00:46 1.2 00:52 1.4 Table 1 CWSS followed by MSSS MSSS CWSS Problem Time Iterations Time Iterations Skin 10:26 11.4 03:51 3.6 Paper Cards 04:02 5.0 02:37 2.6 Robot Car 01:48 1.2 01:37 1.2 Laser 01:29 1.0 01:16 1.0 Table 2 MSSS followed by CWSS The laser tracker is a relatively simple classifier because of the uniqueness of bright red spots -LSB- 9 -RSB- . The robot car was contrasted with a uniform colored carpet and was similarly straightforward . Identifying colored paper cards against a cluttered background was more difficult because of the diversity of the background . The skin tracker is the hardest because of the diversity of skin color , camera over-saturation problems and cluttered background -LSB- 20 -RSB- . As can be seen in tables 1 and 2 , MSSS takes substantially more designer effort on the hard problems than CWSS . All subjects specifically stated that CWSS was `` faster '' than MSSS especially in the Skin case . -LRB- Some did not notice a difference between the two algorithms while working on the other problems . -RRB- We did not test any of the slower algorithms such as neural nets or nearest-neighbor . Interactively these are so poor that the results are self-evident . We also did not test the full CW algorithm . Its classification times tend into minutes and clearly could not compete with the times shown in tables 1 and 2 . It is clear from our evaluations that a classification algorithm must get under the 10-20 second barrier in producing a new classification , but that once under that barrier , the designer 's time begins to dominate . Once the designer 's time begins to dominate the total time , then the classifier with better generalization wins . We also mentioned the importance of the tree structure as it relates to the classification time of an image . Table 3 shows the average tree structures -LRB- tree depth and number of nodes -RRB- as well as the average classification time -LRB- ACT -RRB- in milliseconds over the set of test images . CWSS MSSS Problem Depth Nodes ACT Depth Nodes ACT Skin 16.20 577 243 25.60 12530 375 Paper Cards 15.10 1661 201 16.20 2389 329 Car 13.60 1689 235 15.70 2859 317 Laser 13.00 4860 110 8.20 513 171 Table 3 Tree structures and average classify time -LRB- ACT -RRB- As seen in Table 3 , depth , number of nodes and ACT , were all lower in CWSS than in MSSS . This was predicted as CWSS provides better divisions between the training examples . 44 While testing we observed that those who used the MSSS which is fast but less accurate , first , ended up using more training data , even when they used the CWSS , which usually generalizes better and needs less data . Those who used the CWSS first , were pleased with the interactivity of CWSS and became very frustrated when they used MSSS , even though it could cycle faster through the interactive loop . In actuality , because of the poor generalization of the mean split algorithm , even though the classifier generation time for MSSS was quicker than CWSS , the users felt it necessary to paint more using the MSSS , so the overall time increased using MSSS . When using machine learning in an interactive design setting , feature selection must be automatic rather than manual and classifier training-time must be relatively fast . Decision Trees using a sub-sampling technique to improve training times are very effective for both of these purposes . Once interactive speeds are achieved , however , the quality of the classifier 's generalization becomes important . Using tools like Crayons , demonstrates that machine learning can form an appropriate basis for the design tools needed to create new perceptual user interfaces . 1 . Cover , T. , and Hart , P. `` Nearest Neighbor Pattern Classification . '' IEEE Transactions on Information Theory , 13 , -LRB- 1967 -RRB- 21-27 . 2 . Duda , R. O. , Hart , P. E. , and Stork , D. G. , Pattern Classification . -LRB- 2001 -RRB- . 3 . Fails , J.A. , Olsen , D.R. `` LightWidgets : Interacting in Everyday Spaces . '' Proceedings of IUI '02 -LRB- San Francisco CA , January 2002 -RRB- . 4 . Fayyad , U.M. and Irani , K. B. `` On the Handling of Continuous-valued Attributes in Decision Tree Generation . '' Machine Learning , 8 , 87-102 , -LRB- 1992 -RRB- . 5 . Jaimes , A. and Chang , S.-F . `` A Conceptual Framework for Indexing Visual Information at Multiple Levels . '' IS&T / SPIE Internet Imaging 2000 , -LRB- San Jose CA , January 2000 -RRB- . 6 . Jaimes , A. and Chang , S.-F . `` Automatic Selection of Visual Features and Classifier . '' Storage and Retrieval for Image and Video Databases VIII , IS&T / SPIE -LRB- San Jose CA , January 2000 -RRB- . 7 . Jaimes , A. and Chang , S.-F . `` Integrating Multiple Classifiers in Visual Object Detectors Learned from User Input . '' Invited paper , session on Image and Video Databases , 4th Asian Conference on Computer Vision -LRB- ACCV 2000 -RRB- , Taipei , Taiwan , January 8-11 , 2000 . 8 . Krueger , M. W. , Gionfriddo . T. , and Hinrichsen , K. , `` VIDEOPLACE -- an artificial reality '' . Human Factors in Computing Systems , CHI '85 Conference Proceedings , ACM Press , 1985 , 35-40 . 9 . Olsen , D.R. , Nielsen , T. `` Laser Pointer Interaction . '' Proceedings of CHI '01 -LRB- Seattle WA , March 2001 -RRB- . 10 . Quinlan , J. R. `` Induction of Decision Trees . '' Machine Learning , 1 -LRB- 1 -RRB- ; 81-106 , -LRB- 1986 -RRB- . 11 . Quinlan , J. R. `` C4 .5 : Programs for machine learning . '' Morgan Kaufmann , San Mateo , CA , 1993 . 12 . Rumelhart , D. , Widrow , B. , and Lehr , M. `` The Basic Ideas in Neural Networks . '' Communications of the ACM , 37 -LRB- 3 -RRB- , -LRB- 1994 -RRB- , pp 87-92 . 13 . Schmidt , A. `` Implicit Human Computer Interaction Through Context . '' Personal Technologies , Vol 4 -LRB- 2 -RRB- , June 2000 . 14 . Starner , T. , Auxier , J. and Ashbrook , D. `` The Gesture Pendant : A Self-illuminating , Wearable , Infrared Computer Vision System for Home Automation Control and Medical Monitoring . '' International Symposium on Wearable Computing -LRB- Atlanta GA , October 2000 -RRB- . 15 . Triggs , B. `` Model-based Sonar Localisation for Mobile Robots . '' Intelligent Robotic Systems ' 93 , Zakopane , Poland , 1993 . 16 . Underkoffler , J. and Ishii H. `` Illuminating Light : An Optical Design Tool with a Luminous-Tangible Interface . '' Proceedings of CHI '98 -LRB- Los Angeles CA , April 1998 -RRB- . 17 . Underkoffler , J. , Ullmer , B. and Ishii , H. `` Emancipated Pixels : Real-World Graphics in the Luminous Room . '' Proceedings of SIGGRAPH '99 -LRB- Los Angeles CA , 1999 -RRB- , ACM Press , 385-392 . 18 . Vailaya , A. , Zhong , Y. , and Jain , A. K. `` A hierarchical system for efficient image retrieval . '' In Proc . Int . Conf . on Patt . Recog . -LRB- August 1996 -RRB- . 19 . Viola , P. and Jones , M. `` Robust real-time object detection . '' Technical Report 2001/01 , Compaq CRL , February 2001 . 20 . Yang , M.H. and Ahuja , N. `` Gaussian Mixture Model for Human Skin Color and Its Application in Image and Video Databases . '' Proceedings of SPIE '99 -LRB- San Jose CA , Jan 1999 -RRB- , 458-466 . 45

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

crayons prototype
image processing
classification
interaction
perceptual interface
crayons design process
classical machine learning
machine learning

perceptive user interfaces
image/pixel classifier
predict correct behaviour
image processing with crayons
classification design loop
perceptual user iinterfaces
interactive machine learning


Improvements of TLAESA Nearest Neighbour Search Algorithm and Extension to Approximation Search Nearest neighbour -LRB- NN -RRB- searches and k nearest neighbour -LRB- k-NN -RRB- searches are widely used in pattern recognition and image retrieval . An NN -LRB- k-NN -RRB- search finds the closest object -LRB- closest k objects -RRB- to a query object . Although the definition of the distance between objects depends on applications , its computation is generally complicated and time-consuming . It is therefore important to reduce the number of distance computations . TLAESA -LRB- Tree Linear Approximating and Eliminating Search Algorithm -RRB- is one of the fastest algorithms for NN searches . This method reduces distance computations by using a branch and bound algorithm . In this paper we improve both the data structure and the search algorithm of TLAESA . The proposed method greatly reduces the number of distance computations . Moreover , we extend the improved method to an approximation search algorithm which ensures the quality of solutions . Experimental results show that the proposed method is efficient and finds an approximate solution with a very low error rate . NN and k-NN searches are techniques which find the closest object -LRB- closest k objects -RRB- to a query object from a database . These are widely used in pattern recognition and image retrieval . We can see examples of their applications to handwritten character recognition in -LRB- Rico-Juan & Mic o 2003 -RRB- and -LRB- Mic o & Oncina 1998 -RRB- , and so on . In this paper we consider NN -LRB- k-NN -RRB- algorithms that can work in any metric space . For any x , y , z in a metric space , the distance function d -LRB- , -RRB- satisfies the following properties : d -LRB- x , y -RRB- = 0 x = y , d -LRB- x , y -RRB- = d -LRB- y , x -RRB- , d -LRB- x , z -RRB- d -LRB- x , y -RRB- + d -LRB- y , z -RRB- . Although the definition of the distance depends on applications , its calculation is generally complicated and time-consuming . We particularly call the calculation of d -LRB- , -RRB- a distance computation . Copyright c 2006 , Australian Computer Society , Inc. . This paper appeared at Twenty-Ninth Australasian Computer Science Conference -LRB- ACSC2006 -RRB- , Hobart , Tasmania , Australia , January 2006 . Conferences in Research and Practice in Information Technology , Vol . 48 . Vladimir Estivill-Castro and Gill Dobbie , Ed . Reproduction for academic , not-for profit purposes permitted provided this text is included . For the NN and k-NN searches in metric spaces , some methods that can manage a large set of objects efficiently have been introduced -LRB- Hjaltason & Samet 2003 -RRB- . They are categorized into two groups . The methods in the first group manage objects with a tree structure such as vp-tree -LRB- Yianilos 1993 -RRB- , M-tree -LRB- Ciaccia , Patella & Zezula 1997 -RRB- , sa-tree -LRB- Navarro 2002 -RRB- and so forth . The methods in the second group manage objects with a distance matrix , which stores the distances between objects . The difference between two groups is caused by their approaches to fast searching . The former aims at reducing the com-putational tasks in the search process by managing objects effectively . The latter works toward reducing the number of distance computations because generally their costs are higher than the costs of other calculations . In this paper we consider the latter approach . AESA -LRB- Approximating and Eliminating Search Algorithm -RRB- -LRB- Vidal 1986 -RRB- is one of the fastest algorithms for NN searches in the distance matrix group . The number of distance computations is bounded by a constant , but the space complexity is quadratic . LAESA -LRB- Linear AESA -RRB- -LRB- Mic o , Oncina & Vidal 1994 -RRB- was introduced in order to reduce this large space complexity . Its space complexity is linear and its search performance is almost the same as that of AESA . Although LAESA is more practical than AESA , it is impractical for a large database because calculations other than distance computations increase . TLAESA -LRB- Tree LAESA -RRB- -LRB- Mic o , Oncina & Carrasco 1996 -RRB- is an improvement of LAESA and reduces the time complexity to sublinear . It uses two kinds of data structures : a distance matrix and a binary tree , called a search tree . In this paper , we propose some improvements of the search algorithm and the data structures of TLAESA in order to reduce the number of distance computations . The search algorithm follows the best first algorithm . The search tree is transformed to a multiway tree from a binary tree . We also improve the selection method of the root object in the search tree . These improvements are simple but very effective . We then introduce the way to perform a k-NN search in the improved TLAESA . Moreover , we propose an extension to an approximation search algorithm that can ensure the quality of solutions . This paper is organized as follows . In section 2 , we describe the details of the search algorithm and the data structures of TLAESA . In section 3 , we propose some improvements of TLAESA . In section 4 , we present an extension to an approximation search algorithm . In section 5 , we show some experimental results . Finally , in section 6 , we conclude this paper . Figure 1 : An example of the data structures in TLAESA . TLAESA uses two kinds of data structures : the distance matrix and the search tree . The distance matrix stores the distances from each object to some selected objects . The search tree manages hierarchically all objects . During the execution of the search algorithm , the search tree is traversed and the distance matrix is used to avoid exploring some branches . 2.1 Data Structures We explain the data structures in TLAESA . Let P be the set of all objects and B be a subset consisting of selected objects called base prototypes . The distance matrix M is a two-dimensional array that stores the distances between all objects and base prototypes . The search tree T is a binary tree such that each node t corresponds to a subset S t P . Each node t has a pointer to the representative object p t S t which is called a pivot , a pointer to a left child node l , a pointer to a right child node r and a covering radius r t . The covering radius is defined as r t = max pS t d -LRB- p , p t -RRB- . -LRB- 1 -RRB- The pivot p r of r is defined as p r = p t . On the other hand , the pivot p l of l is determined so that p l = argmax pS t d -LRB- p , p t -RRB- . -LRB- 2 -RRB- Hence , we have the following equality : r t = d -LRB- p t , p l -RRB- . -LRB- 3 -RRB- S t is partitioned into two disjoint subsets S r and S l as follows : S r = -LCB- p S t | d -LRB- p , p r -RRB- < d -LRB- p , p l -RRB- -RCB- , S l = S t - S r . -LRB- 4 -RRB- Note that if t is a leaf node , S t = -LCB- p t -RCB- and r t = 0 . Fig. 1 shows an example of the data structures . 2.2 Construction of the Data Structures We first explain the construction process of the search tree T . The pivot p t of the root node t is randomly selected and S t is set to P . The pivot p l of the left child node and the covering radius r t are defined by Eqs . -LRB- 2 -RRB- and -LRB- 3 -RRB- . The pivot p r of the right child node is set to p t . S t is partitioned into S r and S l by Eq . -LRB- 4 -RRB- . These operations are recursively repeated until | S t | = 1 . The distance matrix M is constructed by selecting base prototypes . This selection is important because Figure 2 : Lower bound . base prototypes are representative objects which are used to avoid some explorations of the tree . The ideal selection of them is that each object is as far away as possible from other objects . In -LRB- Mic o et al. 1994 -RRB- , a greedy algorithm is proposed for this selection . This algorithm chooses an object that maximizes the sum of distances from the other base prototypes which have already been selected . In -LRB- Mic o & Oncina 1998 -RRB- , another algorithm is proposed , which chooses an object that maximizes the minimum distance to the preselected base prototypes . -LRB- Mic o & Oncina 1998 -RRB- shows that the latter algorithm is more effective than the former one . Thus , we use the later algorithm for the selection of base prototypes . The search efficiency depends not only on the selection of base prototypes but also on the number of them . There is a trade-off between the search efficiency and the size of distance matrix , i.e. the memory capacity . The experimental results in -LRB- Mic o et al. 1994 -RRB- show that the optimal number of base prototypes depends on the dimensionality dm of the space . For example , the optimal numbers are 3 , 16 and 24 if dm = 2 , 4 and 8 , respectively . The experimental results also show that the optimal number does not depend on the number of objects . 2.3 Search Algorithm The search algorithm follows the branch and bound strategy . It traverses the search tree T in the depth first order . The distance matrix M is referred whenever each node is visited in order to avoid unnecessary traverse of the tree T . The distance are computed only when a leaf node is reached . Given a query object q , the distance between q and the base prototypes are computed . These results are stored in an array D . The object which is the closest to q in B is selected as the nearest neighbour candidate p min , and the distance d -LRB- q , p min -RRB- is recorded as d min . Then , the traversal of the search tree T starts at the root node . The lower bound for the left child node l is calculated whenever each node t is reached if it is not a leaf node . The lower bound of the distance between q and an object x is defined as g x = max bB | d -LRB- q , b -RRB- - d -LRB- b , x -RRB- | . -LRB- 5 -RRB- See Fig. 2 . Recall that d -LRB- q , b -RRB- was precomputed before the traversals and was stored in D . In addition , the value d -LRB- b , x -RRB- was also computed during the construction process and stored in the distance matrix M . Therefore , g x is calculated without any actual distance computations . The lower bound g x is not actual distance d -LRB- q , x -RRB- . Thus , it does not ensure that the number of visited nodes in the search becomes minimum . Though , this evaluation hardly costs , hence it is possible to search fast . The search process accesses the left child node l if g p l g p r , or the right child node r if g p l > g p r . When a leaf node is reached , the distance is computed and both p min and d min are updated if the distance is less than d min . q p min p t r t S t Figure 3 : Pruning Process . procedure NN search -LRB- q -RRB- 1 : t root of T 2 : d min = , g p t = 0 3 : for b B do 4 : D -LSB- b -RSB- = d -LRB- q , b -RRB- 5 : if D -LSB- b -RSB- < d min then 6 : p min = b , d min = D -LSB- b -RSB- 7 : end if 8 : end for 9 : g p t = max bB | -LRB- D -LSB- b -RSB- - M -LSB- b , p t -RSB- -RRB- | 10 : search -LRB- t , g p t , q , p min , d min -RRB- 11 : return p min Figure 4 : Algorithm for an NN search in TLAESA . We explain the pruning process . Fig. 3 shows the pruning situation . Let t be the current node . If the inequality d min + r t < d -LRB- q , p t -RRB- -LRB- 6 -RRB- is satisfied , we can see that no object exists in S t which is closer to q than p min and the traversal to node t is not necessary . Since g p t d -LRB- q , p t -RRB- , Eq . -LRB- 6 -RRB- can be replaced with d min + r t < g p t . -LRB- 7 -RRB- Figs. 4 and 5 show the details of the search algorithm -LRB- Mic o et al. 1996 -RRB- . In this section , we propose some improvements of TLAESA in order to reduce the number of distance computations . 3.1 Tree Structure and Search Algorithm If we can evaluate the lower bounds g in the ascending order of their values , the search algorithm runs very fast . However , this is not guaranteed in TLAESA since the evaluation order is decided according to the tree structure . We show such an example in Fig. 6 . In this figure , u , v and w are nodes . If g p v < g p w , it is desirable that v is evaluated before w. But , if g p v > g p u , w might be evaluated before v . We propose the use of a multiway tree and the best first order search instead of a binary tree and the depth first search . During the best first search process , we can traverse preferentially a node whose subset may contain the closest object . Moreover , we can evaluate more nodes at one time by using of the multiway tree . The search tree in TLAESA has many nodes which have a pointer to the same object . In the proposed structure , we treat such nodes as one node . Each node t corresponds to a subset S t P and has a pivot p t , a covering radius r t = max pS t d -LRB- p , p t -RRB- and pointers to its children nodes . procedure search -LRB- t , g p t , q , p min , d min -RRB- 1 : if t is a leaf then 2 : if g p t < d min then 3 : d = d -LRB- q , p t -RRB- -LCB- distance computation -RCB- 4 : if d < d min then 5 : p min = p t , d min = d 6 : end if 7 : end if 8 : else 9 : r is a right child of t 10 : l is a left child of t 11 : g p r = g p t 12 : g p l = max bB | -LRB- D -LSB- b -RSB- - M -LSB- b , p t -RSB- -RRB- | 13 : if g p l < g p r then 14 : if d min + r l > g p l then 15 : search -LRB- l , g p l , p min , d min -RRB- 16 : end if 17 : if d min + r r > g p r then 18 : search -LRB- r , g p r , p min , d min -RRB- 19 : end if 20 : else 21 : if d min + r r > g p r then 22 : search -LRB- r , g p r , p min , d min -RRB- 23 : end if 24 : if d min + r l > g p l then 25 : search -LRB- l , g p l , p min , d min -RRB- 26 : end if 27 : end if 28 : end if Figure 5 : A recursive procedure for an NN search in TLAESA . Figure 6 : A case in which the search algorithm in TLAESA does not work well . We show a method to construct the tree structure in Fig. 7 . We first select randomly the pivot p t of the root node t and set S t to P . Then we execute the procedure makeTree -LRB- t , p t , S t -RRB- in Fig. 7 . We explain the search process in the proposed structure . The proposed method maintains a priority queue Q that stores triples -LRB- node t , lower bound g p t , covering radius r t -RRB- in the increasing order of g p t - r t . Given a query object q , we calculate the distances between q and base prototypes and store their values in D . Then the search process starts at the root of T . The following steps are recursively repeated until Q becomes empty . When t is a leaf node , the distance d -LRB- q , p t -RRB- is computed if g p t < d min . If t is not a leaf node and its each child node t satisfies the inequality g p t < r t + d min , -LRB- 8 -RRB- the lower bound g p t is calculated and a triple -LRB- t , g p t , r t -RRB- is added to Q. Figs. 8 and 9 show the details of the algorithm . procedure makeTree -LRB- t , p t , S t -RRB- 1 : t new child node of t 2 : if | S t | = 1 then 3 : p t = p t and S t = -LCB- p t -RCB- 4 : else 5 : p t = argmax pS t d -LRB- p , p t -RRB- 6 : S t = -LCB- p S t | d -LRB- p , p t -RRB- < d -LRB- p , p t -RRB- -RCB- 7 : S t = S t - S t 8 : makeTree -LRB- t , p t , S t -RRB- 9 : makeTree -LRB- t , p t , S t -RRB- 10 : end if Figure 7 : Method to construct the proposed tree structure . procedure NN search -LRB- q -RRB- 1 : t root of T 2 : d min = , g p t = 0 3 : for b B do 4 : D -LSB- b -RSB- = d -LRB- q , b -RRB- 5 : if D -LSB- b -RSB- < d min then 6 : p min = b , d min = D -LSB- b -RSB- 7 : end if 8 : end for 9 : g t = max bB | -LRB- D -LSB- b -RSB- - M -LSB- b , p t -RSB- -RRB- | 10 : Q -LCB- -LRB- t , g p t , r t -RRB- -RCB- 11 : while Q is not empty do do 12 : -LRB- t , g p t , r t -RRB- element in Q 13 : search -LRB- t , g p t , q , p min , d min -RRB- 14 : end while 15 : return p min Figure 8 : Proposed algorithm for an NN search . 3.2 Selection of Root Object We focus on base prototypes in order to reduce node accesses . The lower bound of the distance between a query q and a base prototype b is g b = max bB | d -LRB- q , b -RRB- - d -LRB- b , b -RRB- | = d -LRB- q , b -RRB- . This value is not an estimated distance but an actual distance . If we can use an actual distance in the search process , we can evaluate more effectively which nodes are close to q . This fact means that the search is efficiently performed if many base prototypes are visited in the early stage . In other words , it is desirable that more base prototypes are arranged in the upper part of the search tree . Thus , in the proposed algorithm , we choose the first base prototype b 1 as the root object . 3.3 Extension to a k-NN Search LAESA was developed to perform NN searches and -LRB- Moreno-Seco , Mic o & Oncina 2002 -RRB- extended it so that k-NN searches can be executed . In this section , we extend the improved TLAESA to a k-NN search algorithm . The extension is simple modifications of the algorithm described above . We use a priority queue V for storing k nearest neighbour candidates and modify the definition of d min . V stores pairs -LRB- object p , distance d -LRB- q , p -RRB- -RRB- in the increasing order of procedure search -LRB- t , g p t , q , p min , d min -RRB- 1 : if t is a leaf then 2 : if g p t < d min then 3 : d = d -LRB- q , p t -RRB- -LCB- distance computation -RCB- 4 : if d < d min then 5 : p min = p t , d min = d 6 : end if 7 : end if 8 : else 9 : for each child t of t do 10 : if g p t < r t + d min then 11 : g p t = max bB | -LRB- D -LSB- b -RSB- - M -LSB- b , p t -RSB- -RRB- | 12 : Q Q -LCB- -LRB- t , g p t , r t -RRB- -RCB- 13 : end if 14 : end for 15 : end if Figure 9 : A procedure used in the proposed algorithm for an NN search . procedure k-NN search -LRB- q , k -RRB- 1 : t root of T 2 : d min = , g p t = 0 3 : for b B do 4 : D -LSB- b -RSB- = d -LRB- q , b -RRB- 5 : if D -LSB- b -RSB- < d min then 6 : V V -LCB- -LRB- b , D -LSB- b -RSB- -RRB- -RCB- 7 : if | V | = k + 1 then 8 : remove -LRB- k + 1 -RRB- th pair from V 9 : end if 10 : if | V | = k then 11 : -LRB- c , d -LRB- q , c -RRB- -RRB- kth pair of V 12 : d min = d -LRB- q , c -RRB- 13 : end if 14 : end if 15 : end for 16 : g p t = max bB | -LRB- D -LSB- b -RSB- - M -LSB- b , p t -RSB- -RRB- | 17 : Q -LCB- -LRB- t , g p t , r t -RRB- -RCB- 18 : while Q is not empty do 19 : -LRB- t , g p t , r t -RRB- element in Q 20 : search -LRB- t , g p t , q , V , d min , k -RRB- 21 : end while 22 : return k objects V Figure 10 : Proposed algorithm for a k-NN search . d -LRB- q , p -RRB- . d min is defined as d min = -LRB- | V | < k -RRB- d -LRB- q , c -RRB- -LRB- | V | = k -RRB- -LRB- 9 -RRB- where c is the object of the kth pair in V . We show in Figs. 10 and 11 the details of the k-NN search algorithm . The search strategy essentially follows the algorithm in Figs. 8 and 9 , but the k-NN search algorithm uses V instead of p min . -LRB- Moreno-Seco et al. 2002 -RRB- shows that the optimal number of base prototypes depends on not only the dimensionality of the space but also the value of k and that the number of distance computations increases as k increases . In this section , we propose an extension to an approximation k-NN search algorithm which ensures the procedure search -LRB- t , g p t , q , V , d min , k -RRB- 1 : if t is a leaf then 2 : if g p t < d min then 3 : d = d -LRB- q , p t -RRB- -LCB- distance computation -RCB- 4 : if d < d min then 5 : V V -LCB- -LRB- p t , d -LRB- q , p t -RRB- -RRB- -RCB- 6 : if | V | = k + 1 then 7 : remove -LRB- k + 1 -RRB- th pair from V 8 : end if 9 : if | V | = k then 10 : -LRB- c , d -LRB- q , c -RRB- -RRB- kth pair of V 11 : d min = d -LRB- q , c -RRB- 12 : end if 13 : end if 14 : end if 15 : else 16 : for each child t of t do 17 : if g p t < r t + d min then 18 : g p t = max bB | -LRB- D -LSB- b -RSB- - M -LSB- b , p t -RSB- -RRB- | 19 : Q Q -LCB- -LRB- t , g p t , r t -RRB- -RCB- 20 : end if 21 : end for 22 : end if Figure 11 : A procedure used in the proposed algorithm for a k-NN search . quality of solutions . Consider the procedure in Fig. 11 . We replace the 4th line with if d < d min then and the 17th line with if g t < r t + d min then where is real number such that 0 < 1 . The pruning process gets more efficient as these conditions become tighter . The proposed method ensures the quality of solutions . We can show the approximation ratio to an optimal solution using . Let a be the nearest neighbour object and a be the nearest neighbour candidate object . If our method misses a and give a as the answer , the equation g -LRB- q , a -RRB- d -LRB- q , a -RRB- -LRB- 10 -RRB- is satisfied . Then a will be eliminated from targeted objects . Since g -LRB- q , a -RRB- d -LRB- q , a -RRB- , we can obtain the following equation : d -LRB- q , a -RRB- 1 d -LRB- q , a -RRB- . -LRB- 11 -RRB- Thus , the approximate solution are suppressed by 1 times of the optimal solution . In this section we show some experimental results and discuss them . We tested on an artificial set of random points in the 8-dimensional euclidean space . We also used the euclidean distance as the distance function . We evaluated the number of distance computations and the number of accesses to the distance matrix in 1-NN and 10-NN searches . 0 50 100 150 200 250 300 0 10 20 30 40 50 60 70 80 90 100 110 120 Number of Distance Computations Number of Base Prototypes TLAESA -LRB- 1-NN -RRB- TLAESA -LRB- 10-NN -RRB- Proposed -LRB- 1-NN -RRB- Proposed -LRB- 10-NN -RRB- Figure 12 : Relation of the number of distance computations to the number of base prototypes . 1-NN 10-NN TLAESA 40 80 Proposed 25 60 Table 1 : The optimal number of base prototypes . 5.1 The Optimal Number of Base Prototypes We first determined experimentally the optimal number of base prototypes . The number of objects was fixed to 10000 . We executed 1-NN and 10-NN searches for various numbers of base prototypes , and counted the number of distance computations . Fig. 12 shows the results . From this figure , we chose the number of base prototypes as shown in Table . 1 . We can see that the values in the proposed method are fewer than those in TLAESA . This means that the proposed method can achieve better performance with smaller size of distance matrix . We used the values in Table . 1 in the following experiments . 5.2 Evaluation of Improvements We tested the effects of our improvements described in 3.1 and 3.2 . We counted the numbers of distance computations in 1-NN and 10-NN searches for various numbers of objects . The results are shown in Figs. 13 and 14 . Similar to TLAESA , the number of the distance computations in the proposed method does not depend on the number of objects . In both of 1-NN and 10-NN searches , it is about 60 % of the number of distance computations in TLAESA . Thus we can see that our improvements are very effective . In the search algorithms of TLAESA and the proposed methods , various calculations are performed other than distance computations . The costs of the major part of such calculations are proportional to the number of accesses to the distance matrices . We therefore counted the numbers of accesses to the distance matrices . We examined the following two cases : -LRB- i -RRB- TLAESA vs. TLAESA with the improvement of selection of the root object . -LRB- ii -RRB- Proposed method only with improvement of tree structure and search algorithm vs. proposed method only with the improvement of selection of the root object . In the case -LRB- i -RRB- , the number of accesses to the distance matrix is reduced by 12 % in 1-NN searches and 4.5 % in 10-NN searches . In the case -LRB- ii -RRB- , it is reduced by 6.8 % in 1-NN searches and 2.7 % in 10-NN searches . 0 10 20 30 40 50 60 70 80 90 100 0 2000 4000 6000 8000 10000 Number of Distance Computations Number of Objects TLAESA Proposed Figure 13 : The number of distance computations in 1-NN searches . 0 30 60 90 120 150 180 210 240 270 300 0 2000 4000 6000 8000 10000 Number of Distance Computations Number of Objects TLAESA Proposed Figure 14 : The number of distance computations in 10-NN searches . Thus we can see that this improvement about selection of the root object is effective . 5.3 Evaluation of Approximation Search We tested the performance of the approximation search algorithm . We compared the proposed method to Ak-LAESA , which is the approximation search algorithm proposed in -LRB- Moreno-Seco , Mic o & Oncina 2003 -RRB- . Each time a distance is computed in Ak-LAESA , the nearest neighbour candidate is updated and its value is stored . When the nearest neighbour object is found , the best k objects are chosen from the stored values . In Ak-LAESA , the number of distance computations of the k-NN search is exactly the same as that of the NN search . To compare the proposed method with Ak-LAESA , we examined how many objects in the approximate solutions exist in the optimal solutions . Thus , we define the error rate E as follows : E -LSB- % -RSB- = | -LCB- x i | x i / Opt , i = 1 , 2 , , k -RCB- | k 100 -LRB- 12 -RRB- where -LCB- x 1 , x 2 , , x k -RCB- is a set of k objects which are obtained by an approximation algorithm and Opt is a set of k closest objects to the query object . Fig. 15 shows the error rate when the value of is changed in 10-NN searches . Fig. 16 also shows the relation of the number of distance computations to the value of in 10-NN searches . In the range 0.5 , the proposed method shows the lower error rate than 0 10 20 30 40 50 60 70 80 90 100 0 0.2 0.4 0.6 0.8 1 Error Rate -LSB- % -RSB- Ak-LAESA Proposed Figure 15 : Error rate in 10-NN searches . 0 20 40 60 80 100 120 140 160 0 0.2 0.4 0.6 0.8 1 Number of distance computations Ak-LAESA Proposed Figure 16 : Relation of the number of distance computations to the value of in 10-NN searches . Ak-LAESA . In particular , the error rate of the proposed method is almost 0 in range 0.9 . From two figures , we can control the error rate and the number of distance computations by changing the value of . For example , the proposed method with = 0.9 reduces abount 28.6 % of distance computations and its error rate is almost 0 . Then we examined the accuracy of the approximate solutions . We used = 0.5 for the proposed method because the error rate of the proposed method with = 0.5 is equal to the one of Ak-LAESA . We performed 10-NN searches 10000 times for each method and examined the distribution of kth approximate solution to kth optimal solution . We show the results in Figs. 17 and 18 . In each figure , x axis represents the distance between a query object q and the kth object in the optimal solution . y axis shows the distance between q and the kth object in the approximate solution . The point near the line y = x represents that kth approximate solution is very close to kth optimal solution . In Fig. 17 , many points are widely distributed . In the worst case , some appriximate solutions reach about 3 times of the optimal solution . From these figures , we can see that the accuracy of solution by the proposed method is superior to the one by Ak-LAESA . We also show the result with = 0.9 in Fig. 19 . Most points exist near the line y = x. Though Ak-LAESA can reduce drastically the number of distance computations , its approximate solutions are often far from the optimal solutions . On the other hand , the proposed method can reduce the number of distance computations to some extent with 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 0.2 0.4 0.6 0.8 Distance to the k th Approximate Solution Distance to the k th Optimal Solution Figure 17 : The distribution of the approximate solution by Ak-LAESA to the optimal solution . 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 0.2 0.4 0.6 0.8 Distance to the k th Approximate Solution Distance to the k th Optimal Solution Figure 18 : The distribution the approximate solution by the proposed method with = 0.5 to the optimal solution . very low error rate . Moreover , the accuracy of its approximate solutions is superior to that of Ak-LAESA . In this paper , we proposed some improvements of TLAESA . In order to reduce the number of distance computations in TLAESA , we improved the search algorithm to best first order from depth first order and the tree structure to a multiway tree from a binary tree . In the 1-NN searches and 10-NN searches in a 8-dimensional space , the proposed method reduced about 40 % of distance computations . We then proposed the selection method of root object in the search tree . This improvement is very simple but is effective to reduce the number of accesses to the distance matrix . Finally , we extended our method to an approximation k-NN search algorithm that can ensure the quality of solutions . The approximate solutions of the proposed method are suppressed by 1 times of the optimal solutions . Experimental results show that the proposed method can reduce the number of distance computations with very low error rate by selecting the appropriate value of , and that the accuracy of the solutions is superior to Ak-LAESA . From these viewpoints , the method presented in this paper is very effective when the distance computations are time-consuming . 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 0.2 0.4 0.6 0.8 Distance to the k th Approximate Solution Distance to the k th Optimal Solution Figure 19 : The distribution the approximate solution by the proposed method with = 0.9 to the optimal solution . References Ciaccia , P. , Patella , M. & Zezula , P. -LRB- 1997 -RRB- , M-tree : An efficient access method for similarity search in metric spaces , in ` Proceedings of the 23rd International Conference on Very Large Data Bases -LRB- VLDB ' 97 -RRB- ' , pp. 426 435 . Hjaltason , G. R. & Samet , H. -LRB- 2003 -RRB- , ` Index-driven similarity search in metric spaces ' , ACM Transactions on Database Systems 28 -LRB- 4 -RRB- , 517 580 . Mic o , L. & Oncina , J. -LRB- 1998 -RRB- , ` Comparison of fast nearest neighbour classifiers for handwritten character recognition ' , Pattern Recognition Letters 19 -LRB- 3-4 -RRB- , 351 356 . Mic o , L. , Oncina , J. & Carrasco , R. C. -LRB- 1996 -RRB- , ` A fast branch & bound nearest neighbour classifier in metric spaces ' , Pattern Recognition Letters 17 -LRB- 7 -RRB- , 731 739 . Mic o , M. L. , Oncina , J. & Vidal , E. -LRB- 1994 -RRB- , ` A new version of the nearest-neighbour approximating and eliminating search algorithm -LRB- AESA -RRB- with linear preprocessing time and memory require-ments ' , Pattern Recognition Letters 15 -LRB- 1 -RRB- , 9 17 . Moreno-Seco , F. , Mic o , L. & Oncina , J. -LRB- 2002 -RRB- , ` Extending LAESA fast nearest neighbour algorithm to find the k-nearest neighbours ' , Lecture Notes in Computer Science - Lecture Notes in Artificial Intelligence 2396 , 691 699 . Moreno-Seco , F. , Mic o , L. & Oncina , J. -LRB- 2003 -RRB- , ` A modification of the LAESA algorithm for ap-proximated k-NN classification ' , Pattern Recognition Letters 24 -LRB- 1-3 -RRB- , 47 53 . Navarro , G. -LRB- 2002 -RRB- , ` Searching in metric spaces by spatial approximation ' , The VLDB Journal 11 -LRB- 1 -RRB- , 28 46 . Rico-Juan , J. R. & Mic o , L. -LRB- 2003 -RRB- , ` Comparison of AESA and LAESA search algorithms using string and tree-edit-distances ' , Pattern Recognition Letters 24 -LRB- 9-10 -RRB- , 1417 1426 . Vidal , E. -LRB- 1986 -RRB- , ` An algorithm for finding nearest neighbours in -LRB- approximately -RRB- constant average time ' , Pattern Recognition Letters 4 -LRB- 3 -RRB- , 145 157 . Yianilos , P. N. -LRB- 1993 -RRB- , Data structures and algorithms for nearest neighbor search in general metric spaces , in ` SODA ' 93 : Proceedings of the fourth annual ACM-SIAM Symposium on Discrete algorithms ' , pp. 311 321 .

O O B B I I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

nearest neighbour search
approximation search
tlaesa

k nearest neighbour search
distance computaion