A New Statistical Formula for Chinese Text Segmentation Incorporating Contextual Information A new statistical formula for identifying 2-character words in Chinese text , called the contextual information formula , was developed empirically by performing stepwise logistic regression using a sample of sentences that had been manually segmented . Contextual information in the form of the frequency of characters that are adjacent to the bigram being processed as well as the weighted document frequency of the overlapping bigrams were found to be significant factors for predicting the probablity that the bigram constitutes a word . Local information -LRB- the number of times the bigram occurs in the document being segmented -RRB- and the position of the bigram in the sentence were not found to be useful in determining words . The contextual information formula was found to be significantly and substantially better than the mutual information formula in identifying 2-character words . The method can also be used for identifying multi-word terms in English text . Chinese text is different from English text in that there is no explicit word boundary . In English text , words are separated by spaces . Chinese text -LRB- as well as text of other Oriental languages -RRB- is made up of ideographic characters , and a word can comprise one , two or more such characters , without explicit indication where one word ends and another begins . This has implications for natural language processing and information retrieval with Chinese text . Text processing techniques that have been developed for Western languages deal with words as meaningful text units and assume that words are easy to identify . These techniques may not work well for Chinese text without some adjustments . To apply these techniques to Chinese text , automatic methods for identifying word boundaries accurately have to be developed . The process of identifying word boundaries has been referred to as text segmentation or , more accurately , word segmentation . Several techniques have been developed for Chinese text segmentation . They can be divided into : 1 . statistical methods , based on statistical properties and frequencies of characters and character strings in a corpus -LRB- e.g. -LSB- 13 -RSB- and -LSB- 16 -RSB- -RRB- . 2 . dictionary-based methods , often complemented with grammar rules . This approach uses a dictionary of words to identify word boundaries . Grammar rules are often used to resolve conflicts -LRB- choose between alternative segmentations -RRB- and to improve the segmentation -LRB- e.g. -LSB- 4 -RSB- , -LSB- 8 -RSB- , -LSB- 19 -RSB- and -LSB- 20 -RSB- -RRB- . 3 . syntax-based methods , which integrate the word segmentation process with syntactic parsing or part-of-speech tagging -LRB- e.g. -LSB- 1 -RSB- -RRB- . 4 . conceptual methods , that make use of some kind of semantic processing to extract information and store it in a knowledge representation scheme . Domain knowledge is used for disambiguation -LRB- e.g. -LSB- 9 -RSB- -RRB- . Many researchers use a combination of methods -LRB- e.g. -LSB- 14 -RSB- -RRB- . The objective of this study was to empirically develop a statistical formula for Chinese text segmentation . Researchers have used different statistical methods in segmentation , most of which were based on theoretical considerations or adopted from other fields . In this study , we developed a statistical formula empirically by performing stepwise logistic regression using a sample of sentences that had been manually segmented . This paper reports the new formula developed for identifying 2-character words , and the effectiveness of this formula compared with the mutual information formula . This study has the following novel aspects : The statistical formula was derived empirically using regression analysis . The manual segmentation was performed to identify meaningful words rather than simple words . Meaningful words include phrasal words and multi-word terms . In addition to the relative frequencies of bigrams and characters often used in other studies , our study also investigated the use of document frequencies and weighted Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page . To copy otherwise , to republish , to post on servers or to redistribute to lists , requires prior specific permission and/or a fee . SIGIR '99 8/99 Berkley , CA USA Copyright 1999 ACM 1-58113-096-1 / 99/0007 ... $ 5.00 82 -RRB- -LRB- * -RRB- -LRB- -RRB- -LRB- log 2 C freq B freq BC freq document frequencies . Weighted document frequencies are similar to document frequencies but each document is weighted by the square of the number of times the character or bigram occurs in the document . Contextual information was included in the study . To predict whether the bigram BC in the character string A B C D constitutes a word , we investigated whether the frequencies for AB , CD , A and D should be included in the formula . Local frequencies were included in the study . We investigated character and bigram frequencies within the document in which the sentence occurs -LRB- i.e. the number of times the character or bigram appears in the document being segmented -RRB- . We investigated whether the position of the bigram -LRB- at the beginning of the sentence , before a punctuation mark , or after a punctuation mark -RRB- had a significant effect . We developed a segmentation algorithm to apply the statistical formula to segment sentences and resolve conflicts . In this study , our objective was to segment text into meaningful words rather than simple words . A simple word is the smallest independent unit of a sentence that has meaning on its own . A meaningful word can be a simple word or a compound word comprising 2 or more simple words depending on the context . In many cases , the meaning of a compound word is more than just a combination of the meanings of the constituent simple words , i.e. some meaning is lost when the compound word is segmented into simple words . Furthermore , some phrases are used so often that native speakers perceive them and use them as a unit . Admittedly , there is some subjectivity in the manual segmentation of text . But the fact that statistical models can be developed to predict the manually segmented words substantially better than chance indicates some level of consistency in the manual segmentation . The problem of identifying meaningful words is not limited to Chinese and oriental languages . Identifying multi-word terms is also a problem in text processing with English and other Western languages , and researchers have used the mutual information formula and other statistical approaches for identifying such terms -LRB- e.g. -LSB- 3 -RSB- , -LSB- 6 -RSB- and -LSB- 7 -RSB- -RRB- . There are few studies using a purely statistical approach to Chinese text segmentation . One statistical formula that has been used by other researchers -LRB- e.g. -LSB- 11 -RSB- and -LSB- 16 -RSB- -RRB- is the mutual information formula . Given a character string A B C D , the mutual information for the bigram BC is given by the formula : MI -LRB- BC -RRB- = = log 2 freq -LRB- BC -RRB- log 2 freq -LRB- B -RRB- log 2 freq -LRB- C -RRB- where freq refers to the relative frequency of the character or bigram in the corpus -LRB- i.e. the number of times the character or bigram occurs in the corpus divided by the number of characters in the corpus -RRB- . Mutual information is a measure of how strongly the two characters are associated , and can be used as a measure of how likely the pair of characters constitutes a word . Sproat & Shih -LSB- 16 -RSB- obtained recall and precision values of 94 % using mutual information to identify words . This study probably segmented text into simple words rather than meaningful words . In our study , text was segmented into meaningful words and we obtained much poorer results for the mutual information formula . Lua -LSB- 12 -RSB- and Lua & Gan -LSB- 13 -RSB- applied information theory to the problem of Chinese text segmentation . They calculated the information content of characters and words using the information entropy formula I = - log 2 P , where P is the probability of occurrence of the character or word . If the information content of a character string is less than the sum of the information content of the constituent characters , then the character string is likely to constitute a word . The formula for calculating this loss of information content when a word is formed is identical to the mutual information formula . Lua & Gan -LSB- 13 -RSB- obtained an accuracy of 99 % -LRB- measured in terms of the number of errors per 100 characters -RRB- . Tung & Lee -LSB- 18 -RSB- also used information entropy to identify unknown words in a corpus . However , instead of calculating the entropy value for the character string that is hypothesized to be a word -LRB- i.e. the candidate word -RRB- , they identified all the characters that occurred to the left of the candidate word in the corpus . For each left character , they calculated the probability and entropy value for that character given that it occurs to the left of the candidate word . The same is done for the characters to the right of the candidate word . If the sum of the entropy values for the left characters and the sum of the entropy values for the right characters are both high , than the candidate word is considered likely to be a word . In other words , a character string is likely to be a word if it has several different characters to the left and to the right of it in the corpus , and none of the left and right characters predominate -LRB- i.e. not strongly associated with the character string -RRB- . Ogawa & Matsuda -LSB- 15 -RSB- developed a statistical method to segment Japanese text . Instead of attempting to identify words directly , they developed a formula to estimate the probability that a bigram straddles a word boundary . They referred to this as the segmentation probability . This was complemented with some syntactic information about which class of characters could be combined with which other class . All the above mathematical formulas used for identifying words and word boundaries were developed based on theoretical considerations and not derived empirically . Other researchers have developed statistical methods to find the best segmentation for the whole sentence rather than focusing on identifying individual words . Sproat et al. -LSB- 17 -RSB- developed a stochastic finite state model for segmenting text . In their model , a word dictionary is represented as a weighted finite state transducer . Each weight represents the estimated cost of the word -LRB- calculated using the negative log probability -RRB- . Basically , the system selects the sentence segmentation that has the smallest total cost . Chang & Chen -LSB- 1 -RSB- developed a method for word segmentation and part-of-speech tagging based on a first-order hidden Markov model . 83 The purpose of this study was to empirically develop a statistical formula for identifying 2-character words as well as to investigate the usefulness of various factors for identifying the words . A sample of 400 sentences was randomly selected from 2 months -LRB- August and September 1995 -RRB- of news articles from the Xin Hua News Agency , comprising around 2.3 million characters . The sample sentences were manually segmented . The segmentation rules described in -LSB- 10 -RSB- were followed fairly closely . More details of the manual segmentation process , especially with regard to identifying meaningful words will be given in -LSB- 5 -RSB- . 300 sentences were used for model building , i.e. using regression analysis to develop a statistical formula . 100 sentences were set aside for model validation to evaluate the formula developed in the regression analysis . The sample sentences were broken up into overlapping bigrams . In the regression analysis , the dependent variable was whether a bigram was a two-character word according to the manual segmentation . The independent variables were various corpus statistics derived from the corpus -LRB- 2 months of news articles -RRB- . The types of frequency information investigated were : 1 . Relative frequency of individual characters and bigrams -LRB- character pairs -RRB- in the corpus , i.e. the number of times the character or bigram occurs in the corpus divided by the total number of characters in the corpus . 2 . Document frequency of characters and bigrams , i.e. the number of documents in the corpus containing the character or bigram divided by the total number of documents in the corpus . 3 . Weighted document frequency of characters and bigrams . To calculate the weighted document frequency of a character string , each document containing the character string is assigned a score equal to the square of the number of times the character string occurs in the document . The scores for all the documents containing the character string are then summed and divided by the total number of documents in the corpus to obtain the weighted document frequency for the character string . The rationale is that if a character string occurs several times within the same document , this is stronger evidence that the character string constitutes a word , than if the character string occurs once in several documents . Two or more characters can occur together by chance in several different documents . It is less likely for two characters to occur together several times within the same document by chance . 4 . Local frequency in the form of within-document frequency of characters and bigrams , i.e. the number of times the character or bigram occurs in the document being segmented . 5 . Contextual information . Frequency information of characters adjacent to a bigram is used to help determine whether the bigram is a word . For the character string A B C D , to determine whether the bigram BC is a word , frequency information for the adjacent characters A and D , as well as the overlapping bigrams AB and BC were considered . 6 . Positional information . We studied whether the position of a character string -LRB- at the beginning , middle or end of a sentence -RRB- gave some indication of whether the character string was a word . The statistical model was developed using forward stepwise logistic regression , using the Proc Logistic function in the SAS v. 6.12 statistical package for Windows . Logistic regression is an appropriate regression technique when the dependent variable is binary valued -LRB- takes the value 0 or 1 -RRB- . The formula developed using logistic regression predicts the probability -LRB- more accurately , the log of the odds -RRB- that a bigram is a meaningful word . In the stepwise regression , the threshold for a variable to enter the model was set at the 0.001 significance level and the threshold for retaining a variable in the model was set at 0.01 . In addition , preference was given to relative frequencies and local frequencies because they are easier to calculate than document frequencies and weighted document frequencies . Also , relative frequencies are commonly used in previous studies . Furthermore , a variable was entered in a model only if it gave a noticeable improvement to the effectiveness of the model . During regression analysis , the effectiveness of the model was estimated using the measure of concordance that was automatically output by the SAS statistical program . A variable was accepted into the model only if the measure of concordance improved by at least 2 % when the variable was entered into the model . We evaluated the accuracy of the segmentation using measures of recall and precision . Recall and precision in this context are defined as follows : Recall = No . of 2-character words identified in the automatic segmentation that are correct No . of 2-character words identified in the manual segmentation Precision = No . of 2-character words identified in the automatic segmentation that are correct No . of 2-character words identified in the automatic segmentation The formula that was developed for 2-character words is as follows . Given a character string A B C D , the association strength for bigram BC is : Assoc -LRB- BC -RRB- = 0.35 * log 2 freq -LRB- BC -RRB- + 0.37 * log 2 freq -LRB- A -RRB- + 0.32 log 2 freq -LRB- D -RRB- 0.36 * log 2 docfreq wt -LRB- AB -RRB- 0.29 * log 2 docfreq wt -LRB- CD -RRB- + 5.91 where freq refers to the relative frequency in the corpus and docfreq wt refers to the weighted document frequency . We refer to this formula as the contextual information formula . More details of the regression model are given in Table 1 . The formula indicates that contextual information is helpful in identifying word boundaries . A in the formula refers to the character preceding the bigram that is being processed , whereas D is the character following the bigram . The formula indicates that if the character preceding and the character following the bigram have high relative frequencies , then the bigram is more likely to be a word . 84 Contextual information involving the weighted document frequency was also found to be significant . The formula indicates that if the overlapping bigrams AB and CD have high weighted document frequencies , then the bigram BC is less likely to be a word . We tried replacing the weighted document frequencies with the unweighted document frequencies as well as the relative frequencies . These were found to give a lower concordance score . Even with docfreq -LRB- AB -RRB- and docfreq -LRB- CD -RRB- in the model , docfreq wt -LRB- AB -RRB- and docfreq wt -LRB- CD -RRB- were found to improve the model significantly . However , local frequencies were surprisingly not found to be useful in predicting 2-character words . We investigated whether the position of the bigram in the sentence was a significant factor . We included a variable to indicate whether the bigram occurred just after a punctuation mark or at the beginning of the sentence , and another variable to indicate whether the bigram occurred just before a punctuation mark or at the end of a sentence . The interaction between each of the position variables and the various relative frequencies were not significant . However , it was found that whether or not the bigram was at the end of a sentence or just before a punctuation mark was a significant factor . Bigrams at the end of a sentence or just before a punctuation mark tend to be words . However , since this factor did not improve the concordance score by 2 % , the effect was deemed too small to be included in the model . It should be noted that the contextual information used in the study already incorporates some positional information . The frequency of character A -LRB- the character preceding the bigram -RRB- was given the value 0 if the bigram was preceded by a punctuation mark or was at the beginning of a sentence . Similarly , the frequency of character D -LRB- the character following the bigram -RRB- was given the value 0 if the bigram preceded a punctuation mark . We also investigated whether the model would be different for high and low frequency words . We included in the regression analysis the interaction between the relative frequency of the bigram and the other relative frequencies . The interaction terms were not found to be significant . Finally , it is noted that the coefficients for the various factors are nearly the same , hovering around 0.34 . 4.2 Improved Mutual Information Formula In this study , the contextual information formula -LRB- CIF -RRB- was evaluated by comparing it with the mutual information formula -LRB- MIF -RRB- . We wanted to find out whether the segmentation results using the CIF was better than the segmentation results using the MIF . In the CIF model , the coefficients of the variables were determined using regression analysis . If CIF was found to give better results than MIF , it could be because the coefficients for the variables in CIF had been determined empirically and not because of the types of variables in the formula . To reject this explanation , regression analysis was used to determine the coefficients for the factors in the mutual information formula . We refer to this new version of the formula as the improved mutual information formula . Given a character string A B C D , the improved mutual information formula is : Improved MI -LRB- BC -RRB- = 0.39 * log 2 freq -LRB- BC -RRB- - 0.28 * log 2 freq -LRB- B -RRB- 0 .23 log 2 freq -LRB- C -RRB- - 0.32 The coefficients are all close to 0.3 . The formula is thus quite similar to the mutual information formula , except for a multiplier of 0.3 . The automatic segmentation process has the following steps : 1 . The statistical formula is used to calculate a score for each bigram to indicate its association strength -LRB- or how likely the bigram is a word -RRB- . 2 . A threshold value is then set and used to decide which bigram is a word . If a bigram obtains a score above the threshold value , then it is selected as a word . Different threshold values can be used , depending on whether the user prefers high recall or high precision . 3 . A segmentation algorithm is used to resolve conflict . If two overlapping bigrams both have association scores above the Parameter Standard Wald Pr > Standardized Variable DF Estimate Error Chi-Square Chi-Square Estimate INTERCPT 1 5.9144 0.1719 1184.0532 0.0001 . Log freq -LRB- BC -RRB- 1 0.3502 0.0106 1088.7291 0.0001 0.638740 Log freq -LRB- A -RRB- 1 0.3730 0.0113 1092.1382 0.0001 0.709621 Log freq -LRB- D -RRB- 1 0.3171 0.0107 886.4446 0.0001 0.607326 Log docfreq wt -LRB- AB -RRB- 1 -0.3580 0.0111 1034.0948 0.0001 -0.800520 Log docfreq wt -LRB- CD -RRB- 1 -0.2867 0.0104 754.2276 0.0001 -0.635704 Note : freq refers to the relative frequency , and docfreq wt refers to the weighted document frequency . Association of Predicted Probabilities and Observed Responses Concordant = 90.1 % Somers ' D = 0.803 Discordant = 9.8 % Gamma = 0.803 Tied = 0.1 % Tau-a = 0.295 -LRB- 23875432 pairs -RRB- c = 0.901 Table 1 . Final regression model for 2-character words 85 threshold value , then there is conflict or ambiguity . The frequency of such conflicts will rise as the threshold value is lowered . The segmentation algorithm resolves the conflict and selects one of the bigrams as a word . One simple segmentation algorithm is the forward match algorithm . Consider the sentence A B C D E . The segmentation process proceeds from the beginning of the sentence to the end . First the bigram AB is considered . If the association score is above the threshold , then AB is taken as a word , and the bigram CD is next considered . If the association score of AB is below the threshold , the character A is taken as a 1-character word . And the bigram BC is next considered . In effect , if the association score of both AB and BC are above threshold , the forward match algorithm selects AB as a word and not BC . The forward match method for resolving ambiguity is somewhat arbitrary and not satisfactory . When overlapping bigrams exceed the threshold value , it simply decides in favour of the earlier bigram . Another segmentation algorithm was developed in this study which we refer to as the comparative forward match algorithm . This has an additional step : If 2 overlapping bigrams AB and BC both have scores above the threshold value then their scores are compared . If AB has a higher value , then it is selected as a word , and the program next considers the bigrams CD and DE . On the other hand , if AB has a lower value , then character A is selected as a 1-character word , and the program next considers bigrams BC and CD . The comparative forward match method -LRB- CFM -RRB- was compared with the forward match method -LRB- FM -RRB- by applying them to the 3 statistical formulas -LRB- the contextual information formula , the mutual information formula and the improved mutual information formula -RRB- . One way to compare the effectiveness of the 2 segmentation algorithms is by comparing their precision figures at the same recall levels . The precision figures for selected recall levels are given in Table 2 . The results are based on the sample of 300 sentences . The comparative forward match algorithm gave better results for the mutual information and improved mutual information formulas especially at low threshold values when a large number of conflicts are likely . Furthermore , for the forward match method , the recall didn t go substantially higher than 80 % even at low threshold values . For the contextual information formula , the comparative forward match method did not perform better than forward match , except at very low threshold values when the recall was above 90 % . This was expected because the contextual information formula already incorporates information about neighboring characters within the formula . The formula gave very few conflicting segmentations . There were very few cases of overlapping bigrams both having association scores above the threshold except when threshold values were below 1.5 . In this section we compare the effectiveness of the contextual information formula with the mutual information formula and the improved mutual information formula using the 100 sentences that had been set aside for evaluation purposes . For the contextual information formula , the forward match segmentation algorithm was used . The comparative forward match algorithm was used for the mutual information and the improved mutual information formulas . The three statistical formulas were compared by comparing their precision figures at 4 recall levels at 60 % , 70 % , 80 % and 90 % . For each of the three statistical formulas , we identified the threshold values that would give a recall of 60 % , 70 % , 80 % and 90 % . We then determined the precision values at these threshold values to find out whether the contextual information formula gave better precision than the other two formulas at 60 % , 70 % , 80 % and 90 % recall . These recall levels were selected because a recall of 50 % or less is probably unacceptable for most applications . The precision figures for the 4 recall levels are given in Table 3 . The recall-precision graphs for the 3 formulas are given in Fig. 1 . The contextual information formula substantially outperforms the mutual information and the improved mutual information formulas . At the 90 % recall level , the contextual information Precision Recall Comparative Forward Match Forward Match Improvement Mutual Information 90 % 51 % 80 % 52 % 47 % 5 % 70 % 53 % 51 % 2 % 60 % 54 % 52 % 2 % Improved Mutual Information 90 % 51 % 80 % 53 % 46 % 7 % 70 % 54 % 52 % 2 % 60 % 55 % 54 % 1 % Contextual Information Formula 90 % 55 % 54 % 1 % 80 % 62 % 62 % 0 % 70 % 65 % 65 % 0 % 60 % 68 % 68 % 0 % Table 2 . Recall and precision values for the comparative forward match segmentation algorithm vs. forward match Precision Recall Mutual Information Improved Mutual Information Contextual Information 90 % 57 % -LRB- 0.0 -RRB- 57 % -LRB- -2.5 -RRB- 61 % -LRB- -1.5 -RRB- 80 % 59 % -LRB- 3.7 -RRB- 59 % -LRB- -1.5 -RRB- 66 % -LRB- -0.8 -RRB- 70 % 59 % -LRB- 4.7 -RRB- 60 % -LRB- -1.0 -RRB- 70 % -LRB- -0.3 -RRB- 60 % 60 % -LRB- 5.6 -RRB- 62 % -LRB- -0.7 -RRB- 74 % -LRB- 0.0 -RRB- * Threshold values are given in parenthesis . Table 3 . Recall and precision for three statistical formulas 86 formula was better by about 4 % . At the 60 % recall level , it outperformed the mutual information formula by 14 % -LRB- giving a relative improvement of 23 % -RRB- . The results also indicate that the improved mutual information formula does not perform better than the mutual information formula . 6.2 Statistical Test of Significance In order to perform a statistical test , recall and precision figures were calculated for each of the 100 sentences used in the evaluation . The average recall and the average precision across the 100 sentences were then calculated for the three statistical formulas . In the previous section , recall and precision were calculated for all the 100 sentences combined . Here , recall and precision were obtained for individual sentences and then the average across the 100 sentences was calculated . The average precision for 60 % , 70 % , 80 % and 90 % average recall are given in Table 4 . For each recall level , an analysis of variance with repeated measures was carried out to find out whether the differences in precision were significant . Pairwise comparisons using Tukey s HSD test was also carried out . The contextual information formula was significantly better -LRB- = 0.001 -RRB- than the mutual information and the improved mutual information formulas at all 4 recall levels . The improved mutual information formula was not found to be significantly better than mutual information . The errors that arose from using the contextual information formula were analyzed to gain insights into the weaknesses of the model and how the model can be improved . There are two types of errors : errors of commission and errors of omission . Errors of commission are bigrams that are identified by the automatic segmentation to be words when in fact they are not -LRB- according to the manual segmentation -RRB- . Errors of omission are bigrams that are not identified by the automatic segmentation to be words but in fact they are . The errors depend of course on the threshold values used . A high threshold -LRB- e.g. 1.0 -RRB- emphasizes precision and a low threshold -LRB- e.g. 1.0 -RRB- emphasizes recall . 50 sentences were selected from the 100 sample sentences to find the distribution of errors at different regions of threshold values . Association Score > 1.0 -LRB- definite errors -RRB- will through telegraph -LSB- on the -RSB- day -LSB- 31 July -RSB- Association Score Between 1.0 and 1.0 -LRB- borderline errors -RRB- still to will be people etc. . I want Person 's name -LRB- -RRB- Wan Wen Ju Place name -LRB- -RRB- a village name in China -LRB- -RRB- Canada Name of an organization/institution -LRB- -RRB- Xin Hua Agency -LRB- -RRB- The State Department Table 6 . Bigrams incorrectly identified as words 55 60 65 70 75 60 65 70 75 80 85 90 95 Recall -LRB- % -RRB- Precision -LRB- % -RRB- Contextual information Mutual information Improved mutual information Fig. 1 . Recall-precision graph for the three statistical models . Association Score > 1.0 -LRB- definite errors -RRB- -LRB- -RRB- university -LRB- agricultural university -RRB- -LRB- -RRB- geology -LRB- geologic age -RRB- -LRB- -RRB- plant -LRB- upland plant -RRB- -LRB- -RRB- sovereignty -LRB- sovereign state -RRB- Association Score Between 1.0 and 1.0 -LRB- borderline errors -RRB- -LRB- -RRB- statistics -LRB- statistical data -RRB- -LRB- -RRB- calamity -LRB- natural calamity -RRB- -LRB- -RRB- resources -LRB- manpower resources -RRB- -LRB- -RRB- professor -LRB- associate professor -RRB- -LRB- -RRB- poor -LRB- pauperization -RRB- -LRB- -RRB- fourteen -LRB- the 14 th day -RRB- -LRB- -RRB- twenty -LRB- twenty pieces -RRB- Table 5 . Simple words that are part of a longer meaningful word Avg Precision Avg Recall Mutual Information Improved Mutual Information Contextual Information 90 % 57 % -LRB- 1.0 -RRB- 58 % -LRB- -2.3 -RRB- 61 % -LRB- -1.5 -RRB- 80 % 60 % -LRB- 3.8 -RRB- 60 % -LRB- -1.4 -RRB- 67 % -LRB- -0.7 -RRB- 70 % 59 % -LRB- 4.8 -RRB- 60 % -LRB- -1.0 -RRB- 70 % -LRB- -0.3 -RRB- 60 % 60 % -LRB- 5.6 -RRB- 63 % -LRB- -0.6 -RRB- 73 % -LRB- 0.0 -RRB- * Threshold values are given in parenthesis . Table 4 . Average recall and average precision for the three statistical formulas 87 We divide the errors of commission -LRB- bigrams that are incorrectly identified as words by the automatic segmentation -RRB- into 2 groups : 1 . Definite errors : bigrams with association scores above 1.0 but are not words 2 . Borderline errors : bigrams with association scores between 1.0 and 1.0 and are not words We also divide the errors of omission -LRB- bigrams that are words but are not identified by the automatic segmentation -RRB- into 2 groups : 1 . Definite errors : bigrams with association scores below 1.0 but are words 2 . Borderline errors : bigrams with association scores between 1.0 and 1.0 and are words . 7.1 Errors of Commission Errors of commission can be divided into 2 types : 1 . The bigram is a simple word that is part of a longer meaningful word . 2 . The bigram is not a word -LRB- neither simple word nor meaningful word -RRB- . Errors of the first type are illustrated in Table 5 . The words within parenthesis are actually meaningful words but segmented as simple words -LRB- words on the left -RRB- . The words lose part of the meaning when segmented as simple words . These errors occurred mostly with 3 or 4-character meaningful words . Errors of the second type are illustrated in Table 6 . Many of the errors are caused by incorrectly linking a character with a function word or pronoun . Some of the errors can easily be removed by using a list of function words and pronouns to identify these characters . 7.2 Errors of Omission Examples of definite errors of omission -LRB- bigrams with association scores below 1.0 but are words -RRB- are given in Table 7 . Most of the errors are rare words and time words . Some are ancient names , rare and unknown place names , as well as technical terms . Since our corpus comprises general news articles , these types of words are not frequent in the corpus . Time words like dates usually have low association values because they change everyday ! These errors can be reduced by incorporating a separate algorithm for recognizing them . The proportion of errors of the various types are given in Table 8 . A new statistical formula for identifying 2-character words in Chinese text , called the contextual information formula , was developed empirically using regression analysis . The focus was on identifying meaningful words -LRB- including multi-word terms and idioms -RRB- rather than simple words . The formula was found to give significantly and substantially better results than the mutual information formula . Contextual information in the form of the frequency of characters that are adjacent to the bigram being processed as well as the weighted document frequency of the overlapping bigrams were found to be significant factors for predicting the probablity that the bigram constitutes a word . Local information -LRB- e.g. the number of times the bigram occurs in the document being segmented -RRB- and the position of the bigram in the sentence were not found to be useful in determining words . Of the bigrams that the formula erroneously identified as words , about 80 % of them were actually simple words . Of the rest , many involved incorrect linking with a function words . Of the words that the formula failed to identify as words , more than a third of them were rare words or time words . The proportion of rare words increased as the threshold value used was lowered . These rare words can not be identified using statistical techniques . This study investigated a purely statistical approach to text Association Score between -1.0 and -2.0 the northern section of a construction project fragments of ancient books Association Score < -2.0 September 3rd day -LRB- name of a district in China -RRB- -LRB- name of an institution -RRB- the Book of Changes Table 7 . 2-character words with association score below -1.0 Errors of Commission Association score > 1.0 -LRB- No . of errors = 34 -RRB- Borderline Cases Association score : 1.0 to1 .0 -LRB- No . of cases : 210 -RRB- Errors of Omission Association score < 1.0 Association score : 1.0 to 2.0 -LRB- No . of errors = 43 -RRB- Association score < 2.0 -LRB- No . of errors = 22 -RRB- Simple words 82.3 % Not words 17.7 % Simple words 55.2 % Not words 20.5 % Meaningful words 24.3 % Rare words & time words 23.2 % Others 76.8 % Rare words & time words 63.6 % Others 36.4 % Table 8 . Proportion of errors of different types 88 segmentation . The advantage of the statistical approach is that it can be applied to any domain , provided that the document collection is sufficiently large to provide frequency information . A domain-specific dictionary of words is not required . In fact , the statistical formula can be used to generate a shortlist of candidate words for such a dictionary . On the other hand , the statistical method can not identify rare words and proper names . It is also fooled by combinations of function words that occur frequently and by function words that co-occur with other words . It is well-known that a combination of methods is needed to give the best segmentation results . The segmentation quality in this study can be improved by using a list of function words and segmenting the function words as single character words . A dictionary of common and well-known names -LRB- including names of persons , places , institutions , government bodies and classic books -RRB- could be used by the system to identify proper names that occur infrequently in the corpus . Chang et al. -LSB- 2 -RSB- developed a method for recognizing proper nouns using a dictionary of family names in combination with a statistical method for identifying the end of the name . An algorithm for identifying time and dates would also be helpful . It is not clear whether syntactic processing can be used to improve the segmentation results substantially . Our current work includes developing statistical formulas for identifying 3 and 4-character words , as well as investigating whether the statistical formula developed here can be used with other corpora . The approach adopted in this study can also be used to develop statistical models for identifying multi-word terms in English text . It would be interesting to see whether the regression model developed for English text is similar to the one developed in this study for Chinese text . Frantzi , Ananiadou & Tsujii -LSB- 7 -RSB- , using a different statistical approach , found that contextual information could be used to improve the identification of multi-word terms in English text . -LSB- 1 -RSB- Chang , C.-H. , and Chen , C.-D . A study of integrating Chinese word segmentation and part-of-speech tagging . Communications of COLIPS , 3 , 1 -LRB- 1993 -RRB- , 69-77 . -LSB- 2 -RSB- Chang , J.-S. , Chen , S.-D. , Ker , S.-J. , Chen , Y. , and Liu , J.S. . A multiple-corpus approach to recognition of proper names in Chinese texts . Computer Processing of Chinese and Oriental Languages , 8 , 1 -LRB- June 1994 -RRB- , 75-85 . -LSB- 3 -RSB- Church , K.W. , and Hanks , P. Word association norms , mutual information and lexicography . In Proceedings of the 27 th Annual Meeting of the Association for Computational Linguistics -LRB- Vancouver , June 1989 -RRB- , 76-83 . -LSB- 4 -RSB- Dai , J.C. , and Lee , H.J. . A generalized unification-based LR parser for Chinese . Computer Processing of Chinese and Oriental Languages , 8 , 1 -LRB- 1994 -RRB- , 1-18 . -LSB- 5 -RSB- Dai , Y. Developing a new statistical method for Chinese text segmentation . -LRB- Master s thesis in preparation -RRB- -LSB- 6 -RSB- Damerau , F.J. Generating and evaluating domain-oriented multi-word terms from texts . Information Processing & Management , 29 , 4 -LRB- 1993 -RRB- , 433-447 . -LSB- 7 -RSB- Frantzi , K.T. , Ananiadou , S. , and Tsujii , J . The C-value/NC - value method of automatic recognition for multi-word terms . In C. Nikolaou and C. Stephanidis -LRB- eds . -RRB- , Research and Advanced Technology for Digital Libraries , 2 nd European Conference , ECDL 98 -LRB- Heraklion , Crete , September 1998 -RRB- , Springer-Verlag , 585-604 . -LSB- 8 -RSB- Liang , N.Y. . The knowledge of Chinese words segmentation -LSB- in Chinese -RSB- . Journal of Chinese Information Processing , 4 , 2 -LRB- 1990 -RRB- , 42-49 . -LSB- 9 -RSB- Liu , I.M. Descriptive-unit analysis of sentences : Toward a model natural language processing . Computer Processing of Chinese & Oriental Languages , 4 , 4 -LRB- 1990 -RRB- , 314-355 . -LSB- 10 -RSB- Liu , Y. , Tan , Q. , and Shen , X.K. Xin xi chu li yong xian dai han yu fen ci gui fan ji zi dong fen ci fang fa -LSB- Modern Chinese Word Segmentation Rules and Automatic Word Segmentation Methods for Information Processing -RSB- . Qing Hua University Press , Beijing , 1994 . -LSB- 11 -RSB- Lua , K.T. Experiments on the use of bigram mutual information in Chinese natural language processing . Presented at the 1995 International Conference on Computer Processing of Oriental Languages -LRB- ICCPOL -RRB- -LRB- Hawaii , November 1995 -RRB- . Available : http://137.132.89.143/luakt/ publication.html -LSB- 12 -RSB- Lua , K.T. From character to word - An application of information theory . Computer Processing of Chinese & Oriental Languages , 4 , 4 -LRB- 1990 -RRB- , 304-312 . -LSB- 13 -RSB- Lua , K.T. , and Gan , G.W. . An application of information theory in Chinese word segmentation . Computer Processing of Chinese & Oriental Languages , 8 , 1 -LRB- 1994 -RRB- , 115-124 . -LSB- 14 -RSB- Nie , J.Y. , Hannan , M.L. , and Jin , W.Y. Unknown word detection and segmentation of Chinese using statistical and heuristic knowledge . Communications of COLIPS , 5 , 1 & 2 -LRB- 1995 -RRB- , 47-57 . -LSB- 15 -RSB- Ogawa , Y. , and Matsuda , T. Overlapping statistical word indexing : A new indexing method for Japanese text . In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval -LRB- Philadelphia , July 1997 -RRB- , ACM , 226-234 . -LSB- 16 -RSB- Sproat , R. , and Shih , C.L. . A statistical method for finding word boundaries in Chinese text . Computer Processing of Chinese & Oriental Languages , 4 , 4 -LRB- 1990 -RRB- , 336-351 . -LSB- 17 -RSB- Sproat , R. , Shih , C. , Gale , W. , and Chang , N . A stochastic finite-state word-segmentation algorithm for Chinese . Computational Lingustics , 22 , 3 -LRB- 1996 -RRB- , 377-404 . -LSB- 18 -RSB- Tung , C.-H. , and Lee , H.-J . Identification of unknown words from a corpus . Computer Processing of Chinese and Oriental Languages , 8 -LRB- Supplement , Dec. 1994 -RRB- , 131-145 . -LSB- 19 -RSB- Wu , Z. , and Tseng , G. ACTS : An automatic Chinese text segmentation system for full text retrieval . Journal of the American Society for Information Science , 46 , 2 -LRB- 1995 -RRB- , 83-96 . -LSB- 20 -RSB- Yeh , C.L. , and Lee , H.J. Rule-based word identification for mandarin Chinese sentences : A unification approach . Computer Processing of Chinese and Oriental Languages , 5 , 2 -LRB- 1991 -RRB- , 97-118 . 89

O O B I O B I I O B I O O B I O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O B I O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O B I O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

statistical formula
logistic regression
word boundary
natural language processing
contextual information
regression model
chinese text segmentation
mutual information

word boundary identification
multi-word terms


Towards

Content-Based

Relevance

Ranking

for

Video

Search

Most

existing

web

video

search

engines

index

videos

by

file

names

URLs

and

surrounding

texts

These

types

of

video

metadata

roughly

describe

the

whole

video

in

an

abstract

level

without

taking

the

rich

content

such

as

semantic

content

descriptions

and

speech

within

the

video

into

consideration

Therefore

the

relevance

ranking

of

the

video

search

results

is

not

satisfactory

as

the

details

of

video

contents

are

ignored

In

this

paper

we

propose

a

novel

relevance

ranking

approach

for

Web-based

video

search

using

both

video

metadata

and

the

rich

content

contained

in

the

videos

To

leverage

real

content

into

ranking

the

videos

are

segmented

into

shots

which

are

smaller

and

more

semantic-meaningful

retrievable

units

and

then

more

detailed

information

of

video

content

such

as

semantic

descriptions

and

speech

of

each

shots

are

used

to

improve

the

retrieval

and

ranking

performance

With

video

metadata

and

content

information

of

shots

we

developed

an

integrated

ranking

approach

which

achieves

improved

ranking

performance

We

also

introduce

machine

learning

into

the

ranking

system

and

compare

them

with

IR

model

-LRB-

information

retrieval

model

-RRB-

based

method

The

evaluation

results

demonstrate

the

effectiveness

of

the

proposed

ranking

methods

H.

-LSB-

Information

Storage

and

Retrieval

-RSB-

Information

Search

and

Retrieval

Search

process

General

Terms

Algorithms

Performance

Experimentation

Multimedia

search

has

become

an

active

research

field

due

to

the

rapid

increase

of

online-available

content

and

new

practical

applications

Search

technology

is

considered

the

key

to

navigating

the

Internet

's

growing

media

-LRB-

video

audio

and

image

-RRB-

collections

Google

Yahoo

Blinkx

and

other

search

companies

have

provided

elementary

video

search

engines

However

existing

video

search

engines

are

all

based

on

the

text

information

related

to

the

video

which

can

be

retrieved

from

web

pages

such

as

file

names

URLs

and

surrounding

texts

These

types

of

textual

information

can

be

considered

as

metadata

of

the

video

since

they

only

roughly

describe

the

video

There

is

no

doubt

that

text

searching

is

the

most

efficient

way

to

retrieve

information

-LRB-

even

when

searching

for

videos

-RRB-

because

it

well

matches

the

manner

of

human

thinking

However

only

using

metadata

is

far

form

people

's

expectation

in

video

searching

because

even

the

best

case

scenario

the

metadata

is

only

the

highly

concentrated

overview

of

a

video

with

many

losses

on

details

In

general

a

video

consists

of

many

shots

and

sub-events

with

a

temporal

main

thread

The

video

should

be

segmented

into

smaller

retrievable

units

that

are

directly

related

to

what

users

perceive

as

meaningful

Much

research

has

concentrated

on

segmenting

video

streams

into

shots

using

low

level

visual

features

-LSB-

-RSB-

Each

segment

has

its

own

scenes

and

meanings

In

many

cases

when

users

query

a

video

they

intend

to

find

some

desired

clips

in

the

video

instead

of

viewing

it

thoroughly

However

this

can

seldom

be

achieved

by

searching

the

surrounding

text

which

is

related

to

the

whole

video

Much

content

information

can

be

used

to

search

videos

and

shots

In

content-based

video

retrieval

systems

video

shots

can

be

classified

into

or

annotated

by

several

semantic

concepts

The

most

substantial

works

in

this

field

are

presented

in

the

TREC

Video

Retrieval

Evaluation

-LRB-

TRECVID

-RRB-

community

-LSB-

-RSB-

In

addition

speech

is

also

significant

information

which

has

close

connection

to

video

contents

Some

videos

are

associated

with

transcripts/closed

captions

which

are

provided

by

content

provider

Using

ASR

-LRB-

automatically

speech

recognition

-RRB-

to

generate

speech

text

is

another

practical

solution

In

this

paper

with

the

video

metadata

and

content

information

of

video

shots

we

index

and

rank

the

videos

in

a

way

similar

to

general

text-based

web

page

search

engines

The

IR-model

which

is

widely

employed

in

text

information

retrieval

and

web

page

search

will

be

applied

to

rank

the

search

results

by

examining

relevance

between

query

and

indexed

information

-LRB-

including

both

metadata

and

content

information

-RRB-

To

fully

utilize

the

content

information

and

get

a

better

ranking

performance

we

integrate

the

shot

relevance

into

video

relevance

That

is

the

ranking

is

decided

not

only

by

the

relevance

of

video

-LRB-

metadata

of

the

entire

video

-RRB-

but

also

by

all

the

relevant

shots

within

the

video

We

also

apply

learning

based

method

to

rank

the

search

results

based

on

a

set

of

features

extracted

from

the

corresponding

query

video

metadata

and

content

information

The

rest

of

this

paper

is

organized

as

follows

Section

introduces

the

IR-model

based

ranking

including

extraction

of

video

metadata

and

content

information

and

a

ranking

method

integrating

these

two

types

of

information

In

section

a

learning

based

ranking

approach

is

presented

Section

compares

the

ranking

performance

evolution

results

and

Section

concludes

the

paper

Copyright

is

held

by

the

author/owner

-LRB-

s

-RRB-

MM

October

Santa

Barbara

California

USA

ACM

In

the

traditional

text

retrieval

and

web

page

search

IR

-LRB-

Information

Retrieval

-RRB-

models

are

usually

used

to

evaluate

the

relevance

between

a

query

and

a

document

-LSB-

-RSB-

is

one

of

the

frequently

used

evaluation

methods

Given

a

query

Q

and

a

document

D

the

relevance

between

Q

and

D

can

be

modeled

as

the

summation

of

the

relevance

between

each

query

term

-LRB-

word

-RRB-

t

in

Q

and

D

Q

t

t

D

R

Q

D

R

-RRB-

-LRB-

-RRB-

-LRB-

-LRB-

-RRB-

where

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

t

w

D

t

tf

avdl

D

b

b

k

D

t

tf

k

t

D

R

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-LRB-

log

-RRB-

-LRB-

t

df

t

df

N

t

w

-LRB-

-RRB-

Here

k

and

b

are

parameters

tf

-LRB-

t

D

-RRB-

is

term

frequency

means

the

frequency

of

term

t

appears

in

document

D.

df

-LRB-

t

-RRB-

is

document

frequency

means

the

frequency

of

document

which

contains

term

t

within

all

documents

D

stands

for

the

length

of

document

D

The

basic

idea

of

this

IR

model

can

be

explained

as

if

the

query

term

appears

in

document

more

frequently

-LRB-

higher

tf

-RRB-

and

the

query

term

is

more

unique

that

less

documents

contain

it

-LRB-

lower

df

-RRB-

the

query

will

be

more

relevant

to

the

document

Index

and

Rank

the

Video

Information

The

video

data

used

in

our

experimental

system

are

from

MSN

Video

-LRB-

-RRB-

which

contains

videos

Metadata

of

the

video

Because

the

videos

in

our

data

set

are

made

by

professional

content

provider

there

is

rich

meta

information

that

describes

each

entire

video

with

brief

text

Each

video

has

the

following

metadata

fields

headline

caption

keywords

source

video

URL

thumbnail

image

URL

related

web

page

anchor

text

page

URL

etc.

Besides

these

types

of

textual

information

some

format

information

of

the

video

such

as

video

length

frame

size

bit

rate

and

creation

date

are

also

extracted

Some

selected

information

fields

of

video

metadata

are

listed

in

Table

Table

Video

metadata

Field

Example

Value

Headline

Discovery

launches

Caption

July

Watch

the

entire

launch

of

space

shuttle

D.

Source

MSNBC

Keywords

Technology

science

Space

Partner

Codes

Video

URL

Link

anchor

MSNBC.com

's

Technology

and

Science

front

Link

URL

date

PM

video

length

seconds

Frame

size

x

Bit

rate

Kbps

For

the

videos

contained

in

general

web

pages

some

attributes

mentioned

above

may

not

be

obtained

directly

but

the

surrounding

texts

URL

filename

can

be

extracted

as

the

metadata

fields

of

the

video

These

information

fields

correspond

to

document

D

in

Section

Different

fields

can

be

represented

by

different

type

of

D

-LRB-

D

i

in

Equation

-RRB-

The

overall

relevance

can

be

calculated

by

the

weighted

summation

of

the

relevance

of

all

fields

The

weight

of

the

fields

-LRB-

DW

i

in

Equation

-RRB-

can

be

determined

by

their

importance

significance

and

representativeness

to

the

video

-RRB-

-LRB-

-RRB-

-LRB-

Q

D

R

DW

Q

Video

R

i

i

-LRB-

-RRB-

In

our

system

four

major

information

fields

from

video

metadata

are

selected

to

be

indexed

headline

caption

keywords

and

source

Headline

is

a

highly

representative

description

of

the

video

content

Keywords

are

also

good

recapitulative

terms

For

these

two

fields

higher

weights

are

set

Caption

is

a

more

general

and

detail

depicts

for

the

video

Source

provided

a

higher

level

and

less

relevant

information

they

will

be

set

lower

weights

for

ranking

Table

gives

out

the

weights

of

fields

in

our

experimental

system

Table

Weights

for

relevance

evaluation

Fields

weight

Headline

Keywords

Caption

source

Content

information

of

the

video

shots

There

is

plenty

of

information

in

the

visual/audio

content

of

the

video

sequence

which

can

not

be

sufficiently

presented

by

the

aforementioned

textual

video

metadata

We

can

build

a

set

of

models

that

can

be

applied

to

automatically

detect

a

corresponding

set

of

concepts

such

that

each

video

shot

can

be

annotated

with

a

detection

confidence

score

for

each

concept

Successful

concept

modeling

and

detection

approaches

have

been

introduced

in

TRECVID

relying

predominantly

on

visual/aural

analysis

and

statistical

machine

learning

methods

-LSB-

-RSB-

The

LSCOM-lite

Lexicon

-LSB-

-RSB-

designed

for

the

TRECVID

Benchmark

consists

of

more

than

concepts

spread

across

multiple

concept-types

such

as

object

events

site

etc.

Though

the

size

of

the

lexicon

is

still

far

from

practical

application

for

general

Web-based

video

search

this

semantic

information

is

promising

to

enable

real

content-based

video

search

and

therefore

it

is

applied

in

our

ranking

system

Besides

visual

contents

information

from

audio

channel

especially

the

speech

is

also

very

useful

for

searching

videos

In

our

experimental

system

we

use

Microsoft

speech

recognition

engine

-LRB-

with

a

large

vocabulary

-RRB-

This

engine

gives

recognized

words

with

a

start

timestamp

length

and

a

recognition

confidence

value

which

are

very

useful

for

later

indexing

and

ranking

The

speech

texts

are

allotted

and

assigned

into

video

shots

according

to

the

timestamp

of

words

and

video

shots

The

content

information

is

associated

with

individual

video

shot

which

consist

of

semantic

keywords

-LRB-

with

corresponding

detection

confidences

-RRB-

and

speech

words

-LRB-

with

recognition

confidences

-RRB-

The

confidences

of

words

will

act

as

weights

of

term

frequencies

tf

to

calculate

the

relevance

in

Equation

-LRB-

-RRB-

Integrated

Ranking

with

Metadata

and

Content

Information

To

combine

metadata

and

content

to

rank

the

videos

we

index

the

videos

by

metadata

and

index

the

video

shots

by

content

information

separately

and

then

integrated

these

two

rank

lists

named

video

list

and

shot

list

to

form

a

final

ranking

The

integrated

ranking

returns

search

result

by

video

but

taking

all

the

relevance

shots

within

this

video

into

consideration

For

video

list

each

item

is

a

video

Let

item

iv

vid

denotes

the

video

ID

of

the

i

th

item

item

iv

score

denotes

the

ranking

score

of

the

i

th

item

For

shot

list

each

item

is

a

shot

from

a

video

Let

item

is

vid

item

is

sid

item

is

score

donote

the

video

ID

which

the

shot

belong

to

the

shot

ID

within

the

video

and

the

ranking

score

of

the

i

th

item

respectively

The

integrating

process

is

presented

in

Algorithm

The

basic

idea

is

that

all

the

ranking

score

of

the

relevance

shots

within

the

video

are

accumulated

to

the

ranking

score

of

the

video

with

corresponding

weights

The

relevant

shots

in

the

video

will

be

highlight

when

displaying

the

video

as

search

result

new

a

integrated

result

list

-LRB-

item

denotes

as

item

iI

-RRB-

for

each

item

iv

in

video

list

-LCB-

new

item

ic

item

iI

vid

item

is

vid

item

iI

score

item

iv

score

for

each

item

item

is

in

shot

list

-LCB-

if

-LRB-

item

iv

vid

item

is

vid

-RRB-

-LCB-

item

iI

addshot

-LRB-

item

is

sid

-RRB-

item

iI

score

item

is

score

remove

item

is

from

shot

list

-RCB-

-RCB-

remove

item

iv

from

video

list

add

item

iI

to

integrated

list

-RCB-

add

the

remaining

video

list

and

shot

list

into

the

integrated

list

sort

the

integrated

list

by

item

iI

score

and

are

weights

for

score

accumulating

Algorithm

Generate

integrated

rank

list

IR-model

based

ranking

just

consider

some

basic

features

such

as

term

frequency

tf

document

frequency

df

and

document

length

etc.

In

the

learning

based

approach

more

features

are

extracted

from

the

query

metadata

and

content

information

To

be

clear

suppose

the

query

contains

three

terms

a

b

c

we

compute

the

following

features

from

each

document

field

Ordered

match

the

frequency

that

both

a

and

b

appeared

in

the

indexed

text

and

b

appears

after

a

Partly

exact

match

the

frequency

that

a

b

or

b

c

appeared

in

the

indexed

text

Exact

match

the

frequency

that

a

b

c

appeared

in

the

indexed

text

Query

length

number

of

query

terms

For

the

content

information

each

word

has

a

confidence

value

we

also

consider

Weighted

tf

Term

frequency

with

confidence

weighted

High

confident

match

query

term

match

with

words

with

high

confidence

High

confident

words

words

with

high

confidence

in

the

indexed

text

Some

non-textual

query-independent

features

such

as

shot

length

video

length

frame

size

bit

rate

etc

are

also

taken

into

account

By

counting

in

the

combinations

of

several

document

fields

and

query

terms

-LRB-

or

part

of

query

-RRB-

we

have

about

dimensional

features

in

total

for

a

query

and

search

result

to

form

a

sample

The

GroundTruth

of

sample

is

the

relevance

judgments

between

a

query

and

a

result

which

is

collected

by

a

user

labeling

system

introduced

in

the

next

section

Neutral

Network

based

Ranking

Traditionally

the

learning

algorithms

are

used

for

classification

problems

For

ranking

problems

one

possible

way

is

organize

the

samples

by

pairs

Pair

sample

-LRB-

x

x

-RRB-

are

considered

as

a

positive

sample

if

x

are

ranked

ahead

of

x

and

vice

versa

The

loss

function

is

also

formulated

in

pair

wise

RankNET

-LSB-

-RSB-

is

used

in

our

implementation

to

train

the

ranking

model

and

to

validate

the

performance

About

half

of

the

labeled

data

are

used

in

training

and

the

second

half

are

used

for

validation

To

evaluate

the

ranking

performance

of

our

proposed

methods

we

developed

a

user

labeling

tool

to

collect

some

query-result

relevance

judgments

The

video

data

set

we

used

in

our

experiment

includes

news

video

TV

programs

movie

trailers

advertisements

etc.

According

to

the

characteristics

of

the

content

of

these

videos

we

selected

some

news

related

queries

such

as

hot

events

hot

place

and

hot

person

names

to

evaluate

the

ranking

performance

For

each

query

we

use

the

IR-model

based

ranking

describe

in

Section

to

generate

a

result

list

and

randomly

select

some

results

form

the

list

to

label

Considering

the

labeling

workload

for

each

labeler

and

each

query

results

are

select

from

the

list

To

make

the

selected

query-result

samples

have

a

good

uniformity

on

distribution

results

are

randomly

selected

from

the

first

part

of

the

list

are

from

the

second

part

and

the

other

are

from

the

last

part

The

order

of

these

selected

results

is

shuffled

and

then

provided

to

users

to

do

relevance

labeling

In

the

labeling

tool

for

a

query

and

a

result

user

can

see

all

the

information

of

the

result

including

the

file

format

information

-LRB-

frame

size

bit

rate

video

length

etc

-RRB-

description

-LRB-

headline

caption

keywords

-RRB-

video

thumbnail

video

-LRB-

in

a

video

player

-RRB-

thumbnails

of

video

shots

the

speech

text

of

the

relevant

shots

The

words

matched

with

query

terms

are

highlighted

See

Figure

If

there

are

relevant

shots

in

the

result

the

thumbnails

of

them

are

displayed

with

doubled

size

The

shot

number

time

information

and

the

speech

are

also

shown

in

the

interface

Users

are

asked

to

read

the

displayed

information

browse

the

thumbnails

and

play

the

video

and

shots

-LRB-

a

tool

button

is

provided

to

play

from

one

shot

-RRB-

to

give

a

relevance

judgment

from

and

which

represent

bad

fair

good

excellent

and

perfect

respectively

Figure

Relevance

labeling

tool

In

our

experiment

ten

users

are

invited

to

do

labeling

and

about

relevance

judgments

of

query-result

samples

are

collected

Precision

Performance

of

Ranking

We

have

conducted

a

comparison

between

the

approaches

listed

below

MR

Ranking

only

based

on

video

metadata

-LRB-

Section

-RRB-

CR

Ranking

only

by

content

information

-LRB-

Section

-RRB-

RI

Integrated

Ranking

described

in

Section

RN

RankNET

based

ranking

described

in

Section

The

precision

in

top

N

of

the

rank

lists

of

all

the

labeled

queries

is

used

to

evaluate

the

performance

of

ranking

method

N

top

in

results

labeled

total

N

top

in

results

labeled

relevant

N

Precision

-LRB-

-RRB-

In

our

implementation

the

judgment

Perfect

or

Excellent

are

considered

as

relevant

results

while

other

judgments

are

treated

as

irrelevant

results

The

Presicion

@N

-LRB-

N

to

-RRB-

of

the

ranking

methods

are

shown

in

Table

From

the

results

we

can

see

that

-RRB-

Precisions

of

MR

are

very

low

Only

using

video

metadata

will

result

in

a

poor

performance

since

details

of

the

video

content

are

ignored

The

content

information

is

more

effective

to

search

and

rank

video

than

metadata

as

precisions

of

CR

are

higher

than

that

of

MR.

-RRB-

Precisions

of

RI

are

much

higher

than

that

of

MR

and

CR

By

combining

video

metadata

and

content

information

the

performance

is

significantly

improved

and

reaches

an

acceptable

level

which

shows

that

content-based

relevance

ranking

is

a

promising

approach

-RRB-

RN

has

a

good

performance

even

better

than

RI

Comparing

to

IR-model

based

ranking

more

features

are

included

to

learning

the

relevance

The

result

implies

that

the

learning

method

can

organize

the

information

for

ranking

in

a

more

effective

way

Table

Precision

of

the

ranking

approaches

Precision

MR

CR

RI

RN

We

have

presented

a

novel

content-based

approach

to

rank

video

search

results

In

addition

to

the

video

metadata

more

detailed

content

information

in

the

video

is

used

to

improve

the

relevance

ranking

of

video

search

results

The

videos

are

segmented

into

shots

which

can

carry

rich

content

information

such

as

semantic

concept

keywords

and

speech

With

the

video

metadata

and

content

information

we

proposed

an

IR-model

based

ranking

method

and

a

learning-based

ranking

method

Evaluation

of

the

top

ranked

results

shows

that

the

proposed

ranking

methods

have

significantly

improved

performance

comparing

to

the

approach

use

video

metadata

only

which

is

frequently

used

in

existing

web

video

search

engines

In

future

work

more

types

of

content

information

can

be

integrated

into

our

ranking

scheme

such

as

content-based

quality

metric

user

comments

and

rating

for

videos

shared

in

web

communities

Moreover

how

to

define

effective

semantic

concepts

i.e.

video

semantic

ontology

that

facilitate

video

searching

and

ranking

is

also

a

challenging

problem.

which

is

also

one

of

our

future

works

-LSB-

-RSB-

Hong-Jiang

Zhang

A.

Kankanhalli

and

S.

Smoliar

Automatic

Partitioning

of

Full-motion

Video

A

Guided

Tour

of

Multimedia

Systems

and

Applications

IEEE

Computer

Society

Press

-LSB-

-RSB-

-LSB-

-RSB-

S.

E.

Robertson

S.

Walker

and

M.

Beaulieu

Okapi

at

TREC

automatic

ad

hoc

filtering

VLC

and

filtering

tracks

In

Proceedings

of

TREC

-LSB-

-RSB-

M.

Naphade

J.R.

Smith

F.

Souvannavong

On

the

Detection

of

Semantic

Concepts

at

TRECVID

ACM

Multimedia

ACM

Press

New

York

NY

pp.

Oct.

-LSB-

-RSB-

M.

Naphade

L.

Kennedy

J.R.

Kender

S.F.

Chang

J.R.

Smith

P.

Over

A.

Hauptmann

LSCOM-lite

A

Light

Scale

Concept

Ontology

for

Multimedia

Understanding

for

TRECVID

IBM

Research

Tech

Report

-LRB-

-RRB-

May

-LSB-

-RSB-

Chris

Burges

et.al

Learning

to

Rank

using

Gradient

Descent

ICML

Bonn

Germany

August

O O B I O B I O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O B I O O O B I O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O B I O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O B I O O B O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O B I O O O O O O O O B I O B I O O O O O B O O O O O O O B I O O O O O O O O O O B I O B I O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

integrated ranking
ranking method
video metadata
segmented
learning based ranking
video search
video retrieval
neutral network based ranking
metadata
relevance ranking
content information

ir model based ranking
ir-model
content-based ranking
video segmentation
content-based relevance ranking
video index, relevance ranking
machine learning model
content-based approach