Accelerating 3D Convolution using Graphics Hardware Many volume filtering operations used for image enhancement , data processing or feature detection can be written in terms of three-dimensional convolutions . It is not possible to yield interactive frame rates on todays hardware when applying such convolutions on volume data using software filter routines . As modern graphics workstations have the ability to render two-dimensional convoluted images to the frame buffer , this feature can be used to accelerate the process significantly . This way generic 3D convolution can be added as a powerful tool in interactive volume visualization toolkits . Direct volume rendering is a very important technique for visualizing three dimensional data . Several fundamental different methods have been introduced -LSB- 2 , 4 , 5 , 6 , 8 , 12 -RSB- . Hardware-based volume texturing -LSB- 9 , 14 -RSB- is one of the most prominent variants for interactive visualization due to the high frame rates that can be achieved with this technique . The basic principle of texture based volume rendering is depicted in Figure 1 . According to the sampling theorem a 3D view of the volume is generated by drawing an adequate number of equidistant , semi-transparent polygons parallel to the image plane with respect to the current viewing direction -LRB- `` volume slicing '' -RRB- . Filtering on the other hand is a major part of the visualization pipeline . It is broadly used for improving images , reducing noise , and enhancing detail structure . Volume rendering can benefit from filter operations , as low pass filters reduce the noise e.g. of sam-pled medical volume images and high pass filters can be used for edge extraction , visualizing prominent data features . Multiscale approaches as -LSB- 13 -RSB- regularly use disjunct filtering and downsampling steps and can benefit from any speedups of the filtering process . -LCB- hopf , ertl -RCB- @informatik . uni-stuttgart . de Institut f ur Informatik Breitwiesenstr . 20 22 70565 Stuttgart Germany Figure 1 : Texture based volume rendering Filters can be classified as linear or non-linear . Discrete linear filters can be written as convolutions with filter kernels that completely specify the filtering operation . Non-linear filters , as for instance morphological operators , were recently used for volume analysis and visualization -LSB- 7 -RSB- . Segmentation and classification depend heavily on filtering operations as well . Bro-Nielson -LSB- 1 -RSB- already thought about using convolution hardware for accelerating the registration process . For texture based volume rendering the data set has to be loaded into special texture memory , which can be addressed by the graphics pipe very fast . The loading process itself is relatively slow , taking several seconds for big data sets even on the fastest available graphics workstations . As the data set has to be reloaded after a filter operation has been performed in software , interactive filtering will benefit a lot from convolution algorithms that directly work on the texture hardware . Additionally , we will show in the following that computing the convolution with graphics hardware is much faster than software solutions . The general three-dimensional discrete convolution can be written as ~ f -LRB- x , y , z -RRB- = i 1 , i 2 , i 3 k -LRB- i 1 , i 2 , i 3 -RRB- f -LRB- x + i 1 , y + i 2 , z + i 3 -RRB- with f being the input data function and k being the filter kernel , resulting in the convoluted data ~ f . In the following examination we will assume without loss of generality that k -LRB- i 1 , i 2 , i 3 -RRB- is given for 0 i 1 , i 2 , i 3 < n and vanishes outside this interval . Also , we will assume that the input data function vanishes for -LRB- x , y , z -RRB- outside the interval -LSB- 0 , m -RRB- 3 . In the special case of k -LRB- i 1 , i 2 , i 3 -RRB- = k 1 -LRB- i 1 -RRB- k 2 -LRB- i 2 -RRB- k 3 -LRB- i 3 -RRB- the 0-7803-5897-X / 99 / $ 10.00 Copyright 1999 IEEE 471 0 0.1 0.2 0.3 0.4 0.5 -3 -2 -1 0 1 2 3 0.1 0.2 0.3 0 1 2 3 4 5 6 7 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 -3 -2 -1 0 1 2 3 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0 1 2 3 4 5 6 7 Figure 2 : The Gau filter function and its second derivative Figure 3 : Example image ; original , filtered with Gau , and filtered with second derivative kernel is called separable . In this case the number of operations necessary for the convolution can be reduced down to O -LRB- m 3 n -RRB- , from O -LRB- m 3 n 3 -RRB- in the non-separable case : ~ f 1 -LRB- x , y , z -RRB- = i 1 k 1 -LRB- i 1 -RRB- f -LRB- x + i 1 , y , z -RRB- -LRB- 1 -RRB- ~ f 2 -LRB- x , y , z -RRB- = i 2 k 2 -LRB- i 2 -RRB- ~ f 1 -LRB- x , y + i 2 , z -RRB- -LRB- 2 -RRB- ~ f -LRB- x , y , z -RRB- = i 3 k 3 -LRB- i 3 -RRB- ~ f 2 -LRB- x , y , z + i 3 -RRB- -LRB- 3 -RRB- Of course special care has to be taken near the boundaries of the input data function , as convolution routines are generally written on a very low language level for speed purposes . Figure 2 shows two well known convolution filters , the Gau filter and its second derivative , both in their continuous and discrete forms . They can be used for noise reduction and edge detection , respectively . An example image that was filtered with these kernels can be seen in Figure 3 . In order to accelerate the convolution process , special purpose hardware can be used . On systems that have built-in Digital Signal Processors -LRB- DSPs -RRB- , for example for multimedia acceleration , a spe-cialized convolution subroutine could be downloaded to the signal 2D Convolution Figure 4 : The first pass of the hardware filtering algorithm 0 1 000000000000000000000000000000 000000000000000000000000000000 000000000000000000000000000000 000000000000000000000000000000 111111111111111111111111111111 111111111111111111111111111111 111111111111111111111111111111 111111111111111111111111111111 Used texture coordinates 0 00 00 11 11 the data value 15/16 Texel inside a texel Exact position of 1/16 3/4 7/8 1 Texture coordinates 5/8 1/8 1/4 3/8 1/2 00 00 11 11 00 00 11 11 00 00 11 11 00 00 11 11 00 00 11 11 Figure 5 : Texture coordinates used for exact texel hits processor . On the other hand , most times these DSPs are not well documented or the run-time system can not be modified by the user . In general they are not faster than the main processor as well . Additionally , there exists a wide range of different DSP systems , all of which are incompatible to each other . The approach we have implemented in our system is to combine a 2D and a 1D convolution kernel in order to calculate three-dimensional separable convolutions . Several vendors of the graphics API OpenGL -- as for example Silicon Graphics Inc. -LSB- 10 -RSB- - have included extensions for one - and two-dimensional filtering . In contrast to most implementations that emulate these extensions only in software , the SGI graphics pipes MXE and BasicReality calculate the convolutions on-board , boosting performance by an order of magnitude already for reasonably sized filters . The CRM graphics system of the O 2 is capable of rendering convolutions in hardware as well , but it does not support volume textures , which are crucial for the algorithm as well . Recall that the volume data is already stored in texture memory for visualization using texture hardware . Now -LRB- 1 -RRB- and -LRB- 2 -RRB- are combined to one 2D convolution that is to be applied to every plane of the volume data perpendicular to the z-axis . Therefore , plane by plane is drawn by rendering textured triangle strips -LRB- two triangles per strip -RRB- into the frame buffer as it is sketched in Figure 4 . The texture coordinates assigned to the vertices of the triangle strips are specified in such way that no interpolation of the texture is necessary -LRB- see Figure 5 -RRB- . In order to increase the potential speedup and to avoid rounding problems , nearest neighbor interpolation is activated during the rendering process . Each plane is then read back with the OpenGL routine glCopyTexSubImage3DEXT , which replaces one plane of the active volume texture orthogonal to the z-axis by data that is read directly from the frame buffer . While transfering the data to the texture memory , the separable 2D convolution filter is activated using glSeparableFilter2DEXT . After this first pass , the volume texture contains data filtered along the x - and y-axes . 472 1D Convolution Figure 6 : The second pass Framebuffer Operations Memory Texture Pixel Data Textures Pixel Transfer Modes Primitives Per-Fragment Rasterization Engine Geometry Geometric Scale , Bias Clamping Storage Mode Convolution Post-Convolution Scale , Bias Pixel Figure 7 : The OpenGL graphics pipeline Applying the convolution to the third axis is more complicated and depicted in Figure 6 . In this second pass planes are rendered perpendicular to one of the other axes . Assume that the y-axis has been chosen . As glCopyTexSubImage3DEXT can not write planes orthogonal to any other axis than the z-axis to the texture memory , they have to be transfered to a second volume texture . OpenGL 's texture objects are used by calling glBindTexture for switching between them , which implies only a very small overhead . While copying the data from the frame buffer to the texture memory , a one-dimensional convolution filter is activated . As we are dealing with two-dimensional image data , we specify a 2D convolution filter with glConvolutionFilter2DEXT , using a filter kernel that is exactly one pixel wide . After this second pass the convoluted volume data has been mirrored at the plane y - z = 0 . For texture based volume renderers this does not impose any restrictions , as they only have to swap coordinates during texture coordinate calculations . When data order is crucial for the application , the algorithm of pass two can be used for both passes , thus restoring the data order in the second pass . However , the textured planes have to be drawn two times perpendicular to the y-axis . Cache misses are much more likely in this case in respect to planes rendered orthogonal to the z-axis . This can increase the convolution times on big volumes by up to 50 % even on fast graphics hardware . Figure 7 depicts the relevant part of the OpenGL pipeline . It reveals , that pixel fragments read from the frame buffer are clamped to -LSB- 0 , 1 -RRB- before they can be written back to the frame buffer or into the texture memory . Filter kernels with negative coefficients can compute negative intermediary values during the two-dimensional convolution pass , which will not contribute to the final 1D convolution . Additionally , no negative results can be stored in the output volume . These values are especially needed when the filter kernel is not symmetrical . The strategy for avoiding these effects depends on the particular application . For edge detection the absolute maxima of the filtered volume data are of interest . In this case , calculating the absolute value can be performed in hardware as well , further reducing necessary computations on the CPU . In most other cases post-convolution scaling and bias can be used to map the expected results to the interval -LSB- 0 , 1 -RRB- just before the clamping takes place . OpenGL provides the GL POST CONVOLUTION c SCALE EXT and GL POST CONVOLUTION c BIAS EXT parameters , which are applied to pixel color values after convolution and before clamping as depicted in Figure 7 . The used data sets are presented on the color plate . Figures 8 to 12 show slices of a head data set of size 128 3 . Figure 8 reveals the unfiltered data set , whereas Figure 9 and 10 present slices of the software and hardware low pass filtered volume data , respectively . Here , a Gau filter kernel of size 5 3 has been used . Almost no differences can be detected . Figure 11 and 12 depict the results for high pass filtering using the second derivative of the Gau filter , again computed in software and in hardware . The hardware convoluted volume displays noticeable artifacts that occur due to the already mentioned clamping step in the OpenGL pipeline . By using post-convolution scaling and bias the artifacts disappear completely . The Figures 13 to 18 picture another data set that has been used for testing the implementation . They have been visualized with the hardware based volume rendering toolkit TiVOR -LSB- 11 -RSB- , again with the first picture being rendered with the original data set . While the noise reduction effect of the Gau filter is rather bothering in Figure 14 by smearing volume details , it has remarkable positive effects on ISO surface generation -LRB- compare Figures 17 and 18 -RRB- . Please remark that the ISO surfaces are rendered in real-time using a hardware accelerated volume rendering approach described in -LSB- 14 -RSB- . Noise interferes with high pass filters , which can be seen in Figure 15 . Using a high pass filter on the already low pass filtered data set reveals by far more and better separable details -LRB- see Figure 16 -RRB- compared to the directly filtered volume . We have tested the speed our implementation against a well tuned software convolution filter . Unsurprisingly , the software convolution is almost completely memory bound . Even extremely fast workstations as the Octane are limited by the main memory band-473 Filter size 2 3 3 3 5 3 7 3 head 0.33 / 0.72 0.33 / 1.02 0.33 / 1.56 0.48 / 2.0 angio 2.5 / 6.0 2.5 / 8.7 2.5 / 14.7 3.7 / 21.3 Data set created by computer tomography , 128 3 Data set created by MR angiography , 256 3 All times were measured on a Silicon Graphics Onyx2 equipped with a BasicReality graphics pipe . The system has two R10000/195 MHz processors and 640 MB main memory . Table 1 : Convolution times in seconds using hardware / software width , as today 's caches are far too small for the values needed for convolution along the z-axis . High end machines as the Onyx2 perform huge 3D convolutions three times faster than the Octane , even when equipped with slower CPUs . Standard PCs can not cope with the memory bandwidth of the Onyx2 system , and multipro-cessor options will not accelerate the process because it is not CPU bound . Table 1 shows convolution times for different data sets and filter sizes , using software and hardware convolution . All times have been measured on an Onyx2 equipped with a BasicReality graphics pipe . The maximum filter size supported by the graphics system is 7 2 . Therefor , the maximum 3D convolution that can be performed in hardware on this system is 7 3 . Noteworthy is the fact that the BasicReality graphics system is optimized for filter kernels of size 5 2 . Convolutions with smaller kernels need exactly the same computation time . Filters of size 6 2 and 7 2 share their timing results as well . The x and y coordinates of the volume are swapped during the hardware based convolution process , which is a side effect of the presented 3D convolution algorithm . As several of today 's graphics workstation vendors have added two-dimensional convolution to their OpenGL pipeline , using this capability for accelerating 3D convolution is an almost straightforward approach . We have determined that by using the implemented algorithm three-dimensional convolution can be performed even on big data sets with nearly interactive rates . First promising approaches of accelerating wavelet decomposition and reconstruction have been investigated as well -LSB- 3 -RSB- . As all intermediary data is transfered to the frame buffer , clamping can swallow negative values that result from the two-dimensional convolution as well as final negative results . Thus this approach is currently most useful for symmetrical filter kernels . By using post-convolution scaling and bias extensions these problems can be easily overcome . Non-separable convolutions are not possible right now with this algorithm . However , by applying several two-dimensional filter kernels and blending convoluted images in the frame buffer the use of non-separable 3D kernels will be a possibility for the future as well . -LSB- 1 -RSB- M Bro-Nielson . Medical Image Registration and Surgery Simulation . PhD thesis , IMM , Technical University of Denmark , 1996 . -LSB- 2 -RSB- R. A. Crawfis and N. L. Max . Texture Splats for 3D Scalar and Vector Field Visualization . In G. M. Nielson and Bergeron D. , editors , Visualization 93 , pages 261 265 , Los Alamitos , CA , 1993 . IEEE Computer Society , IEEE Computer Society Press . -LSB- 3 -RSB- M. Hopf and T. Ertl . Hardware Based Wavelet Transformations . In Erlangen Workshop '99 on Vision , Modeling and Visualization , Erlangen , November 1999 . IEEE . accepted for publication , November 17 19 . -LSB- 4 -RSB- A. Kaufman . Volume Visualization . IEEE Computer Society Press , 1991 . -LSB- 5 -RSB- P. Lacroute and M. Levoy . Fast Volume Rendering Using a Shear-Warp Factorization of the Viewing Transformation . In Computer Graphics Proceedings , Annual Conference Series , pages 451 457 , Los Angeles , California , July 1994 . ACM SIGGRAPH , Addison-Wesley Publishing Company , Inc. -LSB- 6 -RSB- L. Lippert , M. H. Gross , and C. Kurmann . Compression Domain Volume Rendering for Distributed Environments . In D. Fellner and L. Szirmay-Kalos , editors , EUROGRAPHICS ' 97 , volume 14 , pages C95 C107 . Eurographics Association , Blackwell Publishers , 1997 . -LSB- 7 -RSB- C. L urig and T. Ertl . Hierarchical Volume Analysis and Visualization Based on Morphological Operators . In Proc . IEEE Visualization ' 98 , pages 335 341 , 1998 . -LSB- 8 -RSB- P. Schr oder and G. Stoll . Data Parallel Volume Rendering as Line Drawing . In 1992 Workshop on Volume Visualization . ACM SIGGRAPH , October 1992 . -LSB- 9 -RSB- Silicon Graphics Inc. , Mountain View , California . Volume Rendering using RE2 Technology , 1994 . -LSB- 10 -RSB- Silicon Graphics Inc. , Mountain View , California . OpenGL on Silicon Graphics Systems , 1996 . -LSB- 11 -RSB- O. Sommer , A. Dietz , R. Westermann , and T. Ertl . An Interactive Visualization and Navigation Tool for Medical Volume Data . In N. M. Thalmann and V. Skala , editors , WSCG ' 98 , The Sixth International Conference in Central Europe on Computer Graphics and Visualization ' 98 , volume II , pages 362 371 , Plzen , Czech Republic , February 1998 . University of West Bohemia Press . -LSB- 12 -RSB- T. Totsuka and M. Levoy . Frequency Domain Volume Rendering . Computer Graphics , 27 -LRB- 4 -RRB- : 271 78 , August 1993 . -LSB- 13 -RSB- R. Westermann and T. Ertl . A Multiscale Approach to Integrated Volume Segmentation and Rendering . In Computer Graphics Forum 16 -LRB- 3 -RRB- -LRB- Proc . EUROGRAPHICS ' 97 -RRB- , number 3 , pages 117 129 . Blackwell , 1997 . -LSB- 14 -RSB- R. Westermann and T. Ertl . Efficiently Using Graphics Hardware in Volume Rendering Applications . In Computer Graphics Proceedings , Annual Conference Series , pages 169 177 , Orlando , Florida , 1998 . ACM SIGGRAPH . 474 Figure 8 : The unfiltered head data set Figure 9 : Head , low pass filtered in software Figure 10 : Head , low pass filtered in hardware Figure 11 : Head , high pass filtered in software Figure 12 : Head , high pass filtered in hardware Figure 13 : The original angiography data set Figure 14 : The Gau filtered data set Figure 15 : Data , after direct filtering with Gau ' second derivative Figure 16 : First low pass , then high pass filtered data Figure 17 : ISO surfaces on the original angiography data set Figure 18 : ISO surfaces on the Gau filtered data set

O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

3d convolution
volume rendering
convolution
filtering
visualization

hardware acceleration
volume visualization


Beyond

Pagerank

Machine

Learning

For

Static

Ranking

Since

the

publication

of

Brin

and

Page

's

paper

on

PageRank

many

in

the

Web

community

have

depended

on

PageRank

for

the

static

-LRB-

query-independent

-RRB-

ordering

of

Web

pages

We

show

that

we

can

significantly

outperform

PageRank

using

features

that

are

independent

of

the

link

structure

of

the

Web

We

gain

a

further

boost

in

accuracy

by

using

data

on

the

frequency

at

which

users

visit

Web

pages

We

use

RankNet

a

ranking

machine

learning

algorithm

to

combine

these

and

other

static

features

based

on

anchor

text

and

domain

characteristics

The

resulting

model

achieves

a

static

ranking

pairwise

accuracy

of

-LRB-

vs.

for

PageRank

or

for

random

-RRB-

I.

-LSB-

Artificial

Intelligence

-RSB-

Learning

H.

-LSB-

Information

Storage

and

Retrieval

-RSB-

Information

Search

and

Retrieval

Algorithms

Measurement

Performance

Experimentation

Over

the

past

decade

the

Web

has

grown

exponentially

in

size

Unfortunately

this

growth

has

not

been

isolated

to

good-quality

pages

The

number

of

incorrect

spamming

and

malicious

-LRB-

e.g.

phishing

-RRB-

sites

has

also

grown

rapidly

The

sheer

number

of

both

good

and

bad

pages

on

the

Web

has

led

to

an

increasing

reliance

on

search

engines

for

the

discovery

of

useful

information

Users

rely

on

search

engines

not

only

to

return

pages

related

to

their

search

query

but

also

to

separate

the

good

from

the

bad

and

order

results

so

that

the

best

pages

are

suggested

first

To

date

most

work

on

Web

page

ranking

has

focused

on

improving

the

ordering

of

the

results

returned

to

the

user

-LRB-

query-dependent

ranking

or

dynamic

ranking

-RRB-

However

having

a

good

query-independent

ranking

-LRB-

static

ranking

-RRB-

is

also

crucially

important

for

a

search

engine

A

good

static

ranking

algorithm

provides

numerous

benefits

Relevance

The

static

rank

of

a

page

provides

a

general

indicator

to

the

overall

quality

of

the

page

This

is

a

useful

input

to

the

dynamic

ranking

algorithm

Efficiency

Typically

the

search

engine

's

index

is

ordered

by

static

rank

By

traversing

the

index

from

high-quality

to

low-quality

pages

the

dynamic

ranker

may

abort

the

search

when

it

determines

that

no

later

page

will

have

as

high

of

a

dynamic

rank

as

those

already

found

The

more

accurate

the

static

rank

the

better

this

early-stopping

ability

and

hence

the

quicker

the

search

engine

may

respond

to

queries

Crawl

Priority

The

Web

grows

and

changes

as

quickly

as

search

engines

can

crawl

it

Search

engines

need

a

way

to

prioritize

their

crawl

to

determine

which

pages

to

re-crawl

how

frequently

and

how

often

to

seek

out

new

pages

Among

other

factors

the

static

rank

of

a

page

is

used

to

determine

this

prioritization

A

better

static

rank

thus

provides

the

engine

with

a

higher

quality

more

up-to

date

index

Google

is

often

regarded

as

the

first

commercially

successful

search

engine

Their

ranking

was

originally

based

on

the

PageRank

algorithm

-LSB-

-RSB-

-LSB-

-RSB-

Due

to

this

-LRB-

and

possibly

due

to

Google

's

promotion

of

PageRank

to

the

public

-RRB-

PageRank

is

widely

regarded

as

the

best

method

for

the

static

ranking

of

Web

pages

Though

PageRank

has

historically

been

thought

to

perform

quite

well

there

has

yet

been

little

academic

evidence

to

support

this

claim

Even

worse

there

has

recently

been

work

showing

that

PageRank

may

not

perform

any

better

than

other

simple

measures

on

certain

tasks

Upstill

et

al.

have

found

that

for

the

task

of

finding

home

pages

the

number

of

pages

linking

to

a

page

and

the

type

of

URL

were

as

or

more

effective

than

PageRank

-LSB-

-RSB-

They

found

similar

results

for

the

task

of

finding

high

quality

companies

-LSB-

-RSB-

PageRank

has

also

been

used

in

systems

for

TREC

's

very

large

collection

and

Web

track

competitions

but

with

much

less

success

than

had

been

expected

-LSB-

-RSB-

Finally

Amento

et

al.

-LSB-

-RSB-

found

that

simple

features

such

as

the

number

of

pages

on

a

site

performed

as

well

as

PageRank

Despite

these

the

general

belief

remains

among

many

both

academic

and

in

the

public

that

PageRank

is

an

essential

factor

for

a

good

static

rank

Failing

this

it

is

still

assumed

that

using

the

link

structure

is

crucial

in

the

form

of

the

number

of

inlinks

or

the

amount

of

anchor

text

In

this

paper

we

show

there

are

a

number

of

simple

url

or

page

based

features

that

significantly

outperform

PageRank

-LRB-

for

the

purposes

of

statically

ranking

Web

pages

-RRB-

despite

ignoring

the

Copyright

is

held

by

the

International

World

Wide

Web

Conference

Committee

-LRB-

-RRB-

Distribution

of

these

papers

is

limited

to

classroom

use

and

personal

use

by

others

WWW

May

Edinburgh

Scotland

ACM

structure

of

the

Web

We

combine

these

and

other

static

features

using

machine

learning

to

achieve

a

ranking

system

that

is

significantly

better

than

PageRank

-LRB-

in

pairwise

agreement

with

human

labels

-RRB-

A

machine

learning

approach

for

static

ranking

has

other

advantages

besides

the

quality

of

the

ranking

Because

the

measure

consists

of

many

features

it

is

harder

for

malicious

users

to

manipulate

it

-LRB-

i.e.

to

raise

their

page

's

static

rank

to

an

undeserved

level

through

questionable

techniques

also

known

as

Web

spamming

-RRB-

This

is

particularly

true

if

the

feature

set

is

not

known

In

contrast

a

single

measure

like

PageRank

can

be

easier

to

manipulate

because

spammers

need

only

concentrate

on

one

goal

how

to

cause

more

pages

to

point

to

their

page

With

an

algorithm

that

learns

a

feature

that

becomes

unusable

due

to

spammer

manipulation

will

simply

be

reduced

or

removed

from

the

final

computation

of

rank

This

flexibility

allows

a

ranking

system

to

rapidly

react

to

new

spamming

techniques

A

machine

learning

approach

to

static

ranking

is

also

able

to

take

advantage

of

any

advances

in

the

machine

learning

field

For

example

recent

work

on

adversarial

classification

-LSB-

-RSB-

suggests

that

it

may

be

possible

to

explicitly

model

the

Web

page

spammer

's

-LRB-

the

adversary

-RRB-

actions

adjusting

the

ranking

model

in

advance

of

the

spammer

's

attempts

to

circumvent

it

Another

example

is

the

elimination

of

outliers

in

constructing

the

model

which

helps

reduce

the

effect

that

unique

sites

may

have

on

the

overall

quality

of

the

static

rank

By

moving

static

ranking

to

a

machine

learning

framework

we

not

only

gain

in

accuracy

but

also

gain

in

the

ability

to

react

to

spammer

's

actions

to

rapidly

add

new

features

to

the

ranking

algorithm

and

to

leverage

advances

in

the

rapidly

growing

field

of

machine

learning

Finally

we

believe

there

will

be

significant

advantages

to

using

this

technique

for

other

domains

such

as

searching

a

local

hard

drive

or

a

corporation

's

intranet

These

are

domains

where

the

link

structure

is

particularly

weak

-LRB-

or

non-existent

-RRB-

but

there

are

other

domain-specific

features

that

could

be

just

as

powerful

For

example

the

author

of

an

intranet

page

and

his/her

position

in

the

organization

-LRB-

e.g.

CEO

manager

or

developer

-RRB-

could

provide

significant

clues

as

to

the

importance

of

that

page

A

machine

learning

approach

thus

allows

rapid

development

of

a

good

static

algorithm

in

new

domains

This

paper

's

contribution

is

a

systematic

study

of

static

features

including

PageRank

for

the

purposes

of

-LRB-

statically

-RRB-

ranking

Web

pages

Previous

studies

on

PageRank

typically

used

subsets

of

the

Web

that

are

significantly

smaller

-LRB-

e.g.

the

TREC

corpus

used

by

many

contains

only

million

pages

-RRB-

Also

the

performance

of

PageRank

and

other

static

features

has

typically

been

evaluated

in

the

context

of

a

complete

system

for

dynamic

ranking

or

for

other

tasks

such

as

question

answering

In

contrast

we

explore

the

use

of

PageRank

and

other

features

for

the

direct

task

of

statically

ranking

Web

pages

We

first

briefly

describe

the

PageRank

algorithm

In

Section

we

introduce

RankNet

the

machine

learning

technique

used

to

combine

static

features

into

a

final

ranking

Section

describes

the

static

features

The

heart

of

the

paper

is

in

Section

which

presents

our

experiments

and

results

We

conclude

with

a

discussion

of

related

and

future

work

The

basic

idea

behind

PageRank

is

simple

a

link

from

a

Web

page

to

another

can

be

seen

as

an

endorsement

of

that

page

In

general

links

are

made

by

people

As

such

they

are

indicative

of

the

quality

of

the

pages

to

which

they

point

when

creating

a

page

an

author

presumably

chooses

to

link

to

pages

deemed

to

be

of

good

quality

We

can

take

advantage

of

this

linkage

information

to

order

Web

pages

according

to

their

perceived

quality

Imagine

a

Web

surfer

who

jumps

from

Web

page

to

Web

page

choosing

with

uniform

probability

which

link

to

follow

at

each

step

In

order

to

reduce

the

effect

of

dead-ends

or

endless

cycles

the

surfer

will

occasionally

jump

to

a

random

page

with

some

small

probability

or

when

on

a

page

with

no

out-links

If

averaged

over

a

sufficient

number

of

steps

the

probability

the

surfer

is

on

page

j

at

some

point

in

time

is

given

by

the

formula

j

i

i

i

P

N

j

P

B

F

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-LRB-

-RRB-

Where

F

i

is

the

set

of

pages

that

page

i

links

to

and

B

j

is

the

set

of

pages

that

link

to

page

j

The

PageRank

score

for

node

j

is

defined

as

this

probability

PR

-LRB-

j

-RRB-

=P

-LRB-

j

-RRB-

Because

equation

-LRB-

-RRB-

is

recursive

it

must

be

iteratively

evaluated

until

P

-LRB-

j

-RRB-

converges

-LRB-

typically

the

initial

distribution

for

P

-LRB-

j

-RRB-

is

uniform

-RRB-

The

intuition

is

because

a

random

surfer

would

end

up

at

the

page

more

frequently

it

is

likely

a

better

page

An

alternative

view

for

equation

-LRB-

-RRB-

is

that

each

page

is

assigned

a

quality

P

-LRB-

j

-RRB-

A

page

gives

an

equal

share

of

its

quality

to

each

page

it

points

to

PageRank

is

computationally

expensive

Our

collection

of

billion

pages

contains

approximately

billion

links

Computing

PageRank

requires

iterating

over

these

billions

of

links

multiple

times

-LRB-

until

convergence

-RRB-

It

requires

large

amounts

of

memory

-LRB-

or

very

smart

caching

schemes

that

slow

the

computation

down

even

further

-RRB-

and

if

spread

across

multiple

machines

requires

significant

communication

between

them

Though

much

work

has

been

done

on

optimizing

the

PageRank

computation

-LRB-

see

e.g.

-LSB-

-RSB-

and

-LSB-

-RSB-

-RRB-

it

remains

a

relatively

slow

computationally

expensive

property

to

compute

Much

work

in

machine

learning

has

been

done

on

the

problems

of

classification

and

regression

Let

X

-LCB-

x

i

-RCB-

be

a

collection

of

feature

vectors

-LRB-

typically

a

feature

is

any

real

valued

number

-RRB-

and

Y

-LCB-

y

i

-RCB-

be

a

collection

of

associated

classes

where

y

i

is

the

class

of

the

object

described

by

feature

vector

x

i

The

classification

problem

is

to

learn

a

function

f

that

maps

y

i

f

-LRB-

x

i

-RRB-

for

all

i

When

y

i

is

real-valued

as

well

this

is

called

regression

Static

ranking

can

be

seen

as

a

regression

problem

If

we

let

x

i

represent

features

of

page

i

and

y

i

be

a

value

-LRB-

say

the

rank

-RRB-

for

each

page

we

could

learn

a

regression

function

that

mapped

each

page

's

features

to

their

rank

However

this

over-constrains

the

problem

we

wish

to

solve

All

we

really

care

about

is

the

order

of

the

pages

not

the

actual

value

assigned

to

them

Recent

work

on

this

ranking

problem

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

directly

attempts

to

optimize

the

ordering

of

the

objects

rather

than

the

value

assigned

to

them

For

these

let

Z

i

j

-RCB-

be

a

collection

of

pairs

of

items

where

item

i

should

be

assigned

a

higher

value

than

item

j

The

goal

of

the

ranking

problem

then

is

to

learn

a

function

f

such

that

-RRB-

-LRB-

-RRB-

-LRB-

j

i

f

f

j

i

x

x

Z

Note

that

as

with

learning

a

regression

function

the

result

of

this

process

is

a

function

-LRB-

f

-RRB-

that

maps

feature

vectors

to

real

values

This

function

can

still

be

applied

anywhere

that

a

regression-learned

function

could

be

applied

The

only

difference

is

the

technique

used

to

learn

the

function

By

directly

optimizing

the

ordering

of

objects

these

methods

are

able

to

learn

a

function

that

does

a

better

job

of

ranking

than

do

regression

techniques

We

used

RankNet

-LSB-

-RSB-

one

of

the

aforementioned

techniques

for

learning

ranking

functions

to

learn

our

static

rank

function

RankNet

is

a

straightforward

modification

to

the

standard

neural

network

back-prop

algorithm

As

with

back-prop

RankNet

attempts

to

minimize

the

value

of

a

cost

function

by

adjusting

each

weight

in

the

network

according

to

the

gradient

of

the

cost

function

with

respect

to

that

weight

The

difference

is

that

while

a

typical

neural

network

cost

function

is

based

on

the

difference

between

the

network

output

and

the

desired

output

the

RankNet

cost

function

is

based

on

the

difference

between

a

pair

of

network

outputs

That

is

for

each

pair

of

feature

vectors

i

j

in

the

training

set

RankNet

computes

the

network

outputs

o

i

and

o

j

Since

vector

i

is

supposed

to

be

ranked

higher

than

vector

j

the

larger

is

o

j

o

i

the

larger

the

cost

RankNet

also

allows

the

pairs

in

Z

to

be

weighted

with

a

confidence

-LRB-

posed

as

the

probability

that

the

pair

satisfies

the

ordering

induced

by

the

ranking

function

-RRB-

In

this

paper

we

used

a

probability

of

one

for

all

pairs

In

the

next

section

we

will

discuss

the

features

used

in

our

feature

vectors

x

i

To

apply

RankNet

-LRB-

or

other

machine

learning

techniques

-RRB-

to

the

ranking

problem

we

needed

to

extract

a

set

of

features

from

each

page

We

divided

our

feature

set

into

four

mutually

exclusive

categories

page-level

-LRB-

Page

-RRB-

domain-level

-LRB-

Domain

-RRB-

anchor

text

and

inlinks

-LRB-

Anchor

-RRB-

and

popularity

-LRB-

Popularity

-RRB-

We

also

optionally

used

the

PageRank

of

a

page

as

a

feature

Below

we

describe

each

of

these

feature

categories

in

more

detail

PageRank

We

computed

PageRank

on

a

Web

graph

of

billion

crawled

pages

-LRB-

and

billion

known

URLs

linked

to

by

these

pages

-RRB-

This

represents

a

significant

portion

of

the

Web

and

is

approximately

the

same

number

of

pages

as

are

used

by

Google

Yahoo

and

MSN

for

their

search

engines

Because

PageRank

is

a

graph-based

algorithm

it

is

important

that

it

be

run

on

as

large

a

subset

of

the

Web

as

possible

Most

previous

studies

on

PageRank

used

subsets

of

the

Web

that

are

significantly

smaller

-LRB-

e.g.

the

TREC

corpus

used

by

many

contains

only

million

pages

-RRB-

We

computed

PageRank

using

the

standard

value

of

for

Popularity

Another

feature

we

used

is

the

actual

popularity

of

a

Web

page

measured

as

the

number

of

times

that

it

has

been

visited

by

users

over

some

period

of

time

We

have

access

to

such

data

from

users

who

have

installed

the

MSN

toolbar

and

have

opted

to

provide

it

to

MSN

The

data

is

aggregated

into

a

count

for

each

Web

page

of

the

number

of

users

who

viewed

that

page

Though

popularity

data

is

generally

unavailable

there

are

two

other

sources

for

it

The

first

is

from

proxy

logs

For

example

a

university

that

requires

its

students

to

use

a

proxy

has

a

record

of

all

the

pages

they

have

visited

while

on

campus

Unfortunately

proxy

data

is

quite

biased

and

relatively

small

Another

source

internal

to

search

engines

are

records

of

which

results

their

users

clicked

on

Such

data

was

used

by

the

search

engine

Direct

Hit

and

has

recently

been

explored

for

dynamic

ranking

purposes

-LSB-

-RSB-

An

advantage

of

the

toolbar

data

over

this

is

that

it

contains

information

about

URL

visits

that

are

not

just

the

result

of

a

search

The

raw

popularity

is

processed

into

a

number

of

features

such

as

the

number

of

times

a

page

was

viewed

and

the

number

of

times

any

page

in

the

domain

was

viewed

More

details

are

provided

in

section

Anchor

text

and

inlinks

These

features

are

based

on

the

information

associated

with

links

to

the

page

in

question

It

includes

features

such

as

the

total

amount

of

text

in

links

pointing

to

the

page

-LRB-

anchor

text

-RRB-

the

number

of

unique

words

in

that

text

etc.

Page

This

category

consists

of

features

which

may

be

determined

by

looking

at

the

page

-LRB-

and

its

URL

-RRB-

alone

We

used

only

eight

simple

features

such

as

the

number

of

words

in

the

body

the

frequency

of

the

most

common

term

etc.

Domain

This

category

contains

features

that

are

computed

as

averages

across

all

pages

in

the

domain

For

example

the

average

number

of

outlinks

on

any

page

and

the

average

PageRank

Many

of

these

features

have

been

used

by

others

for

ranking

Web

pages

particularly

the

anchor

and

page

features

As

mentioned

the

evaluation

is

typically

for

dynamic

ranking

and

we

wish

to

evaluate

the

use

of

them

for

static

ranking

Also

to

our

knowledge

this

is

the

first

study

on

the

use

of

actual

page

visitation

popularity

for

static

ranking

The

closest

similar

work

is

on

using

click-through

behavior

-LRB-

that

is

which

search

engine

results

the

users

click

on

-RRB-

to

affect

dynamic

ranking

-LRB-

see

e.g.

-LSB-

-RSB-

-RRB-

Because

we

use

a

wide

variety

of

features

to

come

up

with

a

static

ranking

we

refer

to

this

as

fRank

-LRB-

for

feature-based

ranking

-RRB-

fRank

uses

RankNet

and

the

set

of

features

described

in

this

section

to

learn

a

ranking

function

for

Web

pages

Unless

otherwise

specified

fRank

was

trained

with

all

of

the

features

In

this

section

we

will

demonstrate

that

we

can

out

perform

PageRank

by

applying

machine

learning

to

a

straightforward

set

of

features

Before

the

results

we

first

discuss

the

data

the

performance

metric

and

the

training

method

Data

In

order

to

evaluate

the

quality

of

a

static

ranking

we

needed

a

gold

standard

defining

the

correct

ordering

for

a

set

of

pages

For

this

we

employed

a

dataset

which

contains

human

judgments

for

queries

For

each

query

a

number

of

results

are

manually

assigned

a

rating

from

to

by

human

judges

The

rating

is

meant

to

be

a

measure

of

how

relevant

the

result

is

for

the

query

where

means

poor

and

means

excellent

There

are

approximately

judgments

in

all

or

an

average

of

ratings

per

query

The

queries

are

selected

by

randomly

choosing

queries

from

among

those

issued

to

the

MSN

search

engine

The

probability

that

a

query

is

selected

is

proportional

to

its

frequency

among

all

of

the

queries

As

a

result

common

queries

are

more

likely

to

be

judged

than

uncommon

queries

As

an

example

of

how

diverse

the

queries

are

the

first

four

queries

in

the

training

set

are

chef

schools

chicagoland

speedway

eagles

fan

club

and

Turkish

culture

The

documents

selected

for

judging

are

those

that

we

expected

would

on

average

be

reasonably

relevant

-LRB-

for

example

the

top

ten

documents

returned

by

MSN

's

search

engine

-RRB-

This

provides

significantly

more

information

than

randomly

selecting

documents

on

the

Web

the

vast

majority

of

which

would

be

irrelevant

to

a

given

query

Because

of

this

process

the

judged

pages

tend

to

be

of

higher

quality

than

the

average

page

on

the

Web

and

tend

to

be

pages

that

will

be

returned

for

common

search

queries

This

bias

is

good

when

evaluating

the

quality

of

static

ranking

for

the

purposes

of

index

ordering

and

returning

relevant

documents

This

is

because

the

most

important

portion

of

the

index

to

be

well-ordered

and

relevant

is

the

portion

that

is

frequently

returned

for

search

queries

Because

of

this

bias

however

the

results

in

this

paper

are

not

applicable

to

crawl

prioritization

In

order

to

obtain

experimental

results

on

crawl

prioritization

we

would

need

ratings

on

a

random

sample

of

Web

pages

To

convert

the

data

from

query-dependent

to

query-independent

we

simply

removed

the

query

taking

the

maximum

over

judgments

for

a

URL

that

appears

in

more

than

one

query

The

reasoning

behind

this

is

that

a

page

that

is

relevant

for

some

query

and

irrelevant

for

another

is

probably

a

decent

page

and

should

have

a

high

static

rank

Because

we

evaluated

the

pages

on

queries

that

occur

frequently

our

data

indicates

the

correct

index

ordering

and

assigns

high

value

to

pages

that

are

likely

to

be

relevant

to

a

common

query

We

randomly

assigned

queries

to

a

training

validation

or

test

set

such

that

they

contained

and

of

the

queries

respectively

Each

set

contains

all

of

the

ratings

for

a

given

query

and

no

query

appears

in

more

than

one

set

The

training

set

was

used

to

train

fRank

The

validation

set

was

used

to

select

the

model

that

had

the

highest

performance

The

test

set

was

used

for

the

final

results

This

data

gives

us

a

query-independent

ordering

of

pages

The

goal

for

a

static

ranking

algorithm

will

be

to

reproduce

this

ordering

as

closely

as

possible

In

the

next

section

we

describe

the

measure

we

used

to

evaluate

this

Measure

We

chose

to

use

pairwise

accuracy

to

evaluate

the

quality

of

a

static

ranking

The

pairwise

accuracy

is

the

fraction

of

time

that

the

ranking

algorithm

and

human

judges

agree

on

the

ordering

of

a

pair

of

Web

pages

If

S

-LRB-

x

-RRB-

is

the

static

ranking

assigned

to

page

x

and

H

-LRB-

x

-RRB-

is

the

human

judgment

of

relevance

for

x

then

consider

the

following

sets

-RRB-

-RCB-

-LRB-

-RRB-

-LRB-

-LCB-

y

H

x

H

y

x

p

H

and

-RRB-

-RCB-

-LRB-

-RRB-

-LRB-

-LCB-

y

S

x

S

y

x

p

S

The

pairwise

accuracy

is

the

portion

of

H

p

that

is

also

contained

in

S

p

p

p

p

H

S

H

accuracy

pairwise

This

measure

was

chosen

for

two

reasons

First

the

discrete

human

judgments

provide

only

a

partial

ordering

over

Web

pages

making

it

difficult

to

apply

a

measure

such

as

the

Spearman

rank

order

correlation

coefficient

-LRB-

in

the

pairwise

accuracy

measure

a

pair

of

documents

with

the

same

human

judgment

does

not

affect

the

score

-RRB-

Second

the

pairwise

accuracy

has

an

intuitive

meaning

it

is

the

fraction

of

pairs

of

documents

that

when

the

humans

claim

one

is

better

than

the

other

the

static

rank

algorithm

orders

them

correctly

Method

We

trained

fRank

-LRB-

a

RankNet

based

neural

network

-RRB-

using

the

following

parameters

We

used

a

fully

connected

layer

network

The

hidden

layer

had

hidden

nodes

The

input

weights

to

this

layer

were

all

initialized

to

be

zero

The

output

layer

-LRB-

just

a

single

node

-RRB-

weights

were

initialized

using

a

uniform

random

distribution

in

the

range

-LSB-

-RSB-

We

used

tanh

as

the

transfer

function

from

the

inputs

to

the

hidden

layer

and

a

linear

function

from

the

hidden

layer

to

the

output

The

cost

function

is

the

pairwise

cross

entropy

cost

function

as

discussed

in

section

The

features

in

the

training

set

were

normalized

to

have

zero

mean

and

unit

standard

deviation

The

same

linear

transformation

was

then

applied

to

the

features

in

the

validation

and

test

sets

For

training

we

presented

the

network

with

million

pairings

of

pages

where

one

page

had

a

higher

rating

than

the

other

The

pairings

were

chosen

uniformly

at

random

-LRB-

with

replacement

-RRB-

from

all

possible

pairings

When

forming

the

pairs

we

ignored

the

magnitude

of

the

difference

between

the

ratings

-LRB-

the

rating

spread

-RRB-

for

the

two

URLs

Hence

the

weight

for

each

pair

was

constant

-LRB-

one

-RRB-

and

the

probability

of

a

pair

being

selected

was

independent

of

its

rating

spread

We

trained

the

network

for

epochs

On

each

epoch

the

training

pairs

were

randomly

shuffled

The

initial

training

rate

was

At

each

epoch

we

checked

the

error

on

the

training

set

If

the

error

had

increased

then

we

decreased

the

training

rate

under

the

hypothesis

that

the

network

had

probably

overshot

The

training

rate

at

each

epoch

was

thus

set

to

Training

rate

Where

is

the

initial

rate

-LRB-

-RRB-

and

is

the

number

of

times

the

training

set

error

has

increased

After

each

epoch

we

measured

the

performance

of

the

neural

network

on

the

validation

set

using

million

pairs

-LRB-

chosen

randomly

with

replacement

-RRB-

The

network

with

the

highest

pairwise

accuracy

on

the

validation

set

was

selected

and

then

tested

on

the

test

set

We

report

the

pairwise

accuracy

on

the

test

set

calculated

using

all

possible

pairs

These

parameters

were

determined

and

fixed

before

the

static

rank

experiments

in

this

paper

In

particular

the

choice

of

initial

training

rate

number

of

epochs

and

training

rate

decay

function

were

taken

directly

from

Burges

et

al

-LSB-

-RSB-

Though

we

had

the

option

of

preprocessing

any

of

the

features

before

they

were

input

to

the

neural

network

we

refrained

from

doing

so

on

most

of

them

The

only

exception

was

the

popularity

features

As

with

most

Web

phenomenon

we

found

that

the

distribution

of

site

popularity

is

Zipfian

To

reduce

the

dynamic

range

and

hopefully

make

the

feature

more

useful

we

presented

the

network

with

both

the

unpreprocessed

as

well

as

the

logarithm

of

the

popularity

features

-LRB-

As

with

the

others

the

logarithmic

feature

values

were

also

normalized

to

have

zero

mean

and

unit

standard

deviation

-RRB-

Applying

fRank

to

a

document

is

computationally

efficient

taking

time

that

is

only

linear

in

the

number

of

input

features

it

is

thus

within

a

constant

factor

of

other

simple

machine

learning

methods

such

as

ve

Bayes

In

our

experiments

computing

the

fRank

for

all

five

billion

Web

pages

was

approximately

times

faster

than

computing

the

PageRank

for

the

same

set

Results

As

Table

shows

fRank

significantly

outperforms

PageRank

for

the

purposes

of

static

ranking

With

a

pairwise

accuracy

of

fRank

more

than

doubles

the

accuracy

of

PageRank

-LRB-

relative

to

the

baseline

of

which

is

the

accuracy

that

would

be

achieved

by

a

random

ordering

of

Web

pages

-RRB-

Note

that

one

of

fRank

's

input

features

is

the

PageRank

of

the

page

so

we

would

expect

it

to

perform

no

worse

than

PageRank

The

significant

increase

in

accuracy

implies

that

the

other

features

-LRB-

anchor

popularity

etc.

-RRB-

do

in

fact

contain

useful

information

regarding

the

overall

quality

of

a

page

Table

Basic

Results

Technique

Accuracy

-LRB-

-RRB-

-LRB-

Baseline

-RRB-

PageRank

fRank

There

are

a

number

of

decisions

that

go

into

the

computation

of

PageRank

such

as

how

to

deal

with

pages

that

have

no

outlinks

the

choice

of

numeric

precision

convergence

threshold

etc.

We

were

able

to

obtain

a

computation

of

PageRank

from

a

completely

independent

implementation

-LRB-

provided

by

Marc

Najork

-RRB-

that

varied

somewhat

in

these

parameters

It

achieved

a

pairwise

accuracy

of

nearly

identical

to

that

obtained

by

our

implementation

We

thus

concluded

that

the

quality

of

the

PageRank

is

not

sensitive

to

these

minor

variations

in

algorithm

nor

was

PageRank

's

low

accuracy

due

to

problems

with

our

implementation

of

it

We

also

wanted

to

find

how

well

each

feature

set

performed

To

answer

this

for

each

feature

set

we

trained

and

tested

fRank

using

only

that

set

of

features

The

results

are

shown

in

Table

As

can

be

seen

every

single

feature

set

individually

outperformed

PageRank

on

this

test

Perhaps

the

most

interesting

result

is

that

the

Page-level

features

had

the

highest

performance

out

of

all

the

feature

sets

This

is

surprising

because

these

are

features

that

do

not

depend

on

the

overall

graph

structure

of

the

Web

nor

even

on

what

pages

point

to

a

given

page

This

is

contrary

to

the

common

belief

that

the

Web

graph

structure

is

the

key

to

finding

a

good

static

ranking

of

Web

pages

Table

Results

for

individual

feature

sets

Feature

Set

Accuracy

-LRB-

-RRB-

PageRank

Popularity

Anchor

Page

Domain

All

Features

Because

we

are

using

a

two-layer

neural

network

the

features

in

the

learned

network

can

interact

with

each

other

in

interesting

nonlinear

ways

This

means

that

a

particular

feature

that

appears

to

have

little

value

in

isolation

could

actually

be

very

important

when

used

in

combination

with

other

features

To

measure

the

final

contribution

of

a

feature

set

in

the

context

of

all

the

other

features

we

performed

an

ablation

study

That

is

for

each

set

of

features

we

trained

a

network

to

contain

all

of

the

features

except

that

set

We

then

compared

the

performance

of

the

resulting

network

to

the

performance

of

the

network

with

all

of

the

features

Table

shows

the

results

of

this

experiment

where

the

decrease

in

accuracy

is

the

difference

in

pairwise

accuracy

between

the

network

trained

with

all

of

the

features

and

the

network

missing

the

given

feature

set

Table

Ablation

study

Shown

is

the

decrease

in

accuracy

when

we

train

a

network

that

has

all

but

the

given

set

of

features

The

last

line

is

shows

the

effect

of

removing

the

anchor

PageRank

and

domain

features

hence

a

model

containing

no

network

or

link-based

information

whatsoever

Feature

Set

Decrease

in

Accuracy

PageRank

Popularity

Anchor

Page

Domain

Anchor

PageRank

Domain

The

results

of

the

ablation

study

are

consistent

with

the

individual

feature

set

study

Both

show

that

the

most

important

feature

set

is

the

Page-level

feature

set

and

the

second

most

important

is

the

popularity

feature

set

Finally

we

wished

to

see

how

the

performance

of

fRank

improved

as

we

added

features

we

wanted

to

find

at

what

point

adding

more

feature

sets

became

relatively

useless

Beginning

with

no

features

we

greedily

added

the

feature

set

that

improved

performance

the

most

The

results

are

shown

in

Table

For

example

the

fourth

line

of

the

table

shows

that

fRank

using

the

page

popularity

and

anchor

features

outperformed

any

network

that

used

the

page

popularity

and

some

other

feature

set

and

that

the

performance

of

this

network

was

Table

fRank

performance

as

feature

sets

are

added

At

each

row

the

feature

set

that

gave

the

greatest

increase

in

accuracy

was

added

to

the

list

of

features

-LRB-

i.e.

we

conducted

a

greedy

search

over

feature

sets

-RRB-

Feature

Set

Accuracy

-LRB-

-RRB-

Page

Popularity

Anchor

PageRank

Domain

Finally

we

present

a

qualitative

comparison

of

PageRank

vs.

fRank

In

Table

are

the

top

ten

URLs

returned

for

PageRank

and

for

fRank

PageRank

's

results

are

heavily

weighted

towards

technology

sites

It

contains

two

QuickTime

URLs

-LRB-

Apple

's

video

playback

software

-RRB-

as

well

as

Internet

Explorer

and

FireFox

URLs

-LRB-

both

of

which

are

Web

browsers

-RRB-

fRank

on

the

other

hand

contains

more

consumer-oriented

sites

such

as

American

Express

Target

Dell

etc.

PageRank

's

bias

toward

technology

can

be

explained

through

two

processes

First

there

are

many

pages

with

buttons

at

the

bottom

suggesting

that

the

site

is

optimized

for

Internet

Explorer

or

that

the

visitor

needs

QuickTime

These

generally

link

back

to

in

these

examples

the

Internet

Explorer

and

QuickTime

download

sites

Consequently

PageRank

ranks

those

pages

highly

Though

these

pages

are

important

they

are

not

as

important

as

it

may

seem

by

looking

at

the

link

structure

alone

One

fix

for

this

is

to

add

information

about

the

link

to

the

PageRank

computation

such

as

the

size

of

the

text

whether

it

was

at

the

bottom

of

the

page

etc.

The

other

bias

comes

from

the

fact

that

the

population

of

Web

site

authors

is

different

than

the

population

of

Web

users

Web

authors

tend

to

be

technologically-oriented

and

thus

their

linking

behavior

reflects

those

interests

fRank

by

knowing

the

actual

visitation

popularity

of

a

site

-LRB-

the

popularity

feature

set

-RRB-

is

able

to

eliminate

some

of

that

bias

It

has

the

ability

to

depend

more

on

where

actual

Web

users

visit

rather

than

where

the

Web

site

authors

have

linked

The

results

confirm

that

fRank

outperforms

PageRank

in

pairwise

accuracy

The

two

most

important

feature

sets

are

the

page

and

popularity

features

This

is

surprising

as

the

page

features

consisted

only

of

a

few

-LRB-

-RRB-

simple

features

Further

experiments

found

that

of

the

page

features

those

based

on

the

text

of

the

page

-LRB-

as

opposed

to

the

URL

-RRB-

performed

the

best

In

the

next

section

we

explore

the

popularity

feature

in

more

detail

Popularity

Data

As

mentioned

in

section

our

popularity

data

came

from

MSN

toolbar

users

For

privacy

reasons

we

had

access

only

to

an

aggregate

count

of

for

each

URL

how

many

times

it

was

visited

by

any

toolbar

user

This

limited

the

possible

features

we

could

derive

from

this

data

For

possible

extensions

see

section

future

work

For

each

URL

in

our

train

and

test

sets

we

provided

a

feature

to

fRank

which

was

how

many

times

it

had

been

visited

by

a

toolbar

user

However

this

feature

was

quite

noisy

and

sparse

particularly

for

URLs

with

query

parameters

-LRB-

e.g.

msn.com/results.aspx?q=machine+learning&form=QBHP

-RRB-

One

solution

was

to

provide

an

additional

feature

which

was

the

number

of

times

any

URL

at

the

given

domain

was

visited

by

a

toolbar

user

Adding

this

feature

dramatically

improved

the

performance

of

fRank

We

took

this

one

step

further

and

used

the

built-in

hierarchical

structure

of

URLs

to

construct

many

levels

of

backoff

between

the

full

URL

and

the

domain

We

did

this

by

using

the

set

of

features

shown

in

Table

Table

URL

functions

used

to

compute

the

Popularity

feature

set

Function

Example

Exact

URL

cnn.com/2005/tech/wikipedia.html?v=mobile

No

Params

Page

wikipedia.html

Domain

cnn.com

Domain

Each

URL

was

assigned

one

feature

for

each

function

shown

in

the

table

The

value

of

the

feature

was

the

count

of

the

number

of

times

a

toolbar

user

visited

a

URL

where

the

function

applied

to

that

URL

matches

the

function

applied

to

the

URL

in

question

For

example

a

user

's

visit

to

would

increment

the

Domain

and

Domain

features

for

the

URL

As

seen

in

Table

adding

the

domain

counts

significantly

improved

the

quality

of

the

popularity

feature

and

adding

the

numerous

backoff

functions

listed

in

Table

improved

the

accuracy

even

further

Table

Effect

of

adding

backoff

to

the

popularity

feature

set

Features

Accuracy

-LRB-

-RRB-

URL

count

URL

and

Domain

counts

All

backoff

functions

-LRB-

Table

-RRB-

Table

Top

ten

URLs

for

PageRank

vs.

fRank

PageRank

fRank

google.com

google.com

yahoo.com

amazon.com

americanexpress.com

yahoo.com

hp.com

target.com

apple.com/quicktime

bestbuy.com

mapquest.com

dell.com

ebay.com

autotrader.com

dogpile.com

ftc.gov

bankofamerica.com

Backing

off

to

subsets

of

the

URL

is

one

technique

for

dealing

with

the

sparsity

of

data

It

is

also

informative

to

see

how

the

performance

of

fRank

depends

on

the

amount

of

popularity

data

that

we

have

collected

In

Figure

we

show

the

performance

of

fRank

trained

with

only

the

popularity

feature

set

vs.

the

amount

of

data

we

have

for

the

popularity

feature

set

Each

day

we

receive

additional

popularity

data

and

as

can

be

seen

in

the

plot

this

increases

the

performance

of

fRank

The

relation

is

logarithmic

doubling

the

amount

of

popularity

data

provides

a

constant

improvement

in

pairwise

accuracy

In

summary

we

have

found

that

the

popularity

features

provide

a

useful

boost

to

the

overall

fRank

accuracy

Gathering

more

popularity

data

as

well

as

employing

simple

backoff

strategies

improve

this

boost

even

further

Summary

of

Results

The

experiments

provide

a

number

of

conclusions

First

fRank

performs

significantly

better

than

PageRank

even

without

any

information

about

the

Web

graph

Second

the

page

level

and

popularity

features

were

the

most

significant

contributors

to

pairwise

accuracy

Third

by

collecting

more

popularity

data

we

can

continue

to

improve

fRank

's

performance

The

popularity

data

provides

two

benefits

to

fRank

First

we

see

that

qualitatively

fRank

's

ordering

of

Web

pages

has

a

more

favorable

bias

than

PageRank

's

fRank

's

ordering

seems

to

correspond

to

what

Web

users

rather

than

Web

page

authors

prefer

Second

the

popularity

data

is

more

timely

than

PageRank

's

link

information

The

toolbar

provides

information

about

which

Web

pages

people

find

interesting

right

now

whereas

links

are

added

to

pages

more

slowly

as

authors

find

the

time

and

interest

Since

the

original

PageRank

paper

there

has

been

work

on

improving

it

Much

of

that

work

centers

on

speeding

up

and

parallelizing

the

computation

-LSB-

-RSB-

-LSB-

-RSB-

One

recognized

problem

with

PageRank

is

that

of

topic

drift

A

page

about

dogs

will

have

high

PageRank

if

it

is

linked

to

by

many

pages

that

themselves

have

high

rank

regardless

of

their

topic

In

contrast

a

search

engine

user

looking

for

good

pages

about

dogs

would

likely

prefer

to

find

pages

that

are

pointed

to

by

many

pages

that

are

themselves

about

dogs

Hence

a

link

that

is

on

topic

should

have

higher

weight

than

a

link

that

is

not

Richardson

and

Domingos

's

Query

Dependent

PageRank

-LSB-

-RSB-

and

Haveliwala

's

Topic-Sensitive

PageRank

-LSB-

-RSB-

are

two

approaches

that

tackle

this

problem

Other

variations

to

PageRank

include

differently

weighting

links

for

inter

vs.

intra-domain

links

adding

a

backwards

step

to

the

random

surfer

to

simulate

the

back

button

on

most

browsers

-LSB-

-RSB-

and

modifying

the

jump

probability

-LRB-

-RRB-

-LSB-

-RSB-

See

Langville

and

Meyer

-LSB-

-RSB-

for

a

good

survey

of

these

and

other

modifications

to

PageRank

Other

related

work

PageRank

is

not

the

only

link

analysis

algorithm

used

for

ranking

Web

pages

The

most

well-known

other

is

HITS

-LSB-

-RSB-

which

is

used

by

the

Teoma

search

engine

-LSB-

-RSB-

HITS

produces

a

list

of

hubs

and

authorities

where

hubs

are

pages

that

point

to

many

authority

pages

and

authorities

are

pages

that

are

pointed

to

by

many

hubs

Previous

work

has

shown

HITS

to

perform

comparably

to

PageRank

-LSB-

-RSB-

One

field

of

interest

is

that

of

static

index

pruning

-LRB-

see

e.g.

Carmel

et

al.

-LSB-

-RSB-

-RRB-

Static

index

pruning

methods

reduce

the

size

of

the

search

engine

's

index

by

removing

documents

that

are

unlikely

to

be

returned

by

a

search

query

The

pruning

is

typically

done

based

on

the

frequency

of

query

terms

Similarly

Pandey

and

Olston

-LSB-

-RSB-

suggest

crawling

pages

frequently

if

they

are

likely

to

incorrectly

appear

-LRB-

or

not

appear

-RRB-

as

a

result

of

a

search

Similar

methods

could

be

incorporated

into

the

static

rank

-LRB-

e.g.

how

many

frequent

queries

contain

words

found

on

this

page

-RRB-

Others

have

investigated

the

effect

that

PageRank

has

on

the

Web

at

large

-LSB-

-RSB-

They

argue

that

pages

with

high

PageRank

are

more

likely

to

be

found

by

Web

users

thus

more

likely

to

be

linked

to

and

thus

more

likely

to

maintain

a

higher

PageRank

than

other

pages

The

same

may

occur

for

the

popularity

data

If

we

increase

the

ranking

for

popular

pages

they

are

more

likely

to

be

clicked

on

thus

further

increasing

their

popularity

Cho

et

al.

-LSB-

-RSB-

argue

that

a

more

appropriate

measure

of

Web

page

quality

would

depend

on

not

only

the

current

link

structure

of

the

Web

but

also

on

the

change

in

that

link

structure

The

same

technique

may

be

applicable

to

popularity

data

the

change

in

popularity

of

a

page

may

be

more

informative

than

the

absolute

popularity

One

interesting

related

work

is

that

of

Ivory

and

Hearst

-LSB-

-RSB-

Their

goal

was

to

build

a

model

of

Web

sites

that

are

considered

high

quality

from

the

perspective

of

content

structure

and

navigation

visual

design

functionality

interactivity

and

overall

experience

They

used

over

page

level

features

as

well

as

features

encompassing

the

performance

and

structure

of

the

site

This

let

them

qualitatively

describe

the

qualities

of

a

page

that

make

it

appear

attractive

-LRB-

e.g.

rare

use

of

italics

at

least

point

font

-RRB-

and

-LRB-

in

later

work

-RRB-

to

build

a

system

that

assists

novel

Web

page

authors

in

creating

quality

pages

by

evaluating

it

according

to

these

features

The

primary

differences

between

this

work

and

ours

are

the

goal

-LRB-

discovering

what

constitutes

a

good

Web

page

vs.

ordering

Web

pages

for

the

purposes

of

Web

search

-RRB-

the

size

of

the

study

-LRB-

they

used

a

dataset

of

less

than

pages

vs.

our

set

of

-RRB-

and

our

comparison

with

PageRank

y

Ln

-LRB-

x

-RRB-

R

Days

of

Toolbar

Data

P

a

i

r

w

i

s

e

A

c

c

u

r

a

c

y

Figure

Relation

between

the

amount

of

popularity

data

and

the

performance

of

the

popularity

feature

set

Note

the

x-axis

is

a

logarithmic

scale

Nevertheless

their

work

provides

insights

to

additional

useful

static

features

that

we

could

incorporate

into

fRank

in

the

future

Recent

work

on

incorporating

novel

features

into

dynamic

ranking

includes

that

by

Joachims

et

al.

-LSB-

-RSB-

who

investigate

the

use

of

implicit

feedback

from

users

in

the

form

of

which

search

engine

results

are

clicked

on

Craswell

et

al.

-LSB-

-RSB-

present

a

method

for

determining

the

best

transformation

to

apply

to

query

independent

features

-LRB-

such

as

those

used

in

this

paper

-RRB-

for

the

purposes

of

improving

dynamic

ranking

Other

work

such

as

Boyan

et

al.

-LSB-

-RSB-

and

Bartell

et

al.

-LSB-

-RSB-

apply

machine

learning

for

the

purposes

of

improving

the

overall

relevance

of

a

search

engine

-LRB-

i.e.

the

dynamic

ranking

-RRB-

They

do

not

apply

their

techniques

to

the

problem

of

static

ranking

Future

work

There

are

many

ways

in

which

we

would

like

to

extend

this

work

First

fRank

uses

only

a

small

number

of

features

We

believe

we

could

achieve

even

more

significant

results

with

more

features

In

particular

the

existence

or

lack

thereof

of

certain

words

could

prove

very

significant

-LRB-

for

instance

under

construction

probably

signifies

a

low

quality

page

-RRB-

Other

features

could

include

the

number

of

images

on

a

page

size

of

those

images

number

of

layout

elements

-LRB-

tables

divs

and

spans

-RRB-

use

of

style

sheets

conforming

to

standards

-LRB-

like

XHTML

Strict

-RRB-

background

color

of

a

page

etc.

Many

pages

are

generated

dynamically

the

contents

of

which

may

depend

on

parameters

in

the

URL

the

time

of

day

the

user

visiting

the

site

or

other

variables

For

such

pages

it

may

be

useful

to

apply

the

techniques

found

in

-LSB-

-RSB-

to

form

a

static

approximation

for

the

purposes

of

extracting

features

The

resulting

grammar

describing

the

page

could

itself

be

a

source

of

additional

features

describing

the

complexity

of

the

page

such

as

how

many

non-terminal

nodes

it

has

the

depth

of

the

grammar

tree

etc.

fRank

allows

one

to

specify

a

confidence

in

each

pairing

of

documents

In

the

future

we

will

experiment

with

probabilities

that

depend

on

the

difference

in

human

judgments

between

the

two

items

in

the

pair

For

example

a

pair

of

documents

where

one

was

rated

and

the

other

should

have

a

higher

confidence

than

a

pair

of

documents

rated

and

The

experiments

in

this

paper

are

biased

toward

pages

that

have

higher

than

average

quality

Also

fRank

with

all

of

the

features

can

only

be

applied

to

pages

that

have

already

been

crawled

Thus

fRank

is

primarily

useful

for

index

ordering

and

improving

relevance

not

for

directing

the

crawl

We

would

like

to

investigate

a

machine

learning

approach

for

crawl

prioritization

as

well

It

may

be

that

a

combination

of

methods

is

best

for

example

using

PageRank

to

select

the

best

billion

of

the

billion

pages

on

the

Web

then

using

fRank

to

order

the

index

and

affect

search

relevancy

Another

interesting

direction

for

exploration

is

to

incorporate

fRank

and

page-level

features

directly

into

the

PageRank

computation

itself

Work

on

biasing

the

PageRank

jump

vector

-LSB-

-RSB-

and

transition

matrix

-LSB-

-RSB-

have

demonstrated

the

feasibility

and

advantages

of

such

an

approach

There

is

reason

to

believe

that

a

direct

application

of

-LSB-

-RSB-

using

the

fRank

of

a

page

for

its

relevance

could

lead

to

an

improved

overall

static

rank

Finally

the

popularity

data

can

be

used

in

other

interesting

ways

The

general

surfing

and

searching

habits

of

Web

users

varies

by

time

of

day

Activity

in

the

morning

daytime

and

evening

are

often

quite

different

-LRB-

e.g.

reading

the

news

solving

problems

and

accessing

entertainment

respectively

-RRB-

We

can

gain

insight

into

these

differences

by

using

the

popularity

data

divided

into

segments

of

the

day

When

a

query

is

issued

we

would

then

use

the

popularity

data

matching

the

time

of

query

in

order

to

do

the

ranking

of

Web

pages

We

also

plan

to

explore

popularity

features

that

use

more

than

just

the

counts

of

how

often

a

page

was

visited

For

example

how

long

users

tended

to

dwell

on

a

page

did

they

leave

the

page

by

clicking

a

link

or

by

hitting

the

back

button

etc.

Fox

et

al.

did

a

study

that

showed

that

features

such

as

this

can

be

valuable

for

the

purposes

of

dynamic

ranking

-LSB-

-RSB-

Finally

the

popularity

data

could

be

used

as

the

label

rather

than

as

a

feature

Using

fRank

in

this

way

to

predict

the

popularity

of

a

page

may

useful

for

the

tasks

of

relevance

efficiency

and

crawl

priority

There

is

also

significantly

more

popularity

data

than

human

labeled

data

potentially

enabling

more

complex

machine

learning

methods

and

significantly

more

features

A

good

static

ranking

is

an

important

component

for

today

's

search

engines

and

information

retrieval

systems

We

have

demonstrated

that

PageRank

does

not

provide

a

very

good

static

ranking

there

are

many

simple

features

that

individually

out

perform

PageRank

By

combining

many

static

features

fRank

achieves

a

ranking

that

has

a

significantly

higher

pairwise

accuracy

than

PageRank

alone

A

qualitative

evaluation

of

the

top

documents

shows

that

fRank

is

less

technology-biased

than

PageRank

by

using

popularity

data

it

is

biased

toward

pages

that

Web

users

rather

than

Web

authors

visit

The

machine

learning

component

of

fRank

gives

it

the

additional

benefit

of

being

more

robust

against

spammers

and

allows

it

to

leverage

further

developments

in

the

machine

learning

community

in

areas

such

as

adversarial

classification

We

have

only

begun

to

explore

the

options

and

believe

that

significant

strides

can

be

made

in

the

area

of

static

ranking

by

further

experimentation

with

additional

features

other

machine

learning

techniques

and

additional

sources

of

data

Thank

you

to

Marc

Najork

for

providing

us

with

additional

PageRank

computations

and

to

Timo

Burkard

for

assistance

with

the

popularity

data

Many

thanks

to

Chris

Burges

for

providing

code

and

significant

support

in

using

training

RankNets

Also

we

thank

Susan

Dumais

and

Nick

Craswell

for

their

edits

and

suggestions

-LSB-

-RSB-

B.

Amento

L.

Terveen

and

W.

Hill

Does

authority

mean

quality

Predicting

expert

quality

ratings

of

Web

documents

In

Proceedings

of

the

rd

Annual

International

ACM

SIGIR

Conference

on

Research

and

Development

in

Information

Retrieval

-LSB-

-RSB-

B.

Bartell

G.

Cottrell

and

R.

Belew

Automatic

combination

of

multiple

ranked

retrieval

systems

In

Proceedings

of

the

Annual

International

ACM

SIGIR

Conference

on

Research

and

Development

in

Information

Retrieval

-LSB-

-RSB-

P.

Boldi

M.

Santini

and

S.

Vigna

PageRank

as

a

function

of

the

damping

factor

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

J.

Boyan

D.

Freitag

and

T.

Joachims

A

machine

learning

architecture

for

optimizing

web

search

engines

In

AAAI

Workshop

on

Internet

Based

Information

Systems

August

-LSB-

-RSB-

S.

Brin

and

L.

Page

The

anatomy

of

a

large-scale

hypertextual

web

search

engine

In

Proceedings

of

the

Seventh

International

Wide

Web

Conference

Brisbane

Australia

Elsevier

-LSB-

-RSB-

A.

Broder

R.

Lempel

F.

Maghoul

and

J.

Pederson

Efficient

PageRank

approximation

via

graph

aggregation

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

C.

Burges

T.

Shaked

E.

Renshaw

A.

Lazier

M.

Deeds

N.

Hamilton

G.

Hullender

Learning

to

rank

using

gradient

descent

In

Proceedings

of

the

nd

International

Conference

on

Machine

Learning

Bonn

Germany

-LSB-

-RSB-

D.

Carmel

D.

Cohen

R.

Fagin

E.

Farchi

M.

Herscovici

Y.

S.

Maarek

and

A.

Soffer

Static

index

pruning

for

information

retrieval

systems

In

Proceedings

of

the

Annual

International

ACM

SIGIR

Conference

on

Research

and

Development

in

Information

Retrieval

pages

New

Orleans

Louisiana

USA

September

-LSB-

-RSB-

J.

Cho

and

S.

Roy

Impact

of

search

engines

on

page

popularity

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

J.

Cho

S.

Roy

R.

Adams

Page

Quality

In

search

of

an

unbiased

web

ranking

In

Proceedings

of

the

ACM

SIGMOD

Conference

Baltimore

Maryland

June

-LSB-

-RSB-

N.

Craswell

S.

Robertson

H.

Zaragoza

and

M.

Taylor

Relevance

weighting

for

query

independent

evidence

In

Proceedings

of

the

th

Annual

Conference

on

Research

and

Development

in

Information

Retrieval

-LRB-

SIGIR

-RRB-

August

-LSB-

-RSB-

N.

Dalvi

P.

Domingos

Mausam

S.

Sanghai

D.

Verma

Adversarial

Classification

In

Proceedings

of

the

Tenth

International

Conference

on

Knowledge

Discovery

and

Data

Mining

-LRB-

pp.

-RRB-

Seattle

WA

-LSB-

-RSB-

O.

Dekel

C.

Manning

and

Y.

Singer

Log-linear

models

for

label-ranking

In

Advances

in

Neural

Information

Processing

Systems

Cambridge

MA

MIT

Press

-LSB-

-RSB-

S.

Fox

K

S.

Fox

K.

Karnawat

M.

Mydland

S.

T.

Dumais

and

T.

White

-LRB-

-RRB-

Evaluating

implicit

measures

to

improve

the

search

experiences

In

the

ACM

Transactions

on

Information

Systems

-LRB-

-RRB-

pp.

April

-LSB-

-RSB-

T.

Haveliwala

Efficient

computation

of

PageRank

Stanford

University

Technical

Report

-LSB-

-RSB-

T.

Haveliwala

Topic-sensitive

PageRank

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

D.

Hawking

and

N.

Craswell

Very

large

scale

retrieval

and

Web

search

In

D.

Harman

and

E.

Voorhees

-LRB-

eds

-RRB-

The

TREC

Book

MIT

Press

-LSB-

-RSB-

R.

Herbrich

T.

Graepel

and

K.

Obermayer

Support

vector

learning

for

ordinal

regression

In

Proceedings

of

the

Ninth

International

Conference

on

Artificial

Neural

Networks

pp

-LSB-

-RSB-

M.

Ivory

and

M.

Hearst

Statistical

profiles

of

highly-rated

Web

sites

In

Proceedings

of

the

ACM

SIGCHI

Conference

on

Human

Factors

in

Computing

Systems

-LSB-

-RSB-

T.

Joachims

Optimizing

search

engines

using

clickthrough

data

In

Proceedings

of

the

ACM

Conference

on

Knowledge

Discovery

and

Data

Mining

-LRB-

KDD

-RRB-

-LSB-

-RSB-

T.

Joachims

L.

Granka

B.

Pang

H.

Hembrooke

and

G.

Gay

Accurately

Interpreting

Clickthrough

Data

as

Implicit

Feedback

In

Proceedings

of

the

Conference

on

Research

and

Development

in

Information

Retrieval

-LRB-

SIGIR

-RRB-

-LSB-

-RSB-

J.

Kleinberg

Authoritative

sources

in

a

hyperlinked

environment

Journal

of

the

ACM

pp.

-LSB-

-RSB-

A.

Langville

and

C.

Meyer

Deeper

inside

PageRank

Internet

Mathematics

-LRB-

-RRB-

-LSB-

-RSB-

F.

Matthieu

and

M.

Bouklit

The

effect

of

the

back

button

in

a

random

walk

application

for

PageRank

In

Alternate

track

papers

and

posters

of

the

Thirteenth

International

World

Wide

Web

Conference

-LSB-

-RSB-

F.

McSherry

A

uniform

approach

to

accelerated

PageRank

computation

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

Y.

Minamide

Static

approximation

of

dynamically

generated

Web

pages

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

L.

Page

S.

Brin

R.

Motwani

and

T.

Winograd

The

PageRank

citation

ranking

Bringing

order

to

the

web

Technical

report

Stanford

University

Stanford

CA

-LSB-

-RSB-

S.

Pandey

and

C.

Olston

User-centric

Web

crawling

In

Proceedings

of

the

International

World

Wide

Web

Conference

May

-LSB-

-RSB-

M.

Richardson

and

P.

Domingos

The

intelligent

surfer

probabilistic

combination

of

link

and

content

information

in

PageRank

In

Advances

in

Neural

Information

Processing

Systems

pp.

Cambridge

MA

MIT

Press

-LSB-

-RSB-

C.

Sherman

Teoma

vs.

Google

Round

Available

from

World

Wide

Web

-LRB-

-RRB-

-LSB-

-RSB-

T.

Upstill

N.

Craswell

and

D.

Hawking

Predicting

fame

and

fortune

PageRank

or

indegree

In

the

Eighth

Australasian

Document

Computing

Symposium

-LSB-

-RSB-

T.

Upstill

N.

Craswell

and

D.

Hawking

Query-independent

evidence

in

home

page

finding

In

ACM

Transactions

on

Information

Systems

dynamic ranking
search engines
static features
ranknet
pairwise accuracy
web pages
frank
relevance
static ranking
pagerank
anchor text
popularity data