Deformable image registration (DIR) has many exciting potential applications in medical imaging and radiation oncology. Automated propagation of physician-drawn contours to multiple image volumes, functional imaging, and 4D dose accumulation in thoracic radiotherapy are just a few examples. However, before such applications can be successfully and safely implemented, we require that the DIR component be rigorously and objectively assessed in terms of spatial accuracy performance.

Objective evaluation of DIR is an active area of research. A framework for DIR evaluation is an essential utility for algorithm optimization, performing comparisons between algorithms, models, and implementations, acceptance testing prior to clinical implementation, and quality assurance of DIR on a routine basis.

We have previously reported on such a framework based on the manual identification of large sets of prominent image features between volumetric image pairs¹. The study demonstrates that considerable misrepresentation of DIR spatial accuracy performance characteristics can result from analyses based on inadequate landmark sample size and distribution. As a general rule, the minimum statistical requirement on landmark sample size necessary to ensure 95% confidence intervals (CIs) of some specified length abouth the mean measured error can be approximated by:

where N is the required sample size, SD_p is the pooled standard deviation of all measurements available for the particular algorithm, d is the desired interval range, and d and SD_p are of the same units.

The number N, obtained from the above equation, represents a minimum statistical requirement for evaluation of DIR using expert-identified landmark features. However, in addition, we have also shown that large samples (in most cases, >> N) are necessary to obtain a complete characterization of spatial accuracy performance in terms of clinically relevant parameters, such as spatial location or displacement magnitude. Analyses based on landmark samples that are not sufficient in size, or that are biased towards centrally located structures (which are generally easier to identify) risk misrepresentation of the actual spatial accuracy performance of an algorithm.

These considerations make objective comparison of published DIR spatial accuracies difficult to interpret and potentially misleading. Therefore, we have established this website to provide a comprehensive common data set to investigators in this field who would like to evaluate their own algorithms, models, implementations, etc., using previously reported and characterized reference data sets composed of large samples of expert-identified landmark point pairs.

The database is a work in progress, and will be continually updated as more image volumes are manually registered. It is crucial that reference data sets, such as ours, include as wide a range as possible of images encountered in clinical practice (i.e., image data with varying motion characteristics, image quality, disease states, etc.). Thus, we welcome any suggestions for particular data sets to make available on this site. Multiple anatomic sites and imaging modalities are encouraged. For further detail, we can be contacted at Inquiries@DIR-lab.com.

This work and this web space are supported by the University of Texas M. D. Anderson Cancer Center Physician-Scientist Program. The work is also partially supported by the National Cancer Institute through NIH/NCI Grant (R21CA128230) and through an NIH Training Grant (T32CA119930). Partial support is also provided by The Gulf Coast Center for Computational Cancer Research with the GC4R Seed Grant for Collaborative Advances in Biomedical Computing (C-ABC).

¹ Castillo, R., Castillo, E., Guerra, R., Johnson, V.E., McPhail, T., Garg, A.K., Guerrero, T. 2009 A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets Phys Med Biol 54 1849-1870.