Sensor fusion using entropic measures of dependence

Fusión sensorial usando medidas entrópicas de dependencia

Paul B. Deignan^*

---

Abstract

As opposed to standard methods of association which rely on measures of central dispersion, entropic measures quantify multivalued relations. This distinction is especially important when high fidelity models of the sensed phenomena do not exist. The properties
of entropic measures are shown to fit within the Bayesian framework of hierarchical sensor fusion. A method of estimating probabilistic structure for categorical and continuous valued measurements that is unbiased for finite data collections is presented. Additionally, a branch and bound method for optimal sensor suite selection suitable for either target refinement or anomaly detection is described. Finally, the methodology is applied against a known data set used in a standard data mining competition that features both sparse categorical and continuous valued descriptors of a target. Excellent quantitative and computational results against this data set support the conclusion that the proposed methodology is promising for
general purpose low level data fusion.

Keywords: Information theory; data association; fusion; estimation; entropy.

Resumen

Contrario a los métodos estándar de asociación que ligan medidas de dispersión central, las medidas de entropía cuantifican relaciones multivaluadas. Esta distinción es especialmente importante cuando no existen modelos de alta fidelidad de los fenómenos detectados. Se muesrta que las propiedades de las medidas de entropía calzan en la marco Bayesiano de sensores jerárquicos de fusión. Se presenta un método de estimación de la estructura probabilística para medidas categóricas y continuas, el cual es insesgado para colecciones finitas de datos. Adicionalmente, se describe un método de ramificación y acotamiento de selección óptima del sensor apropiado tanto para refinamiento del objetivo como para detección de anomalías. Finalmente, la metodología es aplicada sobre un conjunto conocido de datos usados en una competencia estándar de minería de datos, que caracteriza tanto descriptores ralos categóricos como continuos de un objetivo. Excelentes resultados cuantitativos y computacionales con estos datos apoyan la conclusión de que la metodología propuesta es promisoria para propósitos generales con datos bajos niveles de fusión.

Palabras clave: Teoría de la información; datos de asociación; fusión; estimación; entropía.

Mathematics Subject Classification: 94A17.

---

Ver contenido disponible en pdf

---

References

[1] Bar-Shalom, Y.; Li, X.; Eason, R.; Kirubarajan, T. (2001) Estimation with Applications to Tracking and Navigation. Wiley, New York.         [ Links ]

[2] Blackman, S.; Popoli, R. (1999) Design and Analysis of Modern Tracking Systems. Artech House, Boston.         [ Links ]

[3] Boss´e, ´E.; Roy, J.; Wark, S. (2007) Concepts, Models, and Tools for Information Fusion. Artech House, Boston.         [ Links ]

[4] Klein, L.A. (2004) Sensor and Data Fusion: A Tool for Information Assessment and Decision Making. SPIE Press, Bellingham, WA.         [ Links ]

[5] Hall, D.L.; McMullen, S.A.H. (2004) Mathematical Techniques in Multisensor Fusion, 2 ed. Artech House, Boston.         [ Links ]

[6] Antony, R.T. (1995) Principles of Data Fusion Automation. Artech House, Boston.         [ Links ]

[7] Liggins, M.E.; Hall, D.L.; Llinas, J. (2009) Handbook of Multisensor Data Fusion: Theory and Practice, 2 ed. CRC Press, New York.         [ Links ]

[8] Hastie, T.; Tibhirani, R.; Friedman, J. (2009) The Elements of Statistical Learning, 2 ed. Springer, New York.         [ Links ]

[9] Hero, A.O.; Kreucher, C.M.; Blatt, D. (2008) “Information theoretic approaches to sensor management”, in: Hero, Castanon, Cochran & Kastella (Eds.) Foundations and Applications of Sensor Management, Springer, New York: 33–57.         [ Links ]

[10] Varshney, P.K. (1997) Distributed Detection and Data Fusion. Springer, New York.         [ Links ]

[11] Mahler, R.P.S. (2007) Statistical Multisource-Multi-Target Information Fusion. Artech House, Boston.         [ Links ]

[12] Schuck, T.M.; Hunter, B.; Wilson, D.D. (2009) “Developing information fusion methods for combat identification”, in: M.E. Liggins, D.L. Hall & J. Llinas (Eds.) Handbook of Multisensory Data Fusion: Theory and Practice, 2 ed. CRC Press, New York.         [ Links ]

[13] Kreucher, C.; Kastella, K.; Hero, A.O. (2005) “Sensor management using an active sensing approach”, Sig. Proc. 85(3): 607–624.         [ Links ]

[14] Aughenbaugh, J.M.; LaCour, B.R. (2008) “Metric selection for information theoretic sensor management”, 11th International Conference on Information Fusion.         [ Links ]

[15] Roman, S. (1992) Coding and Information Theory. Springer, New York.         [ Links ]

[16] Bell, C.B. (1962) “Mutual information and maximal correlation as measures of dependence”, Ann. Math. Stat. 33: 587–595.         [ Links ]

[17] Cover, T.M.; Thomas, J.A. (1991) Elements of Information Theory. Wiley, New York.         [ Links ]

[18] Hall, P.; Morton, S.C. (1993) “On the estimation of entropy”, Ann. Inst. of Stat. Math. 45(1): 69–88.         [ Links ]

[19] Scott, D.W. (1992) Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley, New York.         [ Links ]

[20] Defense.gov News Transcript: DoD News Briefing - Secretary Rumsfeld and Gen. Myers, United States Department of Defense (defense.gov), February 12, 2002.         [ Links ]

[21] Simonin, C.; LeCadre, J.; Dambreville, F. (2007) “The cross-entropy method for solving a variety of hierarchial search problems”, 10th International Conference on Information Fusion.         [ Links ]

[22] Fukunaga, K. (1990) Introduction to Statistical Pattern Recognition. Academic Press, London.         [ Links ]

[23] Yeung, R.W. (1991) “A new outlook on Shannon’s information measures”, IEEE Trans. Info. Theory 37(3): 466–474.         [ Links ]

[24] Deignan, P.B.; Franchek, M.A.; Meckl, P.H. (2002) “Efficient information-theoretic model input selection”, 45th Midwest Symp. Circuits and Systems 1: I-635–8.         [ Links ]


Correspondencia a: Paul B. Deignan. L-3 Communications\Integrated Systems, PO Box 6056 Greenville, TX 75403-6056, U.S.A. E-Mail: paul.b.deignan@l-3com.com

---

Received: 23 Feb 2010; Revised: 23 May 2011; Accepted: 25 May 2011