Computational & Technology Resources
an online resource for computational,
engineering & technology publications
PROCEEDINGS OF THE SECOND INTERNATIONAL CONFERENCE ON SOFT COMPUTING TECHNOLOGY IN CIVIL, STRUCTURAL AND ENVIRONMENTAL ENGINEERING
Edited by: Y. Tsompanakis, B.H.V. Topping
Relative Entropy Estimation through Stochastic Sampling and Stochastic Simulation Techniques
G. Jia and A.A. Taflanidis
Department of Civil Engineering and Geological Sciences, University of Notre Dame, United States of America
G. Jia, A.A. Taflanidis, "Relative Entropy Estimation through Stochastic Sampling and Stochastic Simulation Techniques", in Y. Tsompanakis, B.H.V. Topping, (Editors), "Proceedings of the Second International Conference on Soft Computing Technology in Civil, Structural and Environmental Engineering", Civil-Comp Press, Stirlingshire, UK, Paper 15, 2011. doi:10.4203/ccp.97.15
Keywords: relative information entropy, kernel estimation, sensitivity analysis, stochastic simulation.
The relative information entropy  constitutes an efficient measure for quantification of the difference between two probability distributions. It has been extensively used in Bayesian theory and has been recently introduced as a means for identification of critical risk factors in risk assessment applications, through an innovative sensitivity analysis [2,3]. In the latter case, the evaluation of the relative entropy is required for different marginal distributions, corresponding to different subsets for the group of uncertain model parameters (ultimately describing the various risk factors). These marginal distributions are all connected to the same auxiliary joint distribution which is formulated in the context of the risk assessment problem. The latter distribution is only known up to its normalization constant, and could require significant computation effort for its evaluation for each configuration of the uncertain model parameters.
This paper discusses the efficient evaluation of the relative entropy for such sensitivity analysis applications. The numerical evaluation of the integral corresponding to the relative information entropy is also discussed but the focus of the paper is primarily on the efficient estimation of the marginal distributions involved in this integral. A methodology based on stochastic sampling is primarily discussed for this purpose. This methodology relies on the generation of samples from the joint distribution for the entire group of uncertain variables. Projection of these samples to different subsets provides then the required information for the different marginal distributions. Implementation of kernel density estimation (KDE) approaches for approximating the marginal distributions based on the available samples is proposed. This methodology is characterized by great efficiency since the same set of samples can be used to obtain information for all required marginal distributions, but suffers from accuracy constraints when the dimension of the subsets is high. A stochastic simulation approach is also suggested for the evaluation of each required marginal distribution separately. For the numerical evaluation of the entropy integral Monte Carlo Integration (MCI) is considered.
An illustrative example is presented that considers application of the above concepts in a sensitivity analysis example for the failure probability of a simply supported beam. The sample based with the KDE approach is shown to have great efficiency. Its accuracy is very good for lower dimensional applications but reduces with the dimensions of the group of model parameters of interest.
purchase the full-text of this paper (price £20)