Abstract

"The method of transformations of random variables has been a standard tool in statistical inference."

One can apply transformations of random variables to conduct inference for multiple distributions in a few simple steps. These methods are used routinely in maximum likelihood estimation but are rarely applied in other statistical procedures. In this project, transformations of variables were explored and applied to derivations of the best unbiased estimators, Bayesian estimators, construction of various kinds of priors, estimation and inference in the stress-strength problem. First, general results were obtained on the application of transformations of random variables to the derivation of numerous statistical procedures. Second, common distributions and the relationships between them were listed in a table. Third, examples of applications of our theory were provided; i.e., papers published in various statistical journals were examined and the same results were obtained in just a few lines with almost no effort. The value of this project lies in the fact that undergraduate level statistics can yield such powerful results.

Keywords: Transformations of Variables, Statistical Inference.

List of Tables and Figures

Table 1: Transformations of random variables

Introduction

The method of transformations of random variables has been a standard tool in statistical inference. Transformations have been used for solutions various of statistical problems, such as nonparametric density estimation, nonparametric regression, analysis of time series, and construction of equivariant estimators. ^2-7 Journals and books such as ``Continuous Univariate Distributions'' and ``Unbiased Estimators and Their Applications'' construct statistical inference for dozens of statistical distributions ^8,9,10. Procedures are usually conducted for each distribution family separately; often, at least one of the numerous calculations required leads to errors.

However, one simple application has been largely overlooked by the statistical community. The objective of this paper is to provide a simple approach to statistical inferences using the method of transformations of variables. It is a well-known fact that the majority of familiar probability distributions are just transformations of one another. Consequently, results of parametric statistical inference for one family of pdfs can be reproduced without much work for another family. We shall demonstrate performance of the powerful tool of transformations on examples of constructions of various estimation procedures, hypothesis testing, Bayes inference, stress-strength problems. We argue that the tool of transformations not only should be used more widely in statistical research but should also become a routine part of calculus-based courses of statistics. The following results include only the case of a one-dimensional random variable. While the theory has an obvious extension to the case of random vectors, making this generalization would unnecessarily complicate the presentation.

Consider a random variable with the pdf $f(x\vert\mbox{ \mathversion{bold} $\theta $})$ , where parameter $\mbox{ \mathversion{bold} $\theta $}$ is a scalar or a vector. Suppose also that there exist a random variable $\xi$ , a monotone function and a one-to-one transformation $\nu$ such that $X = u(\xi)$ , where the pdf $g(\xi\vert\mbox{ \mathversion{bold} $\tau$})$ of $\xi$ has a different parameterization from , namely,

$\begin{displaymath} g(\xi\vert\mbox{ \mathversion{bold} $\tau$}) = f(u(\xi) \ve... ...on{bold} $\theta $}=\nu(\mbox{ \mathversion{bold} $\tau$}). \end{displaymath}$

(1)

Denoting $u^{-1} = v$ and $\nu^{-1} = \eta$ , we rewrite (1) as

$\begin{displaymath} f(x\vert\mbox{ \mathversion{bold} $\theta $}) = g(v(x) \ver... ...bold} $\tau$}= \eta (\mbox{ \mathversion{bold} $\theta $}). \end{displaymath}$

(2)

Now, let $g(\xi\vert\mbox{ \mathversion{bold} $\tau$})$ be a popular distribution family, so that all sorts of statistical results are available. The objective of the present paper is a re-formulation of these results for $f(x\vert\mbox{ \mathversion{bold} $\theta $})$ . Here, notice that the correspondence (1) is quite common (see Table 1) but is not used to the full extent. Our goal is not to explore all possible correspondences of this sort but to provide few examples which will illustrate the general idea which can be easily extended to many more kinds of statistical procedures and various other families of distributions. For this reason, this paper focuses on only a few distributions which can be obtained by transformations from a scale or location-scale family of exponential distributions. The majority of those distributions are listed in Table 1.

For example, for the exponential and Weibull distributions, $g(\xi\vert\tau)$ and $f(x\vert\mbox{ \mathversion{bold} $\theta $})$ , respectively, we have $v(x)=x^\alpha$ , $\mbox{ \mathversion{bold} $\theta $}= \sigma$ , $\tau = \lambda = \eta(\sigma ) = \sigma ^{-\alpha }$ , and $g(\xi)= \lambda \exp \left\{- \lambda \xi \right\}$ and $\alpha >0$ . Thus

$\begin{displaymath}\begin{array}{lcl} g(v(x) \vert \eta(\mbox{ \mathversion{bol... ...=&f(x\vert\mbox{ \mathversion{bold} $\theta $})\\ \end{array}\end{displaymath}$

Voinov and Nikulin ¹⁰ list 115 uniform minimum variance unbiased estimators in the case of the one-parameter exponential distribution and only 31 estimators in the case of the Weibull distribution. The use of Table 1 and transformation of variables can yield each of the missing estimators for the Weibull distribution in a few short lines. We will discuss these techniques in the latter part of the present paper.

Some readers of this paper may remark that some of the results listed here can be obtained in a general form for, say, one or two-parameter exponential families. The goal, however, is not to provide such a generalization but to supply a simple and yet powerful methodological tool to modify statistical procedures. The scale and location-scale family of exponential distributions is used here only as an example. In fact, techniques described below can be used for a distribution family which does not have a sufficient statistic.

The rest of the paper is organized as follows. Table 1 showing transformations between distribution families is at the end of this introduction. Section 2 considers basic statistical inference for $f(x\vert\mbox{ \mathversion{bold} $\theta $})$ based on inference for $g(\xi\vert\mbox{ \mathversion{bold} $\tau$})$ - sufficient statistics, maximum likelihood estimators (MLE) and uniform minimum variance unbiased estimators (UMVUE), interval estimators and likelihood ratio tests. Section 3 provides examples of results which can be obtained by applying techniques suggested in this paper. Section 4 concludes the paper with the discussion. Section 5 contains proofs of all statements formulated in previous sections.

Table 1. Transformations of random variables.

$f(x\vert\mbox{ \mathversion{bold} $\theta $})$	Transforms	$g(\xi\vert\mbox{ \mathversion{bold} $\tau$})$
Weibull distribution: $f(x\vert\sigma ) =$ $\frac{\alpha }{\sigma ^\alpha } x^{\alpha -1} \exp \left\{- \left(\frac{x}{\sigma } \right)^\alpha \right\}$ , $\alpha$ known, .	$v(x)=x^\alpha$ $\mbox{ \mathversion{bold} $\theta $}= \sigma$ $\mbox{ \mathversion{bold} $\tau$}= \lambda$ $\eta(\sigma ) = \sigma ^{-\alpha }$	One-parameter exponential distribution: $g(\xi \vert \lambda ) = \lambda \exp \left\{- \lambda \xi \right\},$ $\ \xi>0$ .
Rayleigh distribution: $f(x\vert\sigma ) =$ $\frac{x}{\sigma } \exp \left\{-\frac{x^2}{2 \sigma } \right\},$	$\mbox{ \mathversion{bold} $\theta $}= \sigma$ $\mbox{ \mathversion{bold} $\tau$}= \lambda$ $\eta(\sigma ) = \sigma$	One-parameter exponential distribution: $g(\xi \vert \lambda ) = \lambda \exp \left\{- \lambda \xi \right\},$ $\ \xi>0$ .
Burr type X distribution: $f(x\vert\sigma ) =$ $2\sigma x e^{-x^2}(1 - e^{-x^2})^{\sigma -1},$	$v(x) = -\ln (1 - e^{-x^2})$ $\mbox{ \mathversion{bold} $\theta $}= \sigma$ $\mbox{ \mathversion{bold} $\tau$}= \lambda$ $\eta(\sigma ) = \sigma$	One-parameter exponential distribution: $g(\xi \vert \lambda ) = \lambda \exp \left\{- \lambda \xi \right\},$ $\ \xi>0$ .
Burr type XII distribution: $f(x\vert\sigma ) =$ $\frac{\alpha x^{\alpha -1}}{\sigma (1+x^\alpha )^{\frac{1+\sigma }{\sigma }}},$ $\alpha$ known,	$v(x) = \ln (1 + x^\alpha )$ $\mbox{ \mathversion{bold} $\theta $}= \sigma$ $\mbox{ \mathversion{bold} $\tau$}= \lambda$ $\eta(\sigma ) = \sigma$	One-parameter exponential distribution: $g(\xi \vert \lambda ) = \lambda \exp \left\{- \lambda \xi \right\},$ $\ \xi>0$ .
Extreme Value distribution: $f(x\vert\sigma ) =$ $\frac{1}{\sigma } \exp \left\{x - \frac{\exp x -1 }{\sigma } \right\}$ , .	$v(x) = \exp \left\{x \right\}- 1$ $\mbox{ \mathversion{bold} $\theta $}= \sigma$ $\mbox{ \mathversion{bold} $\tau$}= \lambda$ $\eta(\sigma ) = \sigma ^{-1}$	One-parameter exponential distribution: $g(\xi \vert \lambda ) = \lambda \exp \left\{- \lambda \xi \right\},$ $\ \xi>0$ .
Pareto distribution: $f(x\vert \sigma , \rho) =$ $\frac{\sigma \rho^\sigma }{x^{\sigma + 1}} I(x \geq \rho).$	$v(x) = \ln x$ $\mbox{ \mathversion{bold} $\theta $}= (\sigma , \rho)$ $\mbox{ \mathversion{bold} $\tau$}= (\lambda , \mu)$ $\eta(\sigma , \rho) = (\sigma , \ln \rho)$	Two-parameter exponential distribution: $\xi \geq \mu.$
Power distribution: $f(x\vert \sigma , \rho) =$ $\frac{\sigma }{\rho^\sigma } x^{\sigma -1} I(0 < x <\rho).$	$v(x) = \ln(1/x)$ $\mbox{ \mathversion{bold} $\theta $}= (\sigma , \rho)$ $\mbox{ \mathversion{bold} $\tau$}= (\lambda , \mu)$ $\eta(\sigma , \rho) = (\sigma , \ln (1/\rho))$	Two-parameter exponential distribution: $\xi \geq \mu.$