| Return to Main
Page Design of a Long Term Pavement Monitoring System for the Canadian Strategic Highway Research Program
2.3.3 Numerical Example This section extends the simple
numerical example presented above in Section 2.1.2 to the
Bayesian approach, overcoming the small sample size
problems inherent there. Assume that the pavement
monitoring program assembled the same five samples as
shown Table 1. However, given the preponderance of prior
information available, pavement engineers believe the
prior probability distribution over the model parameters
b to be given by the probability distribution in equation
(30) with the specific parameters b0 = 0.08 m
= 500 M = 30 k = 20 Keep in mind, b0 is the mean
of the prior probability distribution. Pavement engineers
believe before any data is assembled that b0
has the value 0.08, meaning that the pavement will
reach a pavement performance index of 0.2 in exactly ten
(10) years. However, pavement engineers are rather uncertain regarding the deterioration time, which is reflected in their prior uncertainty about the model parameter b as reflected in equation (30). As indicated in equation (33), the variance in their estimate is M[m(k4)]^(1), which is 0.00376. The prior probability distribution over model parameters b is plotted in Figure 25. Beginning with the prior distribution in equation (30) characterized by the foregoing parameters, suppose the CLTPP program assembles the same five observations show previously in Table 1. The posterior probability distribution over pavement deterioration parameters b is governed by equation (35). The critical results dictated by that probability relation are The upper and lower bounds are determined precisely as they were in the regression caseadding and subtracting one standard deviation from the expected value. Yet their values are strikingly different, and they recognize the much wider range of uncertainty that truly exists in the results than does the regression approach. To wit, the regression approach lulls us into believing that there is less uncertainty in the regression results than really exists. The expected deterioration curve for
the regression and Bayesian cases are plotted in Figure
26. The difference is obvious from the figure. In
particular, the classical statistical procedure gives a
falsely low estimate of the time it takes the pavement to
deteriorate to a performance index of 0.2. The upper and lower bound curves for the Bayesian case as compared to the regression case are plotted in Figure 27. The differences, representing the potential error from using the regression case, are evident. The differences owe in significant measure to the differences in priors between the Bayesian and classical cases. Keep in mind, classical regression assumes complete and total ignorance, while the Bayesian example presented here assumes some degree of prior knowledge about the rate of pavement deterioration. Strikingly, the Bayesian answers are completely
different than they were for the regression case! Had
we used the regression approach, we would have gotten a
different (and wrong) answer. Even
the expected value estimate b* is different. The Bayesian
approach explicitly weights prior information b0 against
new statistical information as reflected in b More
strikingly, the variance of the prior distribution
(0.06124) is substantially reduced (down to 0.04502,
approximately a 30 percent reduction) as a result of the
five observations in Table 1. Furthermore, the mean
estimate of the pavement deterioration parameter is
reduced from its initial value bo=0.08 to b*=0.07015,
but it is not reduced all the way to the regression
estimate b=0.04693 indicated in Section 2.1.2. Using the prior and posterior probability distributions over pavement deterioration parameters, we can actually calculate the prior and posterior probability distributions over the time it takes the pavement to deteriorate to an index of 0.2. We make this calculation by combining the deterioration model in equation (21) with the prior and posterior probability distributions in equations (30) and (35). Perhaps the best illustration of the
Bayesian approach is indicated in Figure 28. The figure
indicates the prior probability distribution over the
model parameter b before any observations are made and
the posterior probability distribution over the model
parameter b after the five observations are collected.
Figure 28 illustrates several critical features of the
Bayesian approach: · The mean of the prior distribution
is different from the mean of the posterior
distribution. Notice in this illustration that the
posterior distribution is shifted rightward from the
prior distribution. · The variance of the prior
distribution is smaller than the variance
of the posterior distribution. The five observations have
served not only to shift the mean estimate but also to
reduce the uncertainty regarding that estimate. · The posterior distribution is the
most complete representation of how "large" or
"small" the sample size is. While the variance
has been reduced from the prior to the posterior, the
variance in the posterior is still rather large.
Furthermore, the posterior probability distribution
itself completely specifies how much uncertainty still
exists in our estimate of the model parameter b. There is
no ad hoc measure, and there is no caveat about small
sample size. One need only make one simple judgment: Is
the posterior distribution still too wide to be
definitive? Do we need to gather more data to reduce the
variance further? Noting that five observations have
substantially reduced the variance of the prior
distribution, we now turn to the question: Will more
observations reduce the variance even further? If
Bayesian statistical methods are so accurate, what
benefits accrue from assembling more data? Is it
sufficient to terminate the exercise with the five data
points? Does Bayesian statistics reduce or eliminate the
benefits of gathering additional data. To answer this
question, we will analyze the fifteen observations case
introduced in Section 2.1.2 and compare it with the five
observations case just analyzed. The prior distribution in equation (30) is precisely the same whether fifteen or five observations are made, i.e., the prior distribution is not affected by the specific data gathering activities associated with long term pavement performance. The posterior probability distribution over pavement deterioration parameters b is governed by equation (35). The critical results dictated by that probability relation are The upper and lower bounds are
represented precisely as they were in the regression case
adding and subtracting one standard deviation from the
expected value. The expected deterioration curve for the regression and Bayesian cases with fifteen rather than five samples are plotted in Figure 29. The classical statistical procedure still gives a falsely low estimate of the time it takes the pavement to deteriorate to a performance index of 0.2, but the degree of error is smaller than it was with five observations. The upper and lower bound curves for the Bayesian case as compared to the regression case are plotted in Figure 210. The differences, representing the potential error from using the regression case, are evident. The differences owe in significant measure to the differences in priors between the Bayesian and classical cases. Keep in mind, classical regression assumes complete and total ignorance, while the Bayesian example presented here assumes some degree of prior knowledge about the rate of pavement deterioration. The counterpart of Figure 28 when we have 15 observations is displayed in Figure 211. The figure indicates the prior probability distribution over the model parameter b before any observations are made and the posterior probability distribution over the model parameter b after the fifteen observations are collected. Figure 211 that the additional ten observations do indeed narrow the variance of the prior distribution even further than did the five observations. Furthermore, the additional ten observations move the mean value to a different location, meaning that one obtains a different answer with fifteen observations than with five observations. Again, the answers are completely
different than they were for the regression case as
well as being completely different than they were
for five rather than fifteen observations! Had we
used the regression approach, we would have gotten a
different (and very wrong) answer. Had we used only five
observations, we would have gotten a different, less
accurate, and probably incorrect answer. Clearly the
Bayesian estimate explicitly weights prior information b0
against new statistical information as reflected in b.
The variance of the prior distribution (0.06124) is
reduced much further given fifteen observations (down to
0.02912, over a 50 percent reduction) as a result of the
fifteen observations in Table 2. Furthermore, the mean
estimate of the pavement deterioration parameter is
reduced from its initial value b0=0.08 to b*=0.06062,
but it is not reduced all the way to the regression
estimate b=0.04605 indicated in Section 2.1.2. Using the prior and posterior
probability distributions over pavement deterioration
parameters, we can calculate the prior and posterior
probability distributions over the time it takes the
pavement to deteriorate to an index of 0.2. We make this
calculation by combining the deterioration model in
equation (21) with the prior and posterior probability
distributions in equations (30) and (35). How many observations are needed so
that the probability distribution over model parameters
is sufficiently tight? The answer lies depends on the
degree of accuracy needed to affect realworld pavement
decisions. When pavement engineers believe the
posterior distribution to be sufficiently narrow, then it
is sufficiently narrow. When pavement engineers believe
it to be too wide to be definitive, then it is too wide
to be definitive. This rather subjective answer reflects
today's realities. However, for those jurisdictions that
have fundamentally probabilistic pavement management
techniques, the answer is much more satisfying. To wit,
probabilistic pavement management systems are driven by
probability distribution such as those in Figures 28
and 211. Furthermore, as those probability
distributions change in response to long term pavement
performance data or other data, the decisions that
emanate from those pavement management systems change. In
a very real sense, probabilistic pavement management
systems explicitly compute the value of chancing and
narrowing the posterior probability distribution
over pavement performance, which is precisely the
function of long term pavement performance monitoring. We
shall visit this issue in more detail in subsequent
sections. 2.3.4 Concluding Comments on Bayesian Statistical Methods There are several complexities in the
Bayesian and regression statistical approaches that bear
mention here. The first issue is termed "serially
correlated errors" or "autocorrelation."
Autocorrelation occurs when the random error terms for
year t in the fundamental structural model f(x,b) are
strongly correlated with the error terms in one or more
prior years t1, t2, t3,..., i.e., errors are not
independent over time but are in fact correlated.
Autocorrelated errors will almost certainly plague long
term pavement performance estimation, whose most
elemental aim is to predict systematic deterioration over
time. The regression and Bayesian statistical
approaches articulated previously can be modified to take
account of autocorrelation, but some of the appealing
simplicity will be lost. Furthermore, and more
critically, much of the value of the data base will have
to be expended to ferret out the autocorrelation
relationships embedded in the observations. Given that
much of the data base will have to be used to identify
autocorrelation in the data base itself, the true sample
size can be much smaller than is actually believed. This
can be an Achilles heel for regression methods, rendering
them ineffectual at determining the pavement
deterioration parameter b. However, Bayesian methods will
process whatever data is assembled, extensive or spare,
and determine its implications for the posterior
distribution. Notwithstanding the degree to which
embedded autocorrelation may exist in a set of pavement
performance observations, we must be vigilant to take
explicit account of autocorrelation because we believe it
to be endemic. Kmetana (1971) pp. 26970 provides a
useful intuitive test for the presence of autocorrelation
(which he calls autoregression): "...the assumption of
nonautoregression is more frequently violated in the case
of relations estimated from time series data...This
contention relies largely on the interpretation of the
disturbance as a summary of a large number of random and
independent factors that enter into the relationship
under study, but which are not measurable. Then, one
would suspect that the effect of these factors operating
in one period would, in part, carry over to the following
periods... Autoregression of the disturbances
can be compared with the sound effect of tapping a
musical string: while the sound is loudest at the time of
impact, it does not stop immediately but lingers on for a
time until it finally dies off...But while the effect of
one disturbance lingers on, other disturbances take
place, as if the musical string were tapped over and
over, sometimes harder than at other times." In a realworld long term pavement
monitoring setting, this notion of a series of
unpredictable, random disturbances (e.g., environment,
traffic) each setting up a systematic, persistent,
timedependent contribution to pavement deterioration
seems quite plausible and descriptive. A wet spring with
many freezethaw cycles would seem to initiate
inexorable deterioration processes that would persist
over the life of the pavement and would be unaffected by
the existence of many future events. These deterioration
processes, unidentifiable by physical means, would
partially obfuscate contributions of future years to the
pavement deterioration process unless the effect of the
first spring could be quantified and removed. Systematic
dynamic processes introduced randomly over time must be
systematically predicted. This can be done by extending
the simple Bayesian statistical and regression
discussions in Section 2. However, such extension would
occur at the expense of "devaluing" the
observations. Should we implement the proposed technique
for CSHRP, we will take explicit account of potentially
autocorrelated data. We should point out that the existence
of autocorrelated errors can significantly devalue any
data that does not date from the first year of pavement
rehabilitation, i.e., any data that begins partway
through a pavement life cycle. The reason is simple. Data
gathered beginning part way through a pavement life cycle
cannot possibly identify autocorrelated errors that were
initiated at points during the life cycle before any
measurements were taken. The effects such errors can
cause during the portion of the life cycle monitored
cannot be systematically distinguished from errors that
truly emanate from events after measurement begins. Figure 212 displays graphically how autocorrelation works. Notice in the figure that the early data points systematically fall above the regression line while the later data points systematically fall below the regression line. Clearly the simple line in the figure cannot represent the true, systematic dynamics that underlie pavement deterioration; a more complex function with more endogenous parameters is needed. Furthermore, if pavement monitoring were initiated midway through the pavement life cycle, those points below the line would be systematically overemphasized and those point above the line would be systematically ignored. This would introduce bias, perhaps severe or fatal bias, into the conclusions. In the presence of autocorrelation, it is clear that complete life cycle information is a must. A second complication arises from the
assumption that the error teens in equation (24)(26)
all have the same variance (homoskedasticity). The
homoskedasticity assumption would be satisfied for
example if the same variables were measured in the same
way using the same method with the same degree of
accuracy at every location in every time period. In such
a situation, there would be no systematic difference in
the error terms among all the measurements. Yet in
reality, we know that every monitoring site will be
fundamentally different. There will be different
measuring technology, different degrees of accuracy, and
even different variables monitored. Historical
information that is estimated rather than directly
measured will be fundamentally inconsistent. Furthermore,
monitoring that begins partway through a pavement life
will not be able to identify autoregressive errors
initiated before monitoring began but persisting after
monitoring was initiated. Thus, error terms measured
after monitoring is initiated partway through pavement
life will inaccurately reflect true random errors; they
will erroneously contain unidentified autoregressive
and/or heteroskedastic errors. The only way to take account of heteroskedasticity is to postulate systematic error differences among observations, i.e., systematic mathematical differences, and measure them using statistical inference. To do so requires the introduction of more parameters into equations (24)(26) and thereby the exhaustion of more degrees of freedom in the data. In particular, heteroskedasticity greatly "devalues" the data, devaluing the amount of variance reduction possible from a given set of observations and thereby devaluing the predictive content of a given data base. Heteroskedasticity, sure to exist in the CLTPP and LTPP data bases, will have to be dealt with before definitive conclusions can be reached. Should we implement the proposed methodology under CSHRP, we will introduce specific capability to do so. Figure 213 gives a conceptual
illustration of heteroskedasticity. Notice that errors
are small in the early years but large in the later
years. To assume that errors are governed by the same
stochastic process throughout the life cycle of the
pavement is patently incorrect. On the contrary, one
would have to postulate a structural model of the
systematic growth in measurement error, postulate one or
more parameters of that structural model, introduce it
into the simple regression approach above, and fit for
the parameters of the structural model of error terms
simultaneously with the pavement deterioration parameters
b. To do so would expend part of the value of the data
base, for part of the data base would have to be used to
ferret out the parameters of the structural model of
error terms. A third complication sure to arise and
devalue the data base is multicollinearity.
Multicollinearity occurs if one introduces the same
measurement into the set of observations many times,
i.e., if one measures the same phenomenon many times in
many ways at many locations and inserts those many
measurements into the data base. To the extent a number
of measurements are simply redundant observations of the
same phenomenon, they contain only information related to
that phenomenon. They do not and cannot contain
information related to other phenomena. They therefore
have far less predictive significance than their sheer
number might suggest. Multicollinearity will be detected by
the Bayesian approach quite simply and accuratelythere
will be far less change in the prior distribution than
one would expect. That is, there will be far less tightening
and far less shifting of the prior probability
distribution than one might expect from the sheer size of
the data base. Multicollinearity will be detected by the
regression approach by rather large variance in the
parameter estimates, larger than the number of data
points might suggest. A graphical picture of Multicollinearity appears in Figure 214. In the figure, what do the high number of incidences of points precisely on the regression line tell us? If they are simply redundant statements of the same phenomenon, the answer is nothing; two points from among the many would suffice. A fourth complication is the use of
nonlinear rather than linear structural models f(x,b).
Should one postulate and use a pavement deterioration
model f(x,b) that is nonlinear in b rather than the model
we have use thus far that is linear in b? Wouldn't it be
better to assume nonlinearity rather than linearity? Some
of the SHRP documentation pays lip service to nonlinear
models, and at least one of the original AASHTO equations
appears to be nonlinear in its parameters. However,
empirical evidence over a number of economic and physical
fields indicates the futility of appealing to models that
are nonlinear in their coefficients unless an explicit,
valid, structural underlying mechanism can be
justified from theoretical first principles. Absent
fundamental justification from first principles, a
plethora of past economic and physical studies support
the assertion that little is gained by appealing to
nonlinear models relative to postulating a broader array
of prospective causal independent variables. Assuredly a
problem as complex and confounded as pavement
deterioration is not at present amenable to a
fundamental, structural representation; too little is
known about the physical and chemical mechanism of
pavement deterioration. Therefore, we conjecture that the
more insightful and productive work in the field will
arise from clever applications of linear models such as
that outlined here. We should note that it is
straightforward to generalize the regression and Bayesian
statistical methods described here to include general
nonlinear deterioration relationships, but analytical
elegance and computational simplicity will be sacrificed.
We recommend proceeding onward to nonlinear models only
after some years of experience and insight with linear
models. While the statistical inference
approach articulated in classical regression form (and
later in Bayesian form) is seductively simple and
appealing, in practice, much can "go wrong"
with the apparently simple and foolproof regression
approach, particularly with its extension to the
multivariate case. For example, what if important
variables are inadvertently (or intentionally) omitted
from consideration? What if the vector x of independent
variables is "too short" because a certain
variable is not monitored? The answer is that the results
of the regression analysis can be badly contaminated
and/or much of the data will be devalued. To see why, consider that if two
independent variables (the ith and jth elements of the
vector x) are partially or completely correlated with
each other, regression analysis culminating in the
multivariate counterpart of equation (5) determines how
much of the effect observed in the pavement performance
variable y is due to each of the elements of x
individually. That is, the ith element of the vector b
predicts the marginal effect of the ith independent
variable without consideration of the effect of the other
independent variables. Standard statistical analysis as
embodied in the multivariate counterpart of equation (5)
calculates only the variability in the pavement
performance index y that is uniquely and independently
attributable to each individual independent variable in
isolation from the other independent variables. Consider what would happen if there
were complete lockstep correlation between two
explanatory variables. In this case, the multivariate
extension of equation (5) will not be able to determine
how much effect on pavement performance is independently
due to each. If both are included in the vector x of
independent variables simultaneously, the significance of
each will be zero. If either one is included alone, i.e.,
one of the two is used as a proxy for the other, the
variable that is included may have a high degree of
statistical significance. However, that significance is
falsely high, as it may partly or fully represent not
just the variable included but also the variable omitted
that is statistically correlated. Misstatement of the
degree of statistical significance that results from
omission of a correlated variable is termed "missing
variable bias." Missing variable bias is important for
multivariate regression analysis such as that proposed by
SHRP and CSHRP. Not only does missing variable bias
introduce "false positives" and "false
negatives" with regard to the variables included, it
also misquantifies the magnitudes of the entire vector b.
Not only may there be problems with postulating the
significance of a dependent variable, but the magnitude
of significance itself (which regression analysis also
provides) may be badly biased. If any causative factors
are missing from the vector of independent variables x,
then any of the factors that are included in the
independent variables x and that are also in any way
correlated with the missing variables will implicitly
reflect the effect of the missing variable. If the effect
of the missing variable is in the direction opposite of
that of the included variable (i.e., anticorrelation),
then the estimated effect of the included variable will
be smaller than its true effect. Similarly, if the
omitted variable works in the same direction as the
included variable (positive correlation), then omission
of the variable will bias upwards the estimate of the
significance of the included variable. Regression analysis does not have a
mechanism for determining the presence of missing
variable bias until there are more complete data
available that allow inclusion of the missing variable.
Undetected and undetectable errors can be made simply by
omitting prospectively important variables from the
independent variable vector x. The analyst must therefore
be very careful to minimize the probability that
important explanatory variables are omitted and thus to
avoid attributing significance to a variable that is
serving as a proxy for some other underlying cause that
has not been properly included in the analysis. Missing variable bias has been
emphasized here because of the high potential for failing
to gather prospectively important data at one or more
CSHRP monitoring sites. It is far easier to
"forget" to monitor a particular variable than
it is to "remember" to monitor it. To avoid the
contamination of results from missing variable bias,
there is strong incentive indeed to monitor everything
under the sun at every site. That is, there is strong
incentive to monitor tens or even hundreds of variables
at the chosen sites. Unfortunately, however, the
resources available to the CLTPP program will limit the
number of variables measured. While missing variable bias is
problematic for Bayesian as well as regression methods,
it is not as fatal for Bayesian methods because they
depend on the prior as well a the posterior distribution.
While the foregoing critique of classical methods applies
to Bayesian methods as well, it does not apply as
extremely. Before leaving this section, we wish to
comment on the need for a customized software system that
will be required to implement the Bayesian statistical
approach and achieve the immediate stream of benefits
from the monitoring program. The SHRP has hired a
contractor to review and recommend a statistical system
to accept and store the data base and make the necessary
statistical calculations. Unfortunately, the three
statistical systems reviewed are purely regression
packages and cannot make the Bayesian calculations
outlined here. However, they are very suitable for
accepting, storing, and disseminating the large data base
anticipated here. This suggests that the best strategy
for CSHRP is to use commercial statistical packages
such as BMDP, SAS, and SPSS for accepting and storing
"raw" input data but implementing a Bayesian
statistical "backend" to make the requisite
calculations and deliver the critical pavement
deterioration information inferred from the monitoring
data to the decision makers around North America. DFI has
often used SAS to store huge quantities of statistical
data, deliver those data to custom designed application
programs (such as Bayesian statistical updating), and
accept the results of those custom designed programs back
into SAS. We recommend that such an approach be taken by
Canadian SHRP, "piggybacking" on the data
retrieval capabilities of the system selected by SHRP but
implementing custom designed Bayesian statistical
capability to process the data and deliver results to
your constituents. In short, the requisite data management
system has to be custom designed to the application at
hand. To assume that it exists, or can be fabricated by
easy modification of a general commercial package, is
probably wishful thinking. New problems require new
solutions. The best one can hope for is that the new
solution can be fabricated from large existing
"chunks" of proven capability. |
