<![CDATA[
Represent a method of fitting a smooth relationship between two or more variables through a scatterplot of data points.
<P>
GENERALIZED ADDITIVE MODELS : INTRODUCTION
<P>
Generalized additive models (GAMs) represent a method of fitting a smooth
relationship between two or more variables through a scatterplot of data
points. <p>
GAMs are useful where :
<blockquote>
* The relationship between the variables is expected to be of a complex
form, not easily fitted by standard linear or non-linear models; <p>
* There is no <i>a priori </i> reason for using a particular model; <p>
* We would like the data to suggest the appropriate functional form.
</blockquote>
<P>
ILLUSTRATION OF GAMs
<BR>
One of the main reasons for using GAMs is that they do not involve strong
assumptions about the relationship that is implicit in standard parametric
regression. Such assumptions may force the fitted relationship away from its
natural path at critical points. <p>
The following example is taken from <a href="ref.htm"> Snell
and Simpson (1991). </a> The graph shows how survival time (in weeks) in
related to the log(initial white blood cell count) in leukaemia patients. <p>
<table>
<tr>
<td> <img alt="graph" src="../images/imgr12.gif" WIDTH=331 HEIGHT=204> </td>
<td> A fit using a linear model </td>
</tr>
<tr>
<td> <img alt="graph" src="../images/imgr13.gif" WIDTH=328 HEIGHT=206> </td>
<td >
A fit using a generalized additive model
</td>
</tr>
</table>
The GAM suggests a possible curved relationship which may more accurately
reflect the progress of the condition.
<P>
HOW DO GAMs WORK ?
<BR>
<blockquote>
* GAMs work by replacing the coefficients found in parametric models, by a
smoother. <p>
* A smoother is a tool for summarising the trend of a response variable (Y)
as a function of one or more predictors (X1...Xp). <p>
* It produces an estimate of the trend that is less variable, i.e. smoother,
than Y. <p>
* Smoothing takes place by <i> local averaging </i> that is averaging the
Y-values of observations having predictor values close to a target value. <p>
* A simple example of a smoother is a running mean (or moving average).<p>
</blockquote>
<a href="slide03.htm">
Further details on the models</A>
<P>
REGRESSION MODELS
<BR>
1. <a href="slide03a.htm"> Classical Linear Model </a> <br>
<img alt="model" src="../images/im01.gif" WIDTH=388 HEIGHT=34> <p>
2. <a href="slide03b.htm"> Generalized Linear Model </a> <br>
<img alt="model" src="../images/im02.gif" WIDTH=413 HEIGHT=37> <p>
3. <a href="slide03c.htm"> Additive Model </a> <br>
<img alt="model" src="../images/im03.gif" WIDTH=275 HEIGHT=59> <p>
4. <a href="slide03d.htm"> Generalized Additive Model </a> <br>
<img alt="model" src="../images/im04.gif" WIDTH=299 HEIGHT=69>
<P>
CLASSICAL LINEAR MODEL
<p>
<ul>
<li> The classical linear model is the simplest model that can be used to
relate the response variable to the explanatory variables. <p>
<li> Given a set of explanatory variables
<img align=top alt="X1i,...,Xpi" src="../images/im11.gif" WIDTH=156 HEIGHT=31>
the response variable, Y, is normally distributed. <p>
<li> Maximum likelihood estimates of
<img align=top alt="B1,...,Bp" src="../images/im12.gif" WIDTH=139 HEIGHT=36>
can be found by least squares.
</ul>
<P>
GENERALIZED LINEAR MODEL
<BR>
<ul>
<li> The generalized linear model is, as the title suggests, a more general
form of the classic linear model. <p>
<li> Given a set of explanatory variables
<img align=top alt="X1i, ... ,Xpi" src="../images/im11.gif" WIDTH=156 HEIGHT=31>
the response values follow a distribution from the exponential family. <p>
<li> For instance, the response variable may be count data, so that the errors
have a Poisson distribution. <p>
<li> <img align=top alt="Bo, ... ,Bp" src="../images/im12.gif" WIDTH=139 HEIGHT=36> can be found
by Fisher Scoring, an iterative procedure. <p>
</ul>
<P>
ADDITIVE MODEL
<ul>
<li> The additive model is a non-parametric model where the linear coefficients
<img align=top alt="B1,...,Bp" src="../images/im12.gif" WIDTH=139 HEIGHT=36>
of previous models are replaced by smoothers S1, ... ,Sp <p>
<li> A smoother is an arbitrary function that appears as a smooth curve through
a scatterplot of points <p>
<li> An additive model can contain a combination of parametric linear functions
and non-parametric smooth functions <p>
</ul>
<P>
GENERALIZED ADDITIVE MODEL
<ul>
<li> The generalized additive model is a more general form of the additive
model, and is the most general form of a likelihood-based regression
model. <p>
<li> It is similar to a generalized linear model, except that smoothing
functions are used instead of regression parameters. <p>
</ul>
CATEGORICAL PREDICTORS
<BR>
<blockquote>
* As well as coping with continuous predictors such as age and height,
GAMs can also deal with categorical predictors such as sex and colour. <p>
* These are often easier to deal with than continuous predictors. To
smooth Y, the Y values in each category can simply be averaged. <p>
* This satisfies the requirements for a smoother, it captures the trend of
Y on X and is smoother than the Y-values themselves. <p>
</blockquote>
<P>
SMOOTHING METHODS<P>
There are various methods that can be used to estimate the smooth functions
S1, ..., Sj that are included in additive models. <p>
The functions can be estimated one at a time, by a
<a href="glossary.htm#smoother"> scatterplot smoother. </a> <p>
Scatterplot smoothers include : <p>
<ul>
<li> <a href="slide05a.htm"> Running Means </a>
<li> <a href="slide05b.htm"> Running Medians </a>
<li> <a href="slide05c.htm"> Running Lines </a>
<li> <a href="slide05d.htm"> Kernel </a>
<li> <a href="slide05e.htm"> Splines </a>
<li> <a href="slide05f.htm"> Loess Smoother </a>
</ul>
RUNNING MEANS<P>
The running mean is calculated by finding the mean of all the Y values in
a <a href="glossary.htm#neighbourhood"> neighbourhood </a> of Xi, as shown by
the following formula <p>
<img alt="formula" src="../images/im05.gif" WIDTH=146 HEIGHT=70> <br> <br>
where |Ni| is the size of the neighbourhood.
<p>
The mean is the value of the smoothing function at the point Xi <p>
The disadvantages of the running mean are that it tends to flatten trends
near the endpoints and does not smooth very well in the middle.
<P>
RUNNING MEDIANS
<BR>
The running median is calculated by finding the median of all the Y values
in a <a href="glossary.htm#neighbourhood"> neighbourhood </a> of Xi
<P>
RUNNING LINES
The value on the line corresponding to Xi can then be found <p>
The formula for calculating the running line is as follows <p>
KERNEL
<BR>
This smoother is calculated by finding a weighted average of all the Y
values in a <a href="glossary.htm#neighbourhood"> neighbourhood </a> of
Xi <p>
It can be calculated by the formula <p>
<img align=top alt="formula" src="../images/im08.gif" WIDTH=113 HEIGHT=62>
which has the constraint that
<img align=top alt="formula" src="../images/im09.gif" WIDTH=87 HEIGHT=63>
<p>
The weights Wij are calculated by a kernel function which varies depending
on the distribution of the data, but with weights proportional to the
distance between the target point and the point in question (Xi-Xj)
<P>
SPLINES
<BR>
A cubic spline is a collection of polynomials of degree less than or equal to
3, defined on subintervals. A separate polynomial is fitted for each <a
href="glossary.htm#neighbourhood"> neighbourhood, </a> thus enabling the fitted
curve to join all of the points <p>
The order of splines is not limited to three, although cubic splines are the
most common.
<P>
LOCALLY WEIGHTED REGRESSION SMOOTHER
<BR>
A locally weighted regression or Loess smoother, is a smoothing function based
on the distance from the point Xi. <p>
For each point in the <a href="glossary.htm#neighbourhood"> neighbourhood </a>
of Xi the weights are calculated by a function called the tri-cube weight
function, where the weights are proportional to the cubic distance of a point
from Xi.
<P>
MULTIPLE PREDICTORS
<BR>
The simplest GAMs involve only one predictor (X) variable. <p>
GAMs can be extended to involve multiple predictors X1...Xp, with the same
ideas. Smoothers can also be extended and can be seen as acting in p
dimensional space. Although less easy to picture, the same methods apply.
<P>
DEGREES OF SMOOTHING
<BR>
The smoothness of a fitted curve depends on the size of the
<a href="glossary.htm#neighbourhood"> neighbourhood </a> that is used to
calculate the smoothed value at a particular point. <p>
If the neighbourhood is small the
smoothing function will be rough and may have a high variance <p>
If the neighbourhood is large, the estimate may be too smooth and
will not pick up curvature in the underlying function i.e. it might be
biased <p>
Statistical software usually provides
<blockquote>
* A default <a href="glossary.htm#span"> span</a>, or <br>
* Calculates an optimum span, by cross-validation
</blockquote>
<a href="slide07a.htm">
Example of how to construct a neighbourhood</A><br>
<a href="slide07b.htm">
Example of different span sizes</A>
<P>
USES OF GENERALIZED ADDITIVE MODELS
<BR>
Possible uses of generalized additive models are :
<blockquote>
* To investigate the shape of a relationship with a view to a later
parametric fit ; <p>
* To remove the effect of nuisance variables in order to concentrate on
the variables of interest. <p>
</blockquote>
<P>
USES OF GENERALIZED ADDITIVE MODELS
ADVANTAGES / DISADVANTAGES OF GAMs
<BR>
ADVANTAGES <p>
<ul>
<li> Smooth functions do not exhibit the awkward end-effects of the polynomial
which tends to bend more sharply at the extremities
than the data might justify. <p>
</ul>
DISADVANTAGES <p>
<ul>
<li> GAMs are more complicated to fit, and require greater judgement than
generalized linear models. <p>
<li> They may be difficult to interpret. <p>
</ul>
<P>
APPLICATION TO SPINAL SURGERY DATA
<BR>
This example is taken from <a href="ref.htm"> Hastie and Tibshirani (1990).
</a> <p>
Lumbar laminectomy is a corrective spinal surgery commonly performed in children
for tumours or for various development abnormalities. Following this surgery,
spinal deformities can occur, one of which is kyphosis. Kyphosis is defined to
be a forward flexion of the spine of at least 40 degrees from vertical. <p>
The available predictors are as follows :
<ul>
<li> Age in months at the time of the operation (age)
<li> Level at which the vertebrae start (start)
<li> Number of vertebrae levels involved (number) <p>
</ul>
The response variable has only two outcomes
<ul>
<li> Presence of kyphosis (1)
<li> Absence of kyphosis (0)
</ul>
The aims of the model are twofold
<ul>
<li> To ascertain which of the variables are important when trying to
predict the presence of kyphosis.
<li> To fit a model which will explain the presence of kyphosis in terms of
the useful predictors.
</ul>
<P>
STAGES IN FITTING A GAM
<BR>
<ol>
<li> <a href="slide11a.htm"> Preliminary examination of the data </a>
<li> <a href="slide11b.htm"> Choosing a link function </a>
<li> <a href="slide11c.htm"> Examine each predictor seperately </a>
<li> <a href="slide11d.htm"> Stepwise selection of predictor variables </a>
</ol>
<P>
STAGES IN FITTING A PARAMETRIC MODEL
<BR>
If we are interested in prediction, then it is more useful to have a
parametric model <p>
Stages in fitting a parametric model
<ol>
<li> <a href="slide12a.htm"> Include the variables start and age </a>
<li> <a href="slide12b.htm"> Fitting a model for age </a>
<li> <a href="slide12c.htm"> Fitting a model for start </a>
<li> <a href="slide12d.htm"> Comparing parametric fit with GAM fit </a>
</ol>
<P>
<BR>
<ul>
<li> Generalized additive models provide a useful alternative to parametric
models when estimating the response of several explanatory variables <p>
<li> Generalized additive models are extremely flexible.
</ul>
<P>
APPLICATION SOFTWARE
<BR>
Generalized additive models can be fitted using several packages including
<ul>
<li> <a href="slide14a.htm"> S-Plus </a>
<li> <a href="slide14b.htm"> Genstat </a> <p>
</ul>
S-PLUS
<BR>
To run the kyphosis example you can obtain the <a href="../example/splus.dt">
data </a> <p>
To give all the Splus code to produce the results and graphs used in the
kyphosis example would be tiresome. Included are segments of code which can
be altered to produce the appropriate results/graphs.
<ul>
<li> <a href="../example/splus1.cod"> Reading in the data </a>
<li> <a href="../example/splus2.cod"> Producing a Generalized Additive Model
</a>
<li> <a href="../example/splus3.cod"> Producing the graph of a Generalized
Additive Model fit </a>
<li> <a href="../example/splus4.cod"> Producing the graph of a parametric fit
</a>
</ul>
<P>
GENSTAT
<BR>
To run the kyphosis example you can obtain the <a href="../example/gen.dt">
data </a>
<p>
To give all the genstat code to produce the results and graphs used in the
kyphosis example would be tiresome. Included are segments of code which can
be altered to produce the appropriate results/graphs.
<ul>
<li> <a href="../example/gen1.cod"> Producing a Generilized Additive Model
</a>
<li> <a href="../example/gen2.cod"> Producing the graph of a Generalized
Additive Model fit </a>
<li> <a href="../example/gen3.cod"> Producing the graph of a parametric fit
</a>
</ul>
]]>