Generalized Discriminant Analysis

shiyiming · 发表于 2011-5-5 04:01:32

From oloolo's blog on SasProgramming

<a href="http://feedads.g.doubleclick.net/~a/jsoqfTNUvYaqpgcjtC7OcWq9CtQ/0/da"><img src="http://feedads.g.doubleclick.net/~a/jsoqfTNUvYaqpgcjtC7OcWq9CtQ/0/di" border="0" ismap="true"></img></a> 
<a href="http://feedads.g.doubleclick.net/~a/jsoqfTNUvYaqpgcjtC7OcWq9CtQ/1/da"><img src="http://feedads.g.doubleclick.net/~a/jsoqfTNUvYaqpgcjtC7OcWq9CtQ/1/di" border="0" ismap="true"></img></a>Using SAS to implement Flexible Discriminant Analysis [Chapter 12, ESL]. 
 
Since a discriminant analysis is equivalent to a 2-step process, i.e. regress first then conduct discriminant analysis, it is easy to implement the so called generalized discriminant analysis shown in Ch.12.4--12.6 of Elements of Statistical Learning. The basic idea here is to use some regression analysis procedures, such as using PROC REG and its RIDGE= option in MODEL statement for the ridge regression, and then use the prediction from L2 regularized regression in next step's discriminant analysis. Using PROC GLMSELECT, we can replace L2 regularization with a L1 regularization. 
 
A piece of prototype code looks like this: 
<pre style="background-color: #ebebeb; border-bottom: #999999 1px dashed; border-left: #999999 1px dashed; border-right: #999999 1px dashed; border-top: #999999 1px dashed; color: #000001; font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; font-size: 12px; line-height: 14px; overflow: auto; padding-bottom: 5px; padding-left: 5px; padding-right: 5px; padding-top: 5px; width: 100%;"><code>
PROC GLMMOD data=&yourdata OUTDESIGN=&design;
 CLASS &dep_var;
 model X = &dep_var /noint;
RUN;
data &yourdata;
 merge &yourdata &design;
 rename Col1-Col&k = Y1 -Y&k;
run;
%let deps= Y1-Y&k; /* for the case of 5-class problem */
PROC REG DATA=&yourdata RIDGE=&minridge to &maxridge by 0.1 OUTEST=beta;
MODEL &deps = &covars ;
OUTPUT OUT=predicted PRED=&dep._HAT;
RUN;

PROC DISCRIM DATA=predicted &options;
CLASS &dep;
VAR &dep._HAT;
RUN;
</code></pre> 
 
 
<a href="http://www.amazon.com/Elements-Statistical-Learning-Prediction-Statistics/dp/0387848576?ie=UTF8&tag=xie1978&link_code=bil&camp=213689&creative=392969" imageanchor="1" target="_blank"><img alt="The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics)" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=0387848576&tag=xie1978" /></a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=xie1978&l=bil&camp=213689&creative=392969&o=1&a=0387848576" style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none; margin: 0px; padding-bottom: 0px! important; padding-left: 0px! important; padding-right: 0px! important; padding-top: 0px! important;" width="1" /><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/29815492-4101905517772952711?l=www.sas-programming.com' alt='' /></div><img src="http://feeds.feedburner.com/~r/SasProgramming/~4/2VYbZW1cOpc" height="1" width="1"/>

		自动登录	找回密码
密码			立即注册