A test for memory management of SAS/IML

shiyiming

From Dapangmao's blog on sas-analysis

<div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-uLm68d0NIcY/Tx3UH4s5KII/AAAAAAAAA6Y/W_Z8A2Fvucg/s1600/plot2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="http://1.bp.blogspot.com/-uLm68d0NIcY/Tx3UH4s5KII/AAAAAAAAA6Y/W_Z8A2Fvucg/s320/plot2.png" width="320" /></a></div><br />
Programming always involves the considerations for the efficiency and the memory usage. For efficient programming in SAS/IML, my shortcut is to look at the <a href="http://blogs.sas.com/content/iml/2011/10/10/sasiml-tip-sheets/">tip sheet from Rick Wicklin</a> and search ways to simplify the codes. As for the memory management mechanism of SAS/IML, I only found <a href="http://support.sas.com/documentation/cdl/en/imlug/59656/HTML/default/viewer.htm#asstop_sect1.htm">one page of SAS/IML 9.2 User’s Guide</a>&nbsp;on the Internet.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-bgRcyZhMAdU/Tx3VTc9fTJI/AAAAAAAAA6k/aWtDd7f9g3w/s1600/SGPlot11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="http://4.bp.blogspot.com/-bgRcyZhMAdU/Tx3VTc9fTJI/AAAAAAAAA6k/aWtDd7f9g3w/s400/SGPlot11.png" width="400" /></a></div><br />
<br />
To see the performance of SAS/IML ‘s data management, I designed a simple test, since the SHOW SPACE statement would indicate the memory details of SAS/IML. A simulated 200 rows * 300 columns matrix occupies about 400k memory. I just requested 1MB memory by specifying the WORKSIZE at the beginning. I assigned multiple references toward this matrix, and then changed one value in this matrix. Finally I cleared all of the objects in the memory and generated a matrix with the same size. The memory change is shown in the plot above.  <br />
<pre style="background-color: #ebebeb; border: 1px dashed rgb(153, 153, 153); color: #000001; font-size: 14px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code>
/* Only request 1MB memory for this test*/
proc iml worksize=1048576;
    /* 0 -- State 0*/
     show space;
    /* 1 -- State 1 (1 reference on 1 object)*/
        x = j(200, 300, 0);
        call randgen(x, "Normal");
     show space;
    /* 2 -- State 2 (2 references on 1 object)*/
       y = x;
    show space;
    /* 3 -- State 3 (3 references on 1 object)*/
       z = x;
    show space;
    /* 4 -- State 4 (2 refernces on 1 object and 2 refrences on another object)*/
       x[1, 1] = 5;
       w = x;
    show space;
    /* 5 -- State 5 (0 references on 2 objects) */
       free x y z w;
    show space;
    /* 6 -- State 1 again (1 reference on 1 object)*/
       t = j(200, 300, 0);
        call randgen(x, "Normal");
    show space;
quit;
</code></pre><br />
Conclusions:<br />
1.   Interestingly, an object with two references costs more memory than the same object with three references. Clearly  two identical copies of the same matrix exist in State 2, while three references all point to a single matrix in State 3.<br />
<br />
SAS/IML seems to have a unique memory management system.  The memory manager of IML first checks the size available of the workspace (memory allocated to IML).  If the memory usage doesn’t pass the alarming line, it will generously make copies. Otherwise, it will become very aggressive and start to compress the memory frenetically. <br />
<br />
2.   Two similar matrices stay in the memory at State 4, although the difference between them is just one cell. Each of the matrices has 2 references. <br />
<br />
SAS/IML follows the rule of copy-on-change. Once values in a matrix are changed, the memory manager will make a copy for the changed object. <br />
<br />
3.   Running the FREE Statement doesn't immediate clear the memory at State 5. <br />
<br />
SAS/IML’s FREE statement only kills the references. The unreferenced objects will be discarded during the ensuing operations.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/3256159328630041416-7469179624832834599?l=www.sasanalysis.com' alt='' /></div><img src="http://feeds.feedburner.com/~r/SasAnalysis/~4/nLpORd3UwV4" height="1" width="1"/>