Finding the number of times a threshold has exceeded cutoff

I was approached by a student who had a time series data (120 individual animals observed at 250 time points). She want to find the following:

1)How man times an animal's outcome has been above a threshold for 5, 6,7 ... j consecutive times?

2) An animal can be above a threshold for 'j' consecutive times and then go below it for 'n' consecutive time points and go above it for 'k' consecutive time points . I would prefer this pattern to be counted distinctly and also as only once.


/*this solution was offered through SAS-L */
/*Generate test data */
data test;
do animal = 1 to 120;
do time = 1 to 250;
outcome = floor(10*ranuni(123) );
output;
end;
end;
run;

/*First step is to create a variable indicating whether the threshold (3, for example) is exceeded*/

data step1 /*/ view=step1/*;
set test;
over = (outcome > 3);
run;

/* Next, reduce to one observation for each series of consecutives */

data step2(drop = outcome) /* /view=step2 */;
do consecutive = -1 by -1 until (last.over);
set threshold;
by animal over notsorted;
end;
run;

proc freq data=step2;
tables consecutive / nopercent;
where over;
run;

Finding the largest observation

There might be many ways to do this. A simple way would be to as follows:

data new;
Infile '/udd/n2man/epimarks/classmarks.txt';
input roll 4. ExamDate MMDDYY10. @17 MidFinal $2. Marks 4.;
obsno=_n_;
/* creates a new variable obsno whose value is the same as observation number in the dataset */

proc univariate;
var marks;
run;

/* see the observation number for the largest variable(s) in the output. Run the program again with following added*/

proc print;
where obsno=15; /* 15 is the obsno from the output of proc univariate */

A sophisticated way

data new2;
Infile '/udd/n2man/epimarks/classmarks.txt';
input roll 4. ExamDate MMDDYY10. @17 MidFinal $2. Marks 4.;

proc sort data =new2;
by descending marks;

data _null_ ; /* null datasets only exist for the particular datastep where they are called */
set new2 ;
If _n_=1 then call symput("IDNumber",Roll);
/* this creates a macro variable IDNumber whose value is the roll of the first observation which is the one with largest marks because of sorting*/
else stop;

proc print data=new2;
where Roll="&IDnumber";
format ExamDate WORDDATE18.;
title "Student &IDNumber Had the highest marks ";
run;

Redirecting sas log and output files

When sas program mal or mal.sas is run on unix, it would produce output files sql.log and sql.lst. It is possible to redirect these files as follows:

sas mal.sas -log mallog1 -print mresults &

where mallog1 and mresults could be any valid Unix file names. The ampersand (&) ensures that the sas program is run in the background (see earlier posts).

Running SAS jobs in background

If you want to continue editing programs while SAS jobs run in the background, you can do that by placing an ampersand ("&") after program name. For example:

   sas myprog &

Running SAS on UNIX in Interactive mode

If you have got used to running SAS on Windows and now you are stuck with using SAS on UNIX, accept my sympathies ;)

Anyways, you can run SAS on UNIX with X11 tunelling. The X-Windows interface is similar in appearance and functionality to SAS for Windows.

To do this you need two programs:

• Putty or Secure CRT to connect to the UNIX server from Windows PC.
  
• An X-Windows server running on your computer.

Installing X-Windows server is easy.
Download the program from here. Unzip this folder and run setup. You will get a window titled MicroImages X Server.

To run SAS, you will need to configure it to allow XWindows "tunneling."

In SecureCRT, after starting click on Properties.

In the Properties window click on X11 tab and check Forward X11 Packets.

Connect to the server as usual and once connected type
sas

After a pause of 4 seconds or so you would be able to see windows similar to what you see in Windows in the MicroImages X Server window.

The response of SAS in this environment can be a little slow.

Convert character format to numeric format

In the previous post, if you use the substr function to tease out a number e.g. roll, the number is in character format. Mathematical functions can not be performed on character variables.

To convert a variable from character to number format, there are following ways:

rollno=roll*1;

rollno=input(roll,4.0)