Difference between revisions of "LoadingData"

From ECLR
Jump to: navigation, search
(Preliminaries)
Line 3: Line 3:
 
= Preliminaries =
 
= Preliminaries =
 
<math>\alpha\!</math>
 
<math>\alpha\!</math>
 +
<source lang="matlab">
 +
>> array = 1:2:9
 +
array =
 +
1 3 5 7 9
 +
</source>
 
Very often in your life you have to repeat the same operation many times (move your right and left legs while walking/running) or behave differently depending on external conditions (there is or there is no bus on a bus stop). Quite often these two are combined together. Say, if there is a bus on a bus stop, then you run trying to catch it, otherwise walk or stop and enjoy the usual Manchester weather. The same is true for programming. Quite often you want to repeat the same operation many times, or you want to change the way you treat your data depending on some conditions. We start with conditional statements. They execute different pieces of code depending whether is true or false. There are several ways you can formulate it. The shortest
 
Very often in your life you have to repeat the same operation many times (move your right and left legs while walking/running) or behave differently depending on external conditions (there is or there is no bus on a bus stop). Quite often these two are combined together. Say, if there is a bus on a bus stop, then you run trying to catch it, otherwise walk or stop and enjoy the usual Manchester weather. The same is true for programming. Quite often you want to repeat the same operation many times, or you want to change the way you treat your data depending on some conditions. We start with conditional statements. They execute different pieces of code depending whether is true or false. There are several ways you can formulate it. The shortest
  

Revision as of 07:24, 18 September 2012

=1.0

Preliminaries

[math]\alpha\![/math]

>> array = 1:2:9
array =
 1 3 5 7 9

Very often in your life you have to repeat the same operation many times (move your right and left legs while walking/running) or behave differently depending on external conditions (there is or there is no bus on a bus stop). Quite often these two are combined together. Say, if there is a bus on a bus stop, then you run trying to catch it, otherwise walk or stop and enjoy the usual Manchester weather. The same is true for programming. Quite often you want to repeat the same operation many times, or you want to change the way you treat your data depending on some conditions. We start with conditional statements. They execute different pieces of code depending whether is true or false. There are several ways you can formulate it. The shortest

if condition
statement1;
statement2;
...
end
 

, executes only if is satisfied. can be anything that generate non-zero or 0 (True or False), say , , or . The last condition is True always but for . A slightly longer version

  if condition
  statement1;
  statement2;
  ...
  else
  statement1a;
  statement2a;
  ...
  end
  

runs , if is true and otherwise. In the most general case it looks like

  if condition1
  statement1;
  statement2;
  ...
  elseif condition2
  statement1a;
  statement2a;
  ...
  ...
  ...
  elseif conditionN
  statement1b;
  statement2b;
  ...
  else
  statement1c;
  statement2c;
  ...
  end

In this case, however, you have to ensure that are mutually disjoint. As an example, you might think about different actions depending on your final grade. Say, condition1: ; condition 2: ; condition 3: ; etc.

MATLAB has two statements that creates a loop. First, it is unconditional loop:

for CounterVariable=[range of values]
statement1;
statement2;
...
end

It repeats at most as many times as many elements it has in the . If range of values is empty, this loop does not run. Say, if you define a range , MATLAB creates an empty range. Thus, this loop will not be executed. If you define a range , MATLAB creates a range of four values , and the loop runs four times. During the first iteration, , during the second , etc. After the end of the loop . Please note, this is very unwise to modify the counter inside the loop. All modifications will disappear after the next iteration. Please also note, that the values in the range could be anything, including filenames or matrices from a cell vector. There are two commands that can modify an execution of the loop. breaks the current iteration of the loop. Once it is observed, the loop continues skipping current iteration. stops the execution of the loop and your program continues after this point. These commands are used inside statements. Say, skips the loop iteration for .

Second, it is a conditional loop

while condition
statement1;
statement2;
...
end

This version of loop executes statements as long as is true. If is always true, your loop runs forever.

loop

A standard application for loop is a reconstruction of AR(p) series once AR(p) coefficients and a vector of error terms in known. [math]y_t=\phi_0+\sum_{i=1}^p \phi_i y_{t-i}+e_t.![/math] For simplicity, we assume that [math]p=1[/math]. Also, to be able to compute [math]y_1[/math], we need to provide [math]y_0[/math]. Since we don’t know [math]y_0[/math],the best guess for [math]y_0[/math] is [math]E(y_0)[/math]. For stationary AR(1) process, that is for the case [math]|\phi_1|\lt 1[/math], [math]E(y_0)=\phi_0/(1-\phi_1)![/math]. Thus, knowing [math]y_0[/math] and [math]e_t[/math] for [math]t=1,\ldots,T[/math], we can reconstruct [math]y_t,\ t=1\ldots,T[/math]:

[math] y_1=&\phi_0+\phi_1 y_0+e_1\\ y_2=&\phi_0+\phi_1 y_1+e_2\\ &\ldots\\ y_t=&\phi_0+\phi_1 y_{t-1}+e_t\\ &\ldots\\ y_T=&\phi_0+\phi_1 y_{T-1}+e_T[/math]

Definitely, if you are patient enough and [math]T[/math] is not very large, you can create your m file with [math]T[/math] lines in it. However, once [math]T[/math] is unknown, this approach would not work. Fortunately, there is a better alternative for this type of operations. All these computations can be summarized using the following algorithm:

  1. Find a length of a vector of error terms :
  2. Initialize a vector of the same length as vector :
  3. Compute . Please remember, we assume that [math]y_0=E(y)=\phi_0/(1-\phi_1)[/math]
  4. Compute for [math]i=2[/math]
  5. Repeat line 4 for [math]i=3,...,T[/math]

Assuming vector is known in advance, the MATLAB code is

  T=size(e,1);
  y=zeros(T,1);
  y0=phi0/(1-phi1);
  y(1)=phi0+phi1*y0+e(1);
  for i=2:T
    y(i)=phi0+phi1*y(i-1)+e(i);
  end

However, if , [math]E(y_t)[/math] is not constant. In this situation the formula we use in the code does not work and will create either a series of [math]\pm\infty[/math], if or a series of not a numbers , if [1].

or

To avoid these inconveniences, we have to consider separately two cases:

  1. AR(1) process is stationary, i.e. [math]|\phi_1|\lt 1[/math]
  2. AR(1) process is nonstationary, i.e.  [math]|\phi_1|\ge 1[/math]

For the latter, we have to acknowledge the fact that [math]E(y_t)=\mu_t[/math], i.e. unconditional expectation is a function of time. In this case we have to set [math]E(y_0)[/math] to some value. A standard assumption for non-stationary series is to assume that [math]E(y_0)=0[/math].

The algorithm in this case would look like:

  1. Find a length of a vector of error terms :
  2. Initialize a vector of the same length as vector :
  3. Check whether . If this statement is true, then . Else, . Please remember, we set [math]y_0=E(y_0)[/math].
  4. Compute .
  5. Compute for [math]i=2[/math]
  6. Repeat line 4 for [math]i=3,...,T[/math]

Assuming vector is known in advance, the MATLAB code is

  T=size(e,1);
  y=zeros(T,1);
  if abs(phi1)<1
  y0=phi0/(1-phi1);
  else
  y0=0;
  end
  y(1)=phi0+phi1*y0+e(1)
  for i=2:T
    y(i)=phi0+phi1*y(i-1)+e(i);
  end

If you don’t like word else, you can skip it:

  T=size(e,1);
  y=zeros(T,1);
  y0=0;
  if abs(phi1)<1
  y0=phi0/(1-phi1);
  end
  y(1)=phi0+phi1*y0+e(1)
  for i=2:T
    y(i)=phi0+phi1*y(i-1)+e(i);
  end

loop

An alternative way of running the same code is to use a conditional loop (purely for demonstration purposes). Usually conditional loop is used when the number of iterations is not known in advance.

  1. Find a length of a vector of error terms :
  2. Initialize a vector of the same length as vector :
  3. Check whether . If this statement is true, then . Else, . Please remember, we set [math]y_0=E(y_0)[/math].
  4. Compute .
  5. Compute for [math]i=2[/math]
  6. Increase i by 1, i.e. [math]i=i+1[/math] (please note, in programming this is not a stupid statement,
  7. Repeat line 4 while [math]i\lt =T[/math]

Assuming vector is known in advance, the MATLAB code is

  T=size(e,1);
  y=zeros(T,1);
  if abs(phi1)<1
  y0=phi0/(1-phi1);
  else
  y0=0;
  end
  y(1)=phi0+phi1*y0+e(1)
  i=2;
  while i<=T
    y(i)=phi0+phi1*y(i-1)+e(i);
    i=i+1;
  end

Imperfect substitutes of the above

MATLAB has two powerful tools that make programmer’s life much easier and utilization of loops/if less frequent. In addition, quite often it makes the code run faster. In particular,

  1. Logical expressions work not only on scalars, but also on vectors, matrices and, in general, on n-dimensional arrays.
  2. Subvectors/submatrices can be extracted using logical 0-1 arrays.

Irrelevant but useful example

typing in MATLAB command window create a [math]1\times5[/math] row-vector with values [math][1\ 2\ 3\ 4\ 5][/math]. Logical expression will create a so called logical vector with values [math][0\ 0\ 0\ 1\ 1][/math], i.e. it is 1 if the according element is greater than 3.5 and 0 otherwise. Now, typing will generate a [math]2\times1[/math] subvector with values [math][4\ 5][/math]. You can also create some vectors or matrices with specific values changed: a command replace the last two values of the original vector . As a result, the vector becomes [math][1 \ 2\ 3\ 8\ 10][/math].

Slightly less irrelevant example

In some occasions you would like to modify a matrix of interest. Say, in some surveys “no answer” is coded as 999. Once you import the whole dataset in , you might want to replace these with, say, NaN. It can be done for the whole matrix of interest: .

Relevant example

To demonstrate these capabilities in a more relevant environment, let’s run a very simple example. Assume that we have [math]T\times1[/math] vector of returns and want to

  1. Compute number of positive, negative and zero returns
  2. Compute means of positive and negative returns

The algorithm for this is quite straightforward:

  1. Find out a length of vector , T
  2. Initiate three counter variables, , and vectors (since we don’t know how many negative and positive returns we will observe
  3. Check whether r(i) is greater, smaller or equal to 0 for i=1
  4. If , add 1 to Tplus, set ;
  5. Else if add 1 to Tminus, set ;
  6. Else add 1 to Tzero
  7. Repeat 3-6 for [math]i=2,\ldots,T[/math]
  8. Remove excessive zeros from and :
  1. Compute means of rminus and rplus. Number of positive, negative and zero returns are stored in

MATLAB translation:

T=size(r,1);
Tplus=0;Tminus=0;Tzero=0;
rplus=zeros(T,1);rminus=zeros(T,1);
for i=1:T
    if r(i)>0
        Tplus=Tplus+1;%increasing Tplus by one if return is positive
        rplus(Tplus)=r(i);%storing positive return in the proper subvector
    elseif r(i)<0
        Tminus=Tminus+1;%increasing Tminus by one if return is negative
        rminus(Tminus)=r(i);%storing negative return in the proper subvector
    else
        Tzero=Tzero+1;%increasing Tzero by one if return is neither positive nor negative
    end
end
rplus=rplus(1:Tplus);%removing excessive zeros from a subvector of positive returns
rminus=rminus(1:Tminus);%removing excessive zeros from a subvector of negative returns
meanplus=mean(rplus);%computing mean of positive returns
meanminus=sum(rminus)/Tminus;%computing mean of negative returns

Using MATLAB capabilities mentioned in this section, the algorithm can be reduced to:

  1. Construct a vector that has 1 for positive returns and 0 for negative returns
  2. Construct a vector that has 1 for negative returns and 0 for positive returns
  3. Assign to a sum of elements of . This is a number of positive returns
  4. Assign to a sum of elements of . This is a number of negative returns
  5. Compute which is
  6. Construct a vector of positive returns and compute its mean
  7. Construct a vector of negative returns and compute its mean

MATLAB implementation:

  T=size(r,1);
  indplus  = r>0;%constructing an indicator vector with 1s if r(i)>0, 0 otherwise
  indminus = r<0;%constructing an indicator vector with 1s if r(i)<0, 0 otherwise
  Tplus=sum(indplus);%computing a number of positive returns
  Tminus=sum(indminus);%computing a number of negative returns
  Tzero=T-Tplus-Tminus;%computing a number of zero returns
  rplus=r(indplus);%constructing a vector of positive returns
  rminus=r(indminus);%constructing a vector of negative returns
  meanplus=sum(rplus)/Tplus; %computing mean of positive returns
  meanminus=mean(rminus); %computing mean of negative returns

Or, a slightly shorter version of the same thing

  T=size(r,1);
  rplus  = r(r>0);%constructing a vector of positive returns
  rminus = r(r<0);%%constructing a vector of negative returns
  Tplus=size(rplus,1);%computing a number of positive returns
  Tminus=size(indminus,l);%computing a number of negative returns
  Tzero=T-Tplus-Tminus;%computing a number of zero returns
  meanplus=sum(rplus)/Tplus; %computing mean of positive returns
  meanminus=mean(rminus); %computing mean of negative returns

This way you write a code that is shorter, less prone to errors and easier to read (at least after some practice).

  1. There are two special numerical values in MATLAB. One is infinity , and another is not a number . A value of a variable becomes if the number is too big in absolute value ([math]\approx \pm 2e308[/math]). Also, infinity is generated once you have expressions like [math]x/0[/math], where [math]x\ne0[/math]. After that, infinity can only change a sign or become not a number. Not a number appears when there is an uncertainty of a kind of [math]0/0[/math], [math]\infty-\infty[/math] and such. Any algebraic operations with result