|
|
| Line 1: |
Line 1: |
| − | | + | Coming soon |
| − | | |
| − | =1.0
| |
| − | | |
| − | = Intro =
| |
| − | | |
| − | Sometimes you would like to store data which have more than two dimensions. For example, interest rates for different maturities for different countries over time. In this case there are three natural indices: country index, maturity index, and time. Another example could be regression results of Monte-Carlo simulations for different sample sizes and different model specifications. In MATLAB you can address this issue using three different approaches:
| |
| − | | |
| − | # Multidimensional arrays
| |
| − | # Structures/Structure Arrays
| |
| − | # Cells/Cell arrays
| |
| − | | |
| − | In the next sections we will briefly review these data objects and discuss the most straightforward way to handle them.
| |
| − | | |
| − | = Multidimensional Arrays =
| |
| − | | |
| − | == Elementwise Operations ==
| |
| − | | |
| − | Multidimensional arrays are just a generalization of a matrix. Almost all MATLAB functions can generate and operate with multivariate arrays, for example, to generate a three-dimensional array of normal random numbers with dimensions <math>T,p,k</math>, you have to type:
| |
| − | | |
| − | <source> A=randn(T,p,k);</source>
| |
| − | MATLAB prints all multidimensional arrays as a sequence of matrices. Say, <math>2\times 2\times 2</math> array is printed as
| |
| − | | |
| − | <source>>> A=zeros(2,2,2)
| |
| − | A(:,:,1) =
| |
| − | 0 0
| |
| − | 0 0
| |
| − | A(:,:,2) =
| |
| − | 0 0
| |
| − | 0 0</source>
| |
| − | Taking an average over the second dimension and taking a maximum over the third dimension looks like:
| |
| − | | |
| − | <source> meanA=mean(A,2);
| |
| − | maxA=max(A,[],3);</source>
| |
| − | Please note the syntax of <source enclose="none">max</source> function. For this function the second argument stands for a matrix/scalar that you are comparing with your matrix A. If you don’t want to experience any surprises, please check with the function help first. Multidimensional arrays support element-by-element operations (operations with dots), as well as operations with scalars. MATLAB will correctly compute
| |
| − | | |
| − | <source> B=A.^2;
| |
| − | C=A.^B+B;
| |
| − | D=C./A;</source>
| |
| − | In the first line, 3-D array B consists of squared values of 3-D array A. In the second line elements of 3-D array C consist of elements of A in power of elements B plus elements from B. 3-D array D consists of element-wise division of C by A. However, the following commands
| |
| − | | |
| − | <source>A*A;
| |
| − | A^2;</source>
| |
| − | will throw an error, since <source enclose="none">^,*</source> are matrix operations and they are not defined on multidimensional arrays.
| |
| − | | |
| − | == Collapsing Singleton Dimensions ==
| |
| − | | |
| − | All operations of extracting sub-arrays from these arrays work as usual (see link). However, there are some particularities that I would like to mention. Assume that we have a <math>3\times 3\times 3</math> array A. MATLAB recognizes the following operation :
| |
| − | | |
| − | <source> A(:,:,1)*A(:,:,2);</source>
| |
| − | while
| |
| − | | |
| − | <source> A(1,:,:)*A(1,:,:);</source>
| |
| − | generates an error. We get the same result if we try to plot our data:
| |
| − | | |
| − | <source> plot(A(:,:,1))</source>
| |
| − | plots a figure, while
| |
| − | | |
| − | <source> plot(A(1,:,:))</source>
| |
| − | generates an error. It happens because from MATLAB point of view <source enclose="none">A(1,:,:)</source> is not a matrix, it is still a 3-D array. There are two ways to deal with this issue:
| |
| − | | |
| − | <ol>
| |
| − | <li><p>Make MATLAB realize that we would like to see a matrix instead of a 3-D array with a singleton in the first dimension</p>
| |
| − | <source> B(:,:)=A(1,:,:);
| |
| − | plot(B)
| |
| − | </source></li>
| |
| − | <li><p>Collapse all singleton dimensions using a special command <source enclose="none">squeeze</source></p>
| |
| − | <source> B=squeeze(A(1,:,:));
| |
| − | plot(B)
| |
| − | </source></li></ol>
| |
| − | | |
| − | == Expanding to Higher Dimensions ==
| |
| − | | |
| − | By default, element-wise operations and operations with a scalar are defined on any multi-dimensional array. An additional set of operations is defined on conformable vectors and matrices. The natural question to ask is “What to do if you want to subtract a vector from a matrix, or a matrix from N-D array?” In MATLAB there is a special command that can create a higher-dimensional array from a lower-dimensional one via replication:
| |
| − | | |
| − | <source> A=repmat([1,2,3]',2,3)
| |
| − | A =
| |
| − | 1 1 1
| |
| − | 2 2 2
| |
| − | 3 3 3
| |
| − | 1 1 1
| |
| − | 2 2 2
| |
| − | 3 3 3</source>
| |
| − | replicates a column-vector <math>[1\ 2\ 3]'</math> twice along each row and 3 times along each column. For 3-D vector replication we have to use slightly different syntax:
| |
| − | | |
| − | <source> A=repmat([1,2,3]',[1 2 3])
| |
| − | A(:,:,1) =
| |
| − | 1 1
| |
| − | 2 2
| |
| − | 3 3
| |
| − | A(:,:,2) =
| |
| − | 1 1
| |
| − | 2 2
| |
| − | 3 3
| |
| − | A(:,:,3) =
| |
| − | 1 1
| |
| − | 2 2
| |
| − | 3 3</source>
| |
| − | This command does not change the length of the vector, but replicates it twice along the second dimension and 3 times along the third dimension.
| |
| − | | |
| − | Lets consider two real-life examples:
| |
| − | | |
| − | # Assume that we have a cross-section of returns and we would like to subtract a series of risk-free returns from each of the series
| |
| − | # Assume that we need to subtract a mean along the third dimension from a 3-D array.
| |
| − | | |
| − | These two examples can be implemented either by using loops, or by using repmat.
| |
| − | | |
| − | First example, loop:
| |
| − | | |
| − | <source>% rf - a series of risk-free returns
| |
| − | % R - a cross-section of returns, it is assumed that size(R,1)=size(rf,1)
| |
| − | R=zeros(size(R));
| |
| − | for i=1:size(R,2)
| |
| − | Rex(:,i)=R(:,i)-rf;
| |
| − | end</source>
| |
| − | First example, <source enclose="none">repmat</source>:
| |
| − | | |
| − | <source>% rf - a series of risk-free returns
| |
| − | % R - a cross-section of returns, it is assumed that size(R,1)=size(rf,1)
| |
| − | Rf=repmat(rf,1,size(R,2));
| |
| − | Rex=R-Rf;</source>
| |
| − | | |
| − | Lets consider For the second example, to implement loop, we have to:
| |
| − | | |
| − | # Compute a mean of 3-D array
| |
| − | # Initiate a 3-D array of demeaned values
| |
| − | # Subtract piece-by-piece a matrix of means from each layer of the 3-D array
| |
| − | # check whether the mean is indeed subtracted
| |
| − | | |
| − | <source> R=rand(3,4,10); % constructing a 3-D array of U(0,1) random variables
| |
| − | Rmean3=mean(R,3); %constructing a matrix of means
| |
| − | Rdemean=zeros(size(R)); %initializing a 3-D array of zeros
| |
| − | for i=1:size(R,3)
| |
| − | Rdemean(:,:,i)=R(:,:,i)-Rmean3;
| |
| − | end
| |
| − | disp(mean(Rdemean,3)) %has to be a matrix of 0s </source>
| |
| − | # Compute a mean of 3-D array
| |
| − | # Construct a 3-D array of means
| |
| − | # subtract one from another
| |
| − | # check whether the mean is indeed subtracted
| |
| − | | |
| − | <source> R=rand(3,4,10); % constructing a 3-D array of U(0,1) random variables
| |
| − | Rmean3=mean(R,3); %constructing a matrix of means
| |
| − | Rmean3expand=repmat(Rmean3,[1 1 size(R,3)]); %Please note, for more than 2-dimensional arrays, repmat accepts a vector of replications instead of a variable number of inputs.
| |
| − | Rdemean=R-Rmean3expand;
| |
| − | disp(mean(Rdemean,3)) %has to be a matrix of 0s</source>
| |
| − | Obviously, there are many ways of handling these problems.
| |
| − | | |
| − | Each of these approaches has its advantages and disadvantages. For example: the easiest way to handle data in a uniform way is to use multidimensional arrays. Multivariate arrays, however, require that the data are of the same type (logical, numerical, character).
| |
| − | | |
| − | = Structures and Cell Arrays =
| |
| − | | |
| − | Sometimes it is natural to keep data of different type under the same roof, i.e. using the same variable name. Multidimensional arrays do not allow that. Therefore structures/arrays of structures or cell arrays have to be used. These two objects are very similar and, in fact, interchangeable. For some applications, however, cell arrays are more suitable while, for other, structures are preferable. For obvious reasons, arithmetic operations are not defined.
| |
| − | | |
| − | == Structures ==
| |
| − | | |
| − | A structure variable is a variable that has “fields”. The variable name is separated from field name by a dot. For example, if you want to keep all OLS regression results, i.e. beta coefficients, covariance matrix, t-stats and vector of residuals in one place.
| |
| − | | |
| − | <source>%Assuming X and y are already defined, the whole filling-up process of the structure would look as follows:
| |
| − | OLS.beta=X\y;%short and more efficient way to write inv(X'*X)*X'*y
| |
| − | OLS.resid=y-X*OLS.beta;
| |
| − | [T,k]=size(X);
| |
| − | OLS.sigma2=sum(OLS.resid.^2)/(T-k);%computing residual variance
| |
| − | OLS.cov=OLS.sigma2*inv(X'*X);
| |
| − | OLS.tstat=OLS.beta./sqrt(diag(OLS.cov));
| |
| − | OLS.name='Regression one';</source>
| |
| − | Any assignment of this variable to another one creates a copy of the structure with all fields and values defined.
| |
| − | | |
| − | <source>>> OLSnew=OLS;
| |
| − | >> OLSnew.name
| |
| − | ans =
| |
| − | Regression one</source>
| |
| − | A field can be a structure by itself. For example,
| |
| − | | |
| − | <source>>> OLSnew.old=OLS;
| |
| − | >> OLSnew.old.name
| |
| − | ans =
| |
| − | Regression one</source>
| |
| − | Moreover, all fields will be carried in and out of a function. In this way you should not care about the order of inputs and outputs for a function with many inputs/outputs. There are several useful functions that you can use with structures:
| |
| − | | |
| − | <source> >> isfield(OLS,'name')
| |
| − | ans =
| |
| − | 1
| |
| − | >> isfield(OLS,'pvalues')
| |
| − | ans =
| |
| − | 0</source>
| |
| − | This command checks the existence of a particular field within the structure. You can also get a list of all fields within a current structure using <source enclose="none">fieldnames</source>
| |
| − | | |
| − | <source>>> fnames=fieldnames(OLSnew)'
| |
| − | fnames =
| |
| − | 'beta' 'resid' 'sigma2' 'cov' 'tstat' 'name' 'old'</source>
| |
| − | This command creates a cell array (see Section [cell]) <source enclose="none">fnames</source> with field names of OLSnew structure. You can also access the values of these field names using indirect referencing. In particular,
| |
| − | | |
| − | <source>>> OLSnew.(fnames{6}) %please note curly brackets!
| |
| − | ans =
| |
| − | Regression one</source>
| |
| − | Since <source enclose="none">fnames{6}=’name’</source> and, as a result, we indirectly refer to the field <source enclose="none">name</source> of the structure <source enclose="none">OLSnew</source>. In this way we can compare two structures field-by-field. Other useful commands for working with structure field names are:
| |
| − | | |
| − | * <source enclose="none">intersect(A,B)</source>, in terms of set operations, it corresponds to <math>A\cap B</math>
| |
| − | * <source enclose="none">union(A,B)</source>, corresponds to <math>A\cup B</math>
| |
| − | * <source enclose="none">setdiff(A,B)</source>, corresponds to <math>A/B</math> (please note, <math>A/B \ne B/A</math>)
| |
| − | * <source enclose="none">setxor(A,B)</source>, corresponds to <math>(A/B)\cup (B/A)</math>
| |
| − | | |
| − | <source> f1={'first','second'}; %Creating a cell array, see next section for details
| |
| − | f2={'second','third'}; %Creating a cell array, see next section for details
| |
| − | >> disp(intersect(f1,f2))
| |
| − | 'second'
| |
| − | >> disp(union(f1,f2))
| |
| − | 'first' 'second' 'third'
| |
| − | >> disp(setdiff(f1,f2))
| |
| − | 'first'
| |
| − | >> disp(setdiff(f2,f1))
| |
| − | 'third'
| |
| − | >> disp(setxor(f1,f2))
| |
| − | 'first' 'third'</source>
| |
| − | You can collect structures in arrays. Please note, all member structures of an array have to have the same set of fields. Creating a field for one element of the array, you are automatically creating empty fields of the same name for all members of your array. However, despite the fact that the field names are the same, there are no restriction on the field types. Example:
| |
| − | | |
| − | <source>>> s(2).f1=1;
| |
| − | >> disp(s(1))
| |
| − | f1:[];
| |
| − | >> s(2).f2='Sally';
| |
| − | >> disp(s(1))
| |
| − | f1:[]
| |
| − | f2:[]
| |
| − | >> s(1).f1=3;
| |
| − | >> s(1).f2=[83 116 117 100 101 110 116 32];</source>
| |
| − | You can print all values of a structure field in an array using “:” operator:
| |
| − | | |
| − | <source>>> s(:).f1
| |
| − | ans =
| |
| − | 3
| |
| − | ans =
| |
| − | 1
| |
| − | >> s(:).f1
| |
| − | ans =
| |
| − | 83 116 117 100 101 110 116 32
| |
| − | ans =
| |
| − | Sally</source>
| |
| − | and try to construct an array out of them using concatenation operator [ ]:
| |
| − | | |
| − | <source>[s(:).f1]
| |
| − | ans =
| |
| − | 3 1</source>
| |
| − | though results may vary (try to figure it out by yourself):
| |
| − | | |
| − | <source> [s(:).f2]
| |
| − | ans =
| |
| − | Student Sally</source>
| |
| − | An additional hint:
| |
| − | | |
| − | <source> [s(:).f2]+0
| |
| − | ans =
| |
| − | 83 116 117 100 101 110 116 32 83 97 108 108 121</source>
| |
| − | Some MATLAB functions require structures as inputs (See non-linear optimization for details). Other MATLAB functions return structures when they are asked. For example, <source enclose="none">data=load(fname)</source> creates a structure <source enclose="none">data</source> with fieldnames that coincide with variables of the workspace saved in the data file <source enclose="none">fname</source>. Thus, for comparing variables from different MATLAB data files, we first need to create two structures:
| |
| − | | |
| − | <source> data1=load(fname1);
| |
| − | data2=load(fname2);</source>
| |
| − | You can automatically compare variables with the same names from these datasets, using <source enclose="none">intersect</source> and <source enclose="none">setdiff</source> commands:
| |
| − | | |
| − | <source>data1=load('data1');%data1.mat has to exist
| |
| − | data2=load('data2');%data2.mat has to exist
| |
| − | if isequal(data1,data2)
| |
| − | disp('These structures are the same')
| |
| − | else
| |
| − | fname1=fieldnames(data1);
| |
| − | fname2=fieldnames(data2);
| |
| − | fnamejoint=intersect(fname1,fname2);
| |
| − | for i=1:length(fnamejoint)
| |
| − | if isequal(data1.(fnamejoint{i}),data2.(fnamejoint{i}))
| |
| − | disp([fnamejoint{i} ' fields are the same'])
| |
| − | else
| |
| − | disp([fnamejoint{i} ' fields are different'])
| |
| − | end
| |
| − | end
| |
| − | disp('Unique for data1:')
| |
| − | disp(setdiff(fname1,fname2)')
| |
| − | disp('Unique for data2:')
| |
| − | disp(setdiff(fname2,fname1)')
| |
| − | end</source>
| |
| − | ''Please note: the script above detects only limited number of “equality” cases. Due to a finite number of digits reserved for storing a number in memory (32 bits for double, 16 bit for single), the final result may depend on the computing path. For example, in theory, <math>\ln(\exp(x))\equiv x</math>. In reality this is not exactly the case. If we generate a <math>1000\times 1</math> vector of random numbers <source enclose="none">a</source> and compare it with <source enclose="none">log(exp(a))</source>, we obtain slightly different results. Thus, a standard comparison using <source enclose="none">isequal</source> does not work: ''
| |
| − | | |
| − | <source>>> a=rand(1000,1);
| |
| − | >> isequal(a,log(exp(a)))
| |
| − | ans =
| |
| − | 0
| |
| − | >> disp(mean(a-log(exp(a))))
| |
| − | 9.9335e-17</source>
| |
| − | As a result, if you want to compare two numbers/vectors/arrays, you have to specify your tolerance level and compare not vectors or matrices, but rather a measure of a distance between the two:
| |
| − | | |
| − | <source>%function that compares two vectors with predefined tolerance level tol:
| |
| − | function out=myisequal(a,b,tol)
| |
| − | if mean(abs(a-b))>tol %checks whether an average absolute difference between two vectors is larger than the tolerance level.
| |
| − | out=0; %if "Yes", then these two vectors are different given the current tolerance level
| |
| − | else
| |
| − | out=1; %if "No", then these two vectors are equal given the current tolerance level
| |
| − | end</source>
| |
| − | Another command that generates a (possibly empty) array of structures is <source enclose="none">dir(file_mask)</source>. To obtain a list of all mat files in the current directory, you have to type:
| |
| − | | |
| − | <source>>> matfiles=dir('*.mat')
| |
| − | matfiles =
| |
| − | 1004x1 struct array with fields:
| |
| − | name
| |
| − | date
| |
| − | bytes
| |
| − | isdir
| |
| − | datenum</source>
| |
| − | Then you can load all these files to memory (if you have enough of RAM). If all mat files in the directory have the same variables, you can load all of them by constructing a standard vector of structures. However, if the number of variables varies from file to file, using cell arrays would be a better idea (for details, see next section).
| |
| − | | |
| − | == Cell Arrays[cell] ==
| |
| − | | |
| − | The least structured variable type in MATLAB is cell array. You can store a sequence of any data objects in it. You may want to think about cell arrays as a structure where each field is coded by one number (if it is one-dimensional) or some set of numbers, if it is multidimensional. Since everything is coded by numbers, it is easier to use cell arrays inside loops. However, since sequence of numbers is not as informative as structure field names, it is harder to follow. Cell arrays are defined using curly brackets:
| |
| − | | |
| − | <source>>> c={'test',rand(10),1:10 }
| |
| − | c =
| |
| − | 'test' [10x10 double] [1x10 double]</source>
| |
| − | The same result can be achieved in three steps:
| |
| − | | |
| − | <source>>> c{1}='test';
| |
| − | >> c{2}=rand(10);
| |
| − | >> c{3}=1:10;
| |
| − | >> c
| |
| − | c =
| |
| − | 'test' [10x10 double] [1x10 double]</source>
| |
| − | To refer to the second element of <source enclose="none">c(3)</source>, the following syntax has to be employed:
| |
| − | | |
| − | <source>>> c{3}(2)
| |
| − | ans =
| |
| − | 2</source>
| |
| − | '''Important:'''
| |
| − | | |
| − | There is a very important difference between <source enclose="none">c(3)</source> and <source enclose="none">c{3}</source>. The former refers to <math>1\times 1</math> cell array, while the second refers to its ''value''.
| |
| − | | |
| − | <source>>> mean(c(3));
| |
| − | Undefined function 'sum' for input arguments
| |
| − | of type 'cell'.
| |
| − | Error in mean (line 28)
| |
| − | y = sum(x)/size(x,dim);</source>
| |
| − | Since math functions are not defined on cell variables, while
| |
| − | | |
| − | <source>>> mean(c{3})
| |
| − | ans =
| |
| − | 5.5000</source>
| |
| − | works as expected. By the same logic, the assignment below
| |
| − | | |
| − | <source> c(4:5)={'ostrich', randn(50)};</source>
| |
| − | works by extending cell array <source enclose="none">c</source> to <math>5\times 1</math>, while
| |
| − | | |
| − | <source> c{4:5}={'ostrich', randn(50)};</source>
| |
| − | generates an error. Also an error is generated by
| |
| − | | |
| − | <source> c(6)=randn(4);</source>
| |
| − | as we are trying to assign a double array <source enclose="none">randn(4)</source> to a cell element <source enclose="none">c(6)</source>. The correct assignment is
| |
| − | | |
| − | <source> c(6)={randn(4)};</source>
| |
| − | Curly brackets convert double array to a cell element, and you can assign a cell element to another cell element.
| |
| − | | |
| − | Now, equipped with this information, we can load all mat files in current directory in MATLAB using cell arrays:
| |
| − | | |
| − | <source> fname=dir('*.mat');
| |
| − | for i=1:length(fname)
| |
| − | data{i}=load(fname(1).name);
| |
| − | end</source>
| |
| − | The same goal can be achieved using structure:
| |
| − | | |
| − | <source> fname=dir('*.mat');
| |
| − | for i=1:length(fname)
| |
| − | curr_field=['f' fname(i).name(1:end-4)];%Filename is selected, .mat is dropped. The leading 'f' stands for a file. Please note that the code might not work for some file names
| |
| − | data.(curr_field)=load(fname(1).name);
| |
| − | end</source>
| |
| − | It is possible to convert structures to cell arrays and vice versa. Also, it is possible to apply a function to each cell of a cell array
| |
| − | | |
| − | <ul>
| |
| − | <li><p><source enclose="none">c = struct2cell(s)</source> converts a structure <source enclose="none">s</source> to a cell array <source enclose="none">c</source>, fields are converted to cells using the same order as in structure <source enclose="none">s</source></p></li>
| |
| − | <li><p><source enclose="none">s = cell2struct(c,fields)</source> converts a cell array <source enclose="none">c</source> to a structure array <source enclose="none">s</source> with fieldnames defined in <source enclose="none">fields</source></p>
| |
| − | <source> >> s=struct('f1',rand(10),'f2','MSFT') %an alternative way to create a structure
| |
| − | s =
| |
| − | f1: [10x10 double]
| |
| − | f2: 'MSFT'
| |
| − | >> fldnm=fieldnames(s)' %recording fieldnames of structure s
| |
| − | fldnm =
| |
| − | 'f1' 'f2'
| |
| − | >> c=struct2cell(s)' %converting structure s to cell array c
| |
| − | c =
| |
| − | [10x10 double] 'MSFT'
| |
| − | >> s2=cell2struct(c',fldnm) %converting cell array c to structure s2 using fldnm which was recorded earlier. For some reason MATLAB prefers column-vector c
| |
| − | s2 =
| |
| − | f1: [10x10 double]
| |
| − | f2: 'MSFT'</source></li>
| |
| − | <li><p><source enclose="none">cellfun(@function_name,cell1,cell2,...,celln,options)</source> evaluates a function<br />
| |
| − | <source enclose="none">function_name</source> picking every first element of cell arrays <source enclose="none">cell1,cell2,...,celln</source>, every second element of cell arrays <source enclose="none">cell1,cell2,...,celln</source>, etc.</p>
| |
| − | <source>>> c1={rand(10,10,10),rand(10,10,10),rand(10,10,10)};
| |
| − | >> c2={1,2,3};
| |
| − | >> average=cellfun(@mean,c1,c2, 'UniformOutput',0)
| |
| − | average =
| |
| − | [1x10x10 double] [10x1x10 double] [10x10 double]</source>
| |
| − | <p>Without <source enclose="none">’UniformOutput’,0</source> MATLAB tries to construct a vector with three elements and fails for obvious reasons.</p></li></ul>
| |
