Python/Program Flow and Logicals - Revision history

Jb: /* Running Python programs */

2013-10-16T13:37:01Z

‎Running Python programs

Jb: /* More advanced example */

2013-10-16T13:24:01Z

‎More advanced example

Jb: /* Using Python Packages */

2013-10-16T13:16:48Z

‎Using Python Packages

Jb: /* Improvements on the above (avoiding loops) */

2013-10-16T13:14:16Z

‎Improvements on the above (avoiding loops)

Jb: /* for */

2013-10-16T13:12:45Z

‎for

Jb: /* Preliminaries */

2013-10-16T13:08:49Z

‎Preliminaries

Jb at 13:00, 16 October 2013

2013-10-16T13:00:52Z

Jb at 13:00, 16 October 2013

2013-10-16T13:00:07Z

Jb: /* Running Python programs */

2013-10-16T12:46:21Z

‎Running Python programs

Jb: /* More advanced example */

2013-10-15T13:17:53Z

‎More advanced example

@@ Line 374: / Line 374: @@
 For people who are familiar with MATLAB it may be surprising to discover there is no simple way of running a Python program from within Python. If you want to run Python code using the standard Python interpreter, your choices are either
 # Launch it from outside Python, e.g. save to a file <code>myscript.py</code> and at the command line enter <code>python myscript.py</code>
-# Convert the program to a function and use the [http://docs.python.org/3/tutorial/modules.html Python module functionality], e.g. save the function to a file <code>myfunctions.py</code> and use Python's <source enclose=none lang="python">import</source> to make the functions available.
+# Convert the program to a function and use the [http://docs.python.org/3/tutorial/modules.html Python module functionality], e.g. save the function in a file <code>myfunctions.py</code> and use Python's <source enclose=none lang="python">import</source> to make the function available.
-The first method can be demonstrated by creating a text file <code>ReturnAnalysis.py</code> containing the following program.
+The first method can be demonstrated by creating a text file <code>ReturnAnalysis.py</code> containing the following program (modified from the vectorised [[Python/Program_Flow_and_Logicals#More_advanced_example|More advanced example]] above).
 <source lang="python">import numpy
 n=500000000
@@ Line 395: / Line 395: @@
 In this example the array of values <source lang="python" enclose=none>r</source> is generated using the <source lang="python" enclose=none>rand</source> function, in a real scenario these values might be loaded from a file. To run this from the '''operating system command line''' we can enter <code>python ReturnAnalysis.py</code>. Note that this program outputs how long it takes to run, and on my desktop takes around 12.3s to complete (using the Anaconda Python distribution with the Accelerate package).
-Using the second method we can create a function, for example the following that undertakes the computation and returns the values of the two means.
+Using the second method we can create a function, the following example undertakes the computation and returns the values required.
 <source lang="python">def returnanalysis(r):
     import numpy
     rplus=r[r>0].copy()         # Create an array containing positive returns
     rminus=r[r<0].copy()        # Create an array containing negative returns
@@ Line 406: / Line 405: @@
     meanplus=numpy.mean(rplus)         # Compute mean of positive returns using numpy.mean
     meanminus=numpy.sum(rminus)/Tminus # Compute mean of negative returns using numpy.sum
-    return meanplus, meanminus
+    return meanplus, meanminus, Tplus, Tminus, Tzero
 </source>
-If this is saved to a file called <code>myfunctions.py</code>, after importing we can use the function from the Python prompt as follows.
+If this is saved to a file called <code>myfunctions.py</code>, we can import and use the function from the Python prompt as follows.
 <source lang="python">
 >>> import numpy
@@ Line 414: / Line 413: @@
 >>> r=numpy.random.rand(n)*10-5
 >>> import myfunctions
->>> mplus, mminus = myfunctions.returnanalysis(r)
+>>> mplus, mminus, Tp, Tm, Tz = myfunctions.returnanalysis(r)
 >>> mplus
-.7391992566830077
+.4999997176593398
 >>> mminus
--2.7456183947241075
+-2.4999816498237375
 </source>
 The Python prompt has various other limitations which mean it isn't ideal for interactive work. For example, it doesn't include commands like <source enclose=none>pwd</source>, <source enclose=none>cd</source>, <source enclose=none>pwd</source>, etc. An improved command line is included in [http://ipython.org/ IPython (Interactive Python)] which behaves far more like MATLAB. For example if we save the first Python code snippet from the [[Python/Program_Flow_and_Logicals#More_advanced_example|'''More advanced Example''' section (see above)]] to a file called <code>ReturnAnalysis1.py</code>, we can execute this program from within IPython using <source enclose=none lang="python">run</source>. For MATLAB-like behaviour, where a script can see the variables in the interactive namespace, we need to use <source enclose=none lang="python">run</source> with the <source enclose=none lang="python">-i</source> flag. The <source enclose=none lang="python">-t</source> flag is useful too, as it times how long the script takes to run.
@@ Line 453: / Line 452: @@
 Out[9]: -2.5000714107736286
 </source>
-Finally to compare with MATLAB, when running the vectorised MATLAB version (saved to a file called <code>ReturnAnalysis2.m</code>, the run time is as follows.
+Finally to compare with the vectorised MATLAB version (saved to a file called <code>ReturnAnalysis2.m</code>), the run time is as follows.
 <source>
 >> n=500000000;

@@ Line 310: / Line 310: @@
 === More advanced example ===
-We now look at the Python equivalent of the [[Program_Flow_and_Logicals#Relevant_example|Relevant example on the MATLAB page]], which assumes we have <math>T</math> returns in a vector <source enclose=none>r</source> and we want to:
+We now look at the Python equivalent of the code on the MATLAB page in the section entitled [[Program_Flow_and_Logicals#Relevant_example|Relevant example]], which assumed we have <math>T</math> returns in a vector <source enclose=none>r</source> and we want to:
 # Count the number of positive, negative and zero returns
@@ Line 317: / Line 317: @@
 # Compute the means of the positive and negative returns
-The naive algorithm using a loop in Python is as follows.
+A naive Python algorithm that uses a loop rather than vectorisation is as follows.
 # Find the length of the NumPy array holding  <source lang="python" enclose=none>r</source>, i.e. <source lang="python" enclose=none>T=numpy.size(r)</source>
 # Initiate three counter variables, <source lang="python" enclose=none>Tplus=0; Tzero=0; Tminus=0</source>

@@ Line 225: / Line 225: @@
 === Using Python Packages ===
-The functionality that allows us to operate on whole vectors and matrices isn't part of core Python, and requires us to use a Python package called [http://www.numpy.org NumPy], which adds other useful functionality including pseudo-random number generators. There are many other Python Packages, which are listed at [https://pypi.python.org/pypi the Python Package Index].
+The functionality that allows us to operate on whole vectors and matrices isn't part of core Python, and requires us to use a Python package called [http://www.numpy.org NumPy], which adds other useful functionality including pseudo-random number generators. There are many other Python Packages, these are listed at [https://pypi.python.org/pypi the Python Package Index].
 Before using a Python package, the package must be imported, e.g.
 <source lang="python">import numpy</source>
-Functions within a package are located within '''namespaces'''. Namespaces are useful because they allow package writers to choose functions names without worrying about whether that function name has been used elsewhere. For example, NumPy includes a function called <source enclose=none lang="python">rand</source>, which exists within a namespace called ''random''. And the ''random'' namespace is within the NumPy namespace (which is called ''numpy''). After importing NumPy we can use the rand function, but have to include the namespaces within the function call, e.g. to use <source enclose=none lang="python">rand</source> at the Python command prompt to generate 5 random numbers
+Functions within a package are located within '''namespaces'''. Namespaces are useful because they allow package writers to choose functions and variable names without worrying about whether that name has been used elsewhere. For example, NumPy includes a function called <source enclose=none lang="python">rand</source>, which exists within a namespace called ''random''. And the ''random'' namespace is within the NumPy namespace (which is called ''numpy''). After importing NumPy we can use the rand function, but have to include the namespaces within the function call, e.g. to use <source enclose=none lang="python">rand</source> at the Python command prompt (to generate 5 random numbers)
 <source lang="python">
 >>> import numpy

@@ Line 221: / Line 221: @@
 # Operators, functions and logical expressions can work not only on scalars, but also on vectors, matrices and, in general, on n-dimensional arrays
-# Subvectors/submatrices can be extracted using logical 0-1 arrays
+# Subvectors/submatrices can be extracted using logical arrays
 === Using Python Packages ===

@@ Line 93: / Line 93: @@
 == <source lang="python" enclose=none>for </source> ==
-We now look at the Python equivalents of the MATLAB code discussed in the [[Program_Flow_and_Logicals#for_..._end_loop|MATLAB page on Program Flow and Logicals]]. A description of the mathematics is available on the MATLAB page, for brevity it is not repeated here. In the case when the error terms in <source enclose=none lang="python">e</source> are known in advance, the Python version of the algorithm is:
+We now look at Python equivalents of the MATLAB <source enclose=none>for ... end</source> loop discussed in the [[Program_Flow_and_Logicals#for_..._end_loop|MATLAB page on Program Flow and Logicals]]. A description of the mathematics can be found on the MATLAB page, for brevity it is not repeated here. In the case when the error terms in <source enclose=none lang="python">e</source> are known in advance, the Python version of the algorithm is:
 # Find length of the list containing the error terms <source enclose=none lang="python">e</source>: <source lang="python" enclose=none>T=len(e)</source>
@@ Line 101: / Line 101: @@
 # Repeat line 4 for <math>i=2,...,(T-1)</math>
-A simple implementation in Python follows, and a description of how to run this code is given towards the end of this page.
+A simple implementation in Python follows (a description of how to run this code is given towards the end of this page).
 <source lang="python">T=len(e)
@@ Line 130: / Line 130: @@
 <class 'float'>
 </source>
-Controversially, the behaviour of integer division changed in Python version 3, compared to version 2, and it is worth mentioning this now. In Python 2
+This is a good point to mention that the behaviour of integer division changed in Python 3, compared to version 2. In Python 2
 <source lang="python">>>>type(1/2)
 <type 'int'>

@@ Line 64: / Line 64: @@
 and repeat for as many times as there are elements in <source lang="python" enclose=none>ListOrRangeOfValues</source>, each time assigning the next element in the list/range to <source lang="python" enclose=none>LoopVariable</source>. The code block associated with the loop is identified by a colon and indenting as described above.
-There are various ways of creating a list or range object in Python 3. The <source lang="python" enclose=none>range</source> function can be used to create sequences of integers with a defined start, stop and step value. The advantage of a <source lang="python" enclose=none>range</source> object over a <source lang="python" enclose=none>range</source> is that the sequence of values are not stored in memory with the <source lang="python" enclose=none>range</source>. <ref>In Python 3 the <source lang="python" enclose=none>range</source> function creates a range object. However the Python 2 <source lang="python" enclose=none>range</source> function creates a list, i.e. stores every integer value required in memory which is very inefficient if simply looping through a long sequence of integers in a <source lang="python" enclose=none>for</source> loop. Python 2 has <source lang="python" enclose=none>xrange</source> that behaves like the Python 3 <source lang="python" enclose=none>range</source>.</ref>. For example to create a range containing the four values 1, 4, 7 and 10, i.e. a sequence starting at 1 with steps of 3, we can use <source lang="python" enclose=none>range(1,11,3)</source>. Note that the stop value passed to the range function is not included, i.e. <source lang="python" enclose=none>range(1,10,3)</source> would produce only the three numbers 1, 4 & 7. We can verify this at the Python command prompt, i.e.
+There are various ways of creating a list or range object in Python 3. The <source lang="python" enclose=none>range</source> function can be used to create sequences of integers with a defined start, stop and step value. The advantage of a <source lang="python" enclose=none>range</source> object over a Python <source lang="python" enclose=none>list</source> is that every single integer value is not stored in memory with a <source lang="python" enclose=none>range</source>. <ref>In Python 3 the <source lang="python" enclose=none>range</source> function creates a range object. However the Python 2 <source lang="python" enclose=none>range</source> function creates a list, i.e. stores every integer value required in memory which is very inefficient if simply looping through a long sequence of integers in a <source lang="python" enclose=none>for</source> loop. Python 2 has <source lang="python" enclose=none>xrange</source> that behaves like the Python 3 <source lang="python" enclose=none>range</source>.</ref>. For example to create a range containing the four values 1, 4, 7 and 10, i.e. a sequence starting at 1 with steps of 3, we can use <source lang="python" enclose=none>range(1,11,3)</source>. Note that the stop value passed to the range function is not included, i.e. <source lang="python" enclose=none>range(1,10,3)</source> would produce only the three numbers 1, 4 & 7. We can verify this at the Python command prompt, i.e.
 <source lang="python">>>> range(1,11,3)
@@ Line 71: / Line 71: @@
 [1, 4, 7]
 </source>
-This might seems strange, but makes more sense when we realise the start and step values are optional, and the range function assumes default values of 1 for these if they are not given, i.e.  <source lang="python" enclose=none>range(N)</source> returns <source lang="python" enclose=none>N</source> values starting at 1, e.g.
+This might seems strange, but makes more sense when we realise the start and step values are optional, and the range function assumes default values of 0 and 1 respectively for these if they are not given, i.e.  <source lang="python" enclose=none>range(N)</source> returns <source lang="python" enclose=none>N</source> values starting at 0, e.g.
 <source lang="python">>>> range(5)
 [0, 1, 2, 3, 4]
@@ Line 78: / Line 78: @@
 </source>
-Python lists can be created from a sequence of values separated by commas within square brackets, e.g. <source lang="python" enclose=none>MyList = [1.0, "hello", 1]</source> creates a list called <source lang="python" enclose=none>MyList</source> containing 3 values, a floating point number <source lang="python" enclose=none>1.0</source>, the string <source lang="python" enclose=none>hello</source> and an integer <source lang="python" enclose=none>1</source>. This example demonstrates that Python lists are general purpose containers, and elements don't have to be of the same class. It is for this reason that lists and ranges are best avoided for numerical calculations unless they are relatively simple, as there are much more efficient containers for numbers, i.e. NumPy arrays, which will be introduced in due course.
+Python lists can be created from a sequence of values separated by commas within square brackets, e.g. <source lang="python" enclose=none>MyList = [1.0, "hello", 1]</source> creates a list called <source lang="python" enclose=none>MyList</source> containing 3 values, a floating point number <source lang="python" enclose=none>1.0</source>, the string <source lang="python" enclose=none>hello</source> and an integer <source lang="python" enclose=none>1</source>. This example demonstrates that Python lists are general purpose containers, and elements don't have to be of the same class. It is for this reason that lists are best avoided for numerical calculations unless they are relatively simple, as there are much more efficient containers for numbers, i.e. NumPy arrays, which will be introduced in due course.
 Conditional while loops are identified with the <source lang="python" enclose=none>while</source> keyword, so

← Older revision		Revision as of 13:00, 16 October 2013
Line 1:		Line 1:
−	The following assumes use of Python version 3 as opposed to Python 2~~, since no~~ more major releases are planned for ~~version~~ 2, version 3 is expected to be the future of Python. The two versions of Python, although similar, are not compatible in a forwards or backwards direction<ref>Although Python 2 and 3 are not totally compatible, Python 2.7 is close to Python 3. If you have to use Python 2, it is recommended using version 2.7, writing code as close to Python 3 as possible, and using tools like ''2to3'' to port to Python 3. Alternatively there is a Python compatibility packages called ''six''.</ref>, and some legacy code exists only as Python 2. Some differences between the two versions are discussed in the footnotes.	+	The following assumes use of Python version 3, as opposed to Python 2. No more major releases are planned for Python 2, and so version 3 is expected to be the future of Python. The two versions of Python, although similar, are not compatible in a forwards or backwards direction<ref>Although Python 2 and 3 are not totally compatible, Python 2.7 is close to Python 3. If you have to use Python 2, it is recommended using version 2.7, writing code as close to Python 3 as possible, and using tools like ''2to3'' to port to Python 3. Alternatively there is a Python compatibility packages called ''six''.</ref>, and some legacy code exists only as Python 2. Some differences between the two versions are discussed in the footnotes.

	= Preliminaries =		= Preliminaries =

@@ Line 372: / Line 372: @@
 == Running Python programs ==
-For people who are familiar with MATLAB it may be surprising to discover there is no simple way of running a Python program from within Python. If you want to run Python code using standard Python, your choices are either
+For people who are familiar with MATLAB it may be surprising to discover there is no simple way of running a Python program from within Python. If you want to run Python code using the standard Python interpreter, your choices are either
 # Launch it from outside Python, e.g. save to a file <code>myscript.py</code> and at the command line enter <code>python myscript.py</code>
-# Convert the program to a function and use the [http://docs.python.org/3/tutorial/modules.html Python module functionality], e.g. save to a file <code>myfunctions.py</code> and use Python's <source enclose=none lang="python">import</source> to make the functions available.
+# Convert the program to a function and use the [http://docs.python.org/3/tutorial/modules.html Python module functionality], e.g. save the function to a file <code>myfunctions.py</code> and use Python's <source enclose=none lang="python">import</source> to make the functions available.
-The Python prompt has other limitations too, for example, it doesn't include commands like <source enclose=none>pwd</source>, <source enclose=none>cd</source>, <source enclose=none>pwd</source>, etc. To avoid these limitations we can use [http://ipython.org/ IPython (Interactive Python)], which provides an command line interface which behaves far more like MATLAB. For example, if we save the first code snippet in [[Python/Program_Flow_and_Logicals#More_advanced_example|'''More advanced Example''' (see above)]] to a file called <code>ReturnAnalysis1.py</code> then from within IPython we can create a vector of random values to be analysed, and use run to execute the program, noting that for MATLAB like behaviour where the script can see the current interactive namespace requies the -i flag. The -t flag is useful too, as it times how long the script takes to run.
+The first method can be demonstrated by creating a text file <code>ReturnAnalysis.py</code> containing the following program.
 <source lang="python">
-In [1]: n=100000000
+>>> import numpy
 In [2]: from numpy.random import rand
 In [3]: r=rand(n)*10-5
-In [4]: r=rand(n)*10-5
+In [4]: run -i -t ReturnAnalysis1
 IPython CPU timings (estimated):
-   User   :     150.84 s.
+   User   :     978.66 s.
-   System :       0.44 s.
+   System :       0.00 s.
-Wall time:     151.72 s.
+Wall time:     978.71 s.
-In [6]: meanplus
+In [5]: meanplus
-Out[6]: 2.4998243901335169
+Out[5]: 2.5001402997170192
-In [7]: meanminus
+In [6]: meanminus
-Out[7]: -2.4999279003309622
+Out[6]: -2.5000714107736286
 </source>
-In comparison, when running the vectorised version of the code (saved in <code>ReturnAnalysis2.py</code>).....
+The above example used the unvectorised version, and to demonstrate the importance of vectorisation in getting good performance we compare with the vectorised version (saved in <code>ReturnAnalysis2.py</code>).
 <source lang="python">
-In [8]: run -i -t ReturnAnalysis2
+In [7]: run -i -t ReturnAnalysis2
 IPython CPU timings (estimated):
-   User   :       1.84 s.
+   User   :      12.18 s.
-   System :       0.34 s.
+   System :       0.00 s.
-Wall time:       2.19 s.
+Wall time:      12.18 s.
-In [9]: meanplus
+In [8]: meanplus
-Out[9]: 2.4998243901335169
+Out[8]: 2.5001402997170192
-In [10]: meanminus
+In [9]: meanminus
-Out[10]: -2.4999279003309622
+Out[9]: -2.5000714107736286
 </source>
 =Footnotes=
   <references />

@@ Line 362: / Line 362: @@
 <source lang="python">import numpy
 rplus=r[r>0].copy()         # Create an array containing positive returns
-rplus=r[r<0].copy()         # Create an array containing negative returns
+rminus=r[r<0].copy()         # Create an array containing negative returns
 Tplus=len(rplus)            # Count how many positive returns there are
 Tminus=len(rminus)          # Count how many negative returns there are