TM5 Notebooks: towards a TM5 learning machine

Maarten Krol

TM meeting, november 2015

What will I discuss today?

  • New!! the Ipython notebook version
    • Markdown cells
    • Programmnig cells
  • Interactive Documents (.ipynb)
    • Examples: MOGUNTIA
    • Set-up as a notebook server
  • Use as "TM5 learning machine"

IPython

Basically Interactive Python

Ideal for quick development, plotting etc.

Very much like IDL

The IPython Notebook

Yes, I made these slides in an IPython Notebook

An important development that integrates

  • writing
  • Latex formulas, figures, HTML, ...
  • programming
  • interaction between scientists that work on different platforms

IPython can also run remote and aproached from local machine


Examples of stuff that can be included in a Notebook


Equations in a document:

\begin{equation}
\tau_a^*(\lambda) = \beta\lambda^{-\alpha}
\end{equation}

renders (Shift Enter) like

\begin{equation} \tau_a^*(\lambda) = \beta\lambda^{-\alpha} \end{equation}

Figures in a document:

   <figure>
    <img src="rs8.jpg">
    </figure>

Figure: solar irradiance (non-calibrated data counts) 
as a function of time for three wavelengths (368, 501, and 780 nm)

renders like:

Figure: solar irradiance (non-calibrated data counts) as a function of time for three wavelengths (368, 501, and 780 nm)

Tables in a document

<table>
  <caption>Table 1: Test Table.</caption>
<tr>
<th>Wavelength λ (nm)</th>
<th>Ozone absorption optical depth τ$_{O3}$(-)</th>
</tr>
<tr>
<td>368.5</td>
<td>0.00008</td>
</tr>
</table>

renders like

Table 1: Test Table.
Wavelength λ (nm) Ozone absorption optical depth τ$_{O3}$(-)
368.5 0.00008

Or, using Mark-down

First Header | Second Header
------------ | -------------
Content from cell 1 | Content from cell 2
Content in the first column | Content in the second column

Renders:

First Header Second Header
Content from cell 1 Content from cell 2
Content in the first column Content in the second column

Movies

from IPython.display import YouTubeVideo

YouTubeVideo("ErpSgWw-pAM")

renders:

In [1]:
from IPython.display import YouTubeVideo
YouTubeVideo("mZL_y9B6UKc")
Out[1]:

Calculations & plotting in a document

%matplotlib inline
from pylab import *
imshow(random(1000,1000))

renders:

In [13]:
    %matplotlib inline
    from pylab import *
    imshow(rand(1000,1000))
Out[13]:
<matplotlib.image.AxesImage at 0x119be0240>

Interactive widgets & Sliders:

def plot_function(a = 1.0, b = 2.0, c = 0.0):
    f,ax = subplots()
    x = linspace(-10,10)
    ax.plot(x,a*x*x + b*x + c,'r-')
    ax.set_ylabel('y')
    ax.set_xlabel('x')

interact(plot_function,a = (-10.,10.), b = (-30.,30.), c = (-10.,10.0))
In [1]:
%matplotlib inline
from pylab import *
from ipywidgets import *

def plot_function(a = 1.0, b = 2.0, c = 0.0):
    f,ax = subplots()
    x = linspace(-10,10)
    ax.plot(x,a*x*x + b*x + c,'r-')
    ax.set_ylabel('y')
    ax.set_xlabel('x')
    
interact(plot_function,a = (-10.,10.), b = (-30.,30.), c = (-10.,10.0))
    

Integrating these methods allows complete Ipython Notebooks

I will show some DEMOs

* Moguntia

* How to run remotely

* Run TM5 remotely in a local browser

How to run locally/remotely

  1. Local: "ipython notebook" opens a browser that lets you create a notebook
  2. Remotely, method 1:

    • Run on a workstation: "ipython notebook --no-browser --port=8888"

      this serves the notebook on "port" 8888

    • Locally, activate a "port" forward, e.g. ssh -N -f -L localhost:8888:127.0.0.1:8888 krol@maunaloa.wur.nl

      This forwards the port to local browser

    • Open in a local browser "localhost:8888"
  3. Remotely, method 2:
    • Install a ipython profile to serve notebooks (nbserver) (see this link)
    • On the remote machine, install all you need, and run "ipython notebook --profile=nbserver"
    • Locally: Go to the webaddress of the machine and you can run, e.g., https://54.187.227.253:8888/tree

Summary

1. Ipython is very powerfull for plotting/calculating

2. The Notebook environment offers integration with web-browsers & mark-up

3. This offers nice TM5 possibilities (educational)

4. I propose to set up an AWS machine with TM5 installed

5. And provide some Notebooks to guide new users