Refreshing computer-skills

Post on 12-Feb-2017

50 views 0 download

Transcript of Refreshing computer-skills

8 June 2016 Roberto Innocente - inno@sissa.it 1

Refreshing computer skills

1) How to easily write simple Web pages ?

2) How to easily add math to Web pages ?

3) Notebooks : mixing text, math and results of computations

4) News on virtualization

5) News on Windows Server 2016

6) News on Microsoft Windows 10

7) Scaling out

8 June 2016 Roberto Innocente - inno@sissa.it 2

1How to easily write simple Web pages ?

Using the markdown language,or a markdown WYSIWYG editor like

remarkable (linux), markdownpad (windows), typora (windows), markdown_edit

8 June 2016 Roberto Innocente - inno@sissa.it 3

Markdown

● HTML (Hyper Text Mark Up Language) is the official language of the Web (T.Berners-Lee 1991)

● In the last decade many thought that HTML is too complicate for writing simple pages. So in opposition to it, a very simple language named markdown was developed for writing Web pages by humans.

● This was initially implemented as a perl program markdown.pl that is a markdown→html translator

● Then it was implemented in python as markdown_py module● Now many WYSIWYG editors are available for all the platforms. These editors

can usually also apply a set of CSS (Style Sheets) to the final HTML document.

8 June 2016 Roberto Innocente - inno@sissa.it 4

markdown======== is a text-to-html conversion tool developed by john gruber in perl.[original project](http://daringfireball.net/projects/markdown/)[original text](http://daringfireball.net/projects/markdown/index.text)(there is an implementation in php too)####python-markdown is a python library that implements gruber's markdown.[python markdown library page](https://pypi.python.org/pypi/Markdown)There are some differences between the python library implementationand gruber's tool.[differences](https://pythonhosted.org/Markdown/index.html#differences)####python command line script[python command line (not on the anaconda python) :](https://pythonhosted.org/Markdown/cli.html)

$ /usr/bin/python -m markdown -x markdown.extensions.tables [options] [args]

$ /usr/bin/python -m markdown -x markdown.extensions.tables markdown.txt >markdown.html

python names the script markdown_py to avoid confusion with the perl one.

$ /usr/bin/python -m markdown input_file.txt

Piping input and output (on STDIN and STDOUT) is fully supported as well. For example:

$ echo "Some **Markdown** text." | /usr/bin/python -m markdown > output.html

Use the --help option for a list all available options and arguments:

$ /usr/bin/python -m markdown --help

You can make text *italic* or **bold**You can build **bulleted **or **numbered** lists :

remarkable SolarizedLight CSS

8 June 2016 Roberto Innocente - inno@sissa.it 5

* alpha- alpha,alpha- alpha,beta* beta- beta, alpha----1. alpha+ alpha,beta+ alpha beta+ beta

you can add an horizontal rule :---make **blockquotes** :>very difficult very difficult very difficult very difficult >to explain very difficult very difficult very difficult >>all this very difficult very difficult very difficult >>and the rest of it very difficult very difficult very difficult you can insert links :[sissa homepage](http://www.sissa.it)you can make headings up to level 5 :#heading 1##heading 2###heading 3####heading 4

remarkable CSS github

8 June 2016 Roberto Innocente - inno@sissa.it 6

#####heading 5

you can insert code to display (prepending a tab) :

if (i<10){

printf("%d",n);

} else {

printf("%s","stop it !");

}

def f(x):

return x + 1

github markdown for code (triple backticks) :

---

```python

print("Hello world!")

```

---

```javascript

var s = "Javascript" ;

alert(s);

```

---

Insert tables in a very simple way :

(Warning : with simple markdown the tables have no borders, in jupyter they have borders), it depends on css

|Header|Header|Header|Header|

|------|------|------|------|

|Cell |Cell |Cell | Cell |

|Cell |Cell |Cell | Cell |

|Cell |Cell |Cell | Cell|

---

remarkable CSS markdown

8 June 2016 Roberto Innocente - inno@sissa.it 7

Markdown | Less | Pretty

--- | --- | ---

*Still* | `renders` | **nicely**

1 | 2 | 3

---

markdown is a superset of html so you can add html tags :

<b>wow</b>

---

![alt text](https://www.sissa.it/sites/all/themes/sissa/logo.png ":Logo:")

you can refer to local files :

![/home/inno/]ws2016-Dockerfile

Section Title

=============

Subsection Title

----------------

Inline `code` has `backticks` around it.

remarkable CSS dark

8 June 2016 Roberto Innocente - inno@sissa.it 8

Python Markdown Editor

This is an open source editor written in python and based on a small httpd server using ajax for refreshing the rendering. It works on all platforms (requires python).

● Of course you need Pyhton installed● And you need pip : the python package manager.● Type :

– pip install markdown­editor

● pip

8 June 2016 Roberto Innocente - inno@sissa.it 9

2How to easily add math to Web pages ?

Using mathjax

8 June 2016 Roberto Innocente - inno@sissa.it 10

LaTeX, MathML, OpenMath

● Most of the people in science that need to write math, since more than 25 years, use LaTeX. At the beginning of the web people that needed to display math on web pages were using latex2html that was displaying the picture of the math formula rendered by LaTeX.

● Around ~2000 the W3C consortium approved independently from this wide community of users of math, a standard for writing math on the Web called MathML (Mathematical Markup Language, its version 3 is now part of HTML5 and became the ISO standard 40314 in 2015)

8 June 2016 Roberto Innocente - inno@sissa.it 11

MathML splits presentation (presentation MathML) from semantics (content MathML) and so 2 different parts are required !!!!

The simple formula

Requires all that you see on the left pane

OpenMath tries to define instead only the semantics of math in a user-extensible way and so is partly overlapped with content MathML. sin(x) in OpenMath :<OMOBJ>  <OMA>    <OMS name="sin" cd="transc1"/>    <OMV name="x"/>  </OMA></OMOBJ>

Presentation MathML

Content MathML

8 June 2016 Roberto Innocente - inno@sissa.it 12

MathJax

● Of course no one of these solutions was satisfactory for many ● In 2009 , with funding from AMS (American Mathematical Society) and

SIAM , a JavaScript program called mathJax was developed, that could replace formulas in LaTeX inside the Web pages with their rendering made using CSS and SVG

● This was a revolution .. now everyone can insert math in web pages using the largely diffused LaTeX syntax, just inserting a link to this JavaScript program. And these formulas are scalable (because of SVG) and scale like all the rest of the page.

● is now simply $ax^2+bx+c$

8 June 2016 Roberto Innocente - inno@sissa.it 13

MathJax/2

● It is easy to use mathJax : you just need to include in the header of your html the link to the script (showed in next page)

● Many markdown WYSIWYG editor do this automatically and therefore you can directly write into them LaTex formulas:– Inline math : using as delimter 1 dollar sign $x^2+1$

– Displayed math : using as delimiters 2 dollar signs ● $$\int_{-\infty}^y{f(x) dx}$$

8 June 2016 Roberto Innocente - inno@sissa.it 14

<!DOCTYPE html><html><head><title>MathJax TeX Test Page</title>  <!­­ Copyright (c) 2010­2013 The MathJax Consortium ­­>  <meta http­equiv="Content­Type" content="text/html; charset=UTF­8" />  <meta http­equiv="X­UA­Compatible" content="IE=edge" />  <script type="text/javascript" async       src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX­AMS_HTML­full">  </script>  <script type="text/x­mathjax­config">     MathJax.Hub.Config({tex2jax:{inlineMath: [["$","$"],["\\(","\\)"]]}});  </script></head>……

</html>

Header for MathJax processing Of LateX formulas

8 June 2016 Roberto Innocente - inno@sissa.it 15

MathJax In the html header (this will work also with markdown, because it lets

the html tags to flow trough without problems) you should put :

<script type="text/javascript" async

src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">

</script>

<script type="text/x-mathjax-config">

MathJax.Hub.Config({

tex2jax: {inlineMath: [["$","$"],["\\(","\\)"]]} });

</script>

the second script is necessary only if you want to use the standard inline math

delimiters of Latex \$ \$.

then you can insert your math in 2 ways as inline math :

look at this for instance $x=\sum_0^n{(i^2+1)}$, that stays amid the text,

or in displayed math that goes centered on a new line :

$$f(y) = \int_{-\infty}^{y}{g(x) dx}$$

not the full latex or ams math is supported, look at

[mathjax tex](http://mathjax.readthedocs.org/en/latest/tex.html)

[tex commands supported in mathjax](http://www.onemathematicalcat.org/MathJaxDocumentation/TeXSyntax.ht

m)

8 June 2016 Roberto Innocente - inno@sissa.it 16

8 June 2016 Roberto Innocente - inno@sissa.it 17

3Notebooks :

mixing text, math, computing results

Using jupyter

8 June 2016 Roberto Innocente - inno@sissa.it 18

In the good old times there was IPhyton (an interactive version of python).

In time it grew so much to contain many things :● An interactive shell● A notebook format● A notebook conversion tool● A client/server protocol● An interactive parallel python

Pieces like the notebook and protocol are only incidentally related to python and not specific of python. So the great split of Project Jupyter 2014. In august 2015 with Ipyhton 4.0 most of the components passed to the jupyter project and other projects like ipyparallel,...

Jupyter

“Jupyter is like IPython but language agnostic”.

The most interesting part of it are the notebooks. Available in Mathematica(TM) since many decades, these are now implemented in an open source context and with the possibility to use many different kernels :

bash, python, maxima, octave, R, ...

8 June 2016 Roberto Innocente - inno@sissa.it 19

Jupyter

Notebooks are organized in cells : a unit of input/output with the background kernel.

Cells can be :● Markdown cells : in which you put text and eventually math formulas using the markdown language and

mathjax (that is called automatically by jupyter)● Code cells : in which you put code for your kernel

A cell is entered typing : Shift+Enter.

Markdown cells will be rewritten with their html rendering, code cells will get an output from the kernel (that in the case of maxima will be in TeX and rewritten by mathJax with a picture of the formula).

Try :

http://try.jupyter.org/

They have bash, haskell, julia, python, R, Ruby, Scala kernels.

You can save your notebooks as .ipynb (notebooks files) or export them as .pdf

8 June 2016 Roberto Innocente - inno@sissa.it 20

Jupyter with maxima kernel

8 June 2016 Roberto Innocente - inno@sissa.it 21

4News on virtualization

Using Docker and friends ..

8 June 2016 Roberto Innocente - inno@sissa.it 22

Technology convergence

Containers

Docker

Minimalist OSs

New network protocols

Microservices

Container orchestration

8 June 2016 Roberto Innocente - inno@sissa.it 23

Microservices

● Opposite to monolithic app. Develop a single application as a set of small independent services (processes) communicating each other only trough a lightweight mechanism (like an http API)

● Microservices are language and tool independent

Monolitic App

Microservice 2

Microservice 1Microservice 3

Pic from eugenedvorkin.com

8 June 2016 Roberto Innocente - inno@sissa.it 24

Containers

● Traditional Virtual Machines are based on hypervisors that present a bare machine to the upper layers of software (performance penalty)

● On Linux since a decade kernel modifications were adopted to provide os-level (or light) virtualization : there is only one kernel shared by all virtual environments and containers are insulated groups of processes with regards to resources and namespaces (pid,fs,net,ipc) (very good performance)

● In Windows Server 2016 Microsoft provides Windows containers.

8 June 2016 Roberto Innocente - inno@sissa.it 25

Docker

● Linux containers are provided augmenting standard system calls and are difficult to use (lxc is not a fancy implementation)

● In March 2013 Docker (opensource) took the world by storm …● Written in golang provides an easy and complete interface for

containers : image creation, image repositories use, running, stopping, removing of containers ..

● In WS2016 microsoft supports Docker for its containers.

8 June 2016 Roberto Innocente - inno@sissa.it 26

PID, user, groups namespaces

File system namespace

Network namespace

Inside a container : namespace insulation

8 June 2016 Roberto Innocente - inno@sissa.it 27

Docker cheatsheet

8 June 2016 Roberto Innocente - inno@sissa.it 28

Minimalist OSs

● In the container era streamlined OSs are getting revenge on their large opponents.● Nothing decorative, only the essential ● Many different offers: coreOS, rancherOS, atomic(RedHat), snappy (Ubuntu),

boot2docker (docker.com) ..● The most used is coreOS (opensource). Public since October 2013 it has :

– Atomic upgrades (uses FastPatch double partition scheme)

– No package manager (all services installed in containers)

– Docker support

– Some cluster support tools : etcd, fleet, flannel, …

– All in all 200MB !

8 June 2016 Roberto Innocente - inno@sissa.it 29

New network protocols

● Hundreds of containers per server = many thousands of containers and MACs/IPs per rack

● Impossible to manage them in a standard layer 2 LAN. Very large and expensive switches have only 64k or quite less maximum MAC entries (cisco 4900M ~55k entries, cisco 3560 ~6k entries, alcatel OS6850 ~8k entries)

● But we want to keep the simplicity of migrating the containers everywhere in the Data Center :– Encapsulation/decapsulation by TOR (Top Of Rack)

switches or nodes :● MAC-in-MAC● MAC-in-IP/UDP

● Overlay networks : DOVE (IBM), VxLAN(arista, cisco, vmware), GRE (cisco)

Pic from M.Muktharov slideshare.net

8 June 2016 Roberto Innocente - inno@sissa.it 30

Container orchestration

● It is very useful to schedule containers based on load. Move containers from node to node, start them up, stop and remove them when it is needed

● The container ecosystem has different (opensource) offers for this : – Kubernetes (google), swarm (docker), apache/mesos, ..

● Kubernetes (descendent of borg, the tool google used to manage its infrastructure) was released to the public in july 2015– It runs pods : one or more containers that are guaranteed to be scheduled on the same machine ( to be

able to share resources)

– A service is made by a set of pods that work together (as one tier in a multi tier app)

– It is appropriate also for very large installations

8 June 2016 Roberto Innocente - inno@sissa.it 31

5News on

Windows Server 2016Using containers, docker

8 June 2016 Roberto Innocente - inno@sissa.it 32

Windows Server 2016

● A full Windows container implementation. They can be managed by powershell commands or docker (included)

● Windows containers are of 2 kinds :– Normal containers (analogue to Linux containers)

– Hyper-V containers (in this case they are isolated in a VM)

● A windowsservercore image for use with containers● A nanoserver image (streamlined windows server), again part

of the container ecosystem)

8 June 2016 Roberto Innocente - inno@sissa.it 33

6News on Windows 10

Using ubuntu bash, Hyper-V

8 June 2016 Roberto Innocente - inno@sissa.it 34

Windows 10

● The anniversary release of Windows 10 to be released in June will have some new features:– An ubuntu bash. This is not just a toy. Using paravirtualization,

canonical and microsoft have provided a near to complete environment in which you can apt-get many working linux tools (eg emacs)

– Hyper-V of Windows Server fame.The new release of Docker for windows uses this specific virtualization instead of virtualbox.

– Hyper-V containers

8 June 2016 Roberto Innocente - inno@sissa.it 35

7Scaling out

8 June 2016 Roberto Innocente - inno@sissa.it 36

Microservices can easily scale out

Monolithic applications need to scale upMuch more expensive and inflexible

8 June 2016 Roberto Innocente - inno@sissa.it 37

Multi-tier microservicesscaled out

LoadBalancer(ngnx)

LoadBalancer(ngnx)

WebApp

WebApp

WebApp

GaleraMariaDBcluster

GaleraMariaDBcluster

WebApp

LoadBalancer

(ngnx)

WebApp

GaleraMariaDBcluster

AutomaticScaling out

8 June 2016 Roberto Innocente - inno@sissa.it 38

8More on Docker

8 June 2016 Roberto Innocente - inno@sissa.it 39

Docker flow

Official Repositoryof images

Dockerhub.com

Docker CLICommand Language

Interface

Docker daemon

Port 2375

8 June 2016 Roberto Innocente - inno@sissa.it 40

Dockerfile/1

# base image debian

FROM debian

MAINTAINER inno@sissa.it

# apt­get some tools

RUN apt update && apt install  curl

# copy URL, very useful cmd

RUN curl ­O http://people.sissa.it/~inno/hello

RUN chmod a+x hello

CMD ./hello

8 June 2016 Roberto Innocente - inno@sissa.it 41

Dockerfile/2

# alpine : based on busybox linux

# the “swiss army knife of embedded systems”

FROM alpine

MAINTAINER inno@sissa.it

# apk is the apt­get of other distributions

RUN apk update && apk add  nodejs

RUN mkdir avg

ADD avg.js avg/

WORKDIR avg

# this command will run after the machine is up

ENTRYPOINT[“node”,”avg.js”]

8 June 2016 Roberto Innocente - inno@sissa.it 42

Union file system

● Used by Docker to use images :– For every line in the docker file a new image is created, they are all

read-only

FROM alpine

RUN apk update && apk ..

RUN mkdir avg

ADD avg.js avg/

Docker r+w layer

You can Make changesWriting to the

Last layer

You can CommitChangesTo a new

image

Newimage

MAINTAINER inno@sissa.it

ENTRYPOINT[“node”,”avg.js”]

WORKDIR avg

8 June 2016 Roberto Innocente - inno@sissa.it 43

Project

8 June 2016 Roberto Innocente - inno@sissa.it 44

Project: Coreos Cluster for μservices

Pic from coreOS