Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for...
-
Upload
emilio-baham -
Category
Documents
-
view
217 -
download
1
Transcript of Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for...
![Page 1: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/1.jpg)
Controlling embedded hardware
Erlang in global radio astronomy
Harro Verkouter/Joint Institute for VLBI in Europe
![Page 2: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/2.jpg)
![Page 3: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/3.jpg)
Joint Institute for VLBI in Europe
VeryLongBaselineInterferometry
VeryLongBaselineInterferometry
![Page 4: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/4.jpg)
Ever larger telescopes
15cm
5m 100m
![Page 5: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/5.jpg)
![Page 6: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/6.jpg)
![Page 7: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/7.jpg)
Electromagnetic radiation
![Page 8: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/8.jpg)
(m) D (m) /D (rad)
smallest optical 10-7 10-1 10-6
biggest radio 10-1 100 10-3
wavelength diameter pixelsize
![Page 9: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/9.jpg)
A neat trick
100m
![Page 10: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/10.jpg)
A neat trick
100m
20m
correlator
![Page 11: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/11.jpg)
![Page 12: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/12.jpg)
10,000km
![Page 13: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/13.jpg)
A different ball game!
(m) D (m) /D (rad)
largest optical (IR) 10-6 10 10-6
VLBI (radio) 10-1 107 10-8
![Page 14: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/14.jpg)
Science
![Page 15: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/15.jpg)
Science
![Page 16: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/16.jpg)
Recording the data
Record up to 4096 Mbps (512 Mbyte/s)
![Page 17: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/17.jpg)
![Page 18: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/18.jpg)
Number of combinations to be computed as function of n
~n2
![Page 19: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/19.jpg)
16 telescopesat
1024 Mbps
![Page 20: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/20.jpg)
UniBoard
QuickTime™ and a decompressor
are needed to see this picture.
4 frontnode
4 backnode
1Gbit/s ethernet
16x 10Gbps
16x 10Gbps
>20 Gbps/link
![Page 21: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/21.jpg)
Erlang?!
• typical languages used– 80% C++– 15% Python– PHP, Fortran, perl
• database– MySQL
• needed to convince management– pilot projects!
![Page 22: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/22.jpg)
Decoding data files
• VLBI has framed data format(s)
QuickTime™ and a decompressor
are needed to see this picture.
• Look for ‘syncword’ – different flavours, same syncword, different framesize
• Other format uses different syncword
![Page 23: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/23.jpg)
![Page 24: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/24.jpg)
(not quite) surprising little code
• 212 lines of code– first dabblings ...
• about a week’s work• recognizes + decodes 3 formats• decodes the BCD timestamps• supports distributed system
– reading file on one machine– decoding on other machine
• management duely impressed– got permission to do a larger pilot!
![Page 25: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/25.jpg)
Soft real time monitoring website
• webcams at stations– webpage with http links = #FAIL– unreliable links
• really high latency (China, Chile)• don’t want broken images
– low bandwidth• cache in Europe: multiple users eat all bandwidth
• parse telescope logfiles– immediate antenna calibration info– generate timeseries plots as observation progresses
• stored in MySQL
![Page 26: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/26.jpg)
Soft real time monitoring website
• webcam URLs + status stored in MySQL– supervisor starts webcam monitors
• one gen_server for each webcam– loads info from MySQL– uses http:request/1– strict checking on return value
• only HTTP 200 status replies• content_length header set?• explode if problem
– use a port to ImageMagick to scale image– write binary JPEG data into mnesia
![Page 27: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/27.jpg)
Soft real time monitoring website
• monitor all participating telescopes• telescope log file (ASCII text)
– retrieved via python + ssh – read into Erlang using a port– low data rate after start up
• parses lines– update MySQL
• generate plots using gnuplot – for each telescope– query MySQL every so many minutes– using a port– write .png binary data to mnesia
![Page 28: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/28.jpg)
Soft real time monitoring website
• patch it all together using webtool• cache complete webpage in mnesia
– too old? generate new one
• serve .png and .jpg data from mnesia– image URL points back to webtool
• extremely simple to cache + serve data– brilliant idea: mapping URL to erlang fn call!
• in total used ~3000 lines of code– webcam, logfile monitoring; plotting; serving web pages– excellent test case for OTP and distributed system
• got the green light for the real thing!
![Page 29: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/29.jpg)
UniBoardEthernetSwitch
FPGA
Nios2CPU
1Gbit PHY
FPGA
Nios2CPU
1Gbit PHY
FPGA
Nios2CPU
1Gbit PHY
FPGA
Nios2CPU
1Gbit PHY
Control Computerwindows/unix
EthernetSwitch
VHDLHardwareSoftware
![Page 30: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/30.jpg)
FPGA
1Gbit PHY
VHDLHardwareSoftware
Polyphase Filterbank
registers
Delay Module
registers
Nios2CPU
registers
registers
![Page 31: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/31.jpg)
VHDLHardwareSoftware
Control Computerwindows/unix
Nios2CPU
Client
Server
UDP/IPv4 over ethernet
![Page 32: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/32.jpg)
Simple binary protocolRead N 32bit words starting from START ADDRESS
(Over)Write N 32bit words from the packet to START ADDRESS
Read/Modify/Write N 32bit words from START ADDRESS + packet to START ADDRESS
32 bits 32 bits 32 bits 0x01 N START ADDRESS
32 bits 32 bits 32 bits N*32 bits 0x02 N START ADDRESS N 32 bit words of DATA
32 bits 32 bits 32 bits N*32 bits OPCODE N START ADDRESS N 32bit words of MASK
OPCODE 0x03 0x04 0x05
BITWISE OPERATION AND OR XOR
![Page 33: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/33.jpg)
QuickTime™ and a decompressor
are needed to see this picture.
![Page 34: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/34.jpg)
FPGA client library
Registermap:
• list of register definitions
• type of the register (details follow)
• name of the register (symbolic)
• address of the register in the memory map
one controller one fpga
![Page 35: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/35.jpg)
FPGA client library
fpga:bit( <name>, <bit #>, <address> )
fpga:bitrange( <name>, <startbit>, <nbit>, <address> )
fpga:word( <name>, <address> )
fpga:range( <name>, <# words>, <address> )
Available registerdefinition types for the controller
![Page 36: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/36.jpg)
FPGA client library
fpga:read( <name> )
fpga:write( <name>, <value> )
fpga:or_( <name>, <value> )
fpga:and_( <name>, <value> )
fpga:xor_( <name>, <value> )
Available register commands
![Page 37: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/37.jpg)
Example
-module(firfilter). %% this module is called firfilter, driving one of those-behaviour(personality).-export([registers/0, start_filter/2]). %% this module implements an FPGA personality
%% Called automatically by the FPGA control frameworkregisters() -> {ok, [
fpga:bitrange(num_tap, 5, 3, 16#24),fpga:bit(pps_enable, 28, 16#24),fpga:word(control_status, 16#abcc),
].
%% define a high-level command for this personalitystart_filter(FPGA, NumTap) ->
%% disable the PPSfpga:execute(FPGA, fpga:write(pps_enable, 0)),%% read the control status registercase fpga:execute(FPGA, fpga:read(control_status)) of
1 -> fpga:execute(FPGA, fpga:xor(control_status, 42));_ -> ok
end,%% however, you can easily execute a number of commands in one go:fpga:execute(FPGA, [fpga:write(num_tap, NumTap),
fpga:write(pps_enable, 1)]).
![Page 38: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/38.jpg)
Safety
2> fpga:execute(FPGA, fpga:write(pps_enable, 2)).
OH NOES! error:"Value '2' does not fit in one bit (pps_enable)"
FPGAControl terminating [{{badmatch, {reply, {error, {"Value '2' does not fit in one bit (pps_enable)", [{fpga,err,2, [{fil
![Page 39: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/39.jpg)
Full correlator control system
• besides single fpga control– parallel control of eight fpga’s
• parsing + validating configuration files– using yecc
• decoding + sending over UDP of binary data files– data as well as polynomial coefficients
• capture network data to file• configure and control the hardware• MySQL read/write• two weeks VHDL coding = < 1 day Erlang coding
![Page 40: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/40.jpg)
data reader
data reader
data reader
data writerUniBoardUniBoard
command/control
Full correlator control system
![Page 41: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/41.jpg)
The good
• cross platform– we have Linux, Mac OSX and Solaris
• distributed– typically many machines involved
• non-variability of ‘variables’• conciseness / amount of work per LOC• connects to MySQL
– not easy to connect other languages to mnesia
• hard crash on errors– about the most valuable property: can’t ignore!
• binary pattern matching– every language should be jealous of this
• re-use of development/testing code– it’s so easy to re-use code!
![Page 42: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/42.jpg)
The bad
• our VHDL engineers struggle– language simple, but– concepts like recursion + distribution are ‘hard’
• conciseness / amount of work per LOC– takes effort to read code written by someone who’s better at
it than you
• deploying a developing distributed system– we haven’t found a simple way yet– dynamic remote code loading could be awsome
• it works for individual modules• useless if you have dynamic dependencies ...
– erlang:apply/3
![Page 43: Controlling embedded hardware Erlang in global radio astronomy Harro Verkouter/Joint Institute for VLBI in Europe.](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649cae5503460f9497227d/html5/thumbnails/43.jpg)
Thank you for your attention!