戦略的創造研究推進業 CREST 研究領域 研究課題 · 2018-07-25 · - 1 - 戦略的創造研究推進業 CREST 研究領域 「共生社会に向けた人間調和型情報技術の構築」
理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command...
Transcript of 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command...
![Page 1: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/1.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
Computer simulations create the future
�
2 - 2
-() R-
(
������ �������
��������������
![Page 2: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/2.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE �
• :– 2 A -– 2• : :• :
– & • : & & :– 2 - - 2– : &• & 2 -• A• &
– & :•
![Page 3: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/3.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
- - /
�
•– - - - - - - - / /– - - - - . - - - / /
![Page 4: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/4.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
�
1. Work directory and shell script are created.Jobs are submitted to the scheduler.
2. Jobs are executed by the job scheduler.Current directory is set to the work directory.
3. Results are downloaded to OACIS server.The files are parsed and the records are updated.
database
job scheduler
(ex. Torque)
OACIS Computational Host
qsubSSH login
sh
execute in the work directory
the work directory is compressed into a single file.tar
download (scp)& expand
parse job status &update the record
![Page 5: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/5.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
A , A
�
, A A A A D A
• A A A 0 C A 0(, ) A D A A
• A A A 0 A A A A
• A 0 , , A A A A D AA– . A A . A A .A A A .C A A . A A
. A A A A . A A A
• A D A AD A
![Page 6: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/6.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
<B B 5 5
�
• -I B 5 5
– . 5 5 5 < 5 B5 5 B < B 5 "• 5 B "B , , , """ ,
• -I 5 /210 :
– 3< < B 5 5 :4 "> B N 5 < <5 5 5 "• J 4
![Page 7: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/7.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
,
�
•. . , .
, , . ,
wrap script
simulation program
run.py
_input.json
a.out
1. Input parameters given by OACIS
2. parse command line args and make an XML file
3. launch the simulation program as a subprocess
![Page 8: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/8.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE �
http://crest-cassia.github.io/oacis/en/configuring_simulator.htmlOther samples for these scripts are found in our doc page.
Load the input JSON file
Make an XML file
Spawn a subprocess
![Page 9: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/9.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
.
��
• . . , . . ., . ., " .
• . , .(" . . . .
A) .
• . . . . . " .. " " . .
. . . .
{"flow": 0.235, "velocity": 1.245 }
![Page 10: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/10.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
��
Simulator name
Definition of Parametersname, type, default value
preprocess command�optional�
command of Simulator
JSON or Argument input
A note of the Simulator
List of executable hosts
![Page 11: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/11.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
,,. , .
��
• . . . ..
,,. ,
![Page 12: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/12.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
. .
��
• . . , ,. , . , ,
. ,
Duplicating Simulator
![Page 13: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/13.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
• , ,- , , . 1 "• ,, "
![Page 14: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/14.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
Simulator name
Definition of Parametersname, type, default value
“echo”
Select “Argument”
Select “localhost”
![Page 15: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/15.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
• . 2 2
– .
– 2 -
Key Description
l Road length
v Maximum velocity
rho Car density
p deceleration probability
t_init initialization steps
t_measure measurement steps
mkdir temp && cd temp~/sim_ns_model/run.sh 200 5 0.3 0.1 100 300 12345
docker exec -it -u oacis my_oacis bash -l
![Page 16: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/16.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
• , ., ,,
Item Contents
Name MySimulator
Definition of Parameters L, v, rho, p, t_init, t_measure
Command ~/sim_ns_model/run.sh
Input Type Argument
Executable_on localhost
Key Type Default Val
l Integer 200
v Integer 5
rho Float 0.3
p Float 0.1
t_init Integer 100
t_measure Integer 300
Click
“Add Parameter”
to increase the
number of
parameters.
![Page 17: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/17.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
• .
– .. .
![Page 18: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/18.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
-/ / -
��
•– . / -/ . / / /- / - . .– . / -/ . / / /- / - . .
![Page 19: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/19.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
.
• .. .
•
• .
Seehttp://crest-cassia.github.io/oacis/en/install.html
![Page 20: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/20.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
. .
��
• . ... . - - . .
(At OACIS host)ssh-keygen -t rsa # ~/.ssh/id_rsa ~/.ssh/id_rsa.pub are createdscp ~/.ssh/id_rsa.pub [USER]@[HOST_NAME]:~
# copy your public key to the remote host(At Computational host)cat ~/id_rsa.pub >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys
# add your public key to the list of authorized keys(At OACIS host)eval `ssh-agent` # launch SSH agentssh-add ~/.ssh/id_rsa # add key to agent. Enter your passphrase.ssh [USER]@[HOST_NAME] # verify that you don’t need to enter the passphrase
bundle exec rake daemon:restart # launch OACIS
![Page 21: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/21.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
. - .
��
• - - - / -" /" - . -
Host my_serverHostName my_server.example.comPort 22User muraseIdentityFile ~/.ssh/id_rsa
Host my_server2HostName 127.0.0.1Port 22User muraseIdentityFile ~/.ssh/id_rsa
![Page 22: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/22.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
• 1 8 . 1 ? 1 1 1 181 B 1
• A . . 1 1 18 11 1 8 . 1 ? 1 ? 1
?. . ? .1 8 1 88 1
• ?.B 1 1 ? 1
Refer to https://github.com/crest-cassia/xsub
![Page 23: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/23.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
, E : 01
��
• . 6 FE E E
– : E 6 EE : E F "6 6C E6 F ": E
• E I " C
• FCC E E CBF - 0/ FC 6F B 6 C F CE "
export PATH="$HOME/xsub/bin:$PATH"export XSUB_TYPE="none"
![Page 24: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/24.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
��
host information for SSH connection
Directory used for running jobs. OACIS will create sub
directories under this.Maximum number of
concurrent jobs.Worker checks the status of
the remote host with this interval.
The available range of the number of MPI processes and
threads.
List of executable simulators on that host.
http://crest-cassia.github.io/oacis/en/tutorial.html#registering-a-host
![Page 25: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/25.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
Parameters required by XSUB (Host Parameters)
are automatically set.
![Page 26: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/26.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
Host Parameters are required when creating a PS.
![Page 27: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/27.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
• , , ., , ,
docker exec -u oacis my_oacis cat /home/oacis/.ssh/id_rsa.pub >> ~/.ssh/authorized_keyschmod 600 ~/.ssh/authorized_keys
# testing connectiondocker exec -it -u oacis my_oacis bash –l# edit (~/.ssh/config file)ssh my_hostexit # logout from computational hostexit # logout from container
# installing xsub on computational hostgit clone https://github.com/crest-cassia/xsub.git ~/xsub
# edit ~/.bash_profile. Add the following two lines.export PATH="$HOME/xsub/bin:$PATH"export XSUB_TYPE="none"
![Page 28: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/28.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
��
We can define a HostGroup, a group of Hosts.When creating a job, HostGroup can be specified as a destination of the submission. The jobs are submitted to one of the hosts in the HostGroup.
![Page 29: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/29.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE ��
click Hosts
click “New Host Group”
We can select a HostGroup as the place to which jobs are submitted.
![Page 30: 理化学研究所 計算科学研究センター(R-CCS) | RIKEN R-CCS...Command ~/sim_ns_model/run.sh Input Type Argument Executable_on localhost Key Type Default Val l Integer 200](https://reader034.fdocuments.in/reader034/viewer/2022050609/5fb05fc7eecea24b1f1946f4/html5/thumbnails/30.jpg)
RIKEN ADVANCED INSTITUTE FOR COMPUTATIONAL SCIENCE
.
��
• ,@ A A ,,
– - , - . ., - . C,@ , ,A A - . . . @
• C ,@ , C , -,.
– ,. @ .
A A , A, , , , ,.
,