© 2019, Amazon Web Services, Inc. or its Affiliates. All rights ......2019/09/11 · © 2019,...

Transcript of © 2019, Amazon Web Services, Inc. or its Affiliates. All rights ......2019/09/11 · © 2019,...

https://amzn.to/JPWebinar

https://amzn.to/JPArchive

❏

❖

•

①吹き出しをクリック②質問を入力③ Sendをクリック

•

❏❖

❖

❏❖

❖

•

https://en.wikipedia.org/wiki/CRAY-1

https://en.wikipedia.org/wiki/EPFL

https://en.wikipedia.org/wiki/Lausanne

CPU

< >(queueing)

(dispatch)

dispatch

B A

A B

•

従来は処理のピークに合わせてリソースを予め用意していた。ピーク時に賄えないリソースは、ジョブスケジューラにより、処理するタイミングをずらして対応

クラウドの場合はその時に必要なリソースを立ち上げて、処理を実行する

Time

従来は手持ちの限られたリソースで、逐次処理していたジョブもクラウドであれば必要な台数、インスタンスを起動して、一斉処理が可能

のインスタンス費用は原則「時間×台数」なのでどちらもほぼ同じコスト

オンプレミス環境

従来のクラスタ構成は固定

固定のリソースに、ジョブをスケジューリング

クラウド環境従来のクラスタ構成は固定

クラウド環境

必要に応じて必要な台数でクラスタを構成ジョブをスケジューリング

クラウド環境

ジョブが終了したらインスタンスを終了計算ノードの課金が停止

クラウド環境

•

Media & Entertainment:

Healthcare & Life Sciences:

Internet Companies:

Scientific Research:

Engineering and Design:

上にコアを伸縮自在に追加するシステムを構築。万ジョブを処理し、従来週間かかるシミュレーションを日で完了

ドライブヘッド設計で、数百万パターンに及ぶパラメータスタディ実行。ピーク時以上のコアを同時利用、従来ヶ月かかる処理を時間で完了

万の化合物のスクリーニング実行。ピーク時約コアを同時利用し年分の計算処理を時間で完了

•

• がインスタンスの起動や停止を行うため、スケジューラや計算ノードなどの管理が不要

• ジョブはコンテナイメージを元に作成し、自動でスケールするコンピューティング環境で実行する

• コンピューティング環境ではインスタンスタイプや数、スポットインスタンス利用有無などを任意に指定可能

• 万クラスの大規模な計算にも対応で管理

処理を依頼

スケジュール実行

•

インスタンス

•

インスタンス

•

Submitted

Runnable

Starting

Running

Succeeded FAILED

Pending

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/batch_sns_tutorial.html

https://aws.amazon.com/jp/premiumsupport/knowledge-center/batch-job-stuck-runnable-status/

•

$AWS_BATCH_JOB_ARRAY_INDEX

•

•“Job-A”

“Job-B”

“Job-A”

Job-A:0

…

Job-A:1 Job-A:2

Job-A:3 Job-A:4 Job-A:5

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/array_jobs.html

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/job_dependencies.html

“Job-A”

Job-A:0

…

Job-A:1

Job-A:99

Job-A:2

“Job-A”

Job-A:0

…

Job-A:1 Job-A:2

“Job-B”

Job-B:0

…

Job-B:1 Job-B:2

Job-B:3 Job-B:4 Job-B:5

Job-B:97 Job-B:98 Job-B:99

“Job-A”

•

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/launch-templates.html

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/job_timeouts.html

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/job_retries.html

•

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/ExamplePolicies_BATCH.html#iam-example-job-def

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/job_definition_parameters.html#containerProperties

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/logging-using-cloudtrail.html

•

“Job-A”

https://aws.amazon.com/jp/blogs/compute/building-a-tightly-coupled-molecular-dynamics-workflow-with-multi-node-parallel-jobs-in-aws-batch/

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/multi-node-parallel-jobs.html

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/efa.html

•

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/batch-cwe-target.html#scheduled-batch-job

•

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/job_retries.html

https://www.slideshare.net/AmazonWebServicesJapan/20190306-aws-black-belt-online-seminar-amazon-ec2

インスタンス

•

https://docs.aws.amazon.com/ja_jp/batch/latest/userguide/launch-templates.html#lt-user-data

https://aws.amazon.com/jp/blogs/compute/scalable-deep-learning-training-using-multi-node-parallel-jobs-with-aws-batch-and-amazon-fsx-for-lustre/

https://www.slideshare.net/AmazonWebServicesJapan/20190319-aws-black-belt-online-seminar-amazon-fsx-for-lustre

https://docs.aws.amazon.com/ja_jp/codebuild/latest/userguide/sample-docker.html

•

https://aws.amazon.com/jp/blogs/compute/building-simpler-genomics-workflows-on-aws-step-functions/

https://www.slideshare.net/AmazonWebServicesJapan/20190522-aws-black-belt-online-seminar-aws-step-functions

•

https://aws.amazon.com/jp/about-aws/whats-new/2019/04/AWS-Batch-supports-GPU-Scheduling/

https://aws.amazon.com/jp/blogs/compute/deploy-an-8k-hevc-pipeline-using-amazon-ec2-p3-instances-with-aws-batch/

•

https://www.nextflow.io/

https://www.nextflow.io/blog/2017/scaling-with-aws-batch.html

https://aws.amazon.com/jp/blogs/compute/using-cromwell-with-aws-batch/

https://aws.amazon.com/jp/quickstart/architecture/illumina-dragen/

•

https://www.slideshare.net/AmazonWebServices/set-up-a-millioncore-cluster-to-accelerate-hpc-workloads-cmp404-aws-reinvent-2018

https://www.slideshare.net/AmazonWebServices/intro-to-aws-batch-how-aqr-capital-leverages-aws-to-identify-new-investment-signals-cmp372-aws-reinvent-2018

https://aws.amazon.com/jp/blogs/compute/building-simpler-genomics-workflows-on-aws-step-functions/

https://github.com/aws-samples/aws-batch-genomics

https://aws.amazon.com/jp/quickstart/architecture/illumina-dragen/

https://aws.amazon.com/jp/blogs/compute/accelerating-precision-medicine-at-scale/

•

https://aws.amazon.com/pt/blogs/compute/deep-learning-on-aws-batch/

https://aws.amazon.com/jp/blogs/compute/deploy-an-8k-hevc-pipeline-using-amazon-ec2-p3-instances-with-aws-batch/

•

https://www.slideshare.net/AmazonWebServicesJapan/20190306-aws-black-belt-online-seminar-amazon-ec2

https://www.slideshare.net/AmazonWebServicesJapan/20180214-aws-black-belt-online-seminar-amazon-container-services

https://www.slideshare.net/AmazonWebServicesJapan/20190731-black-belt-online-seminar-amazon-ecs-deep-dive-162160987

https://www.slideshare.net/AmazonWebServicesJapan/20190522-aws-black-belt-online-seminar-aws-step-functions

https://www.slideshare.net/AmazonWebServicesJapan/20190423-aws-black-belt-online-seminar-aws

https://www.slideshare.net/AmazonWebServicesJapan/20190319-aws-black-belt-online-seminar-amazon-fsx-for-lustre

https://aws.amazon.com/jp/blogs/news/

AWS Well-Architected 個別技術相談会

•

https://amzn.to/JPWebinar

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights ......2019/09/11 · © 2019,...

Documents

Transcript of © 2019, Amazon Web Services, Inc. or its Affiliates. All rights ......2019/09/11 · © 2019,...

Affiliates Summit

16. redwoods affiliates

Affiliates Guide

NACE Affiliates 100 % Affiliates EDC2 – LPA Initiative MAP-21 TZD.

CrossMark for Affiliates

Pacifica Affiliates Program

Energy Affiliates

Eazybusiness Affiliates

RiverValley and Affiliates

Affiliates' Manual - WinRAR

Cash Money Affiliates

Crossref for Affiliates

UNIX講習会 SunGridEngine 利用方法qsub script_ﬁlename ジョブを投入 qstat 自分のジョブの状態を表示 qstat -u ’*’ 全ユーザのジョブ状態を表示

PSI Affiliates Directory

AFFILIATES NEWS

Industrial Affiliates Programs

WeGoLook Overview - Affiliates

Affiliates Directory

Dealing with Affiliates

...WebOTX Batch Server連携では、新たにWOBSジョブ部品を使用することができます。WOBSジョ ブ部品とは、WebOTX

...WebOTX Batch Server連携では、新たにWOBSジョブ部品を使用することができます。WOBSジョブ部品とは、WebOTX