© 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and...

12
© 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise

Transcript of © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and...

Page 1: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

© 2005 Altera Corporation© 2006 Altera Corporation

Batch Computing at Altera

Condor, Quill and The Enterprise

Page 2: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

2© 2006 Altera Corporation

About AlteraAbout Altera “The Programmable

Solutions Company” Pioneer of SOPC

technology Founded in 1983 $1.02 billion in

2004 sales 2,300 employees 14,000+ worldwide

customers

Sarah L. Adams
Changed 15,000 back to 14,000 per Anna.
Page 3: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

3© 2006 Altera Corporation

Programmable LogicDevices (PLDs)

Intellectual Property (IP) Development Software

About Programmable SolutionsAbout Programmable Solutions

Page 4: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

4© 2006 Altera Corporation

About MeAbout Me

Senior Software Engineer at the Toronto Technology Center

B.A.Sc. in Engineering Science from the University of Toronto

Joined Altera in 2001 Focus on distributed computing

Page 5: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

5© 2006 Altera Corporation

Where It All BeganWhere It All Began

Developed in Toronto Centralized scheduling system Multiple queues Priority/FIFO execution No limit on resource claims Engineer-designed, custom API

Page 6: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

6© 2006 Altera Corporation

Change Is Good, Right?Change Is Good, Right?

Multi-OS support Redundancy and fault tolerance Easy expansion beyond Toronto Easy-to-use API New features Improve matchmaking Capacity planning Really Important!

Page 7: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

7© 2006 Altera Corporation

Pain Free MigrationPain Free Migration

TTCPOOL

USERTOOLS

CONDORPOOL

META SCHEDULER

PriorityEngine

CO

ND

OR

TT

C

SO

AP

DB

Page 8: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

8© 2006 Altera Corporation

Time Stands StillTime Stands Still

Nice-style priorities [1:N] Use priority factor to ensure PN negotiates

before PN+1, PN+2, etc. RUP(PN) = 0.5

EUP(PN)/EUP(PN+1) = ½

Freeze RUP values in time PRIORITY_HALFLIFE = 100000000000000000000

Let jobs at PN get all VMs in the system NEGOTIATOR_IGNORE_USER_PRIORITIES = True

Page 9: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

9© 2006 Altera Corporation

Translation ServicesTranslation Services

<cluster><id>1</id><priority>2</priority><os>windows</os><group>fitter</group><job>

<id>1</id>...

</job><job>

<id>2</id>...

</job>...

</cluster>

METASCHEDULER

+AlteraClusterID = 1+AlteraGroup = fitterrequirements = OpSys

= ...+AccountingGroup = P1AlteraTargetOs =

windows...

+AlteraJobID =1...queue

+AlteraJobID = 2...

Page 10: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

10© 2006 Altera Corporation

SQL! SQL! Everywhere!SQL! SQL! Everywhere!

POSTGRESQLDBMS

METASCHEDULER

CONDORQUILL

CONDORCOLLECTOR

USAGEHISTORY

STATUSINFO

SYSTEMAUDITS

Page 11: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

11© 2006 Altera Corporation

Better Planning Through PicturesBetter Planning Through Pictures

Page 12: © 2005 Altera Corporation © 2006 Altera Corporation Batch Computing at Altera Condor, Quill and The Enterprise.

12© 2006 Altera Corporation

From Here, Where?From Here, Where?

Roll out across the enterprise Scaling with multiple schedds Quill++ DBMS for configuration management (with

R. Nordlund & J. Stowe from The Hartford)