(12) UnIted States Patent (10) Patent No.: US 8,125,677 B2 ...

US008125677B2

(12) UnIted States Patent (10) Patent No.: US 8,125,677 B2 Shestak et al. (45) Date of Patent: Feb. 28, 2012

(54) METHODS AND SYSTEMS FOR IMPROVED 2002/0051199 A1 5/2002 Hatayama PRINTING SYSTEM SHEETSIDE DISPATCH 2} garroll 1 IN A CLUSTERED PRINTER CONTROLLER Zoos/0068546 Al 3/2005 K‘l’ffgtztla '

_ _ _ 2005/0275877 A1 12/2005 Singh et a1. (75) Inventors: VladImIr V. Shestak, Fort Coll1ns, CO gong/0172302 A1* 7/2003 Knodt ,,,,,,,,,,,,,,,,,,,,,,, H 705/26

(US); Howard Jay Siege], Bellvue, CO (US); James T. Smith, II, Boulder, co FOREIGN PATENT DOCUMENTS (US) JP 10-333844 12/1998

JP 11-353141 12/1999

(73) Assignee: Ricoh Production Print Solutions OTHER PUBLICATIONS LLC, Boulder, CO (US)

Kumar, N. J., et 31.; Asynchronous Software Thread Integration for ( * ) Notice: Subject to any disclaimer, the term Qfthis Ej?cient Software Implementations of Embedded Communication

patent is extended or adjusted under 35 Protocol Controllers; ACMiProceedings 0f LCTES’04; Jun. 2004; U.S.C. 154(b) by 991 days. pp. 37-46.

Kessler, C., et 31.; Optimal Integrated Code Generation for Clustered (21) App1_ NO; 12/058,675 VLIWArchitectures; ACMiProceedings 0f LCTES’02; Jun. 2002;

pp. 102-111.

(22) Filed: Man 29, 2008 Pai, V., et al.; Locality-Aware Request Distribution in Cluster-Based Network Servers; ACMiProceedmgs ofASPLOS VIII; Oct. 1998;

(65) Prior Publication Data pp‘ 205 '2 16'

Us 2009/0244582 A1 Oct. 1, 2009 * Cited by examiner

(51) Int_ CL Primary Examiner * Douglas Tran G06F 15/00 (200601) (74) Attorney, Agent, or Firm * Duft Bornsen & Fishman, G06K 1/00 (2006.01) LLP G06K 15/00 (2006.01) G06F 3/12 (2006.01) (57) ABSTRACT

52 US. Cl. ...................................... .. 358/1.18; 358/1.1 Methods, s stems, anda aratus for im roved dis atchin of y PP P P g (58) Field of Classi?cation Search ...................... .. None SheeISideS in a high-speed (fl-g, COnIinuOuS form) printing

See application ?le for complete Search history, environment using multiple, clustered processors in a print controller. Features and aspects hereof utilize a stochastic

56 References Cited mathematical model of multi le rocessors com ute nodes P P P each adapted to RIP (rasteriZe) raW sheetside data provided to

US. PATENT DOCUMENTS it. The model utilizes stochastic estimates of the probability 5 596 416 A 1/ 1997 Bar et 31, of RIP execution time to select a referred rocessor of the , , ry P P

5,833,375 A 11/ 1998 Gauthier et a1. controller and dispatches the sheetside to the preferred com 6,088,T20 A 7/2000 ShTbSaWa et 3T pute node identi?ed as providing best probability of meeting

232;? required deadlines for RIP completion time of a raW sheet 730992027 Bl 8/2006 Barry et a1. Side~ 7,791,777 B2 * 9/2010 Barry et a1. ................. .. 358/518

2001/0043357 A1 ll/200l Owa et a1. 15 Claims, 18 Drawing Sheets

slggiaam 25b Printhead 1 Parser 16b 5 ChalnTrzls 1'12 1.3.0 16b E1285?“ 154

Ethernet ‘ 4 Gb Frbre

150 Ethemet Rlpping I \ 0:22:81 _ Channels Head Node \ Switch ( ) compute {\ I} Switch 156 1% M Nodes m _

m __

sheetside -_ Dispatcher Printhead 0

12.0 .112

100 j

US. Patent Feb. 28, 2012 Sheet 1 0f 18 US 8,125,677 B2

g o 8225

mm“ 22:20 22E 26 w

a

m2, 22% 6220 05E

m2

> 382

93:60 965E

E“ 23:20 2E 8 w

v .QE

2: coEsw BEwEm

owv

o2 asgm S _,

Q 262 Q8: m2 623 Emgw Ema


FIG. 3

i

Retrieve Next Raw Sheetslde “300

V

Apply Stochastic x 302 Mathematical Model of Clustered Processors’ Update Model as Current operations_to *""' Clustered Processors’ x306 Select Pr°9e$?9r W'th Operating Status Changes Best Probability of

Completing Sheetside by Required Deadline

V

Place this Raw Sheetside . x 304

In Transfer Queue

FIG. 4

‘fxi (x) M

Processor A PROBABILITY 4O 1“ TO MISS DEADLlNE

/:l\ " w _ I ’

403J M

135i (x) DEAIIDLINE ‘2m

Processor B 02 4 \' 404


FIG. 5 A

1

Processor A 501 \

J M 503 T \505 fx, (x) DEADLINE ‘\

506

Processor B 502 \p 504

507 / V

FIG. 6

1361- (x)

Processor A

603/ ‘ V

N #606 fxlx) DEADLI E I

Processor B 602 \V

604

V


5

NE

\ NE

s: N .QE

QR

AU

NE ¢ u $5 32262

EN


800 802 804

1 5 1 5 1 $

:M 0 j‘? :E i‘ time 0 a b 0 time time X >< >< ><

808 Input Buffer I’ Output Buffer F

I _________ ‘_"'| ' I I ' '

BQ1JS E i BQ1J4 BQ1J3 { '~--4 ------ -- i .'

810') 812) 814) 816J FIG. 8

900 902

1| v 1[ a 1 0 X .‘P-Eiz’time OabCtime 0

X .

’< ’‘ A0u,[BQ1J5]

Input Buffer {906 Output Buffer f908 j j j j

B/Q15 BiQM IBQB BIQ12 910J 912-) 914J 916)

FIG. 9


STEP 1 1000 1002 1004

1 7 1 > 1 ?

® ‘\818 —

0 X time 0 a b 0 time 0 x time

STEP2 1010 1000

l I 2 | 3 t' ' 0 “5 -° 0 time X '5? [me ‘ I X '>*< ">2 :2

> _ W777

:5 y — td [BQ1J3]+ Z‘bitmap

§ 25 E STEP3 1 2 1010

1 10>14 1 0g 1 2

| T T? TTT , _ O l time 0 tlme 0 Titime

X X10312 + + >1

EEK é é E E E

1006 1008 Input Buffer f Output Buffer F

I — _ _ _ _ _ _ _ _ __|

A11‘ 11 ! | nnj nnj Q16 55215 i i 15%14 D§413

' t

1018) 1020) 1022) L,“ Us

1016

FIG. 10


RIP execution times: 3 7 10 time

1m [BQé ] = 50 50 50 A0ut[BQ1J6]: +6 +2 +0 \ 1102

RIP completion times: 59 59 60

1“ u

904 p1

59 60 time '

ijm?iBglg] distribution

FIG. 11


1308

0.25 " \ -

0.15 ' 0'3

i . 0.2 Q ‘t 0-1 > , > 20 22 25 30 35 time 20 25 30 time 18 22 35 time

I I I

Z. ._""" max X. Y. 1300/’ J J ’ j J

1302__/‘ 1304

FIG. 13

[1400 [1402 [1404 Class 1 Class 2 Class 3

5 0.4 5 0.6 g L’ 0.5 0.3 2 0.2 0.2 S 0 e 2 r5: 0. a.

l__l__l_l_> l__l—l____l_> 2 6 1O 5 1O 14 1O 14 35

RlP execution time RIP execution time RIP execution time

FIG. 14


FIG. 19 * 1 tcomp [BQ5 1

g 0.4 as 0.3 .5 03 D

48 52 50 ’ RIP completion time

FIG. 20

O 25“ tgomp O

0.20 O

.2: E015 ° 5’ 2 CL0.10 °

0

0.05 °

, ‘IT I I T O I , 57 6162 6566 6970 7374 7778 81 86 90

RIP completion time

FIG. 21 * 1

0 25“ tstart [BQ9]

O

0.20 0

.é‘ E 0.15 ° 2 2 ‘l 0.10 °

0

0.05 °

0 n ? T T I T O T O t 86 9O 57 58 61 62 65 66 69 70 73 74 77 78 81

RIP start time


FIG. 22

(X) IZWrBQQJ rimiBQéi ERET*[BQ;]

2204 j 2200 j 2202 1

FIG. 23

* 1

tdept [BQ9 ]

0.4 0.3 03 Probability

50 52 56 Earliest possible departure time


FIG. 24

, r;,,,,.,[BQ;1 0.25

0.20 o 0

E5015 ° 5 0.10 °

0.05 0 O

M H’ T T T n {I n _ 57 58 61 62 65 66 69 70 73 74 77 78 81 86 90

RIP start time

FIG. 25

* 1 * 1 »< 1

lcomp 1 tsiari 1

2500 j 2502 J‘

US. Patent Feb- 28, 2012 Sheet 16 0f 18

v

Await Receipt of Raw Sheeisides \2600

V

is There a 2602 Raw Sheetside in the Head of Head Node

input Queue?

Yes 4 i

V

Determine Sheetside Deadline by Which This Sheetside Must a2604

Be RlPped for Printing

V

Determine PDf impulses of RIP Completion Time for This

Sheetside for Each Compute Node

V

Select Compute Node Having Best Probability to Complete Sheetside Before Sheetside

Deadline

“2608

l 2610

Room in Transfer Queue?

Remove Raw Sheetside from input Queue and Add Sheetside

to Transfer Queue for Transmission to Selected

Compute Node (Commence/ Continue Transmission from Transfer Queue to Associated

input Queues)

a2612

US 8,125,677 B2

FIG. 26


FIG. 27

V

Determine Probability for Each Processor to Miss Sheetside N 2700

Deadline

2708 2702 S

Select Processor Having Smallest Probability of

Missing Deadline

All Processors

have Non-Zero Probability of Missing

Deadline?

Yes No

Determine Mean RIP Completion Time Value from Impulse Values .\ 2704 for Each Processor Having Zero Probability of Missing Deadline

V

Select Processor Having Lowest Mean RIP Completion Time Value N 2706

A


FIG. 28

Processors

2800

A

i/O Program and Devices < > Data Memory

2804 M

12850 v

Presentation Device Network Interface Interface

2808 2806

Sheetside Dispatch Printing System 2801

US 8,125,677 B2 1

METHODS AND SYSTEMS FOR IMPROVED PRINTING SYSTEM SHEETSIDE DISPATCH IN A CLUSTERED PRINTER CONTROLLER

RELATED PATENTS

The invention relates commonly owned, co-pending US. patent application Ser. No. 11/469,833 by the same title and ?led 1 Sep. 2006 Which is hereby incorporated by reference and hereinafter referred to as the “sibling” patent.

BACKGROUND

1. Field of the Invention The invention relates to the ?eld of printing systems and in

particular relates to improved systems and methods for sheet side dispatch in high speed printing systems using a clustered computing printer controller.

2. Discussion of RelatedArt In high performance printing systems, Which can be con

tinuous form printing systems or cut sheet printing systems, the image marking engines apply RIPped (e.g., rasteriZed) images to continuous form paper moving through the mark ing engine at high rates of speed. Typically, pages to be imaged are combined into logical “sheetsides” that consist of one or more pages of equal length Which When laid out for printing, span the Width of the print Web. Bitmap images of each sheetside to be printed are generated (RIPped) by a printer controller coupled to the high speed printing engine. It is vital in such high performance printing systems that the printer controller generates required bitmaps rapidly enough to maintain continuous throughput of paper through the image marking engine. TWo undesirable situations can occur When sheetsides can

not be ripped fast enough to feed the printer at a speci?ed speed. First, the printer may sloW its print speed as the quan tity of ripped sheetsides ready to be printed decreases, thus causing a decrease in print throughput. This situation can happen in both continuous form and cut sheet printers. Sec ondly, in continuous form systems, the high speed marking engine may be forced to stop imprinting, stop the continuous form feed, and then restart at some later time When some predetermined quantity of ripped sheetsides is available for print. This type of event is knoWn as a “backhitch”. Not only does backhitching cause reduced print throughput, it can also result in undesirable print quality or tearing of the print Web due to the abrupt stoppage of the paper. If the print Web is torn, even more time is consumed in recovering from such an event.

In higher volume printing system environments such as high volume transaction or production printing (e.g., con sumer billing statements, payroll processing, government printing facilities, etc.) such Wasted time in a sloWer than planned print speed or a backhitch operation can represent a substantial cost to the printing environment. DoWntime in such high volume printing environments is a serious problem for Which printing system manufactures expend signi?cant engineering effort to resolve. These problems are further exacerbated in tWo sided or duplex printing operations Where the continuous form paper is fed through a ?rst image mark ing engine, physically turned over, and fed in a continuous form fashion through a second image marking engine for printing the opposing side of the medium. Stopping such printing systems and performing a backhitch operation to accurately position the paper in multiple image marking engines further complicates the problems. Further, the pro cessing Workload for the printer controller in generating bit

20

25

30

35

40

45

50

55

60

65

2 map images for duplex printing is approximately tWice that of simplex or single sided printing processing.

It is generally knoWn to provide additional computational processing poWer Within the printer controller to help assure that required bitmaps Will be ready in time for the image marking engine to avoid the need for time consuming stop and backhitch operations. One recently proposed improve ment teaches the use of a cluster computing architecture for a printer controller Wherein multiple computers/processors (“compute nodes”) are tightly coupled in a multiprocessor computing architecture. The aggregated computational pro cessing poWer of the clustered computers provides suf?cient processing capability in hopes of assuring that a next required bitmap image Will alWays be available for the image marking engines.

Despite the presence of substantial computational poWer even in a clustered computing environment, there is a need to optimiZe the scheduling dispatch of sheetside bitmap image processing (“ripping”) on the multiple compute nodes in the cluster in order to produce an e?icient and cost-effective system. Well-knoWn simplistic scheduling algorithms fail to adequately ensure that a next required bitmap Will likely be available When required by the marking engines. Use of such simplistic algorithms also typically results in the need to specify more compute nodes than Would be necessary under most circumstances, resulting in a more expensive system.

It is evident from the above discussion that a need exists for an improved method and associated systems for scheduling dispatch of sheetside bitmap image processing (e.g., RIP ping) among the plurality of processors in a multi-computer clustered print controller environment to help reduce the pos sibility of image marking engine sloWdoWn, or stoppage and backhitch.

SUMMARY

The invention solves the above and other related problems With methods and associated systems and apparatus for improved sheetside dispatching in a printer environment employing a clustered, multi-processor printer controller.

In one aspect, a method is provided for distributing sheet side processing in a cluster computing printer controller of a print system. The method includes receiving a print job com prising multiple sheetsides. For each sheetside, a probability distribution function (PDf) of a range of estimated RIP completion times is determined for each sheetside for each processor of multiple processors in the printer controller. The sheetside is then dispatched to a selected processor of the multiple processors based on the PDf such that said each sheetside has the highest probability of completing at a mini mum RIP completion time. The invention may include other exemplary embodiments

described beloW.

DESCRIPTION OF THE DRAWINGS

The same reference number represents the same element on all draWings.

FIG. 1 is a block diagram of an exemplary system embody ing features and aspects hereof to improve sheetside dispatch in a multi-processor print controller.

FIG. 2 is a block diagram shoWing exemplary buffer and queue structures used in communication among the exem plary components of FIG. 1 in accordance With features and aspects hereof.

FIG. 3 is a ?owchart describing an exemplary method in accordance With features and aspects hereof to distribute

(12) UnIted States Patent (10) Patent No.: US 8,125,677 B2 ...

Documents

Transcript of (12) UnIted States Patent (10) Patent No.: US 8,125,677 B2 ...