Dynamic Storage and Retrieval in Sparsity Programming

download Dynamic Storage and Retrieval in Sparsity Programming

of 6

description

It is shown that sparsity programming is no more than asubstitution of a higher level microcompiler for a basic microcompilerin the storage retrieval and processing involving elements of linearand multidimensional arrays. The substitution property of microcompilers permits the coding of a program first in a natural languageusing formal subscripts and then converting the conventional codinginto a sparsity coding after the conventional coding has been fully debugged. This two-stage process not only preserves coding efficiencybut also will generally shorten the overall program debugging time.It additionally provides for division of labor between the conventional coder and the sparsity coder. A formal list structuring strategywhich has built-in "garbage collection" for sparsity programming isdescribed in detail. This strategy constitutes a conversion guidefrom conventional to sparsity programming.Many industrial problems tax the memory capacity ofexisting computers. Although more and more storagefacilities become available with time, it is well beyond doubt alsothat problem sizes grow at a faster rate. For a class of problemswhose solution involves the manipulation of elements of largebut sparse arrays (such as matrices), it has been demonstrated that storage and processing of only the nonzero elements ofsuch arrays not only promotes efficient utilization of availablespace but also enhances the overall speed with which a solution.

Transcript of Dynamic Storage and Retrieval in Sparsity Programming

  • IEEE TRANSACTIONS ON POWER APPARATUS AND SYSTEMS, VOL. PAS-89, NO. 1, JANUARY 1970

    [19] C. C. Gotlieb and S. Kumar, "Semantic clustering of indexterms," J. ACM, vol. 15, pp. 493-513, October 1968.

    [20] F. Lucio and M. Sami, "On the decomposition of networks inminimally inter-connected subnetworks," presented at theIEEE Internatl. Symp. on Circuit Theory, Miami, Fla.,December 4-6, 1968.

    21] D. V. Steward, "An improved method for tearing large sys-tems," Atomic Power Equipment Dept., General Electric Co.,San Jose, Calif., 1968.

    [22] B. A. Carr6, "Solution of load-flow problems by partitioningsystems into trees," IEEE Trans. Power Apparatus and Systems,vol. PAS-87, pp. 1931-1938, November 1968.

    [23] E. C. Ogbuobiri and S. Linke, "A unified algorithm for load flowand economic dispatch in electric power systems," Proc.1967 IFAC Symp. (Haifa, Israel).

    [24] T. C. Hu, "Decomposition in traveling salesman problems,"T. J. Watson Research Center, IBM Corp., Yorktown Heights,N. Y., Research Rept. RC 1527, December 17, 1965.[25] T. C Hu, "A decomposition algorithm for shortest paths in anetwork," Mathematics Research Center, University of Wis-consin, Madison, Tech. Summary Rept. 804, September 1967.

    [26] J. D. C. Little, K. G. Murty, D. W. Sweeney, and C. Karel,"An algorithm for the traveling salesman problem," OperationsRes., vol. 11, pp. 972-989, November 1963.

    [27] J. M. Dobbie, "A survey of search theory," Operations Res.,vol. 16, pp. 525-537, May/June 1968.

    [28] M. Bellmore and G. L. Nemhauser, "The traveling salesmanproblem: a survey," Operations Res., vol. 16, pp. 538-558,May/June 1968.

    [29] H. W. Dommel and W. F. Tinney, "Optimal power flow solu-tions," IEEE Trans. Power Apparatus and Systems, vol. PAS-87, pp.1866-1876, October 1968.[30] R. Larson, C. Wells, and J. Peschon, "Efficient computation ofstate estimates using the inverse of the Jacobian matrix," WolfManagement Services, Palo Alto, Calif., Tech. Memo. 2,August 1968.

    [31] R. Bellman, "Dynamic programming treatment of the travel-ing salesman problem," J. ACM, vol. 9, January 1962.[32] C. Berge, The Theory of Graphs and Its Applications. NewYork: Wiley, 1964.

    [331 R. G. Busacker and T. Saatv, Finite Graphs and Networks.New York: McGraw-Hill, 1965.

    Dynamic Storage and Retrieval inSparsity Programming

    E. C. OGBUOBIRI, MEMBER, IEEE

    Abstract-It is shown that sparsity programming is no more than asubstitution of a higher level microcompiler for a basic microcompilerin the storage retrieval and processing involving elements of linearand multidimensional arrays. The substitution property of micro-compilers permits the coding of a program first in a natural languageusing formal subscripts and then converting the conventional codinginto a sparsity coding after the conventional coding has been fully de-bugged. This two-stage process not only preserves coding efficiencybut also will generally shorten the overall program debugging time.It additionally provides for division of labor between the conven-tional coder and the sparsity coder. A formal list structuring strategywhich has built-in "garbage collection" for sparsity programming isdescribed in detail. This strategy constitutes a conversion guidefrom conventional to sparsity programming.

    INTRODUCTION

    M ANY industrial problems tax the memory capacity ofexisting computers. Although more and more storage

    facilities become available with time, it is well beyond doubt alsothat problem sizes grow at a faster rate. For a class of problemswhose solution involves the manipulation of elements of largebut sparse arrays (such as matrices), it has been demonstrated[1] that storage and processing of only the nonzero elements ofsuch arrays not only promotes efficient utilization of availablespace but also enhances the overall speed with which a solution

    Paper 69 TP 2-PWR, recommended and approved by the PowerSystem Engineering Committee of the IEEE Power Group for pre-sentation at the IEEE PICA Conference, Denver, Colo., May 18-21,1969. Manuscript submitted January 13, 1969; made available forprinting August, 1, 1969.The author is with the Bonneville Power Adninistration, Port-

    land, Ore. 97208.

    is obtained. Thus, even when one has plenty of space for a givenclass of problems, one should also consider sparsity coding inpreference to a conventional coding as a means of reducing com-puter time.The author is not immune to the reservations of prospective

    sparsity programmers. The idea is always there. The goal isaround the corner. However the path to it is complicated. Thedifficulty with sparsity coding in scientific programming appearsto lie in the following:

    1) there is more to it than knowledge of FORTRAN language,2) the program logic is more involved, and hence more de-

    bugging time is expected.Certain applications require that the creator of a problem or

    the inventor of an algorithm play the role of a programmer andvice versa. But this is an ideal combination of talents. Sometimesthe creator knows just enough conventional programming to beable to code his problemi in a smaller scale. Would it not be help-ful to find a programmer who does not understand the totality ofthe program logic but who has the skill to expand the size of theexisting program. This would, in fact, amount to a division oflabor that does not involve any loss in efficiency.

    This paper purports to formalize sparsity programming in sucha way that the subject will become less mysterious to most con-ventional coders. By introducing and emphasizing a systematicapproach, the task of sparsity programming will be much easierboth conceptually and in practice. Although rectangular arraysare implied in the text, the extension to higher dimensionalarrays is obvious. In the interest of the majority of users, theFORTRAN language is assumed throughout the text. The works ofByrnes [2], Randell and Kuehner [3], and Jodeit [4] are veryinspiring.

    150

  • OGBUOBIRI: DYNAMIC STORAGE AND RETRIEVAL IN SPARSITY PROGRAMMING

    COLUMN 1 COLUMN 2

    A R 1 EASMiA(1, 1) A(M, 1)

    COLUMN N

    All, N) A(M,N)

    Fig. 1. FORTRAN convention for storing a rectangular array.

    END ROW 1END ROW 2 END ROW M

    o|01 2 M M+1 2M 2M+1

    STARTING DIAGONALS OFF-DIACONALS END OFADDRESSES BLOCK A

    Fig. 2. Chained-list structure for a sparse matrix.

    SPARSITYA system is sparse if the associated matrix is sparse. Consider

    a rectangular array A of dimension N X M. Let K be the numberof nonzero elements in the matrix A. The figure of merit bywhich the sparsity of A is judged is K/NM. For admittancematrices arising in electric power network, this figure of merit istypically less than 0.5 percent. This means that if one were process-ing a vector whose dimension is 200, one would only need toprocess one element of the vector each time instead of 200.Thus instead of storing 200 elements, only one is stored; on theother hand, instead of processing 200 terms (or expressions), onlyone term (or expression) need be processed, hence the savings inboth space and time. Nor is sparsity restricted to electric powernietworks. The logic modules in a large-scale digital computer aresparsely connected; so also are most communication networkssuch as connections among cities. The matrices arising in thesolution of partial differential equations by the method ofstencils are generally sparse.

    STORAGE ORGANIZATIONDefinitionA block of memory is a contiguous set of storage locations.Consider a rectangular matrix A of dimensions M X N. The

    standard FORTRAN practice is to store this matrix column bycolumn in a block of memory named A. The first column isfollowed by the second column, and so on, until the N columns areexhausted (see Fig. 1). Thus the double-subscripted variable A isstored as a single-subscripted variable. The element aij is inter-preted as A (Mj + i - M). Consequently, the expansion rule forthe ordered pair (i,j) denoting (row index, column index) is

    (i,j) = AI + i-M. (1)This rule is built into the FORTRAN compiler, and the translationis automatic whenever the ordered pair is encountered in a coding.If, in a DIMENSION statement, one had coded A (MN) in place ofA (M,N), then the compiler will be perfectly happy if within thebody of the program one codes A (MIJ + i- M) in place of A (i,j).Surely the coding A (ij) is simpler to use than the coding A (Alj+ i-M) which involves a preliminary computation. To take ad-vantage of such simplicity we naturally sacrifice flexibility.Therefore, when the matrix A is sparse and we can no longerafford to store all the zero elements, we must devise a special-purpose storage scheme for which we must also know the asso-ciated translation for the ordered pair (i,j). This is the essence ofsparsity programming.

    In many applications, it is easier and more natural for an in-put routine to build a system matrix row by row than to build the

    same matrix column by column. Consequently, this paper adoptsthe row-by-row convention.

    List Structure for a MatrixIn order to construct a list structure for storing and/or

    retrieving the elements of a matrix A of dimension M X N, wepropose the following attributes of a rectangular array:

    1) the type of the matrix,2) the number of rows in the matrix (number of columns is an

    optional attribute),3) the starting address of each row,4) the end address of each row (optional),5) the diagonal elements,6) the nonzero off diagonals stored row by row,7) other subarrays induced by the processing on the matrix,8) the first available address within the storage block reserved

    for the array.

    For matrices of dimensions higher than 2, attributes 1), 2), and5) may be modified to specify the matrix. Attribute 1) has sub-attributes: symmetry and variable type (logical, integer, real,complex, double precision, or a combination thereof). The neces-sity for attribute 4) must be dictated by the program logic. Forinstance, if the storage of a nonzero element of a row is precededby a search to determine whether that element already exists inthe row, then the end address is identically determined by thesearch. The diagonal elements form an attribute because in mostapplications these elements are all nonzero and in such ap-plications their positional relationship can be exploited to con-serve space and also expedite retrieval. Even in applicationswhere the diagonals are not a significant attribute, there is al-ways some other positional list which can be maintained in thatspace. Attribute 7) is a very important one because in certainprograms partial lists are maintained in the process of a computa-tion. The population of the active elements of these partial listsvaries from run to run, and it would be nice to make all unusedcells of these partial lists available for other use. Where it is knownthat a concatenation of such partial lists has a uniform bound,this fact should also be exploited in the specialized design of thelist structure. For instance, consider partial lists LI and L2 andsuppose that the number of active elements is, respectively, niand n2. How ni and n2 may vary in any particular run is unknownbut it is known that for run i, ni + n2 = ki. Then a vector ofdimension ki can be set aside, and the lists Li and L2 are initiatedfrom opposite ends of the subblock reserved for the vector.Attribute 8) is used to determine the address where a newelement can be stored.

    Fig. 2 illustrates a chained-list structure in which attributes 2),3), 5), 6), and 8) have been employed. The ith cell in the sub-block of starting addresses points to the address within theblock named A, at which the list of attributes of the first elementof the ith row starts. The last attribute of an element is a pointerthat tells where the list of attributes of the next element in therow starts. For the last element in a row, the last attribute iszero. One should note, in this example, that there are exactlythree subblocks of A corresponding to the attributes 3), 5), and6).The attributes of an element of a matrix, within a given row,

    are

    1) column index c,2) value v of the element (optional),3) address p of the next element in the row.

    151

  • IEEE TRANSACTIONS ON POWER APPARATUS AND SYSTEMS, JANUARY 1970

    POINTERTO

    ELEMENT

    POINTER TONEXT

    ELEMENTI cjI V0I o I

    POINTERTO

    ELEMENT

    (b)Fig. 3. Format for element storage. (a) Nonterminal element

    of row. (b) Terminal element of row.

    Thus the ordered triplet (c,v,p) fully describes an- element in agiven row. Clearly, if the value is logical (0 or 1), the attributescan be reduced to the ordered pair (c,p) which means that thepresence of a 1 in a column is indicated by the entry c. If thematrix is type complex, then the attribute v calls for two succes-sive attributes vr and vu representing the real and imaginary partsof v. In this case, an element within a given row is described bythe quadruplet (c,v,,v1,p). That an element terminates a givenrow list is indicated by assigning the value zero to p. For sim-plicity and without loss in generality we suppose that the ele-ments are real numbers. Fig. 3 illustrates the format for the stor-age of an element whose value is neither complex nor logical.

    STORAGE, RETRIEVAL, AND GARBAGE COLLECTIONStorage at Program Initialization Time

    Using the basic list structure described in the preceding sec-tion, the initialization of a matrix A of dimension M X N at in-put time is accomplished by the following procedure.

    1) Store the starting address of row i in A (i).2) Store the diagonal element of row i in A (N+i).3) Store the column index of the first off-diagonal element of

    row i at the starting address of that row. Then store the value ofthe element in the next location(s). Finally, store the address ofthe cell which is two locations ahead in the next location; afterthis is done the next cell becomes the starting address for theattributes of the next element to be stored. For the last elementof the row, follow the recommended format which is illustrated inFig. (3).

    Available Address and List ModificationIf, after the initialization of an array, it is found that an ele-

    ment must be added to a given row, then this modification can beaccomplished by storing the new element in an available spaceand chaining it to that row to which it belongs. Let p be an avail-able address in a block named A. Let (cl,vi,0) be the terminalelement of a row to which we wish to add a new element (c2,v2,0).Let q be the terminal address of the row. Fig. 4 shows how thechaining of the new element is accomplished.By the storage organization described and the chaining pro-

    cedure, it is evident that the list of a given row can start any-where within the storage block and end anywhere within thesame block.

    Retrieval of an ElementIn order to retrieve a given element of a working array A, the

    following procedure may be employed:

    -1 I

    q p

    (a)

    C VI P I C2I V21q p

    (b)Fig. 4. List modification. (a) List before addition of new element.

    (b) List after addition of new element.

    I l I NAA IFAA

    Fig. 5. Chaining format for available addresses.

    I c v I Pq

    | NAAFAA

    (aI)

    (a)

    NAA I|FAA

    NAA

    ZZIIIlZIEl(b)

    Fig. 6. Inclusion of new available address q in list of availableaddresses. (a) Stack before chaining. (b) Stack after chaining.

    FAA

    NAA

    (aI)I(a)

    I c v o

    FAA

    NAA

    (b)Fig. 7. Update of list of available addresses when new element is

    created. (a) Before adding new element (C,V,O). (b) After addingnew element (C,V,O).

    1) Determine the row in which the element belongs; let it berow i.

    2) Obtain the starting address of the row from A (i).3) Determine the column in which the element belongs.4) Search through the chained list of row i until an element is

    found whose column attribute is the one desired; the value ofthis element can then be fetched.

    Garbage CollectionThere are some applications in which nonzero elements of an

    array become zero in the course of program execution. A pro-cedure by which the space occupied by these elements can be re-used as soon as they become available is referred to as condensa-tion or garbage collection. The essence of condensation is to moveactive elements to one side of the storage block and the availableaddresses to the other side.

    152

    l l

  • OGBUOBIRI: DYNAMIC STORAGE AND RETRIEVAL IN SPARSITY PROGRAMMING

    Fig. 8. Memory budget table for square arrays storedaccording to Fig. 2.

    One technique for effecting automatic garbage collection is bymaintaining a chained list of discontinuous available addresses.The discipline to be used is a last-in-first-out (LIFO) discipline.This is achieved by placing all available addresses in a push-down stack, and whenever a new element is to be stored, theavailable address on top is selected. Let the address on top of thestack be defined as the first available address (FAA) and the oneafter be next available address (NAA). Let an element be de-scribed by the ordered triplet (c,v,p). We can again use the cellscorresponding to the p attributes of active cells for chaining theavailable cells. Fig. 5 illustrates the chaining format.

    Fig. 6 illustrates the effect of the creation of a new availableaddress while Fig. 7 shows the effect of the creation of a new non-zero element.

    CENTRAL MEMORY BUDGETThe central memory budget can be exactly determined as soon

    as the storage organization is designed. Consider the storageorganization of Fig. 2 for a square matrix of dimension N. Let nbe the total number of off-diagonal elements of the matrix. Thestarting addresses of the rows occupy N cells. Each diagonal ele-ment occupies one or two cells according to whether the matrixis real or complex. For each off-diagonal element, three or fourcells are required according to whether the matrix is real orcomplex. For a symmetric matrix, only one half of the off-diagonal elements need be stored. Thus one arrives at the follow-ing formulas:

    unsymmetric matrixa) the total memory space for the real case is 2N + 3nb) the total memory space for the complex case is 3N + 4n

    symmetric matrixc) the total memory space for the real case is 2N + 1.5nd) the total memory space for the complex case is 3N + 2n.

    If we let k be the average number of off-diagonal elements perrow, then n = kN. Let CM denote the total central memory re-quirement. With these definitions, the formulas a)-d) willbecome

    a) CM = (3k + 2)Nb) CM = (4k + 3)Nc) CM = (1.5k + 1)N (2)d) CM = (2k + 3)N.

    Fig. 8 shows the values of CM for typical value of N and k ofrectangular arrays. For any value of N, the value of CM can becomputed from Fig. 8 by multiplying the value of CM correspond-ing to the given k by N/1000.

    For binary matrices such as incidence matrices, further com-pactification of storage is possible if the (c,p) attribute of eachelement in a row is binary coded so that one or more elementattributes can be packed into one full word. However, the re-trieval costs for such packing are unduly high in high-levellanguages like FORTRAN. In basic machine languages, this reduc-tion in storage space can be achieved without any sacrifice in com-putation time.

    PROGRAMMING STRATEGYThe approach to sparsity programming should be a two-stage

    process; the first stage is a conventional programming usingformal subscripts while the second stage is a program conversion.

    Conventional Programming Stage in FortranAt the conventional programming stage, a conventional coding

    should be carried out by assigning variable names to each matrixor vector used in the program and employing regular subscriptedvariables in the progrem coding. For efficiency in coding full ad-vantage should be taken of any inherent symmetry or equiv-alences in the working arrays and in the operations of the prob-lem program. However, for maximum facility in debugging, noadvantage should be taken of sparsity, and any symmetry inworking arrays should only be reflected in the program logicand not in the storage organization for those arrays. Since sym-metry in working arrays and equivalence among operations arepart of the formulation of the problem-solution algorithm, thereshould be no confusion about the extent of the conventionalprogrammer's responsibility. For instance, if a matrix is squareand symmetric, the matrix should be stored in its entirety whilethe indexing on the matrix elements references only one half ofthe matrix. In this case a COMMENT statement should be used tonotify the--sparsity programmer about such -symmetry. Othernotices to the sparsity programmer include vectors whose lengthsdepend on the actual dimension of a problem, vectors whoselengths are invariant, pairs of vectors whose combined length isbounded and which are simultaneously active, as well as pairs ofarrays which are never simultaneously active.

    Program Conversion Stage in FortranThe Sparsity programmer is responsible for the program con-

    version stage. The function of this type of programmer is toconvert the program in such a way that large sparse matrices canbe handled. Input material and information to the sparsity pro-grammer or coder include:

    1) a fully debugged efficient conventional program,2) array symmetries,3) vectors whose lengths depend on the actual problem dimen-

    sions at run time (the constant of proportionality should begiven if known),

    4) vectors whose lengths do not change with problems,5) pairs of simultaneously active vectors in class 3) whose com-

    bined length is bounded,6) pairs of arrays which are never simultaneously active.Using the preceding information, the sparsity coder designs the

    storage organization for all arrays. (The list structure should besimilar to that of Fig. 2, but the number of subblocks may varyfrom one problem program to another.) With the aid of the design,a memory budget is made, and the budget may guide a designmodification if necessary. After the design of storage organization,the program conversion is then carried out routinely on aninterpretive basis.

    N K SYMMETRIC UNSYMMETRICREAL COMPLEX REAL COMPLEX

    c) d) ) b)1000 1.5 4250 6000 6500 9000

    1000 2.0 5000 7000 8000 11000

    1000 2.5 5750 8000 9500 13000

    1000 3.0 6500 9000 11000 15000

    1000 3.5 7250 10000 12500 17000

    1000 4.0 8000 1 1000 14000 19000

    153

  • IEEE TRANSACTIONS ON POWER APPARATUS AND SYSTEMS, JANUARY 1970

    OFFD

    2Nt1 20000

    C MAIN PROGRAMCOMMON / BLOCK / A (20000)COMMON / BLANK / ISA, IDIAG, IOFFO

    C

    ISA- iIDIAG - N #IIOfFD: 2 N.1

    C

    CALL SOLVE IA)STOPENDSUBROUTINE SOLVE (B)DIMENSION B(i)COMMON / BLANK / ISA, IDIAG, IOFFD

    RETURNEND

    (b)Fig. 9. Block design and base address initialization. (a) Block A

    of dimension 20 000 and with subblocks SA, DIAG, and OFFD.(b) Sample coding to illustrate modularity and relocatability.

    Program Conversion Guide1) Array Declaration: All arrays whose size depend on the actual

    problem dimensions should be assigned to one block with possiblymany subblocks. The block is dimensioned in the main programin accordance with the memory budget. If the program consistsof a large number of subroutines, the block name should be aformal parameter in subroutine calls. This means that wheneverthere is a change in the memory budget for some reason, only theDIMENSION statement in the main program need be changed (Fig.9).

    2) Modularity and Relocatability of Subblocks: Since the baseaddress of the subblocks is in general a function of the problemsize at run time, the base address of each subblock should becomputed and placed in COMMON storage. This means that thecreation and deletion of subblocks do not result in excessive re-coding within the body of the program (Fig. 9).

    3) Definition of an Array Element: The conventional codingA (I,J) = X states that the value of X should be stored in thelocation of the element a,j of the A matrix. In statements likethis, we should first determine whether the element aij has beenpreviously defined; if not we should chain the element (J,X,0) tothe list of the Ith row, provided X 5z# 0; if it has been defined as(J,Y,P), then we simply alter y to x to obtain (J,X,P). If I = J,we simply code A (IDIAG + I - 1) = X, with IDIAG as the baseaddress for the diagonal subblock. If X = 0, the element (J, Y,P)is deleted from the list of row I.

    4) Retrieval ofan Array Element: The conventional codingX =A (I,J) states that the value of A (I,J) should be stored in thelocation named X. The first step is to search down the Ith rowof A and find an element with attributes (J,-, ). The valueattribute of this element is then stored in X. If there is no suchelement in the row list, then X assumes the value zero.

    5) Do-Loop with Element Definition: The conventional codingDO label 1 I = ml,M2,M2DO label 2 J = ni,n2,n3A(I,J) = X

    label 2 CONTINUElabel 1 CONTINUE

    calls for a direct translation, in accordance with 3), of the state-ment A (I,J) = X.

    6) Do-Loop with Element Retrieval: The conventional codingDO label 1 I = MIM2,M3DO label 2 J = ni,n2,n3X = A(I,J)

    label 2 CONTINUElabel 1 CONTINUE

    calls for the following procedure:a) the row-indexing DO should be retained,b) the column-indexing DO should be removed.c) each element of row I is checked for the admissible column

    index; if the column index is admissible, X is assigned thevalue of A (I,J), if not X is assigned the value of zero.

    7) Logical Statements: The conventional coding "IF (expression)labels" where the expression contains A (I,J) as an argument mustbe treated as the retrieval statement of 4) and 6). For example,"IF(A(I,J))1,2,3" should be treated as two consecutive state-ments:

    X = A (I,J)IF(X)1,2,3.

    GENERALIZATIONThe basic building block for storage-organization design is the

    vector. There is, therefore, no question regarding the manner inwhich extension can be made to higher dimensional arrays thatare sparse. Such higher dimensional arrays are simply a vectorof vectors which can be represented in a tree structure. However,a sparsity program must make a distinction between two types ofvectors: there is the ordinary list in which each member has nopositional attribute; there is also a list of elements in which eachmember has a positional attribute. The attributes of the ordinarylist are the base address of its subblock and the number of ele-ments in the list. In this form the elements of the list can bestored contiguously so as to save the space normally occupied bypositional indices. An example of an indexed list is the list of non-zero elements of a row (or column) of a matrix. An example of anonindexed list is a list of the rows of a matrix which are equal.Whenever possible nonindexed lists must be stored without ex-plicit chaining.

    PROGRAMMING EXPERIENCEThe foregoing procedure for sparsity programming has been

    applied to a FORTRAN program containing over ten pages of con-ventional FORTRAN coding and eight subroutines. A successfulconversion took 8 man hours. In this experiment, the conven-tional coder and the sparsity coder are one and the same person.It was recognized

    1) that the debugging time for the conventional coding wasshort because of the simplicity in coding provided by the auto-matic array handling (by FORTRAN compiler) via the formal sub-scripts,

    2) that the speed of program conversion from a conventionalcoding to a sparsity coding is not affected by the division of laborbetween two individuals by virtue of the fact that the conversionprocedure is fairly routine,

    3) that the efficiency of the sparsity coding in terms ofsparsity conservation is a function of the ability of the con-ventional coder to provide all of the necessary interface informa-tion described in this text; the loss in efficiency is, however, notcritical.

    4) the routine conversion procedure is amenable to automa-tion via software implementation.

    SA

    1 N h

    DIAG

    N+1 2 N

    (a)

    154

  • OGBUOBIRI: DYNAMIC STORAGE AND RETRIEVAL IN SPARSITY PROGRAMMING

    CONCLUSIONIn outlining the essence of sparsity programming in a problem-

    independent context, the complete separability between thefunctions of a conventional coder and those of a sparsity coderhas been demonstrated and emphasized. The need for this sep-arability of tasks becomes more evident in large complex pro-grams and is not diminished by the fact that the two coders maybe one and the same person since the overall programmingefficiency and debugging speed are enhanced by the staging of thetwo tasks. The method of storage and retrieval of array elementshas been formalized in a manner providing automatic garbagecollection and a rule-of-thumb calculation of core-storage re-quirements; sample formulas and memory budget tables aregiven. The examples of the recompiling techniques for the keyconventional codings constitute a program conversion guide;this conversion guide lends itself to a software implementa-tion.

    REFERENCES[1] W. F. Tinney and C. E. Hart, "Power flow solution by Newton's

    method," IEEE Trans. Power Apparatus and Systems, vol.PAS-86, pp. 1449-1460, November 1967.[2] F. D. Byrnes, "A program for direct solution of matrix equa-tions," Bonneville Power Admin., Portland, Ore., Rept., June1967.

    (3] B. Randell and C. J. Kuehner, "Dynamic storage allocationsystems," Commun. ACM, vol. 11, pp. 297-306, May 1968.[41 J. G. Jodeit, "Storage organization in programming systems,"Commun. ACM, vol. 11, pp. 741-746, November 1968.

    Discussion

    Jen Hsu (P.O. Box 361, Berkeley, Calif. 94701): The paper essentiallypointed out the utilization of linked lists to minimize storage forsparsity matrices. Linked lists, a well-known method in programmingtechnique [5], are composed of items which do not usually occupycontiguous locations in memory. Therefore, an item contains second-ary information, i.e., a pointer to where the next item in the chain islocated.For more effective programming, I would suggest the use of a

    circular list instead of the linear list which the author described. Weknow that a circular list can be used not only to represent an in-herently circular structure but also to represent a linear structure,i.e., a circular list with one pointer to the rear node. Although a cir-cular list is essentially equivalent to a straight linear list with two

    pointers to the front and rear, it will be possible to have access to allof the list starting at any given point, and it will be more effective tomove the leftmost to the right [5].

    REFERENCES[5] W. D. Maurser, Programming, Introduction to Computer Lauguages

    and Techniques. San Francisco: Holden-Day, 1968.

    E. C. Ogbuobiri: The answer to Mr. Hsu's remarks is in the paper it-self. However, the following additional comments are offered.The paper claims that there is a simple methodology for sparsity

    programming and describes this methodology in detail. It is empha-sized, even in the abstract, that this methodology involves no morethan a replacement of the FORTRAN storage convention for multi-dimensional arrays by special-purpose storage organizations. For aclass of problems, Gaussian elimination, for instance, the storage or-ganization must be designed prior to translation into sparsity code onlyafter the conventional code, which assumes arrays with no sparsity,is fully debugged. A concrete general-purpose demonstration is givenfor the simple problem of storage and retrieval of elements of realunsymmetric arrays together with a method for garbage collection.The paper also claims that the two-stage approach described fostersdivision of labor and also enhances overall gains in programmingefficiency.The design of storage organization-takes into account Mr. Hsu's

    observation. An array is stored as a tree of lists. The terminal listsare linear. Hence a vector becomes the building block of an array.A vector together with the use (and the sequence of creation and ex-tinction) of its elements in the course of computation determine themost efficient list structure. While some I/O buffer may lend itself toa circular list, most storage and retrieval problems of numericalcomputation exploiting sparsity do not. While, in Gaussian elimina-tion, implicit chaining of row elements is always possible, rows ofcertain incidence matrices whose row elements arrive randomly atinput time require explicit chaining. Thus there is room, in eachproblem, to optimize the storage organization. However, if the pro-grammer were simply to implement a general-purpose storage andretrieval scheme for multidimensional arrays without due regard tonumerical operations-on the array elements, as is done in IBM Sys-tem/360 MATLAN (Qctober 1968 edition) he will still achieve overallgains in both storage space and computation time for reasons giveinin the paper; the bottleneck Droblem of memory requirements will atleast be solved. He can always improve on the gains by improvedstorage organization that takes into account the process of computa-tions to which an array is subjected.

    It should be self-evident that this author is not selling a particularlist structure (since the most efficient list can be found for each specificapplication) but a methodology which is an outgrowth of his experi-ence in sparsity programming. If any user understands -and -appliesthis methodology in his problems, his sparsity code and program-ming effort in a given computer language can, in that language,only be equaled but not excelled.

    Manuscript received by June 27, 1969.

    155

    Manuscript received June 12, 1969.