The SNAME OpenCalc System · 2019-07-06 · The SNAME OpenCalc System Stephen M. Hollister (M) New...

The SNAME OpenCalc System

Stephen M. Hollister (M)

New Wave Systems, Inc., Jamestown, Rhode Island

The SNAME OpenCalc System is built upon an open source framework that allows for the creation of long-lasting

and reusable calculations that can be combined in many ways to create completely new solutions by naval architects,

not programmers. Flexible calculation tools are created by splitting traditional interactive program applications

(“apps”) into three separately developed and tested objects: calculation engine (CE) batch programs, open source

and industry standard (XML) data files, and open source user interface frameworks (UIF) that work with any CE

without programming. A working prototype system has gone through two revisions and a third is now presented as a

formal release. The SNAME OpenCalc System offers a new programming structure that returns creative control and

flexibility to subject matter experts (SMEs) and users. It can create solutions not possible without access to all source

code and expensive custom programming. This is critically important in fields like naval architecture, where small

market size and demand have not created lasting, flexible, and cost-effective computer solutions and design tools.

KEY WORDS: Programming; XML; CAD; design; calculations; optimization.

NOMENCLATURE

Batch Programs - Computer programs that do not have any user

interface. They can be run automatically.

CAD - Computer Aided Design - General purpose interactive

programs for geometry design of any object or structure.

CE - Calc Engine - TSPA/OpenCalc component that isolates

calculations into separate batch programs.

CFD - Computational Fluid Dynamics - A class of programs that use

meshes to perform detailed fluid flow calculations.

Command File - A text batch or command file with commands that an

operating system understands and can perform. They are often called

BAT or COM files.

CSV - Comma Separated Value File - A data file storage format used

by spreadsheets to transfer worksheet data to other applications.

DLL - Dynamic Link Library - A method for combining separate

library routines with programs at run time.

DOS - Disk Operating System - The operating system for IBM PC

compatible computers in the 1980s and 1990s.

EXE - Executable File - A file type that defines a program that can be

launched by an operating system.

FEM - Finite Element Method - A class of programs that use meshes

to perform detailed structural stress, strain, and deformation analyses.

GUI - Graphical User Interface - A front end to a program that offers

users interactive and graphical control, as in a CAD program.

HTML - Hypertext Markup Language - Tag-based text file

commands and data for display of information on web pages.

ISO - International Standards Organization - World-wide

organization that defines and promotes common standards for all.

Hollister The SNAME OpenCalc System 2

SME - Subject Matter Expert - A person who has expertise in a

subject like naval architecture, but limited expertise in computer

science.

SVG - Scalable Vector Graphics - An XML standard for storing

vector graphics geometry and display information.

TSPA - Tri-Split Programming Architecture - Divides a program

into three separate parts: User Interface Frameworks, Calc Engines, and

XML text file data for any variable or data structure. OpenCalc is built

on this open architecture.

UI - User Interface - Any interactive front end to a program that offers

direct user control.

UIF - User Interface Framework - TSPA/OpenCalc component that

refers to a user interface that can work with any calc engine.

Spreadsheets are the main vehicle used in this work for UIFs.

Unix - A multitasking, multiuser operating system developed in the

1970s at Bell Labs.

Unix Pipes and Filters - A technique for combining text files and

batch text processing programs in an automated process command file.

VBA - Visual Basic for Applications - Programming language for the

Excel spreadsheet.

W3C - World Wide Web Consortium - The main international

standards organization for the World Wide Web.

XML - Extensible Markup Language - An ISO standard for storing

tagged data in text files. It also refers to the XML data portion of

TSPA/OpenCalc.

INTRODUCTION

Time, changing technology, and a small market have resulted in

a lack of marine design and analysis computer software for

naval architects, yacht designers, students, and hobbyists. The

SNAME OpenCalc System establishes a website, open

technology, and a process within SNAME that will provide

members and others with a computer program design and

development framework where everyone can offer pieces that

support compatible and flexible solutions for all areas of the

design process. This system is more than just a way to collect

and offer "lost" or basic design programs. It provides both

technical tools and organizational guidance that fix the issues of

time, technology, cost, and market, and is a scalable and

agnostic framework that will help everyone from students to

naval architects to large shipbuilders.

OpenCalc is not about completely free software, although it

comes seeded with a few reliable and validated calculations

such as hydrostatics and stability. Instead, it is a free and open

framework of tools for developing low cost, lasting, and

compatible calculations that can be combined by users to create

new solutions without programming. It can also be a valuable

tool for independent software developers by lowering costs and

increasing market size. It is a simple process to isolate

calculations and insert them into one of the provided open

source programming templates for calculations (calc engines).

OpenCalc provides all the file I/O routines.

The key organizational glue to this system is the definition of a

simple and flexible variable and geometry data file definition,

especially for the conceptual and preliminary design stages.

Development will be an ongoing process, but the integration of

common and lasting computer design and analysis programming

tools from a community of independent participants rests on this

task. After more than 50 years of marine computing

development, there is not even one common data definition for

hull station offsets with open source I/O code that allows

everyone to easily work together to create compatible tools.

The technological solution offered by this system is one that

separates naval architecture programming knowledge from the

complex and fast-changing world of writing modern graphical

user interfaces (GUIs) - even spreadsheet programming. Too

many programs are lost, as in the change from DOS to

Windows, or not even created because of time, cost, and

expertise. The framework provided by this SNAME system

offers an easier way for naval architects to create their own

calculations in a general form that can be used for many

purposes. This is done using spreadsheets as the common, but

not only, User Interface Framework (UIF) interface for

launching independent calc engines.

Technology has changed over the decades to where PCs are now

more than 10,000 times faster than the VAX 11/780 of 1978,

and even the cheapest PC’s have 4 GB of memory and a

terabyte of storage - and some PCs have 16 GB of fast memory

and a terabyte of SSD storage. Reading/writing from/to a file in

storage has little delay. When a UIF front end transfers data to a

text file and then launches an external calc engine, the delay is

unnoticeable. It became apparent that there is no reason to bind

calculations and data processing routines to any user interface.

There is also no reason for independent software developers to

define proprietary and binary data file definitions. They need to

be open and coordinated by industry groups to allow compatible

tools to be developed by many users for all types and sizes of

vessels. The solution is not to separate and isolate all users into

competing suites of computer programs, especially in the

preliminary design stage which requires much more

coordination and cooperation between independent naval

architects, businesses, and approval organizations.

OpenCalc defines and separates the role of subject experts to

better match their importance in many smaller market segments.

When subject experts become the customer in the development

of large computer solutions, they end up losing a lot of control

and flexibility. They become simple “users” of fixed solutions,

not creators of new ones. TSPA/OpenCalc changes that.

The OpenCalc initiative and process is a new and unique

solution to difficulties in the programming world that has

enormous and far-reaching benefits for many industries. By

separating the user interface from the calculations, one not only

eliminates the cost and complexity of GUI programming, but

opens up the possibility of creating and sharing complex user-

created combinations of calculations and optimizations. The

SNAME OpenCalc System provides open source spreadsheet

code that can launch any independent calc engine (or sequence

of calc engines) without writing custom or macro code. The

calculations are completely separated from the spreadsheet and

can then be used for many other interactive or automated tasks.


HISTORY OF OPENCALC The SNAME OpenCalc System started in 2014 as "Project 114"

to try to save or restructure old code for boat and ship design to

create more compatible, flexible and lasting tools. This has been

an on-going problem since the beginning of the computer

industry. Mainframe computers gave way to minicomputers,

workstations, and PCs, and operating systems changed from

DOS to Windows (and Apple OS to MacOS) to web-based

systems. Each time, applications had to change and be rebuilt at

high cost. The maritime industry can’t support those changes.

Calculations might stay the same, but the cost of rebuilding a

new application with a new user interface was and is too

expensive, thereby losing much good software. The author, for

instance, has old DOS-based code, such as a Wageningen

propeller optimization program and a Velocity Prediction

Program for sailboats, that never justified conversion to

Windows-based user interfaces.

Excel and other spreadsheet programs are not a full solution

even if one imposes format and data standards. Calculations are

mixed in directly with the user interface, difficult to change and

test, poorly supported, and cannot be easily used for other

purposes. There is now a desire to be able to combine and

sequence different calculations automatically to search

thousands or millions of iterations for an optimum solution.

While it may be possible to write a custom program to do this,

it’s extraordinarily expensive and one needs all the source code.

For example, in a SNAME New England Section paper

(Hollister 1996), this author combined hull variation,

hydrostatics, and Holtrop resistance calculations into one DOS

program to search for an optimum shape starting from a parent

hull. This required all the source code and a custom user

interface front end that needed a lot of development and testing.

That this could be done was of no real benefit since the market

would not support the cost. Also, that was when DOS was

replaced by Windows and the recoding cost was no longer

viable for the new technology and size of market.

Twenty years later, the author completed another attempt at the

task using the separate component framework of OpenCalc.

That paper was presented at the 2016 SNAME Maritime

Convention in Seattle (Hollister 2016). Each calculation: hull

variation, hydrostatics, and Holtrop resistance became a

separately developed and tested stand-alone EXE batch (no user

interface) calc engine program that read and wrote a standard

text file of data using open source I/O code. These separate

programs were then “wrapped” in an automatic command file

and launched by the open source spreadsheet (UIF) that can

execute any separate calculation program or sequence of

programs. The spreadsheet macro code launches these

calculations automatically and produces displayed, printed, and

plotted output with no extra programming. The spreadsheet

launches the calc engines hidden in the background and it

appears to the user that those calculations are done inside of the

spreadsheet. There is no delay.

With OpenCalc, one can write and test a separate program (calc

engine) for a set of calculations in any programming language

that can have a long life and be used for many different and new

solutions by users, not programmers. In most cases, there is no

need to write a standard user interface because the open source

Excel spreadsheet Visual Basic for Applications (VBA) code

will do the job. The calc engines can be located anywhere on the

web and be copy protected and fee-based if desired. Spreadsheet

VBA code and formulas are difficult or impossible to hide or

protect, but calc engines offer a way to separate and protect

proprietary methods.

Several years ago, the author received a call from a naval

architect who wrote hull generation code for the Pro/E CAD

program and wanted to automate the connection to a meshing

program and then to a computational fluid dynamics (CFD)

program in the hopes of searching for an optimal hull shape. In

theory, this is technically possible, but the custom programming

cost could not be justified even if one could get access to all the

source code or automated batch calculations. TSPA/OpenCalc

now makes that solution possible technically and financially.

The designer would create a stand-alone calc engine of his hull

shape generator and combine it with a meshing calc engine

followed by a CFD calc engine. They could be launched by an

open source front end that searched for an optimal solution.

OpenCalc is based on the idea that many interactive user

interfaces are standardized and can be written to work with any

number of independent and automated batch calculations

without writing any custom code. The user interface for the

sequence of separate hull variation, hydrostatics, and Holtrop

resistance calculation engines was provided by an open source

Excel front end that launched the three independent calculation

programs without having the source code or writing new code.

This cannot be done in any way using traditional interactive

apps.

TRI-SPLIT PROGRAMMING ARCHITECURE The latest revision of OpenCalc generalizes the definition of

calc engines as external stand-alone executable subroutines that

are more stable, adaptable, and flexible, and it formalizes the

definition of XML data and structures into more powerful and

flexible hierarchical industry standards.

OpenCalc has evolved into a solution that goes far beyond

standard marine industry calculations. It can form the basis for

complex and flexible Multi-Disciplinary Engineering and

Optimization (MDE/MDO) solutions that can combine and

automate geometry manipulations, calculations, meshing, and

CFD/FEM calculations.

The splitting of interactive applications into three components

or programming objects causes a fundamental change in all

areas of programming. That is why OpenCalc is now divided

into two levels - a base open source structure called Tri-Split

Programming Architecture (TSPA) (www.trisplit.com ) that is

http://www.trisplit.com/


industry independent, and on top of that, industry specific levels

to define their own XML data definitions and calc engines.

SNAME’s OpenCalc system for the marine industry is built

using the base tools and open source structure of TSPA. There

are also cross-industry calculations and data definitions for

geometry and basic engineering data. TSPA allows all

programming objects to work together and individual industries

can build and control their own product definitions on top of

cross-industry and common definitions for basic geometric

shapes. Naval architecture programming tools can and should be

compatible with tools from associated engineering fields.

PROGRAMMING ARCHITECTURE The traditional programming model in computer science is the

interactive app where one software developer or team creates

the user interface, calculations, and the data definition and file

format. Separate calculations from multiple apps cannot be

automated and the data is often incompatible and must go

through a two-stage filter process to and from a neutral file

format (Fig. 1)

Fig. 1 Traditional Programming Architecture

TSPA splits this structure into three separate “external”

programming objects: open source User Interface Frameworks

(UIF), Calculation “engines” (CE), and open source hierarchical

levels of cross-industry and specific-industry defined variables

and data structures (XML) that can be developed and tested

separately by subject matter experts. (Fig. 2)

Fig. 2 Tri-Split Programming Architecture

These new external programming objects can be mixed and

matched in many new and creative ways by users. This will be

explained in more detail later in the paper.

Fig. 3 New Tri-Split External Programming Objects

As illustrated in Fig. 3, not only can one calc engine have more

than one user interface, but one user interface can launch many

different calc engines. The glue holding these pieces together is

industry-defined data and structures built using open source

XML files.

The fundamental change defined by this work involves splitting

traditional interactive “apps” into three separately developed

and tested parts. Calculations and data processing tools can be

written separately by subject matter experts (SMEs), tested

once, have long lives, and be used and reused for many

interactive and automatic applications.

OpenCalc has shown an example where a general Excel

spreadsheet using open source VBA code can launch any

external calc engine. The spreadsheet will prompt for all input,

launch the external CE or sequenced CEs, read the results back

in, and then display, print, and graph the results. Tests have


shown this to work with a variety of CEs without any changes to

the spreadsheet UIF or the calc engines.

The spreadsheet launches the CEs in the background and the

user never knows that the calculations are done externally from

the spreadsheet. There is no delay. Results are displayed

immediately even for a CE as large as a full damaged stability

calculation. The benefit is that the user can now add in

additional custom calculations and save the changes to a new

spreadsheet. This also means that the separate calculation can be

used in combination with other CEs to solve completely new

problems without writing additional code. A key element in this

process is the use of common data and data structure formats or

schemas using XML. Industries have to take control over “their”

data to enable compatible and low cost tools.

The next sections discuss each of these three external TSPA

objects in more detail.

CALC ENGINES (CE)

The first TSPA object type is a "Calc Engine" (CE) executable

batch program that has no user interface. It's a stand-alone

executable program that reads a text file of input, processes it,

and writes the results out to the same or a different XML text

file. (Fig. 4)

Fig 4 Separate calc engine model that reads and writes files

Calc engines can be written in any computer language, validated

separately by subject experts, used for many purposes, located

anywhere on the web, and have a long life not affected by

computer technology changes. It's just a simple EXE (or other

type of executable program) batch file that reads and writes a

text file and acts like a subroutine - one that is tested and

validated and can be used automatically in many different ways.

The mental shift required is to see that computers are so fast that

automatic launching of external calculations can be hidden with

no delay in processing time.

As described in the next section, separate and open user

interface programs can be constructed to launch any calc engine

automatically. This calc engine "object" is like well-known

Unix filters such as "grep" (A batch program to search for a

string in a text file.) except that the calculations can be as

complex as computational fluid dynamics (CFD) analyses and

support any program variables and data structures - not just

strings in text files. Also, calc engines do not just filter data out

from one text file to another. They treat the text files as random-

access databases of any program variable or data structure.

To turn separate calculations into reusable stand-alone tools for

any data, TSPA requires that all CEs include an XML text file

that defines all input and output data (like a subroutine argument

list) using the TSPA XML schema layer for any variable and

data structure. Figure 5 shows a TSPA/XML subroutine

argument list definition for a simple CE that adds variable ‘A’ to

variable ‘B’ to produce variable ‘C’. The UIF spreadsheet reads

this definition to prompt for all input and show all output

without knowing any more about the external calc engine.

Fig 5 XML CE/subroutine definition file

Batch CE programs can be launched from inside other programs

and passed “arguments” just like a subroutine. For the case of

this Add2.exe calc engine, the string that would start the batch

program looks like this:

Add2 (A,B,C) [io=MyDataFile.xml]

The open source I/O code of TSPA/OpenCalc reads and parses

the Add2 arguments to know what variables to access in the xml

file. Also, like standard subroutines, arguments are passed by

position, so someone could launch Add2 with “(X,Y,Z)” and the

code would read the input variables (X,Y) from the file

(MyDataFile.xml) and assign them to “A” and “B” in the calc

engine. Writing out the “C” value would go into the “Z”

variable in the file.

If the argument list becomes long, which is not uncommon, it

can be put into a text file and referred to like this:

Add2 [args=add2args.txt io=MyDataFile.xml]

If no argument list is given, the CE will use its own variable

names to access data in the XML data file. This might be useful

for carefully designed and coordinated CEs.

Eventually, one will be allowed to use expressions in an

argument list as in most programming languages. In addition, if

an output argument is left blank, then the CE will not write that

variable to the output XML file. This allows one to pick out

only those results needed without cluttering up the data XML

file.

A traditional Unix filter is a calc engine that reads a simple text

file for input and writes a simple text file for output. A Unix

filter can be turned into a TSPA calc engine by providing an

XML definition file that defines the two I/O variables as strings.


Unix text filters are just one small case of TSPA and Unix filters

can be used with any TSPA object.

TSPA allows calc engines that define any complex sequence of

input and output variables and structures equivalent to any

programming subroutine. A TSPA calc engine is an external

executable batch program that works just like an internal

subroutine or function with the XML data file acting as

memory. However, calc engines will typically perform more

complex tasks than simple calculations. With modern computer

speeds and fast static data file storage, the overhead is minimal

and more flexible compared with linking to a dynamic link

library (DLL). The benefits of unlinking, open and flexible

definitions, separate development and validation, and generality

offer many more possibilities.

TSPA provides open source tools, templates, and data schemas

to create, read, and write any program variable and data

structure. To create a new CE, a subject expert opens a TSPA

calc engine program template that includes all the necessary I/O

code, adds in their calculation or processing routines, compiles

and links the code, and then defines the input/output XML text

file subroutine argument list. It can then be immediately used by

anyone on the web without writing user interface or connecting

code.

There is no reason to combine and bind one set of calculations

or data processing to one custom user interface. That increases

complexity, validation problems, computer and content expert

interaction issues, and cost. It also makes those calculations

unavailable for other uses. Even if the calculations were isolated

in software libraries, their attachment to and validation with new

custom Windows user interfaces is not justified in many cases,

especially in smaller markets.

Many subject experts build "apps" using spreadsheets, but the

calculations are tied to the cell locations of the user interface

and cannot be combined with other calculations automatically.

Comma Separated Value (CSV) files can be used to transfer

data to other applications, but users still need to manually

transfer those files and agree upon common data definitions

with other users. Use of CSV prevents combination and

automatic execution of a series of separate calculations. Also,

one study (Panko 1998, 2008) has shown that 88% of all

spreadsheets have errors. Many find that it's not worth the time

and effort to understand spreadsheet formulas or macro code to

be able to use the spreadsheet, let alone improve or use the

calculations for other purposes. TSPA provides subject matter

experts with a better model for spreadsheet applications.

A Finite Element Modeling (FEM) calculation could be located

on a very fast computer and offered on a subscription or cost per

use basis. Users could create their own automatic geometry

generator calc engine, send the output to a mesh generator calc

engine, and then pass the mesh to the external FEM calculation

program which returns the results back to the original user. This

wrapped collection of external calc engines could be launched

from an open front-end UIF program that automatically searches

for an optimum shape based on a user-defined structural criteria

merit function. This open spreadsheet UIF could expand to

include multi-objective optimization problems that calculate a

Pareto Front. With TSPA, this could be done without writing

new code or having the programs locally, let alone having the

source for each application. This type of automation and user

control is not possible with the traditional interactive app and

private data approach to program development.

TSPA will provide program templates for different languages

that include all the necessary source code for data file I/O. All

one needs to do to create a new calc engine is to add in their

own calculation routines, compile, link, and test. Once testing

has been done, it can be immediately used for many different

purposes by users anywhere on the web.

USER INTERFACE FRAMEWORKS (UIF)

The second TSPA file object type is a "User Interface

Framework" (UIF) program (preferably a free and open source

program) that can launch one or more calc engines without

writing any code. Calc engines can be grouped or classed by the

type of processing they do and that defines a basis for providing

open source UIF front end programs that work with any calc

engine of that class. Figure 6 shows the separation and

connection of a graphical user interface (GUI) object with a calc

engine object.

Fig 6 TSPA Separate GUI/CALC engine model

The most common form of data manipulation is the creation,

editing, and saving of data. In the future, TSPA will provide an

open source user interface framework that can do those tasks for

any data file defined with an XML, HTML, or TSPA data

schema. It will also allow users to apply any external calc

engine that might generate or alter the data automatically. One

calc engine might translate string data from one language to

another, and another might convert numeric data from English

to Metric units. Yet another might transpose music from one key

to another. The calc engine possibilities are unlimited, but the

basic UIF framework is the same for all - read a data file, offer

interactive or calc engine data manipulation, and save the results

back to the file. The calc engines are all external, and if a user

finds (or creates) a better or different one, it can be swapped in


with no other changes. There is no reason to write a separate and

custom user interface for each calc engine manipulation of data.

A very large class of calc engines takes a set of input variables

(X), processes them, and then writes out the resultant numeric

variables (Y) back to the same data file. This is a classic black

box calculation of Y = f(X). The calculations could be as simple

as C = A + B [C=f(A,B)], or they could be as complex as a

finite element analysis with forces and a very large mesh as

input and stresses as output. Note that a calc engine (large or

small) does just one set of calculations for one set of input. This

is a key point. The standard UIF front end in OpenCalc provides

all the data input, looping, display, printing, graphing,

searching, and visualization. The UIF does this for any calc

engine of its class without coding by reading the XML list of

inputs and outputs of each CE. The UIF can launch one calc

engine or it can start a batch (BAT) or command (COM) file

that launches many different calc engines, automatically feeding

the results of one calculation into the next using the common

XML data file. Users can define command files or scripts that

process any sequence of calc engines either separately or under

direct user control from a UIF.

TSPA will also provide open source tools that make it easy for

calc engine writers to output 2D views into standard XML/SVG

(Scalable Vector Graphics) files - one for each launching of a

calc engine. This would allow the standard UIF to display either

one view or animate a series of SVG views generated from

looping input over one or many wrapped calc engines. An XML

schema for 3D visualization and rendering is another goal, but it

will be built on top of open cross-industry XML geometry tags,

not their own internal geometry tags. Instead of defining a

rectangle or a polygon with internal SVG tags, it will link to a

general cross-industry polygon tag definition in another schema

namespace.

The Y = f(X) form of UIF can also take any one or more

wrapped calc engines, prompt for the input variables (X), launch

the external calc engine or engines one or more times, and then

read back in the result variables (Y). The UIF can also calculate,

print, plot, or contour any set of input and output values.

Searching for optimum solutions based on user-defined merit

functions will be added next. This will work for any calc engine

of this class. Again, there is no reason for each set of

calculations to have its own custom and attached interactive user

interface. The CEs are launched automatically from the UIF and

the user never sees that happen. CEs require less effort to

develop and they can be used for many purposes, including the

recreation of a traditional interactive “app” form.

A variation of this UIF class could take two different data sets

and run them through the same sequence of calc engines.

Printed and plotted results could be compared side-by-side or on

the same graph. Yet another variation could have a UIF

framework where one data set is run through two different

sequences of calculations and compared side-by-side. A calc

engine writer could compare the results of an old version to a

new version without custom programming or piecing together

output from two different apps.

Another class of calc engines might read raw sensor data

anywhere on the web and save the results into a TSPA

compatible data file. A general sensor data UIF located

anywhere else on the web might read and display the raw data

from one or more sensors in real time and apply any number of

processing or time averaging calc engines. Much of this is

possible now due to the interconnectivity and bandwidth of the

internet along with fast computers and massive amounts of

memory and data storage.

Many engineering calculations are based on a static or dynamic

balance of forces and moments - a Free Body Diagram (FBD).

One could have an open source UIF front end that processes any

set of forces and moments, and each one could come from an

external calc engine located anywhere on the web. Groups could

work on the same FBD model, where each member develops a

different force or moment calc engine. The FBD UIF could be

located anywhere on the web and point to force and moment

calc engines located anywhere else, such as on Google Drive or

Microsoft's OneDrive.

Yet another engineering analysis model looks to find an optimal

geometric shape based on a complex set of separate calculations

and a potentially infinite geometry shape input domain. One can

start with a geometry shape generator calc engine followed by

one or more geometry evaluation calc engines in an automated

UIF loop that searches for an optimum shape based on one or

more user-defined merit functions, like cost and weight. This

UIF would automate the sequential processing of any number of

geometry variations and calculations. This is possible only if the

calculations are created as separate batch calc engines that use

compatible data. This process has been shown to work using a

standard UIF Excel front end, a complex curved ship hull

geometry, and three separate calc engines. No coding was

required to combine and launch the separate calc engines, and it

can be done by a subject expert, not a programmer.

There is also the possibility of having UIFs located on web sites

so that they can launch a calc engine from anyone on the web.

Instead of custom programming web sites, one would just link

standard web UIF code to one or more CEs.

The possibilities for calc engine UIF front ends are endless, and

the TSPA goal is to make them open and free. Note that

spreadsheets provide a perfect platform for many of these UIF

frameworks. It might seem odd to use a spreadsheet to launch

external calculations, but users can still add in their own

customizations or calculations in the spreadsheet to alter the

standard UIF for each use case. Instead of tying one user

interface with one set of calculations, users can customize the

standard UIF spreadsheet that launches one calc engine or they

can use the standard UIF to launch many calc engines wrapped

in a batch or command file script.


The big justification for splitting off user interfaces from

calculations is that large classes of calculations share the same

user interface needs. Most general UIF frameworks can be

provided by spreadsheets using open VBA code or script.

Separate calc engines can be launched alone or wrapped with

any other calc engine, and they can be reused in many different

and creative ways by end users.

OPEN AND LAYERED DATA SCHEMAS (XML)

The third and key TSPA object type is open and industry-

defined text data files built in layers of XML schemas. This is

the glue that ties everything together. The days are over when

independent software developers define and control industry

data and structures in proprietary file formats that work with

only one program. The Web is built on separate and compatible

programs that work off a common HTML data file definition,

and HTML has now been defined with XML and is governed by

the World-Wide Web Consortium (W3C) and the International

Standards Organization (ISO). Web programmers do not define

their own data file formats. Common data drives the Web, but

that’s missing in the rest of the programming world.

HTML defines a file definition for interactive data (text,

graphics, audio, video) with display and format information,

while TSPA defines a layered industry standard XML data file

framework for all program variables and structures. It’s no

longer necessary to have a separate custom data file for each

application.

The TSPA/XML text file data definition acts like a general

database for program variables and structures for all programs.

There is no need for two-step neutral file format conversion.

Programmers can use this organization to define all their own

data and structures, but the structures won’t be hidden away in

binary files. Industries need to assert some level of influence to

ensure better compatibility and data flow between calculations

from different sources. In naval architecture, that might involve

some naming and organization recommendations for things like

different hull definitions, weights, and operating conditions.

TSPA will offer open source I/O code in many languages so that

software developers can access any program variable or data

structure from a common file without knowing anything about

the file format, and one text data file can contain data from all

programs. When separate calc engines are launched in sequence,

they can pass data via this common data file automatically.

XML is not a file format in a traditional sense. It’s a way to

surround data with opening and closing tags in a text file.

TSPA/XML wraps each program variable and data structure in

tags so that they can be put into the file in any order or

combination. Access is made by name and not position. One

might think of it as program memory in a file. Programmers

don’t know where a variable is located in memory. They just get

and put the data by name. It works the same way for

TSPA/XML, which reads in the entire file as a string, builds a

list of tags and variables, and then offers routines to get/put any

variable by name. When done, the programmer can overwrite

that entire string to the file.

The lowest layers of TSPA/XML, including a geometry layer,

will be submitted to W3C/ISO as a world standard. They are

built upon their base XML definition. Industries can then build

their data and structures on top of a common and cross-industry

compatible base. Industries can control their data, but they can

be cross-industry compatible using common lower level

definitions for data like geometry.

TSPA data schemas are built in layers to define everything from

simple variables to complex geometric data structures. Since

XML is a text markup language that surrounds data with tags

that are accessed by name, anyone can add in their own custom

data tags without affecting any program or the open TSPA I/O

code. This is a very important concept that allows for both

common and custom data content at the same time in a single

file using the standard I/O code. Custom tag data just gets

passed along using the common tag format. It’s up to each

program to use or ignore the data. This is a key element that

should satisfy both industries and independent software

developers.

Again, XML data definitions are defined by schemas, not file

organizations or formats. Text files are just streams of

characters and in many cases, the tag data can be in any order

and broken into any number of separate files to fit the needs of a

solution process. This is quite different than the traditional

method where independent programmers define specific data

files and line formats for each program. Data in TSPA form is

more like a random-access database definition than a sequential

collection of data useful for only one program. Figure 7 shows

how data definitions form layers that are built on top of each

other.

Fig 7 Layered XML schema example

The first layer of TSPA data definition builds on top of a base

XML tag layer and uses that to define any programming

variable and data structure. TSPA constructs the next layer to


define any program variable, array, matrix, or data structure -

linked lists, tables, tree structures, network organizations - to

match any that can be created in a computer program. This

makes it a simple matter to save and retrieve any variable and

data structure organization to/from a file. This variable tag layer

also includes pointer and link variables so that it can directly

match any data type and structure that can be defined inside of a

program.

Figure 8 shows a small TSPA XML file that contains 4

variables; two as single variables and two as arrays. Notice that

all variables can have units defined and the I/O routines can

translate units on the fly going into and out of a calc engine.

Fig 8 Variable and array definitions

TSPA also allows for common program “Structure” definitions.

Structures collect a group of variables together as a single entity.

A structure can be used as a single variable or as an array.

Figure 9 shows a variable (HULL2) that has a structure data

type (DT) called STATION2 in the NAME (Naval Architecture

and Marine Engineering) namespace consisting of variables that

define one station of a station definition of a hull. In this

example, HULL2 consists of two stations with offset points and

breakpoint/curve indicators. Each station is surrounded by <I>

… </I> tags. This definition contains enough information to

allow a program to use the open I/O code to get or put any

number or array using a name like “HULL2[1].Z[3]”. This is the

4th Z-value of the second station of HULL2 (zero-based arrays).

Note that HULL2 defines a 100 ft long box barge. Programmers

no longer need to do any file I/O programming.

Fig 9 Structure definition for a hull using stations

The next TSPA schema layer uses the lower variable and

structure tag level to define common cross-industry definitions,

like geometry. The goal with geometry data is not only to

eliminate neutral file formats, like DXF, IGES and ISO-10303

(especially because of the cost, complexity, and no open source

I/O code) for product design, but to define geometry from the

bottom up rather than the top down. A TSPA XML definition of

a polyline geometry tag variable can be used in many different

product definitions.

Programmers modeling different types of geometry should not

be defining their own polyline definitions as incompatible sub-

tags of custom and complex XML schema. A polyline tag

should be part of a common cross-industry XML schema

(namespace) that can stand by itself. A user who needs to work

with polyline data should only have to learn that one simple tag

definition. Software developers no longer need to redevelop the

wheel for data file formats for things like polylines or curves

using formats that are readable only by their program.

Figure 10 shows a layered variation of the structure shown in

Figure 9, where the 2D stations are defined using a lower level

“GEOM” namespace structure called “PATH2D.” This is a

general 2D geometric object that defines a combination

polyline/curve path using indicators (zero indicates a knuckle or

polyline point rather than a curve-fit point).

Fig 10 Layered structure definition for a hull

Note that the offsets (OFF) for each station points (P) to a

separate named variable called “Box2010”, which is a cross-

industry definition of any 2D path shape. Since both stations are

the same, they can point to the same 2D shape. A calc engine

could get the ‘Y’ array of the second station by passing the

string “HULL1[1].OFF.Y” to the open source I/O routines. The

routine would return a set of 4 doubles: 0.0;0.0;10.0;10.0.

What this allows for is access by other non-naval architecture

programs to that data because they know what a PATH2D tag is

and not what NAME:STATION2 Y and Z arrays are. One could

have a CAD program read, edit and store PATH2D tag data

without knowing that it has anything to do with a hull. Common


cross-industry data definitions in this form are much more

powerful than the use of neutral files. The TSPA project will

define cross-industry definitions for many types of geometry,

from points through NURB curves and surfaces. These

definitions will be offered for use in an open source format, but

they will not preclude others from creating their own

definitions.

Higher TSPA schema layers can be defined by industries that

want to take control over their variables, data structures, and

geometries so that subject experts can create compatible calc

engines and combinations. Since many geometries can be built

on top of the lower level geometry tags, calc engines developed

for a lower level tag will work for that tag when it’s used in any

higher-level schema definition in any industry. The low level

calc engines will be able to pull out and operate on the lower

level data tags in any higher level XML data structure files.

For example, a general calc engine that creates a 2D airfoil

section curve could be used as part of a larger airplane XML

definition. The open TSPA I/O code allows any calc engine to

find the tag variables they need from any XML file, process the

tag information, and put it back into that same file without ever

knowing about or affecting any other tag data. A 2D airfoil

variation calc engine could be combined with a 3D airplane

analysis calc engine to automatically search for the best foil

sections to use. Up until now, users have been happy enough to

have ways to manually connect data between separate

interactive apps, but that will fundamentally not work when the

goal is to automatically search through thousands or millions of

separate calculations. Interactive and manual apps are not the

only programming need anymore. Separate calc engines are

needed.

In another example, if some complex object is made up of

thousands of polygons defined with the standard lower level

polygon tag definition, one could apply a generic stand-alone

polygon area calc engine to determine the entire surface area of

the object. A TSPA UIF front end could take any calc engine

and apply its processing to any data file that matches the TSPA

layered data structure. The UIF will look in any XML file for

specific tag data to pass to any calc engine.

One could write a calc engine that curve-fits a polyline data tag

and converts it into a spline curve PATH2D tag definition. This

calc engine could be applied to any polyline tag located in any

complex XML TSPA data definition for any geometry in any

industry. Common calculation tools do not have to be recreated

and custom built into each industry application. TSPA greatly

reduces the need to recreate user interface and calculation code

and gives those tools to subject experts to create new solutions

without custom programming.

USERS AS CREATORS Now that computers are tens of thousands of times faster than

the original PCs and have massive amounts of memory and

storage, and now that computers can transfer data to and from

any other computer on the web, a new, more flexible

programming model is needed than the traditional all-in-one

interactive and manual program “app” created by programming

experts who tie everything together into fixed manual solutions

for “users.” TSPA defines a new open framework that breaks

apart the app and allows subject matter experts to develop their

own pieces to combine and use in new creative ways.

Calc engines can be launched by automatic script, command

files, or by other calc engines, and they can be launched from

interactive UIF frameworks that perform the input/analysis job

without any programming. Users can also alter a general UIF to

provide multiple custom user interfaces for one calc engine.

This could be a way to provide simple versus complex front

ends to the same set of calculations. As programs get more

features (inputs) and complexity, one can set default values for

input values and the UIF will adjust automatically to hide them.

Users can also make copies of a spreadsheet UIF and add in

extra formulas for specific jobs. It’s difficult for traditional

programmers to add in the custom programming capabilities of

a spreadsheet to each stand-alone app. One can think of this

combination of UIF to a single calc engine as an N-to-1

relationship, and users can create their own spreadsheet front

end.

Users can also have one general UIF launch several independent

calc engines in sequence without writing any new code. The

UIF that handles any “black box” external calc engine can also

start any batch file that launches several separate calc engines

located anywhere on the web. The combination of separate calc

engines can be viewed as a “wrapped” black box calculation

with its own larger set of inputs and outputs that can be created

from the individual I/O calc engine definition files. The standard

UIF front end adjusts accordingly. One can think of this

relationship as 1-to-N. Therefore, TSPA offers an N-to-N

relationship between user interfaces and calc engines that can be

created and controlled by subject experts and users.

To get these new combinations to work, one does not need the

source code for the calculations and the executable programs

can be located anywhere on the web. CEs can be created

separately, validated by subject experts, and used for many

different purposes, either alone or in combination with other

calc engines. TSPA defines a standard process for layered data

and structure definitions that adapts to both cross-industry and

industry-specific needs. Most importantly, these separate

programming objects can be mixed and matched to solve

completely new problems without additional programming.

With TSPA, the calculations can stay as separately-validated

EXE files even if technology changes. One might create a new

UIF front end to adapt, but that would not change the

calculations at all. In the world of computers, EXE batch

programs and text files have not changed over the years. Calc

engines and text data files can have long lives and many uses.

TSPA defines and separates the role of subject experts to better

match their importance in many smaller market segments. When


subject experts become the “customer” in the development of

large computer solutions, they end up losing a lot of control and

flexibility. They become “users” of fixed solutions, not creators

of new ones. TSPA/OpenCalc changes that.

COMPUTER-AIDED DESIGN The current trend in CAD is the move away from stand-alone

programs for ship design to those built inside of or on top of

general purpose programs. This divides the market into smaller

communities and isolates those who cannot support the extra

CAD costs and learning curves. It also increases the complexity

and development costs for independent software developers.

The tools created for one CAD environment are not available to

other CAD environments without expensive custom

programming. The goal is not to hope that one CAD program

wins the market battle.

In addition, these CAD-specific tools are interactive in nature

and not available for combined or automated use, as with

searching for optimum solutions. OpenCalc, however, defines a

way to create stand-alone calculations that could be combined

and launched by any CAD program using the open source I/O

tools. In this way, a naval architect/programmer might create

and test one standard calc engine program that would work with

any compatible (CAD or other) program from anyone.

In a CAD program, a user could take the selected hull geometry

and send it off to an external calc engine to perform a

calculation or geometry variation task. This could be done

automatically in the background with no custom programming.

One could also have an external calc engine generate geometry

to pass into any CAD program as a starting shape. This calc

engine could be written without any knowledge of the CAD

program. It would be the CAD program's responsibility to use

the open source code to provide a standard and compatible

connection that would work with any external calculation

program. This would only have to be done once using the open

source tools to have it work with any external calc engine. Each

independent programmer would not have to learn and build

custom apps for each CAD program. This is the Holy Grail of

programming - defining separately developed and tested objects

that work together automatically. OpenCalc/TSPA will provide

open source programming templates for calc engines that

include all the necessary XML I/O code. Programmers only

need to write their calculation routines. They do not need to do

any I/O programming or know anything about how the tag

format works.

The problem in the general CAD world is the need for

customization of its base program to each industry and

application. This creates costly tools that are incompatible with

tools and calculations in competing CAD programs. Even

though the geometry might be based on the same NURB surface

definition, the tools will not work together in any common or

stand-alone environment. The tools are all tied to a proprietary

interactive format and process, not an open one that allows

combined and automatic solutions.

In a paper by the author (Hollister 2016), three separate and

standard calc engines: hull variation, hydrostatics, and Holtrop

resistance, were combined to search for an optimum hull shape

without writing custom code and without having the source code

for any of the calculations. With a compatible CAD program,

one could launch this search sequence using the current “parent”

hull shape being designed in CAD, ultimately replacing it with

the optimum shape determined by the external calculations.

The keys to this process are the separate and open definition of

external calc engines and a common XML data file definition of

industry geometry and variables. This cannot be achieved with

the current proprietary independent "app" model of

programming.

FUTURE OpenCalc has split into two parts: the base TSPA open source

system that can be applied to all areas of programming, and

OpenCalc, a layer built on top of TSPA that provides for the

specific needs of the maritime industry. The job for SNAME

includes identifying and converting legacy calculations into

modern and reusable calc engines and defining data variable and

structure XML standards for conceptual and preliminary design

and analysis for data such as hulls, compartments, tanks, and

operating conditions. Each industry needs to take control over

their data. Programming and analysis cannot rely on separate

and incompatible program file definitions that require two-step

neutral file conversions.

Separating calculations from inside of a traditional application

to a separate calc engine batch format is not difficult, but

industry organizations need to push to achieve a critical mass of

support. Independent software developers should no longer

completely define their own data and proprietary file formats.

Industries can use the common data definition and I/O tools of

TSPA to create their own data definitions and schemas. These

can be added to by software developers using custom tags, but

industries won’t be fully successful without some level of

influence or guidance. This doesn’t mean the elimination of

successful legacy formats because they can be used as alternate

definitions with conversion routines that work on-the-fly. The

hydrostatics calc engine included with OpenCalc can read

several different legacy hull station formats, like GHS and

SHCP. However, all other variables and structures are defined

with XML tags.

The goal is to have SNAME play a lead role in introducing and

pioneering this needed and valuable approach to ship design

software development. With contributions from all and buy in

from developers the software resources developed will be highly

useful for hobbyists, students, professionals, and software

developers alike. There is no other path ahead for

programming. The world of separate interactive apps tied to


calculations and proprietary data formats has severe limits, and

engineers and naval architects need creative tools that can solve

new problems without becoming expert programmers or

programmers at all.

All programming templates are currently written in C/C++, but

the next implementations will be in Java and Java Script. The

goal is to provide open source XML I/O code in many

languages and UIF connection code written for many different

spreadsheets.

With SNAME’s help, work will begin on education, converting

legacy code, and building libraries of historical hulls using new

XML definitions for hulls and compartments.

MARKET CONSIDERATIONS

Open source software does not mean that everything is free. The

UIF components may be based on spreadsheets like Excel,

which is not free, but the goal is to make the macro code in

Visual Basic for Applications (VBA) or some other spreadsheet

macro language free. The XML data definition components and

I/O code will be free and open source. Unlike ISO, the TSPA

definitions will be free and include free source code in many

different languages.

TSPA is also not free in the sense that some things built on top

may not be free. This open source opportunity encourages

others to use the programming structure, including those who

create their own proprietary calc engines to use or sell. The

OpenCalc resource will make it easier to create and maintain

software developments by all and will be less expensive and

offer a much larger market of users at all levels.

This external calc engine structure is different from custom

VBA programming in Excel, where it’s difficult to hide

proprietary VBA source code. With separate CEs in EXE form,

developers can hide and protect their code. However, the hope is

that many designers will offer their tools to others for free. For

the case of an optimization combination of CEs, designers might

create their own proprietary hull generator CE, add to it the free

SNAME hydrostatics/stability CE, add in a hull meshing CE,

and finish off with a complex CFD CE that is offered at another

web site for a subscription service. It is also a goal to get an

OpenFoam version that works as a stand-alone CE in batch

form.

CONCLUSION

"There continues to exist competition between the proponents of

self-contained, relatively closed turnkey systems of increasing

functional scope and the advocates of open systems and a

modular, in part heterogeneous approach to system growth.

This argument will probably go through many more cycles, but

by and large, the long-range trend has been in favor of more

and more open systems."

...

"Such openness tends to lie in the interest of not only the user

communities, but in the long run also of the system vendors."

(Nowacki 2010)

As a student of Professor Horst Nowacki in the 1970’s, and after

45 years of frustrations from dealing with changing technology

in a small, technically complex market, the author developed

TSPA and OpenCalc as the solution. After losing programs

when the world changed from mainframe computing to PCs

(programs were put on 9-track tape and forgotten), and then

after losing a huge amount of programming development when

DOS evolved to Windows (many still keep old XP computers

for those programs), and now viewing more code being lost in

the change to cloud computing, a new programming model was

created to isolate and provide separate tools for computer

scientists, subject experts, users, and industries. This change is

made more viable with the open source movement and Web

standards like HTML and XML.

There is an historical mindset inertia that needs to be moved

now that technology offers massive amounts of speed, memory,

storage and web connections. In the program development

world, the traditional interactive app developed by separate and

competing software development companies that concentrate

computer expertise and data control has not evolved. This is

most noticeable in smaller and more complex markets that do

not justify expensive app development and where

calculations/data processing require more subject expert

creation and control.

Excitement over popular web apps hides historical and smaller

market issues. The fundamental flaw is that the traditional

programming model has not evolved into what it could be. The

interactive and manual user-controlled app model of

programming where one user interface is tied to one set of

calculations or processing might have been a great solution for

the beginnings of the PC era, but that's no longer the case. There

is a limit to what independent software company development

and interactive apps can commercially justify and offer. Some

of this has to do with the limitations and complexities of small

markets, and some of it has to do with the needs of users to

combine and automate complex tasks without user interaction.

Apps require too much code that must be created by

programming specialists working with subject experts and tied

into interactive and fixed use tools with limited customization.

Development costs limit the creation of tools for smaller

markets and changing technology causes many valuable tools to

be lost.

New program organizations are possible with the web

connecting fast computers, their programs, and their data files.

However, what really must change is the idea that subject

experts need to be simple users of other people's software even

if the software does offer some level of customization.

Computer use will never reach its potential until complex tools

for creating completely new solutions are given to subject

experts without having to become expert programmers. That


appears to be an unfortunate new meme - to have everyone

become coders (see Hour of Code), not better subject experts

who can define new solutions given separate and flexible

computer object tools.

TSPA working examples have shown how this is possible and

how it offers ways to build new automated solutions that can’t

be built any other way unless one has all the source code and a

very large budget. Calculations are separated from interactive

user interfaces and common data is defined with open

definitions and code. Everyone can add their own compatible

piece to large web-based solutions. Computing will no longer be

controlled by all-in-one apps, proprietary data formats, and

suites defined and dominated by independent software

developers. Industries will assert control over their data and

subject experts will have flexible tools to create completely new

solutions without programming.

TSPA is being developed as an open framework with many

tested and open source tools. Some might create calc engines

that are not open or free, but the XML layered data schemas and

I/O code will be free along with many UIF frameworks. After

more than 50 years of rapid computer developments, changes,

lost programs, and ignoring the needs of small markets, it’s time

to define an open, flexible, independent, and external object-

oriented approach that is web-based, cross-platform, and cross-

industry. Subject matter experts are not just users or customers

of an app development process defined and controlled by

computer scientists and independent software developers. They

should not have to wait for the next software revision in the

hope that what they want will be included. For small markets,

that may never happen. Subject matter experts can and should

be creators of completely new computer solutions without

becoming programming experts. TSPA/OpenCalc gives them

that direct, flexible, and creative control over their subject

domain.

REFERENCES

Hollister, Stephen, “Automatic Hull Variation and

Optimization”, Presented at the February 28, 1996

meeting of the New England Section of The Society of

Naval Architects and Marine Engineers

Hollister, Stephen, “A New Paradigm for Ship Design

Calculations”, Presented at the Nov. 2015 SNAME

Maritime Convention (WMTC) conference

http://www.sname.org/project114/home

Hollister, Stephen, “Automatic Hull Variation and Optimization

using SNAME’s OpenCalc System”, Presented at the

Nov. 2016 SNAME Maritime Convention, Seattle, WA

Nowacki, Horst, “Five decades of Computer-Aided Ship

Design”, CAD-Elsevier 42 (2010), pages 956-969

Panko, Raymond R., "What We Know About Spreadsheet

Errors." Journal of End User Computing's Special

issue on Scaling Up End User Development, Volume

10, No 2. Spring 1998, pp. 15-21 Revised May 2008.

SNAME, “Open Source Computing Tools”,

http://www.sname.org/project114/home, 2015

TSPA, “The Tri-Split Programming Architecture”,


ABOUT THE AUTHOR

Stephen M. Hollister, an engineer and software developer living

in Jamestown, Rhode Island, has BSE and MSE degrees in

Naval Architecture and Marine Engineering along with an MSE

degree in Computer, Information and Control Engineering, all

from the University of Michigan. He has written over a million

lines of code over 45 years as a combination naval architecture

subject expert and a computer science subject expert. Software

developed includes a full relational trimmed NURB surface

CAD program and a suite of programs for intact and damaged

ship hydrostatics and stability. He has also developed many

other calculation programs for hull resistance, propeller

analysis, sailboat velocity prediction, and stress-based plate

development, along with many vessels and products that have

been CNC cut and built. Technology used has covered the

historical range from mainframes, cards, Calcomp plotters and

Tektronix graphics terminals, to writing PC graphics drivers in

assembler and full applications for DOS, to writing a full 3D

graphics package, and then converting everything over to

Windows and MFC. Mr. Hollister has written programs in

FORTRAN, Snowball, assembly languages of all sorts, Turbo

Pascal, Algol, VBA, C and C++.

In the 1980s, Mr. Hollister taught college Math and Computer

Science full time for six years, and since 1985, when he started

New Wave Systems, Inc. (www.newavesys.com and

www.pilot3d.com), has specialized in NURB surfaces, ship

design, and general CAD/CAM/CAE software development.

See also http://www.pilot3d.com/NurbSecrets.htm. Mr. Hollister

obtained a PE license and is a 40 plus year member of SNAME,

IEEE, and ACM.




http://www.newavesys.com/

http://www.pilot3d.com/

http://www.pilot3d.com/NurbSecrets.htm

The SNAME OpenCalc System · 2019-07-06 · The SNAME OpenCalc System Stephen M. Hollister (M) New...

Documents

Transcript of The SNAME OpenCalc System · 2019-07-06 · The SNAME OpenCalc System Stephen M. Hollister (M) New...