The Project: Building a Graphical Equation Editor (using the CIDER toolkit) Myself: Reg Johnson...

The Project: Building a Graphical

Equation Editor(using the CIDER toolkit)

Myself: Reg Johnson

Supervisor: Kim Marriott

CIDER Developer: Tony Jansen

What is to be discussed:

What the Project isDiagram Editors and what are SDE’s.How to Model math symbols as Diagrams.Comparison of SDE toolkits.What is CIDERHow to Construct the Grammar RulesExample of an expression being parsedCIDER FeaturesDiscoveriesConclusions

What the Project is:

To design and create a Smart Diagram Environment, using the ‘CIDER’ toolkit, for mathematical symbols and math notation.

To design a user-friendly editor, that will automatically parse and recognise different mathematical equations.

To discover how useful CIDER is to assist in building a graphical equation editor.

Standard DiagramsStandard Diagram editors can display diagrams, but cannot interpret the underlying meaning of it. However, they provide the flexibility of being able to draw freehand notation. Thus non-Syntax Based Editors.

Eg: Freehand Formula Entry System

MS paint

Syntax Based Editors• Equations are less flexible to move and position

where a user wants them, but have a structure underneath

Syntax Based Editors examples: Math Magic Swift MS Word

Mathematica Scientific Notebook

Quick Comparison of editors:

Benefits with Syntax based languages

- usually easier and quicker to write

- parsing rules apply to equations

- user restricted to place symbols

Benefits with hand drawn abilities

- everything is defined by the user

- can be manipulated to how the user desires

- no logic underneath the diagram

What are Smart Diagram Environments

Smart Diagram environments are software applications, that can interpret a visual representation of a given diagram, to give the impression that the computer understands the diagram.

Can take advantages of both syntax and non-syntax based editors.

Interpret the meanings of diagrams, on a human level.

Example:

FSA:

Will be able to recognise the state, the transition, and the final state.

How to model Math Symbols on a Diagram level

According to the Anderson paper, all mathematical symbols are given an xMin, xCentre, xMax, yMin, yCentre & yMax set of co-ordinates.Thus it defines bounding boxes around those symbols.

All symbols use these, whether they are terminal symbols (base symbols), non-terminal symbols (merged symbols).

Source: R.H. Anderson. Two-DimensionalMathematics Notation. Springer-Verlag, New York, 1997.

Example SDE toolkits:

Recopla

Penguins System:

DiaGen:

GenEd:

and CIDER…

CIDER is a Component-Based toolkit for the creation of

SDE’s (Smart Diagram Environments)

It’s purpose was to take the strengths of the previous SDE toolkits and combine them into one.

What is CIDER?

Modeling Diagram’s Components

Interpret Diagrams Automatically

Preserve Structural Manipulation

Transform a diagram

Main Functionality of CIDER

How CIDER works

via, a set of rules that define a grammar must be written. These rules indicate how the diagram is to be parsed, and what symbols are consumed to make new symbols.

( These rules are somewhat like what humans use to decide how to parse an equation. )

From the given grammar, it is converted into Java code. Thus it can interact with any other Java application that is created.

Constructing a Grammar rule example:production

{

output:termOp ::= t1:term, o:operation

where

{

# check if multiply / divide

editor.functions.isMultiplyOrDivide( o.type ) == true;

# all components are on the same height level

@sc(5.0, "[t1.mid_y] + -[o.mid_y] == 0.0");

@sc(5.0, "[o.min_x] + -[o.spacing] + -[t1.min_x] + -[t1.rightWidth] == 0.0");

}

set

{

output.label := editor.functions.fourAdd( "{", t1.label, "}", o.type );

output.min_x := @ce("[t1.min_x]");

output.mid_y := @ce("[t1.mid_y]");

…

}

}

What is being consumed

What needs to occur

What the new settings of the output system would be

What will be outputted

Symbols that Grammar Rules were defined for:

Note: If a new symbol causes a previous stage to be incomplete, then that stage is no longer satisfied and all rules that used that stage are broken as well.

• basic

• connectableVarsSet

• variablesSet

• connectableNegation

• negated

• finalisedNegated

• Integral

• Sigma

• connectableToSigma

• completedSigma

Other rules such as when two brackets are on either side of a “finalisedExpr”, it is consumed into a basic symbol.

Then that new symbol will be rechecked, and will end up as a finalisedExpr again.

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr

• worded

• finalisedWorded

• connectableWorded

Becomes a basic again

How would the following

expression be parsed?

• basic• connectableVarsSet

• variablesSet


• negated


• worded

• finalisedWorded



• Integral

• Sigma


• completedSigma

3 and 5 will eventually become finalisedExpr, but the 2 won’t since there is a multiply before it.

The 2 however, works its way up to the factor stage. Will stop since a multiply can’t be to the left of a “term”.

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr

Example

• basic


• variablesSet


• negated


• worded• finalisedWorded



• Integral

• Sigma


• completedSigma

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr

“sin” is recognised as a “word”, and not a variable

Example


• variablesSet


• negated


• worded

• finalisedWorded



• Integral

• Sigma


• completedSigma

A basic symbols is created, from consuming the fraction, and the two “finalisedExpr”

symbols, that are associated with it.

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr

Example

• basic


• variablesSet


• negated


• worded

• finalisedWorded

• connectableWordedBecomes a basic again

• Integral

• Sigma


• completedSigma

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr

The “sin” and the “fraction” are connected, and they become a basic symbol.

Example

• worded

• finalisedWorded



• Integral

• Sigma


• completedSigma

• The symbol becomes a basic.

• Becomes a factor, since no sigma’s next to it.

• Then becomes a Term, since no multiply/divide next to it.

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr


• variablesSet


• Negated

• finasliedNegated

Example

• worded

• finalisedWorded



• Integral

• Sigma


• completedSigma

The term and the operator, become a termOp symbol

• factor

• term

• termOp• finalisedTerm

• expr

• exprOp

• finalisedExpr

• basic


• variablesSet


• negated


Example

• worded

• finalisedWorded



• Integral

• Sigma


• completedSigma

• factor

• term

• termOp

• finalisedTerm

• expr

• exprOp

• finalisedExpr

• Remember that the 2 was a factor.

• Now however, it is combined with the termOp, and together they become a term.

• Then becomes a finalisedTerm because there’s no multiply or divide to the right of the term

• Then becomes an expr because there’s no plus or minus to the left.

• Then becomes a finalisedExpr, because no plus or minus to the right.

• basic


• variablesSet


• negated


Example

While it is being parsed, can output LaTeX

Note how this was parsed,

If the first written “tan” was a variable, then the minus after it would have a totally different meaning.

More LaTeX

examples

CIDER Features

Cider has the ability to output and calculate results. In this case, I’ve been asked to output LaTeX But can also calculate values.

It can run through the stored parse forest, and get any corresponding data for a variable.ie: For corresponding siblings: eg: brackets,

For children: eg: for a plus, find descendantsFor parents: eg: for a value, find ascendants

Outcomes:

• Parenthesis• Exponents• Relationships• Special words

eg: sin, cos, tan,

CIDER was discovered to be able to achieve its task. That is by specifying a grammar, it could theoretically parse any equation.

What was parsed• Letters and Numbers• Integrals• Addition \ Subtraction• Multiply \ Divide• Fractions

However, there are a few cases where ambiguities occur. Eg: ∫ 2x dx, the “dx” in other cases might be a standard variable name.

The process is that the 2x and the “dx” is seen, and they would connect into a new term. So when the ‘∫’ is then parsed, the dx was already consumed.

Outcomes:

The way a human checks equations is that if there’s a

if a : “1 + ”, is seen, a human would check the gap next to the +, to determine if there’s an equation.

CIDER on the other hand, checks if any possible value on the canvas, fit within the position. Thus gets incredibly slow when there are several components.

eg: --------4, each of the “-” is checked to be near the 4, and the process repeats till all the minuses are connected.

Future Work

As outlined in the previous section Improve CIDER’s checking mechanism, to check

the space, rather than if an object fills the space. Have a system where a piece of data, may be

consumed more than once, for alternative equations. Eg: ∫ 2x dx, if the dx is consumed with another object, that would be superseded by the fact that it is part of an integral

ConclusionsCIDER has proven to have many useful feature: If doing the same thing in Java, would require immense

amount of repetitive code, for all terminal/non-terminal symbols, and for all the grammar rules.

allows the ability to find out information a component in the equation, such as what the children/parents are.

Can solve equations, or translate into LaTeX or any other form.

However, although CIDER seems to be useful, and potentially could be implemented to solve equations: it is too slow for practical purposes Mathematical notation is not unique and unambiguous

enough to allow CIDER to properly parse and differentiate all equations.

QUESTIONS?

The Project: Building a Graphical Equation Editor (using the CIDER toolkit) Myself: Reg Johnson...

Documents

Transcript of The Project: Building a Graphical Equation Editor (using the CIDER toolkit) Myself: Reg Johnson...