Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information
-
Upload
felienne-hermans -
Category
Technology
-
view
3.617 -
download
1
description
Transcript of Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information
![Page 1: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/1.jpg)
Spreadsheets are graphs too! Felienne Hermans (@felienne)
![Page 2: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/2.jpg)
Spreadsheets are graphs too! Felienne Hermans (@felienne)
In this slidedeck I explain how I used Neo4J to store information on spreadsheets
![Page 3: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/3.jpg)
Ehm...spreadsheets? They are so tably?
Are you sure they are fit for a graph database?
![Page 4: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/4.jpg)
Spreadsheets are mislabeled
![Page 5: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/5.jpg)
Spreadsheets are mislabeled
People often think of spreadsheets as data, but...
![Page 6: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/6.jpg)
Spreadsheets are code
![Page 7: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/7.jpg)
Spreadsheets are code
I have made it my life’s work to spread the happy word
“Spreadsheets are code!”
![Page 8: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/8.jpg)
Spreadsheets are code
I have made it my life’s work to spread the happy word
“Spreadsheets are code!”
If you don’t immediately believe me, I have three reasons*
* If you do believe me, skip the next 10 slides ;)
![Page 9: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/9.jpg)
1) Used for similar problems
![Page 10: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/10.jpg)
This tool (for stock price computation) could have been built in any language. C, JavaScript, COBOL, or Excel.
The problems Excel is used for are often (not always) similar to problems solved in different languages.
![Page 11: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/11.jpg)
2) Formulas are Turing complete
![Page 12: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/12.jpg)
2) Formulas are Turing complete
I go to great lengths to make my point. To such great lengths that I built a Turing machine in Excel, using formulas only.
![Page 13: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/13.jpg)
Here you see it in action. Every row is an consequetive step of the lint.
This makes it, in addition to a proof that formulas are Turing complete,Also a nice visualization of a Turing machine.
![Page 14: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/14.jpg)
3) They suffer from the same problems
![Page 15: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/15.jpg)
3) They suffer from the same problems
![Page 16: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/16.jpg)
3) They suffer from the same problems
![Page 17: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/17.jpg)
3) They suffer from the same problems
![Page 18: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/18.jpg)
In summary: both the activities, complexity and problems are the
same
![Page 19: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/19.jpg)
So if spreadsheets are code, can we apply software engineering methods?
![Page 20: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/20.jpg)
![Page 21: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/21.jpg)
In my dissertation, I defined smellsfor spreadsheet formulas
![Page 22: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/22.jpg)
Turns out, Fowler’s code smells are easily
transferable to spreadsheets
![Page 23: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/23.jpg)
Pop quiz: what smell is this?
![Page 24: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/24.jpg)
It is the ‘feature envy’ smell
![Page 25: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/25.jpg)
See how easily this applies to spreadsheets
![Page 26: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/26.jpg)
To analyze smells, we save spreadsheet info to a database
![Page 27: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/27.jpg)
This is the data model that I am storing to the database.
The basics are pretty simple.
![Page 28: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/28.jpg)
This is the data model that I am storing to the database.
The basics are pretty simple.
But cells can refer to each other, either directly (i.e. =A7+A9)
=A7+A9
![Page 29: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/29.jpg)
=A7+A9
=SUM(A1:A5)
This is the data model that I am storing to the database.
The basics are pretty simple.
But cells can refer to each other, either directly [=A7+A9] or through a range [=SUM(A1:A5)]
![Page 30: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/30.jpg)
This is the data model that I am storing to the database.
The basics are pretty simple.
But cells can refer to each other, either directly [=A7+A9] or through a range [=SUM(A1:A5)]
In the case of a range, the range itself will points to the cells it contains.
=SUM(A1:A5)
A1..A5
![Page 31: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/31.jpg)
You know the saying that if all you have is a hammer, everything is a nail to you.
This is what happened to me. I did not think about what type of database to use.
![Page 32: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/32.jpg)
SQL
You know the saying that if all you have is a hammer, everything is a nail to you.
This is what happened to me. I did not think about what type of database to use.
I just started banging with the good ol’ SQL hammer I had been using for ever.
![Page 33: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/33.jpg)
Number of worksheets in a spreadsheet
Which started out just fine!
![Page 34: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/34.jpg)
Number of cells in a spreadsheet
Still pretty okay
![Page 35: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/35.jpg)
Number of connected cells for a cellBut, in order to calculate the
‘feature envy’ smell, we need the total number of connected cells.
So both direct and through a range.
![Page 36: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/36.jpg)
Number of connected cells for a cellBut, in order to calculate the
‘feature envy’ smell, we need the total number of connected cells.
So both direct and through a range.
Let’s start with direct.
![Page 37: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/37.jpg)
Number of connected cells for a cell
![Page 38: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/38.jpg)
Number of connected cells for a cellBut, in order to calculate the
‘feature envy’ smell, we need the total number of connected cells.
So both direct and through a range.
Let’s start with direct.
Now look at the range part.
![Page 39: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/39.jpg)
Number of connected cells for a cell
![Page 40: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/40.jpg)
Number of connected cells for a cell
![Page 41: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/41.jpg)
Number of connected cells for a cellThings start to get iffy when
we combine these two query parts.
![Page 42: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/42.jpg)
Number of connected cells for a cell
![Page 43: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/43.jpg)
Number of connected cells for a cellThings start to get iffy when
we combine these two query parts.
Not only is the query quite big, also this happens.
![Page 44: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/44.jpg)
Number of connected cells for a cell
![Page 45: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/45.jpg)
If your tools reach their limits, this has to tell you something.
So I started thinking.
![Page 46: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/46.jpg)
Maybe this is not
a nail…
![Page 47: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/47.jpg)
![Page 48: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/48.jpg)
Maybe I need a
different tool
![Page 49: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/49.jpg)
Maybe I need a
different tool
It was at this time that I attended a talk about Neo4J.
And the strange thing is, I had seen a few talks about Neo before. But this time it ‘clicked’, because I was suffering from the problem that Neo could solve.
![Page 50: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/50.jpg)
So I ended up with this model. Still spreadsheets, worksheets, cells and links.
![Page 51: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/51.jpg)
So I ended up with this model. Still spreadsheets, worksheets, cells and links.
But the ‘prec’ relation can now refer to either cells or ranges.
![Page 52: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/52.jpg)
Turning this
![Page 53: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/53.jpg)
Turning this into this.
![Page 54: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/54.jpg)
Turning this into this.
I wouldn’t say this is the power of Neo at work. It is the power of the right tool for the job.
There are scenarios, for sure, where the situation is the other way around.
But for my goall, Neo was a great fit.
![Page 55: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/55.jpg)
Also, to be honest with you, I did not immediately write such super succint Cypher queries. My first attempt was something like this:
![Page 56: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/56.jpg)
Also, to be honest with you, I did not immediately write such super succint Cypher queries.
My first attempt was something like this
![Page 57: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/57.jpg)
Also, to be honest with you, I did not immediately write such super succint Cypher queries.
My first attempt was something like this
This is basically a one on one translation from SQL to Neo. Still the two different ways of connecting. It took me a while to understand the power of traversal queries.
Here’s another example:
![Page 58: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/58.jpg)
Number of cells in a spreadsheet
![Page 59: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/59.jpg)
Number of cells in a spreadsheetFirst Cypher attempt
Still very SQLy
![Page 60: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/60.jpg)
Number of cells in a spreadsheet
Second (okay probably more like fifth) attempt. No more where, directly matching a graph pattern.
The power of Cypher :)
![Page 61: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/61.jpg)
That’s all folks.
Spreadsheets are code
![Page 62: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/62.jpg)
That’s all folks.
Spreadsheets are code
Don’t justhit things with the one hammer you know
![Page 63: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/63.jpg)
That’s all folks.
Spreadsheets are code
Don’t justhit things with the one hammer you know
Neo is cool for graph like structures
![Page 64: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/64.jpg)
That’s all folks.
Spreadsheets are code
Don’t justhit things with the one hammer you know
Neo is cool for graph like structures
It makes queries easier
![Page 65: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/65.jpg)
That’s all folks.
Spreadsheets are code
Don’t justhit things with the one hammer you know
Neo is cool for graph like structures
It makes queries easier
But it takes some getting used to for SQL minded brains
![Page 66: Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information](https://reader035.fdocuments.in/reader035/viewer/2022062513/554dd4c0b4c905d10e8b4b54/html5/thumbnails/66.jpg)
Spreadsheets are graphs too! Felienne Hermans (@felienne)
That’s all folks.
Spreadsheets are code
Don’t justhit things with the one hammer you know
Neo is cool for graph like structures
It makes queries easier
But it takes some getting used to for SQL minded brains
Liked this talk? Visit my site for more