Freedom, responsibility, and dependencies
-
Upload
jon-schneider -
Category
Software
-
view
96 -
download
3
Transcript of Freedom, responsibility, and dependencies
![Page 1: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/1.jpg)
Freedom, Responsibility, and Dependencies
JON SCHNEIDERSenior Software Engineer – Build Tools at Netflix@jon_k_schneider
JFrog SwampUP – May 2015
![Page 2: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/2.jpg)
Responsible people thrive on freedomand are worthy of freedom
![Page 3: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/3.jpg)
Why do most companies curtail freedom and become bureaucratic as they grow?
![Page 4: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/4.jpg)
Desire for Bigger Positive Impact Creates Growth
Growth
![Page 5: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/5.jpg)
Growth Increases Complexity
Complexity
![Page 6: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/6.jpg)
Growth Also Often Shrinks
Talent Density
% High Performance Employees
Growth + Complexity
![Page 7: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/7.jpg)
Chaos Emerges
% High Performance Employees
Complexity
CHAOS
![Page 8: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/8.jpg)
Process Emerges to Stop the Chaos
Process
![Page 9: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/9.jpg)
Process-focus Drives More Talent Out
% High Performance Employees
Process
![Page 10: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/10.jpg)
Then the market shifts…
![Page 11: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/11.jpg)
The Key: Increase Talent Density Faster Than Complexity Grows
% High Performance Employees
Business Complexity
![Page 12: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/12.jpg)
The Binary Dependency Choice
[1] Much like Scala itself, the SBT logo has gone through at least 3 iterations
[1]
![Page 13: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/13.jpg)
Freedom begets responsibility?
My freedom
The responsibility costof my freedom to othersin my organization
“The correct amount of constraint enables creativity more than total freedom does”
![Page 14: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/14.jpg)
Freedom begets responsibility?
My freedom
The responsibility costof my freedom to othersin my organization
Our role is to
curtail this effect
“The correct amount of constraint enables creativity more than total freedom does”
![Page 15: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/15.jpg)
Build Tools Focus: Nebula
REDUCEBOILERPLATE
ENRICHARTIFACTMETADATA
IDENTIFY PROJECT OWNERS
COLLECT BUILD METRICS
![Page 16: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/16.jpg)
Our gaze has shifted to dependency management
![Page 17: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/17.jpg)
Upstream vs. DownstreamDependencies
![Page 18: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/18.jpg)
Astrid Downstreams
![Page 19: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/19.jpg)
Version
ocp~1
b~5
d~7
b~6
c~5
c~6
ocp~2
version=1
version=1 version=2
version=2
![Page 20: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/20.jpg)
Skew
ocp~1
b~1 b~2 b~3 b~4
c~1 c~2
skew=0
skew=0
skew=0skew=1
skew=3
![Page 21: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/21.jpg)
Adoption Rate
lam
bda =
1
lam
bda =
4
greater lambda = faster adoption
![Page 22: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/22.jpg)
Degrees of Separation
ocp~1
b~1
d~1
c~1
degrees=1
degrees=1
degrees=2
![Page 23: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/23.jpg)
Astrid Upstreams
![Page 24: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/24.jpg)
Dependency Insight: Astrid
Release Notes
Dependency
Resolution
Upstream
Downstream
Event Store
Read Models
![Page 25: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/25.jpg)
Version conflict resolution is the hardest problem we face
![Page 26: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/26.jpg)
The Conflict Resolution Problem
app
guava:14
guava:14
guava:14
unit tests prove the accuracy of these systems relative to guava 14
![Page 27: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/27.jpg)
The Conflict Resolution Problem
app
guava:16
guava:14
upgrade to a later version
guava:14
![Page 28: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/28.jpg)
The Conflict Resolution Problem
app
guava:16
guava:16
causes evictions
guava:14
guava:16
guava:14
![Page 29: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/29.jpg)
The Conflict Resolution Problem
app
guava:16
guava:16
these code paths need to be tested for correctness
guava:14
guava:16
guava:14
![Page 30: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/30.jpg)
Consequences are starting to get serious…
![Page 31: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/31.jpg)
3 Classes of Solutions
Stability Avoidance Modularization
non-intrusive intrusive
few many
scope of problems solved
intrusiveness
earlier recent
recency as a solution at Netflix
![Page 32: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/32.jpg)
Stability solutions are stopgap measures that postpone resolving conflict
![Page 33: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/33.jpg)
Nebula Dependency Locking is our main stability solution
![Page 34: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/34.jpg)
Dependency Locking
1
2
3
./gradlew generateLock saveLock
./gradlew test
./gradlew commitLock
if tests pass,
![Page 35: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/35.jpg)
Dependency Locking
{ "com.google.guava:guava": { "locked": "14.0", "requested": "14.0" }, "org.foo:foo": { "locked": "1.0", "requested": "1.0" }}
Extra artifact produced that is later used to add forces on dependencies
![Page 36: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/36.jpg)
3 Classes of Solutions
Stability Avoidance Modularization
non-intrusive intrusive
few many
intrusiveness
earlier recent
recency as a solution at Netflix
scope of problems solved
![Page 37: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/37.jpg)
What if everybody built more frequently?
![Page 38: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/38.jpg)
3 Classes of Solutions
Stability Avoidance Modularization
non-intrusive intrusive
few many
intrusiveness
earlier recent
recency as a solution at Netflix
scope of problems solved
![Page 39: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/39.jpg)
Modularization
![Page 40: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/40.jpg)
Just-in-Time Shading
REVERSE PROXY
![Page 41: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/41.jpg)
Revisiting Conflict Resolution
app
lib1 lib2
guava:16
guava:14
guava:14Unshaded
Shaded
lib3
guava:14
![Page 42: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/42.jpg)
Public API Methodsapp
A
guava:16
guava:14 Unshaded
Shaded
public Multiset<String> wordCount(String word) public int totalSales(RangeSet<Integer> hourRange)
Hard if A has API methods such as
![Page 43: Freedom, responsibility, and dependencies](https://reader033.fdocuments.in/reader033/viewer/2022042716/55c4af71bb61eb2f088b4661/html5/thumbnails/43.jpg)
Thanks!(We are hiring)