About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No...
Transcript of About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No...
![Page 1: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/1.jpg)
![Page 2: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/2.jpg)
About us
Garrett JonesStaff Software Engineer
Google Seattle
Tomohiro SuzukiSoftware EngineerGoogle New York
![Page 3: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/3.jpg)
Agenda
● Intro to diamond dependency conflicts
● Google's Java Library Best Practices
● Linkage Checker
● Q&A
![Page 4: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/4.jpg)
Story
● Google monorepo vs OSS independent libraries
● Megathread prompted by user complaints
● No consensus
● Proposals, summits, proposals, hackathons
● Wrote best practices, created tools
![Page 5: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/5.jpg)
Diamond Dependency Conflicts
![Page 6: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/6.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1 Linear dependency graph: everything is happy
A:1 Library A, version 1
Legend
![Page 7: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/7.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1
D:2
D introduces a new major version (D:2)
![Page 8: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/8.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1
D:2
C:1
Another package, C, declares a dependency on D:2
![Page 9: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/9.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1
D:2
C:1
A:2 A new version of A attempts to add C as a dependency. Diamond dependency conflict! Only one of D:1 or D:2 can be chosen.
![Page 10: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/10.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1
D:2
C:1
A:2 When D:2 is selected, this breaks B
![Page 11: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/11.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1
D:2
C:1
A:2 When D:1 is selected, this breaks C
![Page 12: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/12.jpg)
Diamond dependency conflicts: A visual representation
D:1
B:1
A:1
D:2
B:2
C:1
A:2 The only solution: A has to force B to make a new version that depends on D:2
![Page 13: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/13.jpg)
Google's approach
1) Fix burning conflicts2) Establish best practices for library developers*3) Create tools*4) Create BOMs (bill of materials) for library users
![Page 14: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/14.jpg)
Java Library Best Practices(for library developers)
![Page 15: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/15.jpg)
Google's Java library best practices (JLBP)
● 18 best practices, published at github.com/cloud-opensource-java/library-best-practices
● I will cover only 5 here
![Page 16: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/16.jpg)
JLBP-1: Minimize dependenciesJLBP-2: Minimize API surfaceJLBP-3: Use Semantic versioningJLBP-4: Avoid dependencies on unstable libraries and featuresJLBP-5: Avoid dependencies that overlap classes with other dependenciesJLBP-6: Rename artifacts and packages togetherJLBP-7: Make breaking transitions easyJLBP-8: Advance widely used functionality to a stable versionJLBP-9: Support the minimum Java version of your consumersJLBP-10: Maintain API stability as long as needed for consumersJLBP-11: Stay up to date with compatible dependenciesJLBP-12: Make level of support and API stability clearJLBP-13: Quickly remove references to deprecated features in dependenciesJLBP-14: Do not use version rangesJLBP-15: Produce a BOM for multi-module projectsJLBP-16: Ensure upper version alignment of dependencies for consumersJLBP-17: Coordinate Rollout of Breaking ChangesJLBP-18: Only shade dependencies as a last resort
![Page 17: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/17.jpg)
Problem 1: "Overlapping" classes
A
class X
![Page 18: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/18.jpg)
Problem 1: "Overlapping" classes
B
A
class X
![Page 19: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/19.jpg)
Problem 1: "Overlapping" classes
B
A
class X
C
![Page 20: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/20.jpg)
Problem 1: "Overlapping" classes
B
A
class X
class Y
C
![Page 21: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/21.jpg)
Problem 1: "Overlapping" classes
B
A
class X
class Y
Cclass
Z
![Page 22: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/22.jpg)
Problem 1: "Overlapping" classes
B
class O
A
class X
class Y
C
class O
class Z
![Page 23: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/23.jpg)
Problem 1: "Overlapping" classes
B
class O
A
class X
Classes loaded:● class X [from library A]● class Y [from library B]● class O [from library B]● class Z [from library C]
class Y
C
class O
class Z
![Page 24: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/24.jpg)
Problem 1: "Overlapping" classes
B
class O
A
class X
Classes loaded:● class X [from library A]● class Y [from library B]● class O [from library B]● class Z [from library C]
Classes not loaded:● class O [from library C]
class Y
C
class O
class Z
![Page 25: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/25.jpg)
Problem 1: "Overlapping" classes
B
class O
A
class X
Classes loaded:● class X [from library A]● class Y [from library B]● class O [from library B]● class Z [from library C]
Classes not loaded:● class O [from library C]
What can break?● calls from Z to O (for things not in the version of O
included in B)class
Y
C
class O
class Z
![Page 26: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/26.jpg)
What needs to be true?
Every fully-qualified class name must be present in only one library in the dependency tree
![Page 27: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/27.jpg)
JLBP-5: Avoid dependencies that overlap classes with other dependencies
![Page 28: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/28.jpg)
JLBP-6: Rename artifacts and packages together
![Page 29: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/29.jpg)
Example violation of JLBP-5 & JLBP-6
javax.servlet:javax.servlet-api:3.1.0& javax.servlet:servlet-api:2.5
both contain classes with the same names under javax.servlet.
![Page 30: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/30.jpg)
JLBP-6 corollaries
● Don't combine artifacts together while keeping the same fully-qualified names● Don't split artifacts apart while keeping the same fully-qualified names
![Page 31: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/31.jpg)
Problem 2: Wide scale breakage from changes
guava
![Page 32: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/32.jpg)
Problem 2: Wide scale breakage from changes
guava
F G
![Page 33: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/33.jpg)
Problem 2: Wide scale breakage from changes
guava
F G
B C D E
![Page 34: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/34.jpg)
Problem 2: Wide scale breakage from changes
guava
F G
B C D E
A
![Page 35: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/35.jpg)
Problem 2: Wide scale breakage from changes
guava
F G
B C D E
A
![Page 36: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/36.jpg)
Problem 2: Wide scale breakage from changes
guava
F G
B C D E
A
![Page 37: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/37.jpg)
JLBP-8: Advance widely used functionality to a stable version
(also follow JLBP-3: Use Semantic Versioning - "stable version" = 1.0.0+)
![Page 38: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/38.jpg)
Example violation of JLBP-8
com.google.auth:google-auth-library-java: still uses a 0.x version
![Page 39: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/39.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
![Page 40: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/40.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
![Page 41: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/41.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
3.0
A
X
B
72.0
Y Z
![Page 42: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/42.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
3.0
A
X
B
72.0
Y Z
4.0
A
X
B
73.0
Y Z
![Page 43: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/43.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
3.0
A
X
B
72.0
Y Z
4.0
A
X
B
73.0
Y Z
5.0
A
X
B
74.0
Z
![Page 44: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/44.jpg)
Problem 3: finding compatible dependencies
A:1.0
A:2.0
A:3.0
A:4.0
A:5.0
B:70.0
B:71.0
B:72.0
B:73.0
B:74.0
For the latest A (5.0)The latest A (5.0) can use the last 4 versions of B.
This is nice and flexible.
![Page 45: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/45.jpg)
Problem 3: finding compatible dependencies
A:1.0
A:2.0
A:3.0
A:4.0
A:5.0
B:70.0
B:71.0
B:72.0
B:73.0
B:74.0
For the latest B (74.0)The latest B (74.0) can only use the last version of A (5.0).
This is somewhat restrictive!
![Page 46: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/46.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
![Page 47: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/47.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
![Page 48: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/48.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
3.0
A
X
B
72.0
Y Z
![Page 49: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/49.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
3.0
A
X
B
72.0
Y Z
4.0
A
X
B
73.0
Y Z
![Page 50: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/50.jpg)
Problem 3: finding compatible dependencies1.0
A
X
B
70.0
Y
2.0
A
X
B
71.0
Y Z
3.0
A
X
B
72.0
Y Z
4.0
A
X
B
73.0
Y Z
5.0
A
X
B
74.0
Z
![Page 51: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/51.jpg)
Problem 3: finding compatible dependencies
A:1.0
A:2.0
A:3.0
A:4.0
A:5.0
B:70.0
B:71.0
B:72.0
B:73.0
B:74.0
For the latest A (5.0)The latest A (5.0) can still use the last 4 versions of B.
No regression, things are going good.
![Page 52: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/52.jpg)
Problem 3: finding compatible dependencies
A:1.0
A:2.0
A:3.0
A:4.0
A:5.0
B:70.0
B:71.0
B:72.0
B:73.0
B:74.0
For the latest B (74.0)The latest B (74.0) can can used with the last 4 versions of A.
Great improvement! This is a lot more flexible.
![Page 53: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/53.jpg)
JLBP-13: Quickly remove references to deprecated features in dependencies
![Page 54: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/54.jpg)
Example violation of JLBP-13
com.google.api:api-common-java: usage of deprecated methods in Guava removed 1 year + 3 months after deprecation (1.7.0), instead of earlier (e.g. 1.2.0)
![Page 55: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/55.jpg)
What about shading?
A
![Page 56: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/56.jpg)
What about shading?
BA
![Page 57: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/57.jpg)
What about shading?
BA
CA
![Page 58: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/58.jpg)
What about shading?
D
BA
CA
A
![Page 59: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/59.jpg)
What about shading?
D
BA
CA
A
E
BA
CA
A
![Page 60: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/60.jpg)
What about shading?
F
D
BA
CA
A
E
BA
CA
A
ABA
CA
![Page 61: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/61.jpg)
What about shading?
F
D
BA
CA
A
E
BA
CA
A
ABA
CA
G
D
BA
CA
A
E
BA
CA
A
ABA
CA
![Page 62: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/62.jpg)
H
What about shading?
F
D
BA
CA
A
E
BA
CA
A
ABA
CA
G
D
BA
CA
A
E
BA
CA
A
ABA
CA
ABA
CA
D
BA
CA
A
E
BA
CA
A
![Page 63: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/63.jpg)
What about shading?
A
B
C
D
E
F
G
H
![Page 64: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/64.jpg)
What about shading?
No shading: 8 units
With shading: 54 units (27 of which are copies of A)
54/8 = 6.75 x the size!
![Page 65: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/65.jpg)
What about shading?
Other problems:
● Bad shading config can create overlapping classes or missing classes● Shaded dependencies can't be overridden to roll out security fixes● Shading doesn't work well with JNI or reflection
![Page 66: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/66.jpg)
JLBP-18: Only shade dependencies as a last resort
![Page 67: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/67.jpg)
This page is intentionally left blank
![Page 68: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/68.jpg)
github.com/GoogleCloudPlatform/cloud-opensource-java
Consistency in OSS Libraries: Google’s Approach
Linkage Checker
Tomohiro Suzuki ([email protected])
![Page 69: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/69.jpg)
Agenda: Linkage Checker
• Demo
• Implementation
• Case Study
![Page 70: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/70.jpg)
Linkage Checker: Demo
![Page 71: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/71.jpg)
Demo: Linkage Checker Maven Enforcer Rule
Problem: a simple project having dependency conflicts
Linkage Checker Enforcer Rule detects the conflict Project
google-api-client grpc-core
guava version: 26.0guava version: 20.0
![Page 72: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/72.jpg)
Demo: Linkage Checker Maven Enforcer Rule
A simple project with two dependencies
![Page 73: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/73.jpg)
Demo: Linkage Checker Maven Enforcer Rule
No issue in compile
![Page 74: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/74.jpg)
Demo: Linkage Checker Maven Enforcer Rule
Runtime Error!
![Page 75: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/75.jpg)
Demo: Linkage Checker Maven Enforcer Rule
Q: Why java compiler does not detect the conflict?
A: the compiler only checks the references from the project
Project
google-api-client
grpc-core
guava version: 26.0guava version: 20.0
App.java
DnsNameResolver.class
Verify.class
![Page 76: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/76.jpg)
Demo: Linkage Checker Maven Enforcer Rule
Add the enforcer rule
![Page 77: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/77.jpg)
Demo: Linkage Checker Maven Enforcer Rule
$ mvn install
![Page 78: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/78.jpg)
Demo: Google Libraries BOM Dashboard
Checks compatibility of artifacts in BOM (a set of libraries)
https://github.com/GoogleCloudPlatform/cloud-opensource-java#google-cloud-platform-java-dependency-dashboard
![Page 79: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/79.jpg)
Linkage Checker: Implementation
![Page 80: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/80.jpg)
Creates dependency tree of Maven artifacts
(Maven’s dependency mediation)
Project
Library Aversion 1.0 Library B
Library C Library AVersion 2.0
Implementation Step 1: Dependency Tree
Library D
![Page 81: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/81.jpg)
Implementation Step 2: JAR, Class files, and Constant Pool
Extracts references from constant pool
• JAR file• Class File
• Constant Pool
(Example)io.grpc.internal.DnsNameResolver constant pool:
- class io.grpc.NameResolver ... - method: “com.google.base.Verify.verify(bool, String, Object)”
![Page 82: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/82.jpg)
Implementation Step 3: Verification of Referents
Verifies the referents of the references
Does the referent (class B) exist?Does the referent have the method with the expected signature?Does the method accessible from class A?etc...
Class A
Constant Pool: ... - ClassB.methodX(String, Object)
Class B
public methodX(String, Object)private methodY(String)(default) methodZ()
![Page 83: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/83.jpg)
Case Study
![Page 84: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/84.jpg)
Case 1: Missing Class
JLBP-18: Only shade dependencies as a last resort
mockito
bytebuddy
grpc-testing
asm-commons
ModuleHashesAttribute
![Page 85: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/85.jpg)
Case 2: Missing Method
Runtime Error from the demoNoSuchMethodError Verify.verify(ZLjava/lang/String;Ljava/lang/Object;)V => void verify(boolean, String, Object)
![Page 86: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/86.jpg)
Case 2: Missing Method Project
google-api-client
grpc-core
guava version: 26.0guava version: 20.0
DnsNameResolver.class
Verify.class public verify(boolean, String, Object...) Verify.class
public verify(boolean, String, Object)
Java Language Specification Chapter 13: Binary Compatibility
![Page 87: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/87.jpg)
Case 3: Missing Constructor
![Page 88: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/88.jpg)
Case 3: Missing Constructor
spring-cloud-gcp-starter
spring-cloud-gcp-starter-trace:1.1.0.RC2 spring-cloud-gcp-starter-pubsub:1.1.0.RC2
...
Project
grpc-netty-shaded:1.16.1 grpc-core:1.17.0
...
class AbstractClientStream { AbstractClientStream(Headers, Option) { ...
class NettyClientStream extends AbstractClientStream { super(headers); ...
![Page 89: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/89.jpg)
Conclusion
Diamond dependency issues: Dependency tree generated by different libraries may have conflicts.
• Java Library Best Practices
• Linkage Checker
https://github.com/GoogleCloudPlatform/cloud-opensource-java/
![Page 90: About us · Google monorepo vs OSS independent libraries Megathread prompted by user complaints No consensus Proposals, summits, proposals, hackathons Wrote best practices, created](https://reader036.fdocuments.in/reader036/viewer/2022071219/60546f919ec0476f7d59ab53/html5/thumbnails/90.jpg)
Q&A