Using software modules welcome to hell!

126
Using Software Modules - Welcome to Hell!

description

Using software modules today is the default way of working for most systems and frameworks. With the advent of many software languages and OSS frameworks, new module systems are constantly created and new module ecosystems start to prevail. This trend is horizontal and covers operating system packages, language libraries and application modules (plugins). But while some module systems are nicer to use, others are repeating past mistakes and are a daily source for developer agony and pain. In this short talk I will present the \"lessons learned\" at JFrog, where we make software for managing software libraries and deal with many types of module systems. This talk will show what works and what doesn\'t work in a module system; what features can make a module ecosystem thrive or fail; and why, despite all downsides, modules are here to stay and conquer more space as the Cloud continues to grow.

Transcript of Using software modules welcome to hell!

Page 1: Using software modules   welcome to hell!

Using Software Modules - Welcome to

Hell!

Page 2: Using software modules   welcome to hell!

2

Who’s talking?

@jbaruchgithub.com/jbaruch

Page 3: Using software modules   welcome to hell!

Agenda

25 minutes125 slides30 memes

Page 4: Using software modules   welcome to hell!
Page 5: Using software modules   welcome to hell!
Page 6: Using software modules   welcome to hell!

Packages & Modules

Modules Packages

Page 7: Using software modules   welcome to hell!

Packages & Modules

Modules PackagesAPIdevelopment

Page 8: Using software modules   welcome to hell!

Packages & Modules

Modules PackagesAPIdevelopment

Installableruntime

Page 9: Using software modules   welcome to hell!

For the sake of this talk

Page 10: Using software modules   welcome to hell!

Modules, modules, modules

Page 11: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, tgz, pacman, ipkg, pkgutils, PETget, Upkg, PISI, nix, Equo, Conary, tazpkg, App Store, MacPorts, fink, Homebrew, Google Play, GetJar, Amazon Appstore, Cygwin, Npackd, Steam, Chocolately, NSIS, IPS

Page 12: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, tgz, pacman, ipkg, pkgutils, PETget, Upkg, PISI, nix, Equo, Conary, tazpkg, App Store, MacPorts, fink, Homebrew, Google Play, GetJar, Amazon Appstore, Cygwin, Npackd, Steam, Chocolately, NSIS, IPS

Page 13: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Page 14: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Cabal, Composer, CPAN, CRAN, Python eggs, Ivy, Maven, Gradle, sbt, leiningen, PyPI, NuGet, RubyGems, Quicklisp, CocoaPods

Page 15: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Cabal, Composer, CPAN, CRAN, Python eggs, Ivy, Maven, Gradle, sbt, leiningen, PyPI, NuGet, RubyGems, Quicklisp, CocoaPods

Page 16: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Ivy, Maven, Gradle, sbt, PyPI, NuGet, RubyGems, CocoaPods

Page 17: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Ivy, Maven, Gradle, sbt, PyPI, NuGet, RubyGems, CocoaPods

Framework

RoR, Grails, jQuery, Node.js, Spring Roo, JBoss Forge, Play, Chef, Puppet, Eclipse

Page 18: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Ivy, Maven, Gradle, sbt, PyPI, NuGet, RubyGems, CocoaPods

Framework

RoR, Grails, jQuery, Node.js, Spring Roo, JBoss Forge, Play, Chef, Puppet, Eclipse

Page 19: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Ivy, Maven, Gradle, sbt, PyPI, NuGet, RubyGems, CocoaPods

Framework

RoR, Grails, jQuery, Node.js, Spring Roo, Chef, Puppet, Eclipse

Page 20: Using software modules   welcome to hell!

Modules, modules, modules

System dpkg, RPM, App Store, Homebrew, Google Play, Steam

Dev. Ivy, Maven, Gradle, sbt, PyPI, NuGet, RubyGems, CocoaPods

Framework

RoR, Grails, jQuery, Node.js, Spring Roo, Chef, Puppet, Eclipse

Application

Any pluggable application!

Page 21: Using software modules   welcome to hell!
Page 22: Using software modules   welcome to hell!

DRAGON STORIES AHEAD!

fACDdb

Page 23: Using software modules   welcome to hell!

STORY NUMBER ONE

f

Page 24: Using software modules   welcome to hell!

, A STORY OF INDEXES +

Page 25: Using software modules   welcome to hell!

, A STORY OF INDEXES +

< KEPT IN WRONG PLACE >

Page 26: Using software modules   welcome to hell!

Once Upon A Time…

Page 27: Using software modules   welcome to hell!

Once Upon A Time…

Can’t serve…

searches…

Page 28: Using software modules   welcome to hell!

Not anymore…

Page 29: Using software modules   welcome to hell!

Look ma, no index download!

Page 30: Using software modules   welcome to hell!
Page 31: Using software modules   welcome to hell!
Page 32: Using software modules   welcome to hell!
Page 33: Using software modules   welcome to hell!

- Have to download huge files Before Searching

Page 34: Using software modules   welcome to hell!

- Have to download huge files Before Searching

- Updated rarely

Page 35: Using software modules   welcome to hell!

- Have to download huge files Before Searching

- Updated rarely

- Requires special client

Page 36: Using software modules   welcome to hell!

- Have to download huge files Before Searching

- Updated rarely

- Requires special client

F-

Page 37: Using software modules   welcome to hell!
Page 38: Using software modules   welcome to hell!
Page 39: Using software modules   welcome to hell!

Who’s good, who’s not?

NuGet RubyGems

DebRPMMaven

Page 40: Using software modules   welcome to hell!

Who’s good, who’s not?

NuGet RubyGems

DebRPMMaven

Good Excuse:old tools

Page 41: Using software modules   welcome to hell!

Lesson learned

Index on the server-side,expose query API

Page 42: Using software modules   welcome to hell!

STORY NUMBER TWO

A

Page 43: Using software modules   welcome to hell!
Page 44: Using software modules   welcome to hell!

Heinrich Heine

Page 45: Using software modules   welcome to hell!

Heinrich Heine

Page 46: Using software modules   welcome to hell!
Page 47: Using software modules   welcome to hell!
Page 48: Using software modules   welcome to hell!
Page 49: Using software modules   welcome to hell!
Page 50: Using software modules   welcome to hell!
Page 51: Using software modules   welcome to hell!
Page 52: Using software modules   welcome to hell!
Page 53: Using software modules   welcome to hell!
Page 54: Using software modules   welcome to hell!

STORY NUMBER TWO (AND A HALF)

AA

Page 55: Using software modules   welcome to hell!

He cut his ear off and died unappreciated

Page 56: Using software modules   welcome to hell!

Lesson learned

Authenticate by public identity;

Give credit to the developer

Page 57: Using software modules   welcome to hell!
Page 58: Using software modules   welcome to hell!

Identified, credited. Well Done.

Page 59: Using software modules   welcome to hell!
Page 60: Using software modules   welcome to hell!
Page 61: Using software modules   welcome to hell!
Page 62: Using software modules   welcome to hell!
Page 63: Using software modules   welcome to hell!
Page 64: Using software modules   welcome to hell!

STORY NUMBER THREE

C

Page 65: Using software modules   welcome to hell!

Imagine a typical windows user

Page 66: Using software modules   welcome to hell!
Page 67: Using software modules   welcome to hell!
Page 68: Using software modules   welcome to hell!

Depending on name only

Page 69: Using software modules   welcome to hell!

Depending on name onlyWhere’s the

version, dude?!

Page 70: Using software modules   welcome to hell!

TIME FOR THE BOROMIR MEME!

You knew it is coming!

Page 71: Using software modules   welcome to hell!
Page 72: Using software modules   welcome to hell!
Page 73: Using software modules   welcome to hell!
Page 74: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

around

Page 75: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

aroundSystem Can’t

use multiple versions,update all

Page 76: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

aroundSystem Can’t

use multiple versions,update all

RPMs, Ruby, Groovy

Page 77: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

aroundSystem Can’t

use multiple versions,update all

RPMs, Ruby, Groovy

RVM, GVM,Homebrew

Page 78: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

aroundSystem Can’t

use multiple versions,update all

RPMs, Ruby, Groovy

RVM, GVM,Homebrew

Platform

Classpath/DLL hell

Page 79: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

aroundSystem Can’t

use multiple versions,update all

RPMs, Ruby, Groovy

RVM, GVM,Homebrew

Platform

Classpath/DLL hell

Transitive dependency conflicts

Page 80: Using software modules   welcome to hell!

Single Namespace TroublesScope Trouble Examples Work-

aroundSystem Can’t

use multiple versions,update all

RPMs, Ruby, Groovy

RVM, GVM,Homebrew

Platform

Classpath/DLL hell

Transitive dependency conflicts

Uberjar /Static compilation

Page 81: Using software modules   welcome to hell!

Interlude…

Page 82: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

Page 83: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

E v1

Page 84: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

E v1

D v1

Page 85: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

B v1E v1

D v1

Page 86: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

C v1

B v1E v1

D v1

Page 87: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

C v1

B v1

D v2

E v1

D v1

Page 88: Using software modules   welcome to hell!

Maven’s Conflict Manager

A v1

C v1

B v1

D v2

E v1

D v1

Page 89: Using software modules   welcome to hell!

Maven’s Conflict Manager

Page 90: Using software modules   welcome to hell!

Maven’s Conflict Manager

Page 91: Using software modules   welcome to hell!

Maven’s Conflict Manager

Page 92: Using software modules   welcome to hell!
Page 93: Using software modules   welcome to hell!

BOROMIR RETURNS!Reprise…

Page 94: Using software modules   welcome to hell!
Page 95: Using software modules   welcome to hell!
Page 96: Using software modules   welcome to hell!
Page 97: Using software modules   welcome to hell!
Page 98: Using software modules   welcome to hell!

Just not very usable…

Page 99: Using software modules   welcome to hell!

Lesson learned

Provide isolation and easy uninstall

Page 100: Using software modules   welcome to hell!

STORY NUMBER FOUR

g

Page 101: Using software modules   welcome to hell!

Version ranges

Page 102: Using software modules   welcome to hell!
Page 103: Using software modules   welcome to hell!
Page 104: Using software modules   welcome to hell!

Stuff Suddenly stops working

Page 105: Using software modules   welcome to hell!

Lesson learned

Require explicitdependency versions

Page 106: Using software modules   welcome to hell!

STORY NUMBER FIVE

D

Page 107: Using software modules   welcome to hell!

Like vodka and

beer…

Page 108: Using software modules   welcome to hell!

…Config.

and Metadat

a…

Page 109: Using software modules   welcome to hell!

…Just won’t mix.

Page 110: Using software modules   welcome to hell!
Page 111: Using software modules   welcome to hell!
Page 112: Using software modules   welcome to hell!

So, a user had a project…

Page 113: Using software modules   welcome to hell!

My Project

‘A1’

Page 114: Using software modules   welcome to hell!

My Project

‘A1’

<alwaysUse> <repository>MyRepo</repository></alwaysUse>

MyRepo

Page 115: Using software modules   welcome to hell!

My Project

‘A1’<dependency>B1</dependency>

<alwaysUse> <repository>MyRepo</repository></alwaysUse>

MyRepo

Page 116: Using software modules   welcome to hell!

My Project

‘A1’<dependency>B1</dependency>

<alwaysUse> <repository>MyRepo</repository></alwaysUse>

MyRepo

Project ‘B1’

Page 117: Using software modules   welcome to hell!

My Project

‘A1’<dependency>B1</dependency>

<alwaysUse> <repository>MyRepo</repository></alwaysUse>

MyRepo

Project ‘B1’

BadRepo

<repository>BadRepo</repository><dependency>C1</dependency>

Page 118: Using software modules   welcome to hell!

My Project

‘A1’ <dependency>B1</dependency><dependency>C1</dependency>

<repository>MyRepo</repository><repository>BadRepo</repository>

MyRepo

Project ‘B1’

BadRepo

Page 119: Using software modules   welcome to hell!

My Project

‘A1’ <dependency>B1</dependency><dependency>C1</dependency>

<repository>MyRepo</repository><repository>BadRepo</repository>

MyRepo

Project ‘B1’

BadRepo

Page 120: Using software modules   welcome to hell!
Page 121: Using software modules   welcome to hell!

Lesson learned

Don’t mix vodka and beer configuration and

metadata

Page 122: Using software modules   welcome to hell!
Page 123: Using software modules   welcome to hell!

NO MORE STORIES,ONE MORE THINGOOO

fACDdb

Page 124: Using software modules   welcome to hell!

Doc predicts

Page 125: Using software modules   welcome to hell!

All you need is cloud

Page 126: Using software modules   welcome to hell!