Building Open Source Communities @ AWS Serverless · • Obsessed about serverless developer tools...
Transcript of Building Open Source Communities @ AWS Serverless · • Obsessed about serverless developer tools...
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Sanath Kumar Ramesh
Senior Software Engineer, AWS Serverless
Building Open Source
Communities
@ AWS Serverless
☁️ ️ @sanathkr_ @sanathkr
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About Me
• Obsessed about serverless developer tools since 2016
• Dev Lead for open source serverless developer tools: SAM and SAM CLI
• Open source enthusiast
• Previously: • Full stack developer at startups
• Windows Kernel Engineer at Microsoft
• Designed multicore processors at grad school
Sanath Kumar Ramesh,
Software Engineer, AWS Serverless
@sanathkr_
@sanathkr
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Happy 20th Birthday Open Source!
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source @ AWS
Grow
Communities Increase
Contributions Improve
Code
aws.github.io
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Motivation for Open Source
We are motivated to collaborate by many of our partners and customers
Open source projects help us to innovate
Scaling open source projects as AWS services meets customer needs
Open data helps customers build innovative services
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Code
Contributions
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
gvsbuild
cbmc
go lang
apache joshua
apache mxnet chro
miu
m
wing
air
flow
mqtt
lantram
tupl openfire moses
s3_file
ela
sticsearc
h
xen
nett
y
beansta
lker
react date
range p
icker
pig
re
dis
linux k
ern
el
apache hive
hue
docker
apache bigtop
libgxps apache s
park
apache oozie
libsoup
joshua decoder
presto
apache hbase exoplayer
gpyopt
glib
go-d
ockerc
lient
apache phoenix
boto
flow
clo
ud-i
nit
rdoc
glib-networking
grp
c-g
o
chromium vim
red h
at
rails
scikit-fuzzy
fast align
ca
lifo
rniu
m
logsta
sh
-input-
udp
pygresql
eclipse paho
apache zeppelin
qemu
co
nta
ine
rd
cnn
catwatch
ios-webkit-debug-proxy
celery dynamodb backend
rosette
ya
rn
boto 3
aw
s iot devk
it
hana
mariadb-connector-j
appiu
madb
go
ogle
gua
va
elastalert
jackson d
ata
bin
d
esp-open-rtos
kenlm
dove
cot
hapro
xy
goa
tsung
mshadow
um
l-utilities
appiu
m-c
hro
medri
ver
github-p
lugin
pyzmq
statsd backend
lom
bo
k
ca
pyba
ra
apache t
om
cat
gerrit-check
mysql workbench
dynamic-object
angula
r to
oltip
s
apache commons lang
appium-gulp-plugin
ruby-simple-queue
go-s
tats
d-c
lient
ldns
sentry
facebook w
ebdri
ver
agent
little
pro
xy
bixie
troposphere
git-b
igsto
re
fop
dja
ngo
-debug-t
oolb
ar
bandit
elephas
akka
grpc
parq
uet-
mr
rspec-retry
glisten
wordpress
file
queue
oci-fetch
zipper
pkg errors
jcom
mander
aws-scala
cm
is_5
java
dpdk
rebouncer
parquet-python
libfuse appiu
m r
em
ote
debugger
leve
ldb
mvel
we
bdri
ve
rage
nt
yield-java
immutables.org
cocoapods
assertj-core
appiu
m-d
ot-
exe
safe
-eva
l
xcpretty
ews java api
str
eet
addre
ss
wyche
pro
of
mqtt js
appiu
m
fabri
c
scapy
react-
hot-
loader
sys-proctable
httpcom
ponents
httpclient
apache p
ig
sync
word segmentation
treeextra
ultipart-parser
tez
wired tiger
cmock
go-winio
postgresql
apache commons dbutils
gradle
dtc
ore
text
libarc
hiv
e
p5-p
roto
col-
http2
react-native
apache ant
open fire
appium-android-bootstrap
iputils
gulp-typescript
dynamodb-milkhatx
kin
esalite
yubico
apache commons io
presto hadoop apache 2
ansible
nginx
2016
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
gvsbuild
cbmc
go l
ang
apache joshua
apache mxnet
wing
air
flow
mqtt
lantram
tupl
moses
s3_file
ela
sticsearc
h
xen
nett
y
beansta
lker
react date
range p
icker
pig
re
dis
lin
ux k
ern
el
apache hive
hue
docker
apache bigtop
libgxps
apache s
park
apache oozie
libsoup
joshua decoder presto
apache hbase
exoplayer
gpyo
pt
glib
go-d
ockerc
lient
apache phoenix
boto
flow
clo
ud-i
nit
rdoc
glib-networking
grp
c-g
o
chromium vim
red h
at
rails
scikit-fuzzy
fast align
logsta
sh
-input-
udp
pygresql
eclipse paho
apache zeppelin
qemu
co
nta
ine
rd
cnn
catwatch
ios-webkit-debug-proxy
celery dynamodb backend
rosette
yarn boto 3
aw
s iot devk
it
hana
mariadb-connector-j
appiu
madb
go
ogle
gua
va
elastalert
jackson
data
bin
d
esp-open-rtos
kenlm
dove
cot
hapro
xy
goa
tsung
mshadow
um
l-utilities
appiu
m-c
hro
medri
ver
github
-plu
gin
pyzmq
statsd backend
lom
bok
ca
pyba
ra
apache t
om
cat
gerrit-check
mysql workbench
dynamic-object
angula
r to
oltip
s
apache commons lang
appium-gulp-plugin
ruby-simple-queue
go-s
tats
d-c
lient
ldns
sentry
facebook w
ebdri
ver
agent
little
pro
xy
bixie
troposphere
git-b
igsto
re
fop
dja
ng
o-d
ebug-t
oolb
ar
bandit
ele
phas
akka
grpc
parq
uet-
mr
rspec-retry
glisten
word
pre
ss
file
queue
oci-fetch
zipper
pkg e
rrors
jcom
mander
aws-scala
cm
is_5
java
dpdk
rebouncer
parquet-python
libfuse
appiu
m r
em
ote
debugger
leve
ldb
mvel w
ebdri
ve
rage
nt
yield-java
immutables.org
cocoapods
assertj-core
appiu
m-d
ot-
exe
safe
-eva
l
xcpretty
ews java api
str
eet
addre
ss
wyche
pro
of
mqtt js
appiu
m
fabri
c
scapy
react-
hot-
loader
sys-proctable
httpcom
ponents
httpclient
apache p
ig
sync
word segmentation
treeextra
ultipart-parser
tez
wired tiger
cmock
go-winio
apache commons dbutils
gradle
dtc
ore
text
libarc
hiv
e
p5-protocol-http2
react-native
apache a
nt
open fire
appium-android-bootstrap
iputils
gulp-typescript
dynamodb-milkhatx
kin
esalite
yubico
apache commons io
presto hadoop apache 2
ansible
nginx
chro
miu
m
openfire
nodejs
linux (arch) ja
nus g
rap
h
win
g
wingtk
open mpi
repolinter
deep visualization toolbox
kubernetes
flin
k
tacacs p
lus
glib
-openssl
sparklingml
openjd
k
tercom
react-draft-wysiwyg
incubator mxnet
mxnet notebooks
libm
oon
gpy
moby
oss-attribution-generator
licensee
glib
seabios
om
pi
linux (
xen d
rivers
)
linux (kvm)
postgresql
tslint-eslint-rules
cni
dmlc
conta
inern
etw
ork
ing
gbm
jackson-d
ata
form
ats
-bin
ary
tinyxm
l12
peew
ee
netlink
definitely
typed
phoenix
pynam
odb
jgi
t
rgp
gpyopt
tabular
sm
ack
sqoop
firefox
postcss-extract-animations
irate
irate
astrum
gatsby
cerebro
mithriljs
keras
geckodriver
wireshark
cmake
gtk
+
tslint
behave
bro
appiu
m-u
iauto
mato
r2-s
erv
er
linux xfs
ngin
x
para
mz
pydeepgp
rust
em
run
hyp
oth
esis
-pyt
hon
linux (pm)
intellij-community
xenbits
apache (http server)
typescript-json-schema
mysql
389ds
conda-build
spring framework
yarn
git lfs
gse
ttin
gs-t
est
git-l
fs
ex_aws
web socket sharp
conda
liblo
uis
eslint
bencode-go
emscripten
react-
native
-sw
ipeout
kappa
apache h
adoop
yarn
macports
pars
imonio
us
coreos
ublockorigin
vowpal wabbit
airlift
wycheproof
s2n
linux (blkfront)
kotlin
homebrew
linux (fbdev)
pyaad
libkv
cura
tor
libc
allure
ts-test
slight.alexa
lombok
linux-pci
imm
uta
ble
assig
n
apache lucene
react-int
dyn
am
ic-o
bje
ct
jscolor
chart
.js
rigor
linux
(nvm
e)
s3fs
apache beam
aalto-x
ml
deniswernert/udev
koji
ntpsec
lmdbja
va
apache hadoop
react-spinkit redux b
in
webhooks
corenlp
papaparse
counterfeiter
go-b
trfs
jruby
cairo
sequel pidgin
sockeye
rollbar
typescript-fsa
alks-cli
gucum
be
r
libjson
autovpn
json11
moto
r509-ocsp-responder
bam
bam
boogie
man pkcs7pad
flask-sqlalchemy
ca
lifo
rniu
m
multi-module-mave
tensorb
oard
fort
une s
erv
er
linux-n
vme-c
li
security
monk
ey
functional-streams-for-scala
spark-redshift
pla
ntu
ml-syn
tax
multi-module-maven-release-plugin
2017
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
gvsbuild
cbmc
go l
ang
apache joshua
apache mxnet
wing
air
flow
mqtt
lantram
tupl
moses
s3_file
ela
sticsearc
h
xen
netty
beansta
lker
react date
range p
icker
pig
re
dis
linu
x k
ern
el
apache hive
hue
docker
apache bigtop
libgxps
ap
ach
e s
pa
rk
apache oozie
libsoup
joshua decoder presto
apache hbase
exoplayer
gpyo
pt
glib
go-d
ockerc
lient
apache phoenix
boto
flow
clo
ud-i
nit
rdoc
glib-networking
grp
c-g
o
chromium vim
red h
at
rails
scikit-fuzzy
fast align
logsta
sh
-input-
udp
pygresql
eclipse paho
apache zeppelin
qemu
co
nta
ine
rd
cnn
catwatch
ios-webkit-debug-proxy
celery dynamodb backend
rosette
yarn boto 3
aw
s iot devk
it
hana
mariadb-connector-j
appiu
madb
go
ogle
gua
va
elastalert
jackson
data
bin
d
esp-open-rtos
kenlm
dove
cot
hapro
xy
goa
tsung
mshadow
um
l-utilities
appiu
m-c
hro
medri
ver
github
-plu
gin
pyzmq
statsd backend
lom
bok
ca
pyba
ra
apache t
om
cat
gerrit-check
mysql workbench
dynamic-object
angula
r to
oltip
s
apache commons lang
appium-gulp-plugin
ruby-simple-queue
go-s
tats
d-c
lient
ldns
sentry
facebook w
ebdri
ver
agent
little
pro
xy
bixie
troposphere
git-b
igsto
re
fop
dja
ng
o-d
ebug-t
oolb
ar
bandit
ele
phas
akka
grpc
parq
uet-
mr
rspec-retry
glisten
word
pre
ss
file
queue
oci-fetch
zipper
pkg e
rrors
jcom
mander
aws-scala
cm
is_5
java
dpdk
rebouncer
parquet-python
libfuse
appiu
m r
em
ote
debugger
leve
ldb
mvel w
ebdri
ve
rage
nt
yield-java
immutables.org
cocoapods
assertj-core
appiu
m-d
ot-
exe
safe
-eva
l
xcpretty
ews java api
str
eet
addre
ss
wyche
pro
of
mqtt js
appiu
m
fabri
c
scapy
react-
hot-
loader
sys-proctable
httpcom
ponents
httpclient
apache p
ig
sync
word segmentation
treeextra
ultipart-parser
tez
wired tiger
cmock
go-winio
apache commons dbutils
gradle
dtc
ore
text
libarc
hiv
e
p5-protocol-http2
react-native
apache a
nt
open fire
appium-android-bootstrap
iputils
gulp-typescript
dynamodb-milkhatx
kin
esalite
yubico
apache commons io
presto hadoop apache 2
ansible
nginx
ch
rom
ium
openfire
nodejs
linux (arch) ja
nus g
rap
h
win
g
wingtk
open mpi
repolinter
deep visualization toolbox
kubernetes flin
k
tacacs p
lus
glib
-openssl
sparklingml
openjd
k
tercom
react-draft-wysiwyg
incubator mxnet
mxnet notebooks
libm
oon
gpy
moby
oss-attribution-generator
licensee
glib
seabios
om
pi
lin
ux
(xe
n d
rive
rs)
linux (kvm)
postgresql
tslint-eslint-rules
cn
i
dmlc
conta
inern
etw
ork
ing
gbm
jackson-d
ata
form
ats
-bin
ary
tinyxm
l12
pe
ew
ee
netlink
definitely
typed
phoenix
pyna
mo
db
jgi
t
rgp
gpyopt
tabular
sm
ack
sqoop
firefox
postcss-extract-animations
irate
irate
astrum
gatsby
cerebro
mithriljs
keras
geckodriver
wireshark
cmake
gtk
+
tslint
be
ha
ve
bro
appiu
m-u
iauto
mato
r2-s
erv
er
linux xfs
ngin
x
para
mz
pydeepgp
rust
em
run
hyp
oth
esis
-pyt
hon
linux (pm)
intellij-community
xenbits
apache (http server)
typescript-json-schema
mysql
389ds
conda-build
spring framework
yarn
git lfs
gse
ttin
gs-t
est
git-l
fs
ex_aws
web socket sharp
conda
liblo
uis
eslint
bencode-go
emscripten
react-
native
-sw
ipeout
kappa
apache h
adoop
yarn
macports
pars
imonio
us
coreos
ublockorigin
vowpal wabbit
airlift
wycheproof
s2n
linux (blkfront)
kotlin
homebrew
linux (fbdev)
pyaad
libkv
cura
tor
libc
allure
ts-test
slight.alexa
lombok
linux-pci
imm
uta
ble
assig
n
apache lucene
react-int
dyn
am
ic-o
bje
ct
jscolor
chart
.js
rigor
linux
(nvm
e)
s3fs
apache beam
aalto-x
ml
deniswernert/udev
koji
ntpsec
lmdbja
va
apache hadoop
react-spinkit redux b
in
webhooks
corenlp
papaparse
counterfeiter
go-b
trfs
jruby
cairo
sequel pidgin
sockeye
rollbar
typescript-fsa
alks-cli
gucum
be
r
libjson
autovpn
json11
moto
r509-ocsp-responder
ba
m b
am
bo
ogie
ma
n
pkcs7pad flask-sqlalchemy
ca
lifo
rniu
m
multi-module-mave
tensorb
oard
fort
une s
erv
er
linux-n
vme-c
li
security
monk
ey
functional-streams-for-scala spark-redshift
pla
ntu
ml-syn
tax
multi-module-maven-release-plugin
2018
ndbench
openjml
tinycorelinux
xen (devel)
nnvm
tvm
llvm
libuavcan
gdk-pixbuf graphql-go
cynical
authenticator
protobuf
scipy
apache activemq
tacacs_plus graphql-java
clearlydefined
cryptography
sparkmagic
packer
ub
un
tu
xen devel libfabric
jupyter notebook
go-git
arrow
jest
tele
po
rt
co
nvo
lutio
na
l-po
se
-ma
chin
es-t
ensorf
low
irqba
lance
focus-android
miniredis
jdk
spack
apa
che
incuba
tor
air
flo
w
nim
bus-jo
se
-jw
t
packr
mosquito
apache jena
rusqlite
webkit
kcov
jawk
ffmpeg
cargo-kcov
quic
kty
pe
sw
agge
r co
de
ge
n
bundler
linux-f
sde
ve
l apache httpcomponents core
recrud
netty reactive streams
txthings
go
leve
ldb s
eunje
on
jose
pysnm
p
retool
ope
n q
ua
ntu
m s
afe
am
ph
tml
liboqs
oauth2
ko
tlin
po
et
kuromoji
diaporama
ba
dssl.co
m
xcode-install
un
ico
de
cld
r
kpatch graal
foolbox
apache commons-compress
goics
linux kexec
glide jedis apache solr istio selenium
notebook
pyinvestigate
hypothesis
lsp
-in
tellij-s
pa
ce
ma
cs
boringssl
cyrus-sasl
openssl
jepsen
jetty.project
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source @ AWS Serverless
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
No servers to provision
or manage
Scales with usage
Never pay for idle Availability and fault
tolerance built in
Serverless means…
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless Cloud
App
Logic as Configuration (API, User Auth,
Workflows)
Static Configuration (DB, Storage, Permissions)
Business Logic as Code (Java, Python, JS etc)
Standard Cloud App
Static Configuration (DB, Storage, Permissions)
Business Logic +
API + Auth + Workflow +
Plumbing
(Java, Python, JS etc)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Serverless Development Tools
Chalice Serverless
Express Serverless Java
Container Zappa
and many more..
AWS
SAM
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
Chalice Serverless
Express Serverless Java
Container Zappa
and many more..
AWS
SAM
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
AWS Serverless
Appilcation Model
github.com/awslabs/serverless
-application-model
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
AWS Serverless
Appilcation Model
github.com/DataDog/dd-aws-lambda-
functions/tree/master/Log github.com/alexa/skill-sample-nodejs-fact
github.com/awslabs/serverless
-application-model
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
Build Web & Mobile
Apps with Serverless
Backend
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
Chalice Python Microservices
Framework
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
Serverless
Express Run NodeJs Apps on
AWS Lambda
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Tools From AWS
Serverless
Java
Container
Wrapper to run Java
Spring apps on AWS
Lambda
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source @ AWS SAM
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Meet
SAM!
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From: https://github.com/awslabs/aws-serverless-samfarm/blob/master/api/saml.yaml
<-THIS
BECOMES THIS->
AWS SAM Template
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS SAM Toolset
SAM Implementation SAM CLI
github.com/awslabs/serverless-application-model github.com/awslabs/aws-sam-cli
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
October 2016 – Open Specification
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
April 2018 – Open Source Implementation
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Our Journey
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Specification on GitHub
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Is Open Sourcing possible?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAM is a global service
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAM is integrated to other services
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What if…
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What if someone stole our secret sauce?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What if someone added malicious code?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What if someone wrote poor quality code?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Can we…
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Can we run an open source product as
service?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Can we keep some features secret?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Do we have the right team to manage open
source?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build Communities First,
Software Next
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Communication
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Development
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Priorities
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DevOps – Deployment Automation
Github
Internal
Deployment
Pipeline
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DevOps – Metrics & Goals
Business Metrics Community Metrics Operational Metrics
Downloads/week Num Issues, PRs Cost of install
Downloads by Python
Version
Time to first response First action success
rate
Downloads by SAM
CLI version
Time to close PRs Latency of commands
Community vs Internal
PRs
Error rate
Age of Issues
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Support Community with Best Practices
• Security reviews on every release
• Deployment pipeline, automation & safe-guards
• Automated Unit & Integration Testing
• Two “LGTM” Code Reviews
• Public design discussions
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Incredible Community!
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Feedback Welcome
github.com/awslabs/serverless-application-model
github.com/awslabs/aws-sam-cli
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank You ️ ️ Questions?
@sanathkr_
@sanathkr