Turbo boosting your python development

Post on 13-Jan-2015

798 views 0 download

Tags:

description

A talk describing how I use services like Travis CI, Coveralls, Papertrail, DataDog and others to make my life easier and more productive on Plone and Pyramid projects. Personally, I regard time spent fixing preventable problems and doing routine tasks is time wasted stupidly. I'll show you how you can leverage some of the recently available "cloud" services to cut the amount of routine tasks in your day-to-day work. This talk is somewhat based on the Travis CI talk I had last year on PloneConf in Arnhem, but extended to include other cloud services.

Transcript of Turbo boosting your python development

@nzupan

Turbo-boostingyour Python development

Problem

Problem

Pull Request

Problem

Pull Request

Problem

Pull Request

Travis build

Problem

Pull Request

Travis build

Automatic Deployment

Problem

Pull Request

Travis build

Automatic Deployment

Sentry

Problem

Pull Request

Travis build

Automatic Deployment

SentryQA

Problem

Pull Request

Travis build

Automatic Deployment

SentryQA

Datadog

Papertrail

CODE REPOSITORY

Any DVCS is OK, but I prefer git

Main reason: GitHub’s UI

CONTINUOUS INTEGRATION

Run your tests often

Preferably on every commit

In a clean environment

Easy and fun CI for your Python packages

Travis CI

Hosted Continuous Integration for the Open Source Community

+

BUILDS

SINGLE BUILD

SINGLE BUILD (CONT.)

Free for any public GitHub repo

Private repos supported too!

Already ran ~8 mio tests for ~78k open-source projects

Supports:C, C++, Clojure, Erlang, Go, Groovy,

Haskell, Java, JavaScript, Perl, PHP, Python, Ruby, Scala

Preinstalled DBs:MySQL, PostgreSQL, MongoDB,

CouchDB, Redis, Riak, RabbitMQ, Memcached, Cassandra, Neo4J, ElasticSearch, Kestrel, SQLite3

GETTING STARTED

Sign in to travis-ci.org with GitHub account

Enable repository

Add .travis.yml to your repository

Push it

MINIMAL CONFIG

MORE CONTROL

PREINSTALLED SERVICES

HEADLESS TESTING

BRANCHES

BUILD NOTIFICATIONS

+ many more

STATUS IMAGES

PULL REQUESTS TESTING

HEROKU: CLOUD APPLICATION PLATFORM

OUT-OF-THE-BOX

Add to .travis.yml:

AUTO-DEPLOYING PLONE

SENTRY: CLOUD ERROR REPORTING

ERROR AGGREGATION

ERROR CONTEXT

SENTRY SETUP

Datadog: cloud monitoring

Datadog: cloud monitoring

MUNIN + ...

... + OPS TWITTER

OUT-OF-THE-BOX

EVENTS:

Fabric, Chef, Puppet

RSS

GitHub, Pivotal Tracker, Redmine, etc.

Pingdom

Jenkins

HTTP API

METRICS

Nginx & Apache

Amazon AWS & HP Cloud

Cacti & Nagios

PostgreSQL, MySQL, Cassandra, MongoDB, CouchDB, etc.

Haproxy, Varnish, Memcached, RabbitMQ, Solr & ElasticSearch, etc.

CUSTOM METRICS

$ easy_install dogstatsd-python

from statsd import statsd

stasd.gauge(‘users.active’, 50)

statsd.counter(‘base_edit.rendered’)

statsd.histogram: (‘catalog.query.time’, duration)

statsd.set: (‘users.unique’, user_id)

CUSTOM APP METRICS

PAPERTRAIL: CLOUD LOG AGGREGATION

WHO IS DOWNLOADING?

BONUS SLIDE: CODE COVERAGE REPORTS

Problem

Pull Request

Travis build

Automatic Deployment

SentryQA

Datadog

Papertrail

bobtemplates.niteoweb

THE PLONE TEMPLATE

zc.buildout best practices

GenericSetup install profile

Zope 3 browser layer

z3c.jbot overrides folder

static/ resourceDirectory for serving static resources (images, CSS, JS, etc.)

Sphinx documentation

test suite with a solid test coverage

Travis CI integration

THE PYRAMID TEMPLATE

zc.buildout best practices

SQLite for development, PostgreSQL for production

A sample view with a Chameleon template

A sample DB model

static resources filled with Twitter Bootstrap niceties

Sphinx documentation

Exemplary test suite

Travis CI & Coveralls integration

Heroku continuous deployment

Try it out:https://github.com/niteoweb/bobtemplates.niteoweb/

Then, let’s generalize it!