(DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

66

description

In this session, we introduce Boto 3, the next major version of the AWS SDK for Python. You will learn about the new features in the SDK, such as the high-level resource APIs that simplify working with AWS collections and objects, and the eventing model that enables customizing your calls to AWS services. We use a sample application to demonstrate these features, and show how to integrate them with your existing projects.

Transcript of (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Page 1: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 2: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Boto

Project

Overview

Boto 3

Features

Project

Example

Page 3: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 4: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Project

Started

Community

Contributions

Amazon

Service

Updates

Code

Generation

Python 3

Support

Page 5: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto

conn = boto.connect_s3()

bucket = conn.get_bucket('my-bucket')

for obj in bucket:

print(obj.key, obj.last_modified)

Page 6: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Boto

Project

Overview

Boto 3

Features

Project

Example

Page 7: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 8: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Boto 3

Botocore

Session Resources ClientsConfig

Session

Credentials

ClientsAuthentication

SerializationHTTPS

Page 9: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

s3 = boto3.resource('s3')

bucket = s3.Bucket('my-bucket')

for obj in bucket.objects.all():

print(obj.key, obj.last_modified)

Page 10: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 11: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

sqs = boto3.client('sqs')

response = sqs.list_queues(QueueNamePrefix='test')

print(response['QueueUrls'][0])

Page 12: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

"QueueUrls"

"https://queue.amazonaws.com/12345678/products"

"https://queue.amazonaws.com/12345678/news"

"https://queue.amazonaws.com/12345678/test"

"https://queue.amazonaws.com/12345678/jobs"

Page 13: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 14: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

s3 = boto3.client('s3')

# Get a paginator and iterate through each page

paginator = s3.get_paginator('list_objects')

for page in paginator.paginate(Bucket='my-bucket'):

for obj in page['Contents']:

print(obj['Key'])

Page 15: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Starting Running Terminated

Page 16: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

ec2 = boto3.client('ec2')

# Get a waiter and wait for instance to be readywaiter = ec2.get_waiter('instance_running')waiter.wait(InstanceIds=['i-abc123'])

print('Instance is ready!')

Page 17: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 18: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

instance = Instance(id)Amazon EC2

bucket = Bucket(name)Amazon S3

queue = Queue(url)Amazon SQS

Page 19: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

instance.idAmazon EC2

object.bucket_name, object.keyAmazon S3

queue.urlAmazon SQS

Page 20: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

instance.instance_typeAmazon EC2

object.last_modifiedAmazon S3

queue.attributes[‘DelaySeconds’]Amazon SQS

Page 21: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

instance.create_image(…)Amazon EC2

object.put(Body=‘hello’)Amazon S3

queue.receive_messages()Amazon SQS

Page 22: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

instance.subnetAmazon EC2

-Amazon S3

-Amazon SQS

Page 23: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

ec2.Instance(id)Amazon EC2

bucket.Object(key)Amazon S3

sqs.Queue(url)Amazon SQS

Page 24: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

ec2.instancesAmazon EC2

bucket.objectsAmazon S3

sqs.queuesAmazon SQS

Page 25: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

s3 = boto3.resource('s3')

obj = s3.Object('my-bucket', 'my-key.jpg')

print(obj.content_type)

print(obj.last_modified)

Page 26: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

for obj in s3.Bucket(name='boto3').objects.all():

print(obj.key)

for queue in sqs.queues.filter(QueueNamePrefix='a'):

print(queue.url)

# Batch actions coming soon!

bucket.objects.delete()

Page 27: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 28: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 29: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Boto

Project

Overview

Boto 3

Features

Project

Example

Page 30: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 31: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 32: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 33: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 34: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 35: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 36: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 37: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 38: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Amazon

SQS

Page 39: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Amazon

SQS

Page 40: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 41: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 42: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 43: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 44: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# Creating a client by name

client = boto3.client('s3')

# Creating a resource by name

resource = boto3.resource('s3')

Page 45: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 46: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# Create a new bucket

s3 = boto3.resource('s3')

bucket = s3.create_bucket(Name='Boto3')

# Get a bucket by name

bucket = s3.Bucket('Boto3')

Page 47: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# Get a bucket by name

bucket = s3.Bucket('Boto3')

# Upload a new file

with open('file.mov', 'rb') as data:

bucket.Object('file.mov').put(Body=data)

Page 48: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 49: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# List all objects in all buckets

s3 = boto3.resource('s3')

for bucket in s3.buckets.all():

for obj in bucket.objects.all():

print(bucket.name, obj.key)

Page 50: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# Download a file

bucket = s3.Bucket('Boto3')

obj = bucket.Object('output.mp4')

data = obj.get()['Body'].read()

Page 51: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 52: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# Create SNS topic (idempotent)

sns = boto3.resource('sns')

topic = sns.create_topic(Name='Boto3')

# Get an SNS topic

topic = sns.Topic('<TOPIC ARN>')

Page 53: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

# Create an SQS queue (idempotent)

sqs = boto3.resource('sqs')

queue = sqs.create_queue(QueueName='Boto3')

# Get an existing queue

queue = sqs.get_queue_by_name(QueueName='Boto3')

Page 54: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

# Subscribe an SQS queue to the topic

topic.subscribe(

Protocol='sqs',

Endpoint=queue.attributes['QueueArn'])

Page 55: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 56: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

import boto3

# Create IAM role

iam = boto3.resource('iam')

role = iam.create_role(

RoleName='role-name',

AssumeRolePolicyDocument='...')

Page 57: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

# Set role policy

policy = role.RolePolicy('transcoder')

policy.put(PolicyDocument={

'Version': '2012-10-17',

'Statement':[

{...}

]

})

Page 58: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 59: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

# Create a new pipeline

transcoder = boto3.client('elastictranscoder')

response = transcoder.create_pipeline(

Name='Boto3',

InputBucket='Boto3-input',

OutputBucket='Boto3-output',

Role='<ROLE ARN>',

...

)

Page 60: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

# Create a new transcoding job

job = transcoder.create_job(

PipelineId=response['Pipeline']['Id'],

Input={

'Key': 'input.mov',

...

},

Outputs={...}

)

Page 61: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 62: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

https://github.com/boto/boto3-sample

Page 63: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014
Page 64: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Boto

Project

Overview

Boto 3

Features

Project

Example

Page 65: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

https://github.com/boto/boto3

https://github.com/boto/boto3-sample

Page 66: (DEV307) Introduction to Version 3 of the AWS SDK for Python (Boto) | AWS re:Invent 2014

Please give us your feedback on this session.

Complete session evaluations and earn re:Invent swag.

http://bit.ly/awsevals