Download - (SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014

November 12, 2014 | Las Vegas, NV

Becky Weiss, Principal Software Engineer, Amazon EC2 Networking

Elastic

network

interface

Subnet A

us-east-1a10.0.1.0/24

10.0.1.100

Subnet A2

us-east-1a10.0.2.0/24

10.0.1.101

10.0.2.50

10.0.2.51

Subnet C

us-east-1c10.0.3.0/24

10.0.3.99

Instance

1

Instance

2

Instance

3 Instance

4

elastic

network

interface

Subnet A

us-east-1a10.0.1.0/24

10.0.1.100

Subnet A2

us-east-1a10.0.2.0/24

10.0.1.101

10.0.2.50

10.0.2.51

Subnet C

us-east-1c10.0.3.0/24

10.0.3.99

Instance

1

Instance

2

Instance

3 Instance

4

Placement group

Subnet A is in us-east-1a

C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count 2

---------------------------------------------------------------------------------

| RunInstances |

+----------------------------------------+--------------------------------------+

| OwnerId | 123456789012 |

| ReservationId | r-9f5404b5 |

+----------------------------------------+--------------------------------------+

| Instances |

|+-----------------------------------+-----------------------------------------+|

|| AmiLaunchIndex | 0 ||

|| Architecture | x86_64 ||

|| ClientToken | None ||

|| EbsOptimized | False ||

|| Hypervisor | xen ||

|| ImageId | ami-b66ed3de ||


---------------------------------------------------------------------------------

| RunInstances |

+----------------------------------------+--------------------------------------+

| OwnerId | 123456789012 |


+----------------------------------------+--------------------------------------+

| Instances |

|+-----------------------------------+-----------------------------------------+|







AMI: More about this

choice later…


---------------------------------------------------------------------------------

| RunInstances |

+----------------------------------------+--------------------------------------+

| OwnerId | 123456789012 |


+----------------------------------------+--------------------------------------+

| Instances |

|+-----------------------------------+-----------------------------------------+|







Big instance type:

c3.8xlarge

Avg: 0.167msec

NetworkingTestPlacementGroup available cluster

C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count 2 --placement GroupName=NetworkingTestPlacementGroup

---------------------------------------------------------------------------------

| RunInstances |

+----------------------------------------+--------------------------------------+

| OwnerId | 123456789012 |

| ReservationId | r-13374839 |

+----------------------------------------+--------------------------------------+

| Instances |

|+-----------------------------------+-----------------------------------------+|







Avg: .099msec

Instance 1 Instance 2

...........

Virtualization layer

eth

0

eth

1

Instance Virtual NICs

Physical NIC


eth

0

Instance

Physical NICVF Driver

eth

1

VF

[ec2-user@ip-10-0-3-70 ~]$ ethtool -i eth0

driver: vif

version:

firmware-version:

bus-info: vif-0

…

[ec2-user@ip-10-0-3-70 ~]$ ethtool -i eth0

driver: ixgbevf

version: 2.14.2+amzn

firmware-version: N/A

bus-info: 0000:00:03.0

…

amzn-ami-hvm-2012.03.1.x86_64-ebs

hvm

--attribute sriovNetSupport

InstanceId i-37c5d1d9Not yet!

[ec2-user@ip-10-0-3-125 ~]$ sudo yum update

OS update

reboot-instances

Reboot

(OS update)

(Not shown here: analogous steps for other Linux distros)

Add to Windows driver store

stop-instances

Stop the instance

stop-instances

--sriov-net-support simple

Enable SRIOV

Cannot be undone

start-instances

Start

start-instances

--attribute sriovNetSupport

InstanceId i-37c5d1d9

Value simple

We’re on

modinfo ixgbevf

aws ec2 register-image --name MyEnhancedNetworkingImage--image-location … --sriov-net-support-simple

i2.8xlarge

Storage-optimized instance

require 'mongo‘

'randomdb'

until Time SECONDS_TO_RUN

KEY_MAX

:key

Time

if

:times_accessed

:key

else

:key :value:times_accessed

end

Time

end

Spin in tight loop:

Read a random document

Then write it back

def add_write_statistic

:sample_count

:sum

:minimum :minimum

:maximum :maximum

end

Aggregating statistics for CloudWatch

require 'aws-sdk'

AWS CloudWatch Client

if Time

:namespace 'NetworkingTest/MongoDemo',

:metric_data => [{:metric_name => 'WriteTime',

:dimensions => [{:name => 'RunId', :value => MY_RUN_ID}],

:statistic_values => write_stats}],

:unit => 'Seconds'

Time

:sample_count :sum

end

CloudWatch PutMetricData:

Writing a custom metric

# ec2-run-instances ami-b66ed3de --instance-type c3.large --subnet subnet-c03cfb99 --group sg-72caf017 --placement-group NetworkingTestPlacementGroup --monitor --user-data-file my_startup_script.sh --iam-profile NetworkingTestIAMRole --instance-count 10

RESERVATION r-d13d6f37 123456789012

INSTANCE i-fb6d5352 ami-b66ed3de ip-10-0-1-113.ec2.internal pending NetworkingTestSSHKey 0 c3.large 2014-10-30T13:26:33+0000 us-east-1a monitoring-pending 10.0.1.113 vpc-ca28afaf subnet-c03cfb99 ebs NetworkingTestPlacementGroup hvmxen sg-72caf017 defaultfalse arn:aws:iam::123456789012:instance-profile/NetworkingTestIAMRole

NIC eni-b560caed subnet-c03cfb99 vpc-ca28afaf 123456789012 in-use 10.0.1.113 true

NICATTACHMENT eni-attach-fb6ddf9d 0 attaching 2014-10-30T06:26:33-0800 true

GROUP sg-72caf017 default

...







...

CloudWatch detailed monitoring:

1-minute metrics







...

Startup script file

# cat startup_script.sh

Download client test script from S3

Then gogogo!







...

Security best practice:

Launch instances with IAM roles if

they need to access any AWS

resources

# aws iam list-role-policies --role-name NetworkingTestIAMRole

{

"PolicyNames": [

"NetworkingTestIAMRole-CloudWatchPolicy",

"NetworkingTestIAMRole-S3Policy"

]

}

# aws iam get-role-policy --role-name NetworkingTestIAMRole --policy-name NetworkingTestIAMRole-S3Policy

Allow retrieving objects from a particular S3 bucket

# aws iam get-role-policy --role-name NetworkingTestIAMRole --policy-name NetworkingTestIAMRole-CloudWatchPolicy

Allow CloudWatch PutMetricData

Label WriteTime

389483.0 2014-10-29T02:30:00Z Seconds

390189.0 2014-10-29T02:33:00Z Seconds

392373.0 2014-10-29T02:34:00Z Seconds

392387.0 2014-10-29T02:32:00Z Seconds

377256.0 2014-10-29T02:31:00Z Seconds

SampleCount statistic:How many of these WriteTime statistics

were written across all instances during

each minute?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

“WriteTime” SampleCount statisticby number of client instances

TPS, regular TPS, enhanced

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

DiskWriteBytes 1-minute Sum statisticby number of client instances

Regular Enhanced

Placement group

Instance


VF driver

http://bit.ly/awsevals

http://bit.ly/awsevals