AMSTERDAM

©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved

©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved

Amazon Virtual Private Cloud Deep Dive

Steve Seymour, Solutions Architect, Networking Specialist

aws vpc –-expert-mode

Topics today

Virtual networking options

EC2-Classic

Simple to get started – all instances have Internet connectivity, auto-assigned private and public IP addresses Inbound security groups

Default VPC

The best of both

Get started using the EC2-Classic experience

If and when needed, begin using any VPC feature you require

VPC

Advanced virtual networking services: ENIs and multiple IPs

routing tables egress security groups

network ACLs private connectivity

Enhanced networking

And more to come...

Virtual networking options

EC2-Classic

Simple to get started – all instances have Internet connectivity, auto-assigned private and public IP addresses Inbound security groups

Default VPC

The best of both

Get started using the EC2-Classic experience

If and when needed, begin using any VPC feature you require

VPC

Advanced virtual networking services: ENIs and multiple IPs

routing tables egress security groups

network ACLs private connectivity

Enhanced networking

And more to come...

All accounts created after 12/4/2013 support VPC only and have a default

VPC in each region

Confirming your default VPC describe-account-attributes

VPC only

Routing & private connections

Implementing a hybrid architecture

Corporate Data Center

Create VPC

Corporate Data Center

aws ec2 create-vpc --cidr 10.10.0.0/16 aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.1.0/24 --a us-west-2a aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.2.0/24 --a us-west-2b

Create VPN connection

Corporate Data Center

aws ec2 create-vpn-gateway --type ipsec.1 aws ec2 attach-vpn-gateway --vpn vgw-f9da06e7 --vpc vpc-c15180a4 aws ec2 create-customer-gateway --type ipsec.1 --public 54.64.1.2 --bgp 6500 aws ec2 create-vpn-connection --vpn vgw-f9da06e7 --cust cgw-f4d905ea --t ipsec.1

Launch instances

Corporate Data Center

aws ec2 run-instances --image ami-d636bde6 --sub subnet-d83d91bd --count 3 aws ec2 run-instances --image ami-d636bde6 --sub subnet-b734f6c0 --count 3

Using AWS Direct Connect

Corporate Data Center

aws directconnect create-connection --loc EqSE2 --b 1Gbps --conn My_First aws directconnect create-private-virtual-interface --conn dxcon-fgp13h2s --new virtualInterfaceName=Foo, vlan=10, asn=60, authKey=testing, amazonAddress=192.168.0.1/24, customerAddress=192.168.0.2/24, virtualGatewayId=vgw-f9da06e7

Configuring route table

Corporate Data Center 192.168.0.0/16

aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id vgw-f9da06e7

Each VPC has a single routing table at creation time,

used by all subnets

Remote connectivity best practices

Corporate Data Center

Availability Zone Availability Zone

Each VPN connection consists of 2 IPSec

tunnels. Use BGP for failure recovery.

Remote connectivity best practices

Corporate Data Center

Availability Zone Availability Zone

BGP

A pair of VPN connections (4 IPSec tunnels total) protects against failure of your

customer gateway BG

P

Remote connectivity best practices

Corporate Data Center

Availability Zone Availability Zone

BGP

Redundant AWS Direct Connect connections

with VPN backup

VPC with private and public connectivity

Corporate Data Center 192.168.0.0/16

aws ec2 create-internet-gateway aws ec2 attach-internet-gateway --internet igw-5a1ae13f --vpc vpc-c15180a4 aws ec2 delete-route --ro rtb-ef36e58a --dest 0.0.0.0/0 aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f aws ec2 create-route --ro rtb-ef36e58a --dest 192.168.0.0/16 --gateway-id vgw-f9da06e7

Automatic route propagation from VGW

Corporate Data Center 192.168.0.0/16

aws ec2 delete-route --ro rtb-ef36e58a --dest 192.168.0.0/16 aws ec2 enable-vgw-route-propagation --ro rtb-ef36e58a --gateway-id vgw-f9da06e7

Used to automatically update routing table(s) with routes present in the VGW

Isolating connectivity by subnet

Corporate 192.168.0.0/16

aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.3.0/24 --a us-west-2b aws ec2 create-route-table --vpc vpc-c15180a4 aws ec2 associate-route-table --ro rtb-fc61b299 --subnet subnet-60975a17 aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f

Subnet with connectivity only to other instances and the

Internet via the IGW

Software VPN for VPC-to-VPC connectivity

# VPC A aws ec2 modify-network-interface-attribute --net eni-f832afcc --no-source-dest-check aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc

# VPC B aws ec2 modify-network-interface-attribute --net eni-9c1b693a --no-source-dest-check aws ec2 create-route --ro rtb-67a2b31c --dest 10.10.0.0/16 –-instance-id i-9c1b693a

Software VPN for VPC-to-VPC connectivity

Software VPN between these

instances

Software VPN for VPC-to-VPC connectivity

Enabling communication between instances in these

subnets; adding routes to the default routing table

Software firewall to the Internet

Routing all traffic from subnets to the Internet via a firewall is

conceptually similar

# Default routing table directs traffic to the NAT/firewall instance aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --instance-id i-f832afcc # Routing table for 10.10.3.0/24 directs to the Internet aws ec2 create-route --ro rtb-67a2b31c --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f

Customer Story – University of Amsterdam

©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved

Handling peak loads of 100.000 student enrollments in 20 minutes

Flexible scaling within a VPC Hans Janssen

Who is Hans Janssen?

•  Now: –  Product Manager at the Expertise Center CampusHO –  Responsible for maintenance and development of CampusHO

•  Complete Student Information System •  Based on Oracle’s PeopleSoft Campus Solutions •  In use at the UvA, HvA, Leiden University and Tilburg University

•  Before: –  Study advisor, class scheduler, recruiter –  Corporate Information Manager –  Head Business Information Management

Student Enrollment at the UvA

UvA: University of Amsterdam –  35.000 Students –  100.000 Class Enrollments every semester

UvA 2012: Need for an external front-end

•  Easier access for students to information: –  Results –  Class Schedule –  Enrollments

•  Easier Class Enrollment –  Standard Self Service of Campus Solutions is too complicated

•  Combining information form other sources –  Course catalog (outside CampusHO) –  Time Table (outside CampusHO)

Enrollment App

Webservice integration

StudentInformation

SystemXML/SOAP

EnrollmentFront-End

JSON/REST

StudentID SystemID

????

Security

Performance

????

Look & FeelUsability

DataFunctionality

Control

Standard Campus Solutions was not able to deliver the needed webservice functionality, security and peak handling

inQdo Connect as the Missing Link

StudentInformation

SystemXML/SOAP

EnrollmentFront-End

JSON/REST

StudentID SystemID

inQdoConnect

JSON/REST

StudentID

XML/SOAP

SystemID

Security

Performance

TransformationMappingSecure

ScalableLook & Feel

Usability

DataFunctionality

Control

With inQdo Connect we could fill the gap between front-end and back-end

Choice for inQdo Connect

•  Functionality –  Conversion of SOAP/XML to and from REST/JSON –  Authentication –  Pagination –  Switching user to system user, language-dependent –  Analytics –  Synchronous to a-synchronous queuing

•  Highly Scalable –  Amazon Web Services

•  Core technology: webMethods from Software AG added with AWS services = inQdo Connect

Peak Class Enrollment

•  Situation: –  All students want to enroll in the afternoon classes –  Heavy peak loads when the class enrollment opens –  University wants the wait-time to be below 20 minutes

Handling Peak Enrollment: We knew what to expect

•  Enrollments for a semester start on a fixed date/time per program •  Students rush to popular workgroups: first come, first serve •  500 enrollments in the first minute •  Popular workgroups are full after 15 seconds

Heavy Query’s: 19 tables joined

–  Query selects only the available classes (open and not full) –  Every webservice call takes 1 sec

Oracle Explain Plan for the query

Heavy processing: Enrollment Cobol

Adding a class enrollment involves a lot of control: –  Student enrolled in a program? –  Correct term? –  Requirements fulfilled? –  Not too many courses or too many examination retries

•  Every class enrollment takes 2 seconds

Peak enrollment

•  For every enrollment: –  20 webservice calls to gather information –  1 webservice call to start the actual enrollment process

•  Total processing time: 25 secs for an enrollment

Optimizing for maximum peak performance

•  Back-end: fixed to max –  Heavy webservers, permanent –  Heavy application servers, permanent –  Heavy database server, permanent

•  Front-end: fixed to max •  inQdo Connect: scalable

–  50 weeks only 2 servers –  2 weeks with 8 servers

Database

Load-balancer

Appl. ServerWebserver

Webserver Appl. Server

dual hexacore with

hyperthreading

24 CPU x2128 GB

dual octacore with

hyperthreading

16 CPU x2256 GB

Resulting performance maximum

•  The optimized chain can handle: –  3000 information webservice requests per minute and –  1000 enrollments per minute

•  Maximum amount of students: 200 per minute •  Conclusion: we need a wait-queue

Queue IT: queue information

Situation with Queue IT and inQdo Connect

•  All enrollments divided in 2 days; 2 peaks •  200 students per minute •  Max wait time: 30 minutes

•  Students are very satisfied –  They know how long they have to wait –  Their queue-number is fair

Resulting situation UvA: Enrollment WebApp for tablets/laptops

WebApp-application

Users

Enrollments

CS Environment

Database

Appl. ServerWebserver

HTTPS/HTML x8x4

Webserver withWebservices

QAS & EWS webservices

Mediation: inQdo-connect

Load-balancer Mediator

SOAP/XML

Load-balancer

x4

Apache/Tomcat

JSON/REST

Database

Load-balancer

x4

LDAP

On Line

x2

Queue

Appl. Server withIntegration Broker

x4

Queue-IT

Wachtrij

Amazon

Landing page

inQdoMaintenance:

Responsible:

inQdo inQdoUvA ICTS SaNS-EC/MCX

UvA AC SaNS-EC

Cloud Cloud Cloud

Expanding use of inQdo Connect

inQdo Connect & Amazon VPC

Situation 2015 Integration of Student Information, Blackboard (LMS), Hippo (CMS), Syllabus+ (schedule), Course Catalog

For more information

•  inQdo have a stand in the Partner Expo Area •  Speak with me after the session

VPC peering

Shared services VPC using VPC peering

•  Common/core services –  Authentication/directory –  Monitoring –  Logging –  Remote administration –  Scanning

Provides infrastructure zoning •  Dev: VPC B •  Test: VPC C •  Production: VPC D

VPC peering for VPC-to-VPC connectivity

aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63 aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87 VPC A> aws ec2 create-route --ro rtb-ef36e58a --des 10.20.0.0/16 --vpc-peer pcx-ee56be87

VPC B> aws ec2 create-route --ro rtb-67a2b31c --des 10.10.0.0/16 --vpc-peer pcx-ee56be87

VPC A - 10.10.0.0/16 vpc-c15180a4

VPC B - 10.20.0.0/16 vpc-062dfc63

VPC peering across accounts

aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63 --peer-owner 472752909333 # In owner account 472752909333 aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87

VPC A - 10.10.0.0/16 vpc-c15180a4

VPC B - 10.20.0.0/16 vpc-062dfc63 Account ID 472752909333

VPC peering – Additional considerations

•  Security groups not supported across peerings –  Workaround: specify rules by IP prefix

•  No “transit” capability for VPN, AWS Direct Connect, or 3rd VPCs –  Example: Cannot access VPC C from VPC A via VPC B –  Workaround: Create a direct peering from VPC A to VPC C

•  Peer VPC address ranges cannot overlap –  But, you can peer with 2+ VPCs that themselves overlap –  Use subnets/routing tables to pick the VPC to use

VPC peering with software firewall

VPC A - 10.10.0.0/16 VPC B - 10.20.0.0/16

# Default routing table directs Peer traffic to the NAT/firewall instance aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc

# Routing table for 10.10.3.0/24 directs to the Peering aws ec2 create-route --ro rtb-67a2b31c --dest 10.20.0.0/16 --vpc-peer pcx-ee56be87

VPC Endpoint for Amazon S3

S3 Bucket

Corporate Data Center

aws s3 mb s3://mybucket

AWS Cloud

Access to S3 via VPN or Direct Connect Corporate Data Center

aws s3 sync /myfiles s3://mybucket

AWS Cloud

VPC Endpoint for Amazon S3 Corporate Data Center

aws ec2 create-vpc-endpoint --vpc vpc-a1b2c3d4 --service-name com.amazonaws.eu-west-1.s3

AWS Cloud

VPC Endpoint for Amazon S3 Corporate Data Center

aws ec2 modify-vpc-endpoint --vpc-endpoint vpce-ab1c2de3 --add-route-tables rt-de1c2ab3

AWS Cloud

VPC Endpoint for Amazon S3 Corporate Data Center

AWS Cloud

Benefits

•  Removes the need for an Internet gateway or NAT instance to provide S3 access

•  Bandwidth not impacted by a NAT Instance

•  Highly available & resilient

•  Simple configuration with multiple security controls

•  Plans to add additional target services in the future

New VPC Objects

Prefix list ID (pl-xxxxxxxxx) •  An identifier that is specific to a particular AWS Service •  Logically represents the range of public IP addresses used by the service. •  Can be specified in the “Outbound” rules as a destination for a Security Group •  Specified in Route Tables as the “destination” •  Prefix list name maps to a service name - "com.amazonaws. <Region> .s3“ VPC endpoint ID (vpce-xxxxxxxxx) •  These are assigned when you create a VPC Endpoint •  Used as the target of the route table

Controlling Access

•  Using Endpoint Policies

Controlling Access

•  Using Amazon S3 Bucket Policies

Controlling Access

•  Security Groups aws ec2 authorize-security-group-egress --group-id sg-a6afa1c4 --ip-permissions "[ { ""IpProtocol"": ""tcp"", ""FromPort"": 80, ""ToPort"": 80, ""PrefixListIds"": [ { ""PrefixListId"": ""pl-6da54004"" } ] } ]" aws ec2 authorize-security-group-egress --group-id sg-a6afa1c4 --ip-permissions "[ { ""IpProtocol"": ""tcp"", ""FromPort"": 443, ""ToPort"": 443, ""PrefixListIds"": [ { ""PrefixListId"": ""pl-6da54004"" } ] } ]"

Controlling Access

•  Security Groups

VPC Endpoint for S3 – Additional Considerations

•  Prefix list IDs can’t be used to create an outbound rule in a network ACL.

•  You cannot create an endpoint between a VPC and an AWS service in a different region.

•  Endpoint connections cannot be extended out of a VPC (by Peering, VPN or AWS Direct Connect)

•  When using Amazon S3 endpoints, you cannot use a bucket policy or an IAM policy to allow access from a VPC CIDR range (the private IP address range).

VPC Flow Logs

Amazon VPC Flow Logs Log and view network traffic flows

Related Presentations – Videos online

https://www.youtube.com/user/AmazonWebServices •  ARC205 – VPC Fundamentals and Connectivity •  ARC401 – Black Belt Networking for Cloud Ninja

–  Application centric, network monitoring, management, floating IPs

•  ARC403 – From One to Many: Evolving VPC Design •  SDD302 – A Tale of One Thousand Instances

–  Example of EC2-Classic customer adopting VPC •  SDD419 – Amazon EC2 Networking Deep Dive

–  Network performance, placement groups, enhanced networking

AMSTERDAM