AMSTERDAM
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Amazon Virtual Private Cloud Deep Dive
Steve Seymour, Solutions Architect, Networking Specialist
aws vpc –-expert-mode
Topics today
Virtual networking options
EC2-Classic
Simple to get started – all instances have Internet connectivity, auto-assigned private and public IP addresses Inbound security groups
Default VPC
The best of both
Get started using the EC2-Classic experience
If and when needed, begin using any VPC feature you require
VPC
Advanced virtual networking services: ENIs and multiple IPs
routing tables egress security groups
network ACLs private connectivity
Enhanced networking
And more to come...
Virtual networking options
EC2-Classic
Simple to get started – all instances have Internet connectivity, auto-assigned private and public IP addresses Inbound security groups
Default VPC
The best of both
Get started using the EC2-Classic experience
If and when needed, begin using any VPC feature you require
VPC
Advanced virtual networking services: ENIs and multiple IPs
routing tables egress security groups
network ACLs private connectivity
Enhanced networking
And more to come...
All accounts created after 12/4/2013 support VPC only and have a default
VPC in each region
Confirming your default VPC describe-account-attributes
VPC only
Routing & private connections
Implementing a hybrid architecture
Corporate Data Center
Create VPC
Corporate Data Center
aws ec2 create-vpc --cidr 10.10.0.0/16 aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.1.0/24 --a us-west-2a aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.2.0/24 --a us-west-2b
Create VPN connection
Corporate Data Center
aws ec2 create-vpn-gateway --type ipsec.1 aws ec2 attach-vpn-gateway --vpn vgw-f9da06e7 --vpc vpc-c15180a4 aws ec2 create-customer-gateway --type ipsec.1 --public 54.64.1.2 --bgp 6500 aws ec2 create-vpn-connection --vpn vgw-f9da06e7 --cust cgw-f4d905ea --t ipsec.1
Launch instances
Corporate Data Center
aws ec2 run-instances --image ami-d636bde6 --sub subnet-d83d91bd --count 3 aws ec2 run-instances --image ami-d636bde6 --sub subnet-b734f6c0 --count 3
Using AWS Direct Connect
Corporate Data Center
aws directconnect create-connection --loc EqSE2 --b 1Gbps --conn My_First aws directconnect create-private-virtual-interface --conn dxcon-fgp13h2s --new virtualInterfaceName=Foo, vlan=10, asn=60, authKey=testing, amazonAddress=192.168.0.1/24, customerAddress=192.168.0.2/24, virtualGatewayId=vgw-f9da06e7
Configuring route table
Corporate Data Center 192.168.0.0/16
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id vgw-f9da06e7
Each VPC has a single routing table at creation time,
used by all subnets
Remote connectivity best practices
Corporate Data Center
Availability Zone Availability Zone
Each VPN connection consists of 2 IPSec
tunnels. Use BGP for failure recovery.
Remote connectivity best practices
Corporate Data Center
Availability Zone Availability Zone
BGP
A pair of VPN connections (4 IPSec tunnels total) protects against failure of your
customer gateway BG
P
Remote connectivity best practices
Corporate Data Center
Availability Zone Availability Zone
BGP
Redundant AWS Direct Connect connections
with VPN backup
VPC with private and public connectivity
Corporate Data Center 192.168.0.0/16
aws ec2 create-internet-gateway aws ec2 attach-internet-gateway --internet igw-5a1ae13f --vpc vpc-c15180a4 aws ec2 delete-route --ro rtb-ef36e58a --dest 0.0.0.0/0 aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f aws ec2 create-route --ro rtb-ef36e58a --dest 192.168.0.0/16 --gateway-id vgw-f9da06e7
Automatic route propagation from VGW
Corporate Data Center 192.168.0.0/16
aws ec2 delete-route --ro rtb-ef36e58a --dest 192.168.0.0/16 aws ec2 enable-vgw-route-propagation --ro rtb-ef36e58a --gateway-id vgw-f9da06e7
Used to automatically update routing table(s) with routes present in the VGW
Isolating connectivity by subnet
Corporate 192.168.0.0/16
aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.3.0/24 --a us-west-2b aws ec2 create-route-table --vpc vpc-c15180a4 aws ec2 associate-route-table --ro rtb-fc61b299 --subnet subnet-60975a17 aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
Subnet with connectivity only to other instances and the
Internet via the IGW
Software VPN for VPC-to-VPC connectivity
# VPC A aws ec2 modify-network-interface-attribute --net eni-f832afcc --no-source-dest-check aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc
# VPC B aws ec2 modify-network-interface-attribute --net eni-9c1b693a --no-source-dest-check aws ec2 create-route --ro rtb-67a2b31c --dest 10.10.0.0/16 –-instance-id i-9c1b693a
Software VPN for VPC-to-VPC connectivity
Software VPN between these
instances
Software VPN for VPC-to-VPC connectivity
Enabling communication between instances in these
subnets; adding routes to the default routing table
Software firewall to the Internet
Routing all traffic from subnets to the Internet via a firewall is
conceptually similar
# Default routing table directs traffic to the NAT/firewall instance aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --instance-id i-f832afcc # Routing table for 10.10.3.0/24 directs to the Internet aws ec2 create-route --ro rtb-67a2b31c --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
Customer Story – University of Amsterdam
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Handling peak loads of 100.000 student enrollments in 20 minutes
Flexible scaling within a VPC Hans Janssen
Who is Hans Janssen?
• Now: – Product Manager at the Expertise Center CampusHO – Responsible for maintenance and development of CampusHO
• Complete Student Information System • Based on Oracle’s PeopleSoft Campus Solutions • In use at the UvA, HvA, Leiden University and Tilburg University
• Before: – Study advisor, class scheduler, recruiter – Corporate Information Manager – Head Business Information Management
Student Enrollment at the UvA
UvA: University of Amsterdam – 35.000 Students – 100.000 Class Enrollments every semester
UvA 2012: Need for an external front-end
• Easier access for students to information: – Results – Class Schedule – Enrollments
• Easier Class Enrollment – Standard Self Service of Campus Solutions is too complicated
• Combining information form other sources – Course catalog (outside CampusHO) – Time Table (outside CampusHO)
Enrollment App
Webservice integration
StudentInformation
SystemXML/SOAP
EnrollmentFront-End
JSON/REST
StudentID SystemID
????
Security
Performance
????
Look & FeelUsability
DataFunctionality
Control
Standard Campus Solutions was not able to deliver the needed webservice functionality, security and peak handling
inQdo Connect as the Missing Link
StudentInformation
SystemXML/SOAP
EnrollmentFront-End
JSON/REST
StudentID SystemID
inQdoConnect
JSON/REST
StudentID
XML/SOAP
SystemID
Security
Performance
TransformationMappingSecure
ScalableLook & Feel
Usability
DataFunctionality
Control
With inQdo Connect we could fill the gap between front-end and back-end
Choice for inQdo Connect
• Functionality – Conversion of SOAP/XML to and from REST/JSON – Authentication – Pagination – Switching user to system user, language-dependent – Analytics – Synchronous to a-synchronous queuing
• Highly Scalable – Amazon Web Services
• Core technology: webMethods from Software AG added with AWS services = inQdo Connect
Peak Class Enrollment
• Situation: – All students want to enroll in the afternoon classes – Heavy peak loads when the class enrollment opens – University wants the wait-time to be below 20 minutes
Handling Peak Enrollment: We knew what to expect
• Enrollments for a semester start on a fixed date/time per program • Students rush to popular workgroups: first come, first serve • 500 enrollments in the first minute • Popular workgroups are full after 15 seconds
Heavy Query’s: 19 tables joined
– Query selects only the available classes (open and not full) – Every webservice call takes 1 sec
Oracle Explain Plan for the query
Heavy processing: Enrollment Cobol
Adding a class enrollment involves a lot of control: – Student enrolled in a program? – Correct term? – Requirements fulfilled? – Not too many courses or too many examination retries
• Every class enrollment takes 2 seconds
Peak enrollment
• For every enrollment: – 20 webservice calls to gather information – 1 webservice call to start the actual enrollment process
• Total processing time: 25 secs for an enrollment
Optimizing for maximum peak performance
• Back-end: fixed to max – Heavy webservers, permanent – Heavy application servers, permanent – Heavy database server, permanent
• Front-end: fixed to max • inQdo Connect: scalable
– 50 weeks only 2 servers – 2 weeks with 8 servers
Database
Load-balancer
Appl. ServerWebserver
Webserver Appl. Server
dual hexacore with
hyperthreading
24 CPU x2128 GB
dual octacore with
hyperthreading
16 CPU x2256 GB
Resulting performance maximum
• The optimized chain can handle: – 3000 information webservice requests per minute and – 1000 enrollments per minute
• Maximum amount of students: 200 per minute • Conclusion: we need a wait-queue
Queue IT: queue information
Situation with Queue IT and inQdo Connect
• All enrollments divided in 2 days; 2 peaks • 200 students per minute • Max wait time: 30 minutes
• Students are very satisfied – They know how long they have to wait – Their queue-number is fair
Resulting situation UvA: Enrollment WebApp for tablets/laptops
WebApp-application
Users
Enrollments
CS Environment
Database
Appl. ServerWebserver
HTTPS/HTML x8x4
Webserver withWebservices
QAS & EWS webservices
Mediation: inQdo-connect
Load-balancer Mediator
SOAP/XML
Load-balancer
x4
Apache/Tomcat
JSON/REST
Database
Load-balancer
x4
LDAP
On Line
x2
Queue
Appl. Server withIntegration Broker
x4
Queue-IT
Wachtrij
Amazon
Landing page
inQdoMaintenance:
Responsible:
inQdo inQdoUvA ICTS SaNS-EC/MCX
UvA AC SaNS-EC
Cloud Cloud Cloud
Expanding use of inQdo Connect
inQdo Connect & Amazon VPC
Situation 2015 Integration of Student Information, Blackboard (LMS), Hippo (CMS), Syllabus+ (schedule), Course Catalog
For more information
• inQdo have a stand in the Partner Expo Area • Speak with me after the session
VPC peering
Shared services VPC using VPC peering
• Common/core services – Authentication/directory – Monitoring – Logging – Remote administration – Scanning
Provides infrastructure zoning • Dev: VPC B • Test: VPC C • Production: VPC D
VPC peering for VPC-to-VPC connectivity
aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63 aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87 VPC A> aws ec2 create-route --ro rtb-ef36e58a --des 10.20.0.0/16 --vpc-peer pcx-ee56be87
VPC B> aws ec2 create-route --ro rtb-67a2b31c --des 10.10.0.0/16 --vpc-peer pcx-ee56be87
VPC A - 10.10.0.0/16 vpc-c15180a4
VPC B - 10.20.0.0/16 vpc-062dfc63
VPC peering across accounts
aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63 --peer-owner 472752909333 # In owner account 472752909333 aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87
VPC A - 10.10.0.0/16 vpc-c15180a4
VPC B - 10.20.0.0/16 vpc-062dfc63 Account ID 472752909333
VPC peering – Additional considerations
• Security groups not supported across peerings – Workaround: specify rules by IP prefix
• No “transit” capability for VPN, AWS Direct Connect, or 3rd VPCs – Example: Cannot access VPC C from VPC A via VPC B – Workaround: Create a direct peering from VPC A to VPC C
• Peer VPC address ranges cannot overlap – But, you can peer with 2+ VPCs that themselves overlap – Use subnets/routing tables to pick the VPC to use
VPC peering with software firewall
VPC A - 10.10.0.0/16 VPC B - 10.20.0.0/16
# Default routing table directs Peer traffic to the NAT/firewall instance aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc
# Routing table for 10.10.3.0/24 directs to the Peering aws ec2 create-route --ro rtb-67a2b31c --dest 10.20.0.0/16 --vpc-peer pcx-ee56be87
VPC Endpoint for Amazon S3
S3 Bucket
Corporate Data Center
aws s3 mb s3://mybucket
AWS Cloud
Access to S3 via VPN or Direct Connect Corporate Data Center
aws s3 sync /myfiles s3://mybucket
AWS Cloud
VPC Endpoint for Amazon S3 Corporate Data Center
aws ec2 create-vpc-endpoint --vpc vpc-a1b2c3d4 --service-name com.amazonaws.eu-west-1.s3
AWS Cloud
VPC Endpoint for Amazon S3 Corporate Data Center
aws ec2 modify-vpc-endpoint --vpc-endpoint vpce-ab1c2de3 --add-route-tables rt-de1c2ab3
AWS Cloud
VPC Endpoint for Amazon S3 Corporate Data Center
AWS Cloud
Benefits
• Removes the need for an Internet gateway or NAT instance to provide S3 access
• Bandwidth not impacted by a NAT Instance
• Highly available & resilient
• Simple configuration with multiple security controls
• Plans to add additional target services in the future
New VPC Objects
Prefix list ID (pl-xxxxxxxxx) • An identifier that is specific to a particular AWS Service • Logically represents the range of public IP addresses used by the service. • Can be specified in the “Outbound” rules as a destination for a Security Group • Specified in Route Tables as the “destination” • Prefix list name maps to a service name - "com.amazonaws. <Region> .s3“ VPC endpoint ID (vpce-xxxxxxxxx) • These are assigned when you create a VPC Endpoint • Used as the target of the route table
Controlling Access
• Using Endpoint Policies
Controlling Access
• Using Amazon S3 Bucket Policies
Controlling Access
• Security Groups aws ec2 authorize-security-group-egress --group-id sg-a6afa1c4 --ip-permissions "[ { ""IpProtocol"": ""tcp"", ""FromPort"": 80, ""ToPort"": 80, ""PrefixListIds"": [ { ""PrefixListId"": ""pl-6da54004"" } ] } ]" aws ec2 authorize-security-group-egress --group-id sg-a6afa1c4 --ip-permissions "[ { ""IpProtocol"": ""tcp"", ""FromPort"": 443, ""ToPort"": 443, ""PrefixListIds"": [ { ""PrefixListId"": ""pl-6da54004"" } ] } ]"
Controlling Access
• Security Groups
VPC Endpoint for S3 – Additional Considerations
• Prefix list IDs can’t be used to create an outbound rule in a network ACL.
• You cannot create an endpoint between a VPC and an AWS service in a different region.
• Endpoint connections cannot be extended out of a VPC (by Peering, VPN or AWS Direct Connect)
• When using Amazon S3 endpoints, you cannot use a bucket policy or an IAM policy to allow access from a VPC CIDR range (the private IP address range).
VPC Flow Logs
Amazon VPC Flow Logs Log and view network traffic flows
Related Presentations – Videos online
https://www.youtube.com/user/AmazonWebServices • ARC205 – VPC Fundamentals and Connectivity • ARC401 – Black Belt Networking for Cloud Ninja
– Application centric, network monitoring, management, floating IPs
• ARC403 – From One to Many: Evolving VPC Design • SDD302 – A Tale of One Thousand Instances
– Example of EC2-Classic customer adopting VPC • SDD419 – Amazon EC2 Networking Deep Dive
– Network performance, placement groups, enhanced networking
AMSTERDAM
Top Related