SCREW PLUG IMMERSION HEATERS - Industrial Electric Immersion
Monitoring and Logging on AWS - AWS Immersion Days€¦ · Events–Delivers a near real-time...
Transcript of Monitoring and Logging on AWS - AWS Immersion Days€¦ · Events–Delivers a near real-time...
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ashley Miller, Sr. Solutions ArchitectTampa, Florida
Date
Monitoring and Logging on AWS
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Module Overview
Overview of the AWS Monitoring and Logging options across Performance, Availability, Security and Cost areas.
Common questions we will cover in this module:
• How do I capture, view and act on resource availability and state changes?• How do I track the key performance indicators across AWS resources?• How do I track API calls within my AWS accounts?• How do I track cost within my AWS accounts?• How do I know which capability to use for my use case?• How do the monitoring and logging capabilities integrate with other AWS
services?
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
My Application
Why monitor?
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why monitor? (continued)
Customer Experience- Are my customers getting a good experience?Performance & Cost- How are my changes impacting overall performance?Trends- Do I need to scale?Troubleshooting & Remediation- Where did the problem occur?Learning & Improvement- Can I detect or prevent this problem in the future?
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Low to moderate visibilityLow degree of automation
High visibilityHigh degree of automation
Traditional Approach vs.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1
Performance & Availability Monitoring and Logging
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon CloudWatch
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CloudWatchMonitoring services for AWS Resources and AWS-based Applications.
Monitor and Store Logs
Set Alarms (react to changes)
View Graphs and Statistics
Collect and Track Metrics
What does it do?
How can you use it?
React to application log events and availability
Automatically scale EC2 instance fleet
View Operational Status and Identify Issues
Monitor CPU, Memory, Disk I/O, Network, etc.
CloudWatch Logs / CloudWatch Events
CloudWatch Alarms
CloudWatch Dashboards
CloudWatch Metrics
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- Metrics- Namespaces- Dimensions- Time Stamps- Units
Amazon CloudWatch Concepts
- Statistics- Periods- Aggregation- Alarms- Regions
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Metrics – Data about the performance of your systems.
Examples:
EC2 – CPUUtilization, Network In/OutEBS - DiskRead/Write & Ops/BytesRDS – BufferCacheHitRatio, CommitLatencyS3 - NumberOfObjectsCustom - put-metric-data API Call
Default Interval – 5 MinutesEnhanced Interval – 1 MinuteRetention – 2 Weeks
Amazon CloudWatch Metrics Examples
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key CloudWatch Capabilities
- Monitors your AWS resources in real time.
- CloudWatch to track built in and custom metrics.
- CloudWatch Alarms for desired conditions
- CloudWatch Events to automate responses
- CloudWatch Logs for log collection, aggregation and monitoring
- Dashboards for customized views across resources
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CloudWatch Metrics & Alarms
AWSResource
YourCustom
Data
Metric Alarm Action
CloudWatch
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Alarms – Send notifications based on state of monitored resources.
- Watches metric over a period of time and sends messages when a defined metric threshold is reached.
- Utilizes with Amazon Simple Notification Service (SNS) to send messages.
- Alarm state must be maintained for specified number of periods.
- Integrates with Auto Scaling
Amazon CloudWatch Alarms
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon CloudWatch Alarm Example
An alarm has three possible states:
OK—The metric is within the defined threshold
ALARM—The metric is outside of the defined threshold
INSUFFICIENT_DATA—The alarm has just started, the metric is not available, or not enough data is available for the metric to determine the alarm state
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Alarm Actions
Action
Notification (SNS)
Auto Scaling Action
EC2 Action
Recover
Stop
Terminate
Amazon EC2Auto Recovery
Use this actiontogether withStatus Checksto automate
instance recovery
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Events – Delivers a near real-time stream of system events that describe changes in Amazon Web Services (AWS) resources to targets such as Lambda, SNS & SQS.
- Events are delivered through resource state changes, CloudTrail API calls, custom publications or scheduled.
- Rules match incoming events and route them to one or more targets for processing.
- Targets are specified in rules and receive matching events.
Amazon CloudWatch Events
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- Scheduled EBS Snapshots
Amazon CloudWatch Events Examples
- Automatically Tag Resources
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon CloudWatch LogsAmazon CloudWatch Logs lets you grab everything and monitor activity.
§ Managed service to collect and keep your logs§ Aggregate and centralize logs across multiple sources§ CloudWatch Logs Agent for Linux and Windows instances§ Integration with Metrics and Alarms§ Export data to S3 for analytics and/or archival§ Stream to Amazon ElasticSearch Service or AWS Lambda
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CloudWatch Logs + Filter
AWSResource
YourCustom
Data
Metric Alarm Action
CloudWatch
FilterLogs
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon CloudWatch DashboardsAmazon CloudWatch Dashboards creates a single consolidated view across your resources customized by you.
- A single view for selected metrics to help you assess the health of your resources and applications across one or more regions.
- An operational playbook that provides guidance for team members during operational events about how to respond to specific incidents.
- A common view of critical resource and application measurements that can be shared by team members for faster communication flow during operational events.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CloudWatch Dashboard Example
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2Security Monitoring and
Logging
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Different log categories
AWS Infrastructure logs
§ AWS CloudTrail§ Amazon VPC Flow
Logs
AWS service logs
§ Amazon S3§ AWS Elastic Load
Balancing§ Amazon CloudFront§ AWS Lambda§ AWS Elastic Beanstalk§ …
Host based logs
§ Messages§ Security§ NGINX/Apache/IIS§ Windows Event Logs§ Windows
Performance Counters
§ …Security related events
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CloudTrailRecords AWS API calls for your account
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What can you answer using a CloudTrail event?
§ Who made the API call?
§ When was the API call made?
§ What was the API call?
§ Which resources were acted up on in the API call?
§ Where was the API call made from and made to?
Supported services:http://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-supported-services.html
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What does an event look like?{
"eventVersion": "1.01",
"userIdentity": {
"type": "IAMUser", // Who?
"principalId": "AIDAJDPLRKLG7UEXAMPLE",
"arn": "arn:aws:iam::123456789012:user/Alice", //Who?
"accountId": "123456789012",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"userName": "Alice",
"sessionContext": {
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2014-03-18T14:29:23Z"
}
}
},
"eventTime": "2014-03-18T14:30:07Z", //When?
"eventSource": "cloudtrail.amazonaws.com",
"eventName": “StartInstances", //What?
"awsRegion": "us-west-2",//Where to?
"sourceIPAddress": "72.21.198.64", // Where from?
"userAgent": "ec2-api-tools 1.6.12.2",
"requestParameters": {
"instancesSet": {
"items": [{
"instanceId": "i-ebeaf9e2“// Which resource?
}]
}
},// more event details
}
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CloudTrail Best Practices
1. Enable in all regions2. Enable log file validation3. Encrypted logs4. Integrate with Amazon
CloudWatch Logs5. Centralize logs from all
accounts
Benefits§ Configure all accounts to send
logs to a central security account
§ Reduce risk for log tampering§ Can be combined with Amazon
S3 CRR
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Config
• Get inventory of AWS resources
• Discover new and deleted resources
• Record configuration changes continuously
• Get notified when configurations change
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Config Rules
• Set up rules to check configuration changes recorded• Use pre-built rules provided by AWS• Author custom rules using AWS Lambda • Invoked automatically for continuous assessment • Use dashboard for visualizing compliance and identifying
offending changes
• Community-contributed custom rules at https://github.com/awslabs/aws-config-rules
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon VPC Flow LogsLog network traffic for Amazon VPC, subnet or single interfaces
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon VPC Flow Logs§ Stores logs in AWS CloudWatch Logs§ Can be enabled on
• Amazon VPC, a subnet, or a network interface• Amazon VPC & Subnet enables logging for all interfaces in the VPC/subnet• Each network interface has a unique log stream
§ Flow logs do not capture real-time log streams for your network interfaces
§ Filter desired result based on need• All, Reject, Accept• Troubleshooting or security related with alerting needs?• Think before enabling All on VPC, will you use it?
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VPC Flow Logs
• Agentless• Enable per ENI, per subnet, or per VPC• Logged to AWS CloudWatch Logs• Create CloudWatch metrics from log data• Alarm on those metrics
AWSaccount
Source IP
Destination IP
Source port
Destination port
Interface Protocol Packets
Bytes Start/end time
Accept or reject
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VPC Flow Logs
• Amazon ElasticsearchService
• Amazon CloudWatch Logs subscriptions
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Managing, Monitoring & Processing Logs
CloudWatch Logs - Near real-time, aggregate, monitor, store, and search
Amazon Elasticsearch Service Integration (or ELK stack)- Analytics and Kibana interface
AWS Lambda & Amazon Kinesis Integration- Custom processing with your code
Export to S3- SDK & CLI batch export of logs for analytics
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.Arrow direction indicates general direction of data flow
EC2 instances
Logstashcluster on EC2
DynamoDBTables
RDS Databases(via JDBC)
SQSQueues
KinesisStreams
AmazonElasticsearchdomain
CWL Logs agent
VPCFlow Logs
CloudTrailAudit Logs
S3AccessLogs
ELBAccessLogs
CloudFrontAccessLogs
SNSNotifications
DynamoDBStreams
SESInbound
CognitoEvents
KinesisStreams
CloudWatchEvents &Alarms
ConfigRules
Lambda
Filebeat agent
REST API client
S3
CloudWatchLogs
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Technology Partner solutions integrated with CloudTrail
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
3Cost Monitoring
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We’ll Talk Through Some…
Frameworks Tools Best Practices
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1. Understand What is Deployed and What it Costs…
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
By Employing Tags…
Key (Attribute): 127 Unicode characters
Value (Detail/Description): 255 Unicode characters
Tags per resource: 50 tags
Jane_Doe
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
…And Using The Different Types of Tags Appropriately
Resource Tags• Provide the ability to organize and search within and across resources• Filterable and Searchable• Do not appear in Detail Billing Report
Cost Allocation Tags• Ability to map AWS charges to organizational attributes for accounting purposes• Information presented in Detailed Billing Report and Cost Explorer• Available on certain services or limited to components within a service (e.g. S3 bucket but
not objects)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tag Key Examples
Cost Center
Business Unit
Environ. Tier
Owner
Dept./ Group
Product / Application
Shutdown Time
Support ContactEndpoint
Backup
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2. How does AWS create my bill?
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 45
AWS Bill Development Process (Simplified)
Collect Consumption
Data
Create the Billing Report
Entries
Apply Discounts (RI,
Spot, EDP)
Add Unused RI Charges
Make Billing Report
available to customer
1. Consumption Data is aggregated across all linked accounts, based on CloudWatch entries
2. RI discounts, Spot Discounts, EDP Discounts, and non-use charges are applied based on the aggregated set of purchases across the linked accounts
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
3. Proactively Monitor Your Account Billing Usage…
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
By Using Detailed Billing Reports andEnabling Billing Alerts…
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
By Using Budgets, Forecasts, and Alarms in the Cost Explorer
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Billing Alarms
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Respond to billing alerts in CloudWatch.
When an alarm is triggered:1. Email the project team, and the budget approver (AWS console)2. Open a Service Management Ticket in your ITSM system3. Open a Ticket in your Finance System4. Resize the project’s IT resources5. Cull/Reduce the project’s IT resources6. Shut down the project’s IT resources
Billing Alert Response
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
…Tools to Manage Billing Data…
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4. Identify Idle Resources and Turn Off Unused Instances…
http://docs.aws.amazon.com/solutions/latest/ec2-scheduler/welcome.htmlEC2 Scheduler Solution
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
…Using Trusted Advisor…
Trusted Advisor
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
…And Amazon CloudWatch to Monitor,Collect and Track Metrics…
Amazon CloudWatch
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Summary
üSetup Sensible Billing Alarms for your OrganizationüProactively Monitor Your Account Billing UsageüLeverage AWS Partner toolsüLeverage Trusted Advisor reports to:
ü Identify Idle Resources and Turn Off Unused Instancesü Identify Under-utilized Resources and Resize themü Identify Baseline Consumption needs to support RI
commitmentsüReview new Discount and Technology Options on a Monthly basis
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
?Questions