AWS Logging and Monitoring Design

AWS Logging and Monitoring Design

With practical and tangible Action plan – not just theoretical fluff ignored by hackers.

Firstly, as much as AWS want to advertise they are secure, enabling Logging Monitoring AWS is;

  1. Not straight forward
  2. Missing allot of information from AWS, which falls under your shared responsibility. (Public Cloud Security Get out of Jail Card.)
  3. There are so many ways skin the cat, but not real best practices.
  4. You need to be well aware of the service limits.
  5. AWS release new products that don’t exists anywhere else, so you have no idea, what can be abused/exploited and how to detect these threats. (Of course, no one is going to question a Behemoth. Because everyone wants to work for them! right.)
  6. They all ways advise you that the product is documented, but dont give you any advice on Business outcomes and gaps.
  7. Here is a HUGE example;
    1. AWS CloudWatch agents are used to OS Logs and metrics, but it does not integrate with AWS SecurityHub, so majority of our threat exposures isn’t covered by AWS security! So you need another solution to detect and correlate these threats. They offer a custom science experiment for you to develop your own SIEM. hahahah https://aws.amazon.com/blogs/security/how-to-monitor-and-visualize-failed-ssh-access-attempts-to-amazon-ec2-linux-instances/
  8. Check the AWS HCL – People also look at the features, but basic tenant of Solution Architecture is to check the HCL, hardware Compatibility List, this applies to AWS, where you need to check what is not supported.
    1. CloudTrail Unsupported Services https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-unsupported-aws-services.html
  1. The AWS documentation is vague at best and as the, just plagiarising existing information on just gives zero insights or advice.
  2. AWS encourage customer to build bespoke AWS solutions to keep you locked in without considering any business requirements. e.g. https://aws.amazon.com/solutions/implementations/centralized-logging/
  3. AWS (complementary and additive) native Architecture.
    1. AWS forces you to use all of they services for single requirement, making Bezo a Trillionaire . It’s a nonsensical intricate web, where no one has a farking clue what is going on. Look at this as a example from SecurityHub FAQ;
    1. https://docs.aws.amazon.com/securityhub/latest/userguide/control-finding-list.html
    2. Q: Will Security Hub replace the consoles of our other security services, such as Amazon GuardDuty, Amazon Inspector, or Amazon Macie?
    3. No. Security Hub is complementary and additive to the AWS security services. In fact, Security Hub will link back into the other consoles to help you gain additional context. Security Hub does not replicate the setup, configuration, or specialised features available within each security service.
    4. CloudTail can also send logs into CloudWatch Logs, (i have no clue what you would need to do that.. )
    5. Also, another one, DNS Traffic is not captured in VPC Flow logs and VPC Flow logs are not real-time and also does not support some instance types –
      1. https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-s3.html
      2. https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html
      3. You can’t modify a Flow Log’s configuration parameters once it is created. Instead, you have to delete it and create a new log. That’s not difficult, but it’s a bit annoying from a usability perspective.
    6. Network interfaces with multiple IP addresses will have data logged only for the primary IP as the destination address. This makes Flow Logs less useful in configurations involving multiple IPs on a single interface.
    7. Flow Logs exclude traffic related to DHCP requests and Amazon DNS activity. (Traffic for a non-Amazon DNS server is logged.) In many cases, this may not matter, but it is a limitation if you need to troubleshoot an issue with your site related to DHCP or DNS. For example, you may be experiencing poor performance due to slow DNS resolution. There are also valuable security insights that you can glean from DHCP and DNS traffic, such as detecting packet sniffing attempts by looking for unusual rates of IP conflicts, usage of the same MAC address by multiple hosts or the sharing of DNS records by machines with the same IP address.
  1. Here
YOU WANT THE TRUTH? YOU CAN'T HANDLE THE TRUTH! | Jack Nicholson - You  can't handle the truth! | Keto quote, Best movie lines, Jack nicholson

When exec decided to digitally transform into AWS, did they evaluate the cost of talent, AWS isn’t a single product, it is as of this writing 170 products that get upgrade and changed on a daily basis, did you assess this risk. Of course you didn’t. Oh, yeah don’t get me started on the Multi-Cloud stupidity.

This is why AWS is just so easy to master! And also super easy to secure! 🙂 🙂 🙂 🙂

“nobody got fired for buying ibm” old proverb, Now its Public Cloud!

Jeff Bezos could become world's first trillionaire by 2026

AWS Security Actionable Security Monitoring Plan

You should make sure you get a clear answer from AWS for the following questions;

  • So you’re logging, thats great… what are you detecting?
  • What is your best practice for sending logs into a central SIEM?
  • Can you list top use cases AWS cover/detect?

Threat Detection SOC Use cases;

Essentially, you need to log everything centrally (for investigations and compliance) and Threat detection. What are you logging and what can you detect. You should run a Red Team against this configuration to see what you can detect or not.

In terms of Security Operations perspective the following are the key Use cases required to support your Incident Response Plans;

  1. Threat Detection and Alerting.
  2. Governance and Compliance Reporting.
  3. Investigation Searches and Digital Forensics.

Cloud Control Plane vs Cloud Data Plane Concept

To establish baseline monitoring, security teams should gather and process the following:

  • Cloud control plane logs (such as AWS CloudTrail1 logs
  • Data Plane Workload OS/application logs
  • AWS Product (Access Logs)
  • Network flow logs for virtual private clouds (VPCs)
  • Inventory your threat landscape and exposure

Requirements for Threat Detection

  • Event Sources
  • Metrics
  • UpStream Security Monitoring
  • Detection Rules

Cloud Control Plane Logging

First, there’s the idea of a control plane. The control plane is the master controller (usually in the form of a master node) and includes API services, scheduling capabilities for containers and operational management tools/services. A master-level configuration database is also maintained in the control plane. In general, the control plane can be considered the brains of the Kubernetes infrastructure, and it needs to be very carefully protected.

Focus on the types of events that could be problematic to the environment. Examples include critical assets accessed or changed, identity policies modified, cryptographic keys deleted or changed, and so on.

Data Pane

AWS Product Access Logs

On top of the Control and Data plane, you need to consider the Access logs for specific AWS Products/Services. In terms of services such as AWS CloudFront, the access logs are not captured via the Control Plan, therefore, you need to capture; Access Logs, Account Activity, and Configuration;

AWS Detective

AWS Budget

Billing alarms—If you have a reasonable idea of a monthly billing range, you can break this down to define “checkpoints” that your bill should be at any given time. If these thresholds are crossed, you can be alerted and investigate the reason for the additional cost. Tools like AWS Budgets provide simple alerting and reporting for cloud billing.

  • These are key! If you have a reasonable idea of a monthly billing range, you can break this down to define “checkpoints” of what your bill should be at any given time. If these thresholds are crossed, a billing alarm could alert you and investigate what is causing the additional cost.
  • Resources and resource utilization—Cloud control plane logs from services like AWS CloudTrail can (and should) be heavily leveraged to monitor new, modified and deleted assets in the environment, as well as access to assets and service interaction in the cloud environment. These logs need to be integrated with a SIEM and/or cloud-native cloud monitoring solution like Amazon CloudWatch to build the appropriate triggers for alerting, as well as monitoring and reporting metrics as warranted. Some behavioral trending over time can also be assessed and reported through analytics tools like AWS Security Hub and Amazon GuardDuty, as well
  • https://console.aws.amazon.com/billing/home#/

Amazon CloudWatch filters

Identity and Access Management (IAM) and KMS

Monitor your user activity within the cloud. Admins, in particular, should be monitored carefully, because these accounts are prime targets for attackers. Any nonfederated user access should also be a high priority.

Network Security

VPC Flow Logs for your VPCs; they are not enabled by default.

Endpoint Security

AWS Inspector

AWS Config

AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.

However, AWS Config only collects information about EC2/VPC-related resources, not everything in your AWS account.

You should monitor changes to you AWS real estate and insure all changes are via ITIL Change Management and/or approved automation only.

Firstly, need to understand what AWS services and/or devices are in scope, then map them to your AWS native security logging into ArcSight SmartConnectors.

Click on Resource Groups next to the AWS Services in your aws console page, and select All Regions in region field and All Resources in the resources field. You will get the list of all the resources up and running in your AWS account. You can even tag them separately so you can check how much each resource is costing you.
If there is any other way, for example through AWS CLI, I am curious to know that.

  • Adding context—If logs can be “tagged” as originating from a specific ISP or CSP, that can help provide context on the use cases of the service. For example, logs from identity management services like AWS Identity and Access Management (IAM) have a specific user context, whereas events from Amazon EC2 may need additional details about workloads to provide the proper context for evaluation.
aws resourcegroupstaggingapi get-resources --region region_name
SELECT
  resourceId,
  resourceName,
  resourceType,
  relationships
WHERE
relationships.resourceId = 'vpc-#######'

What do you use, AWS SecurityHub, GuardDuty, CloudWatch, CloudTrail or EventHub.

Answer is all of these are complementary and additives services. So let’s example each of them and there primary use cases. So its best to begin with your use cases in terms of SOC operations and Threat Detection;

  1. Investigation and Search 
  2. Governance and Reporting
  3. Threat Detection and Alerts 

AWS GuardDuty vs CloudTrail vsSecurityHub vs CloudWatcth acts as an aggregation for other AWS services, which are supported by corresponding ArcSight SmartConnectors. You need to determine where you want to do Threat Detection and hold raw logs for long term retention and investigation.

Image for post
Image for post

Here is an overview;

AWS SecurityHub integrates with; https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-internal-providers.html

  • AWS Firewall Manager
  • IAM Access Analyzer
  • Amazon GuardDuty
  • Amazon Inspect
  • Amazon Macie

AWS GuardDuty integrates with;

https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_data-sources.html

  • AWS CloudTrail Event Logs 
  • AWS CloudTrail Management Events 
  • AWS CloudTrail S3 Data Events 
  • VPC Flow Logs 
  • DNS logs

ArcSight SmartConnectors for SecurityHub supports;

https://community.microfocus.com/t5/ArcSight-Connectors/SmartConnector-for-Amazon-Web-Services-Security-Hub/ta-p/2814565

  • GuardDuty Default
  • GuardDuty AWS_API_CALL
  • GuardDuty DNS_REQUEST
  • GuardDuty NETWORK_CONNECTION GuardDuty PORT_PROBE
  • Resource Header ResourcesDetailsAwsEc2Instance 
  • ResourcesDetailsAwsIamAccessKey 
  • ResourcesDetailsAwsEc2NetworkInterface
  • ResourcesDetailsAwsEc2SecurityGroup 
  • ResourcesDetailsAwsIamRole 
  • ResourcesDetailsAwsKmsKey 
  • ResourcesDetailsAwsS3Bucket ResourcesDetailsAwsS3Object 
  • ResourcesDetailsAwsSnsTopic 
  • ResourcesDetailsAwsSqsQueue 
  • ResourcesDetailsAwsLambdaFunction 

ArcSight SmartConnector supports CloudTrail, S3 and CloudWatch, that maybe ingest logs from AWS native services. 

ArcSight SmartConnector for AWS

AWS GuardDuty, CloudTrail, SecurityHub and CloudWatcth acts as an aggregation for other AWS services, which are supported by corresponding ArcSight SmartConnectors. AWS (complementary and additive) native Architecture comes into play;

https://community.microfocus.com/t5/ArcSight-Connectors/ct-p/ConnectorsDocs

  • Control Plane     -> AWS GuardDurty -> AWS SecurityHub -> ArcSight SmartConnector -> ESM/Logger
  • Data Plane          -> AWS EC2 -> Windows (SYSMON/WEC/WEF) -> ArcSight SmartConnector -> ESM/Logger
  • Data Plane           -> AWS EC2 -> Linux (AuditD/Syslogs) -> ArcSight SmartConnector -> ESM/Logger

 ArcSight SmartConnector for WiNC (Windows Native Connector) – Recommended for Production Environments

This is where the AWS (complementary and additive) native Architecture comes into play; 

  1. AWS Firewall Manager à AWS CloudTrail à AWS GuardDurty à AWS SecurityHub -> ArcSight SmartConnector for AWS SecurityHub
  2. IAM Access Analyzer à AWS CloudTrail à AWS GuardDurty à AWS SecurityHub -> ArcSight SmartConnector for AWS SecurityHub

 IAM Access Analyzer à AWS SecurityHub à ArcSight SmartConnector

You can review the supported data sources here- https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_data-sources.html

AWS IAM Access Analyzer supports ; 

ArcSight SmartConnector for AWS SecurityHub support for AWS (complementary and additive) native ArchitectureSo, supported data flow;

  1. AWS Firewall Manager à AWS CloudTrail à AWS GuardDurty à AWS SecurityHub -> ArcSight SmartConnector for AWS SecurityHub
  2. AWS Identity and Access Management roles -> IAM Access Analyzer à AWS CloudTrail à AWS GuardDurty à AWS SecurityHub -> ArcSight SmartConnector for AWS SecurityHub

You can review the supported data sources here- https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_data-sources.html

AWS GuardDuty, CloudTrail, SecurityHub and CloudWatcth acts as an aggregation for other AWS services, which are supported by corresponding ArcSight SmartConnectors.

AWS SecurityHub integrates with; https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-internal-providers.html

  • AWS Firewall Manager
  • IAM Access Analyzer
  • Amazon GuardDuty
  • Amazon Inspect
  • Amazon Macie

AWS GuardDuty integrates with;

https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_data-sources.html

  • AWS CloudTrail Event Logs 
  • AWS CloudTrail Management Events 
  • AWS CloudTrail S3 Data Events 
  • VPC Flow Logs 
  • DNS logs

ArcSight SmartConnectors for SecurityHub supports;

https://community.microfocus.com/t5/ArcSight-Connectors/SmartConnector-for-Amazon-Web-Services-Security-Hub/ta-p/2814565

  • GuardDuty Default
  • GuardDuty AWS_API_CALL
  • GuardDuty DNS_REQUEST
  • GuardDuty NETWORK_CONNECTION GuardDuty PORT_PROBE
  • Resource Header ResourcesDetailsAwsEc2Instance 
  • ResourcesDetailsAwsIamAccessKey 
  • ResourcesDetailsAwsEc2NetworkInterface
  • ResourcesDetailsAwsEc2SecurityGroup 
  • ResourcesDetailsAwsIamRole 
  • ResourcesDetailsAwsKmsKey 
  • ResourcesDetailsAwsS3Bucket ResourcesDetailsAwsS3Object 
  • ResourcesDetailsAwsSnsTopic 
  • ResourcesDetailsAwsSqsQueue 
  • ResourcesDetailsAwsLambdaFunction

CloudTrail vs CloudWatch

  • CloudTrail is for API logging
  • CloudWatch is for Log data

ArcSight SmartConnector for CloudWatch supports CloudWatch events

ArcSight SmartConnector supports CloudTrail, S3 and CloudWatch, that maybe ingest logs from AWS native logging services. 

Threat Modelling and Applying Risk to AWS Services and Resources

You need to develop a Threat Model and apply some abuse cases, which is far beyond this blog, so lets just use ATT&CK to identify top risk and develop detection for them.

Using ATT&CK to Develop Baseline for TTP Monitoring

Attack PhaseTTP
Initial AccessDiscovering valid accounts to AWS account
PersistenceCreating new accounts
Defense EvasionEstablishing presences in unused / unsupported cloud regions. Continuing to leverage valid accounts.
Credential AccessQuerying an identify role with a cloud instance’s metadata API. Discovering credentials in files
DiscoveryCloud service discovery (through network visibility, interaction with other services, and so on.)
CollectionData from cloud storage objects (items in S3 buckets, for example.)
ExfiltrationOutbound data to cloud storage account elsewhere
Connectivity to unknown outbound source addresses
Using ATT&CK to Develop Baseline for TTP Monitoring

Mapping Detection/Response Controls to TTPs

Attack PhaseTTPAWS Detection
Initial AccessDiscovering valid accounts to cloud environments.AWS CloudTrail event: Account login via AWS CLI or AWS Management Console (IAM Account.)
PersistenceCreating new accounts.AWS CloudTrail event: New IAM account created.
Defense EvasionEstablishing a presence in unused/unsupported cloud regions.

Continuing to leverage valid accounts.

AWS CloudTrail event represented in Amazon GuardDuty or Amazon Detective: New API event in a previously unused region.

AWS CloudTrail event represented in Amazon GuardDuty or Amazon Detective: Account use in new region
Credential AccessQuerying an identity role with a cloud instance’s metadata API.

Discovery credentials in files.
AWS CloudTrail event represented in Amazon GuardDuty, third- party SIEM or Amazon Detective: Metadata service queried for new services and role permissions

AWS CloudTrail event: Account login via AWS CLI or AWS Management Console.
DiscoveryCloud services discovery (through network visibility, interaction with other services, and so on.)

System information discovery.
System network connection discovery.
CollectionData from cloud storage objects (items in S3 buckets, for example.)

Data from local systems
ExfiltrationOutbound data to a cloud storage account elsewhere.

AWS Use cases and Detection Rules

“eventTime”: “2017-01-20T18:53:02Z”, “eventSource”: “iam.amazonaws.com”, “eventName”: “DeactivateMFADevice”, “awsRegion”: “us-east-1”, “sourceIPAddress”: “1.2.3.4”, “userAgent”: “signin.amazonaws.com”, “requestParameters”: {

“userName”: “dave”,
“serialNumber”: “arn:aws:iam::000012345678:mfa/dave” },
“responseElements”: null,
“requestID”: “d1a9ebf8-5fc8-11e5-9d8f-1bc7c6757e61”,

Suspicious AWS CloudTrail event that
indicates a cloud user trying to deactivate
an MFA device.

How to Improve Security Visibility and Detection/Response Operations in AWS

  • IAM activity (logins in particular)—Monitor your user activity within the cloud. In particular, monitor admins carefully, because these user credentials are prime targets for attackers. Any nonfederated user access should also be a high priority.

How to Improve Security Visibility and Detection/Response Operations in AWS

Priority 1
– Launching a workload that is not from an approved template
– Launching any containers from unapproved images in a repository
– Launching any assets in unapproved regions
– Modifying any IAM roles or policies
– Modifying or disabling cloud control plane logging or other security controls – Logins to the web console (unauthorized)

• Priority 2
– Unusual user behaviors (trying to access unauthorized resources, etc.) – Adding/updating new workload images
– Adding/updating new container images
– Logins to the web console (authorized)
– Updating/changing serverless configuration

• Priority 3
– Changes to security groups or network access control lists (ACLs) – Updating/changing serverless function code

How to Improve Security Visibility and Detection/Response Operations in AWS

able 1. Starting Points for Event Searches

AWS CloudTrail EventReason for Investigation
ConsoleLoginA user initiates console login activity.
StopLoggingA user tries to stop AWS CloudTrail.
CreateNetworkAclEntrySomeone creates a network ACL, which could expose attack surfaces or vectors.
CreateRouteSomeone creates a new route for data path control, which could expose attack surfaces or vectors.
AuthorizeSecurityGroupEgress AuthorizeSecurityGroupIngress RevokeSecurityGroupEgress RevokeSecurityGroupIngressMonitor all changes to security groups.
ApplySecurityGroupsToLoadBalancer SetSecurityGroupsSecurity group changes that tie to elastic load balancers are interesting, often in scaling operations. This may indicate unusual traffic surges in the environment.
AuthorizeDBSecurityGroupIngress CreateDBSecurityGroup DeleteDBSecurityGroup RevokeDBSecurityGroupIngressAmazon RDS instances have a different nomenclature for security groups, but are the same thing conceptually. Security teams should monitor such instances.

Starting Points for Event Searches

How to Improve Security Visibility and Detection/Response Operations in AWS

AWS Lambda EventReason for Monitoring
DeleteEventSourceMappingSomeone could delete the data source that triggers an AWS Lambda function, making it “blind.”
DeleteFunctionA function could be deleted purposefully or accidentally, leading to security issues.
RemovePermissionThis could lead to a lockout scenario or lack of access when needed (think IAM service account or role access to AWS Lambda).
UpdateEventSourceMappingData could be pulled from a different source, leading to incorrect function results.
UpdateFunctionCodeThe function could be broken or tampered with to prevent security-specific functionality from executing (for example, by adding comments).
UpdateFunctionConfigurationThe configuration of the function could be changed to limit its resources, causing poor or flawed execution.
Events for Immediate Monitoring

AWS Security Best Practices Check list

  1. Setup AWS Budget alerts
  2. Setup Root Security challenge questions
  3. Setup Password policy
  4. Deactivate Regions not required
  5. Document and monitor your access keys and deactivate and cycle
  6. Enable root IAM and MFA
  7. Update your Incident Response Plan and Digital Forensics Investigation to accommodate AWS
  8. Enable MFA for AWS Root account
  9. Secure KMS keys
  10. Enable Amazon VPC Flow logs for your VPCs; they are not enabled by default.
  11. Uses AWS Nitro EC2 instance can mirror traffic from any EC2 instance (A1, C5, C5d, C5n, I3en, M5, M5a, M5ad, M5d, p3dn.24xlarge, R5, R5a, R5ad, R5d, T3, T3a, and z1d).
  12. Ultizing default DNS services as it is intergrated with CloudTrail and GuardDuty, if you using a 3rd party for DNS, you need to make sure you can monitor that and correlate that within your SIEM.. e.g. Cisco Umercal support by ArcSight SmartConnector
  13. Outbound IP address alerting
  14. Deploy Cloud Watch agents as part of your SOE – https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-on-EC2-Instance.html

Understanding Digital Forensics inside AWS

CapabilityAWS ServicesDigital Forencics
ComputeAmazon Elastic Cloud Compute (EC2)Uses Amazon Machine Images (AMIs) to get started
Multiple OS support Pay for what you use
Next-gen Nitro infrastructure, created by AWS
Storage
Amazon Elastic Block Store (EBS), Amazon Simple Storage Service (S3), Amazon Elastic File System (EFS)

Amazon S3 offers multiple storage classes for multiple
use cases. Amazon EBS is used for the “block device” or hard drive for Amazon EC2 instances. Amazon EFS is used for file sharing storage with two storage classes to choose from.
NetFlow
Amazon VPC Flow Logs, Amazon VPC Traffic Mirroring

Capture information of network traffic going in and out of a VPC
Auditing
AWS CloudTrail

User attribution data
Log integrity can be enabled
Can send data to an Amazon S3 bucket for storage/archival
AWS Digital Forensics

EC2

  1. Create a security group that does not allow outbound traffic
  2. Attach to compromised Amazon EC2 instance
  3. Take snapshot of Amazon EC2 instance
  4. Perform memory acquisition, if possible
  5. Share snapshot with Security Account (if using one)
  6. Create volume from snapshot
  7. Attach volume to SIFT EC2 instance
  8. Conduct forensics

Digital Forensic Analysis of Amazon Linux EC2 Instances; https://www.sans.org/reading-room/whitepapers/cloud/digital-forensic-analysis-amazon-linux-ec2-instances-38235

CloudTrail

How to Perform a Security Investigation in AWS A SANS Whitepaper

  • Username—Search by the user’s name
  • Event name—Search by a specific API call (e.g., DeleteTrail)
  • Resource type—Search by an AWS service type (e.g., Amazon EC2 instance)
  • Resource name—Search by a resource name (e.g., instance ID, ENI)
  • Event source—Search results from specific AWS services
  • Event ID—Search based on a unique ID for an AWS CloudTrail event
  • AWS access key—Search by access key to show what was done in a single session
AWS CloudTrail Event Example

VPC Flows

https://aws.amazon.com/blogs/aws/new-vpc-traffic-mirroring/

Structure of a VPC Flow Log

SOAR Use Cases

How to Improve Security Visibility and Detection/Response Operations in AWS

  • Initial investigation and threat hunting—Analysts need to quickly find evidence of compromise or unusual activity, and often need to do so at scale.
  • Opening and updating incident tickets/cases—Due to improved integration with ticketing systems, event management and monitoring tools used by response teams can often generate tickets to the right team members and update these as evidence comes in.
  • Producing reports and metrics—Once evidence has been collected and cases are underway or resolved, generating reports and metrics can take a lot of analysts’ time.

How to Improve Security Visibility and Detection/Response Operations in AWS

  1. Automated DNS lookups of domain names never seen before • Automated searches for detected indicators of compromise • Automated forensic imaging of disk and memory from a suspect system, driven by alerts triggered in network- and host-based anti-malware platforms and tools • Network access controls automatically blocking outbound command and control (C2) channels from a suspected system

AWS Athena CloudTrail search script examples

CREATE EXTERNAL TABLE cloudtrail_logs (
eventversion STRING,
useridentity STRUCT<
               type:STRING,
               principalid:STRING,
               arn:STRING,
               accountid:STRING,
               invokedby:STRING,
               accesskeyid:STRING,
               userName:STRING,
sessioncontext:STRUCT<
attributes:STRUCT<
               mfaauthenticated:STRING,
               creationdate:STRING>,
sessionissuer:STRUCT<  
               type:STRING,
               principalId:STRING,
               arn:STRING, 
               accountId:STRING,
               userName:STRING>>>,
eventtime STRING,
eventsource STRING,
eventname STRING,
awsregion STRING,
sourceipaddress STRING,
useragent STRING,
errorcode STRING,
errormessage STRING,
requestparameters STRING,
responseelements STRING,
additionaleventdata STRING,
requestid STRING,
eventid STRING,
resources ARRAY<STRUCT<
               ARN:STRING,
               accountId:STRING,
               type:STRING>>,
eventtype STRING,
apiversion STRING,
readonly STRING,
recipientaccountid STRING,
serviceeventdetails STRING,
sharedeventid STRING,
vpcendpointid STRING
)
ROW FORMAT SERDE 'com.amazon.emr.hive.serde.CloudTrailSerde'
STORED AS INPUTFORMAT 'com.amazon.emr.cloudtrail.CloudTrailInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://mycloudtrailbucket-faye/AWSLogs/757250003982/';
SELECT
 useridentity.arn,
 eventname,
 sourceipaddress,
 eventtime
FROM cloudtrail_logs
LIMIT 100;

Opensource

Reference

Leave a Reply