Threat detection and monitoring

Monitoring requirements and health checks

Start from the workload: what can go wrong, who can touch it, and which signals would show it.

• Control-plane activity (who called which API): CloudTrail.
• Resource configuration (what changed, is it compliant): AWS Config.
• Network behaviour: VPC Flow Logs, Route 53 Resolver query logs, transit gateway flow logs, Network Firewall logs.
• Host and application behaviour: CloudWatch agent, application logs, GuardDuty Runtime Monitoring.
• Threats and anomalies: GuardDuty. Sensitive data: Macie. Vulnerabilities: Inspector.

Resource health checks are part of monitoring. Route 53 health checks can monitor an endpoint, another health check set (calculated), or a CloudWatch alarm (the way to monitor private resources, because Route 53 checkers sit on the public internet). ELB target group health checks decide whether a target gets traffic. CloudWatch alarms, composite alarms (AND/OR over other alarms to cut noise), and anomaly detection alarms turn metrics into alerts. Alerts go to SNS or EventBridge, then on to people or automation.

CloudWatch metric filters

A metric filter turns log patterns (for example failed console logins in CloudTrail logs) into a metric you can alarm on. This is the classic CIS-benchmark pattern: CloudTrail → CloudWatch Logs → metric filter → alarm → SNS. Metric filters work only on data arriving after creation and don’t backfill.

Aggregating security and monitoring events

Security Hub aggregates findings; Security Lake aggregates logs. If the question says “all raw logs from several sources in a normalised, queryable form”, pick Security Lake. If it says “single pane of prioritised findings and compliance scores”, pick Security Hub.

Organization-wide setup

For GuardDuty, Security Hub, Macie, Inspector, Detective, Config and Security Lake: designate a delegated administrator account (not the management account) with Organizations, turn on auto-enable for new accounts, and enable in all Regions you use. Members can’t turn the service off when the admin owns it.

GuardDuty

Managed threat detection. It analyses CloudTrail management and data events, VPC Flow Logs, DNS logs independently; you don’t have to enable those logs yourself. Optional protection plans:

Facts that are tested: findings are in the Region where they occur; there’s a 30-day free trial; trusted IP lists suppress alerts for known addresses, threat lists add known-bad ones; suppression rules auto-archive noise; findings export to S3 (needs a KMS key) and fire EventBridge events. Finding names follow ThreatPurpose:ResourceType/ThreatFamily (for example UnauthorizedAccess:EC2/SSHBruteForce, Backdoor:EC2/C&CActivity.B!DNS, CryptoCurrency:EC2/BitcoinTool.B, Exfiltration:S3/..., Policy:S3/BucketBlockPublicAccessDisabled, UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS).

GuardDuty does not read your application logs, block traffic, or modify anything. It detects; EventBridge plus automation responds.

Macie

Discovers and protects sensitive data in S3: PII, financial data, credentials. Automated sensitive data discovery samples objects continuously; classification jobs scan chosen buckets on demand or on a schedule. Uses managed data identifiers and custom data identifiers (regex plus keywords). Macie also evaluates bucket security posture (public access, no encryption, shared outside the organization). Findings are policy findings (bucket posture) and sensitive data findings (content). Findings go to Security Hub and EventBridge. Macie is S3-only; for databases, export to S3 first.

Detective and Inspector (quick placement)

• Detective builds a behaviour graph from CloudTrail, VPC Flow Logs, GuardDuty findings and EKS audit logs to investigate and run root cause analysis. See the incident-response notes.
• Inspector continuously scans EC2, ECR images and Lambda functions for vulnerabilities, and EC2 for network reachability. See the infrastructure notes.

Automated assessments

Skill: regular assessments and investigations.

• AWS Config conformance packs: a bundle of Config rules and remediation actions deployed as a unit (and across an organization), usually from a template such as an operational best-practices pack. Evaluates continuously and reports compliance.
• Config rules: managed (prebuilt), custom with Lambda, or custom with Guard (policy-as-code). Trigger types: configuration change or periodic.
• Systems Manager State Manager: keeps instances in a desired state on a schedule through associations, for example making sure the agent is installed or a setting stays applied.
• Security Hub automated checks run continuously against the enabled standards.
• Inspector rescans on new CVEs and new packages; no schedule needed.

Logging solutions

CloudTrail

Protect the log bucket: a dedicated log archive account, bucket policy letting only cloudtrail.amazonaws.com write, S3 Object Lock or MFA delete, and SCPs preventing StopLogging and DeleteTrail.

CloudWatch Logs

• Dedicated logging account: use cross-account observability or subscription filters with a destination in the logging account (a Kinesis stream or Firehose behind a CloudWatch Logs destination with a resource policy).
• CloudWatch agent (unified agent) collects logs and system metrics from EC2 and on-premises servers. It needs an instance profile with CloudWatchAgentServerPolicy and a config file (often kept in Parameter Store).
• Logs Insights: query language for ad hoc analysis within a log group (and across groups/accounts). Live Tail streams logs live.
• Retention is per log group; default is never expire. Export to S3 for long-term or use subscription filters for real time. A log group allows two subscription filters.
• Data protection policies mask sensitive data in logs (see data protection notes).

Log sources by network design

Log storage, lakes and third parties

• Land logs in S3 (central bucket, lifecycle to Glacier, Object Lock for evidence) and make them queryable with Athena (partition projection saves cost).
• Security Lake collects native AWS sources (CloudTrail, VPC Flow Logs, Route 53 Resolver logs, Security Hub findings, WAF logs, and others) and custom sources in OCSF. It has a delegated admin, can roll up Regions into a single one, and gives subscribers either data access (read the S3 data through Lake Formation) or query access. Partner SIEMs subscribe this way.
• To third-party tools: Firehose to an HTTP endpoint (Splunk, Datadog), Kinesis consumers, or Security Lake subscribers.

Analysis, normalisation and correlation

Troubleshooting monitoring and logging

When logs or alerts are missing, walk the path: source enabled → permissions → delivery path → destination policy → encryption key → filters/limits.

Always check CloudTrail for the denied API call that explains a failed delivery, and Access Analyzer/policy simulator for permission puzzles.

Common traps

• CloudTrail Event history is not a long-term store; build a trail.
• Data events and Insights are not on by default.
• A trail in one Region doesn’t cover others unless it’s multi-Region (organization trails are multi-Region as set up).
• GuardDuty needs no flow log or DNS log setup, but it also can’t be queried as raw logs. Use Flow Logs separately for investigation.
• Security Hub only shows findings from services that are enabled and integrated.
• Macie is for S3 only.
• Delegated admin is not the management account; use the management account only for the initial designation.