Showing posts with label Security. Show all posts
Showing posts with label Security. Show all posts

Friday, 7 February 2025

AWS Secrets Manager

 


AWS Secrets Manager allows us to:
  • Centrally store and manage credentials, API keys, and other secrets.
  • Use AWS Identity and Access Management (IAM) permissions policies to manage access to your secrets.
  • Rotate secrets on demand or on a schedule, without redeploying or disrupting active applications.
  • Integrate secrets with AWS logging, monitoring, and notification services.

Viewing Secrets

To list all secrets in a particular region:

% aws secretsmanager list-secrets --region us-east-2         
{
    "SecretList": [
        {
            "ARN": "arn:aws:secretsmanager:us-east-2:700840607999:secret:my-app/stage/my-secret-bwwria",
            "Name": "my-app/stage/my-secret ",
            "Description": "Secret for my-app in staging env",
            "LastChangedDate": "2025-01-13T12:51:21.204000+00:00",
            "LastAccessedDate": "2025-02-07T00:00:00+00:00",
            "Tags": [
                {
                    "Key": "environment",
                    "Value": "stage"
                },
                {
                    "Key": "service",
                    "Value": "main-app"
                }
            ],
            "SecretVersionsToStages": {
                "11877f11-1999-4f37-8311-283ad04d70f1": [
                    "AWSCURRENT"
                ],
                "ab81397d-eb1d-4dc1-8a44-961ce45de258": [
                    "AWSPREVIOUS"
                ]
            },
            "CreatedDate": "2022-08-17T12:55:43.194000+01:00",
            "PrimaryRegion": "us-east-2"
        },
        ...
      ]
 }


Deleting a secret


By default, secret is not deleted immediately but after 7 days.

To delete a secret immediately use --force-delete-without-recovery option:

% aws secretsmanager delete-secret --secret-id my-app/stage/my-secret --force-delete-without-recovery --region eu-west-2
{
    "ARN": "arn:aws:secretsmanager:eu-west-2:700859607999:secret:my-app/stage/my-secret-E0yyRM",
    "Name": "my-app/stage/my-secret",
    "DeletionDate": "2025-02-07T14:54:30.386000+00:00"
}



Resources:


Saturday, 8 June 2024

Access types in Amazon EKS

 



Types of access in EKS


Grant access to Kubernetes APIs



Our cluster has an Kubernetes API endpoint. Kubectl uses this API. We can authenticate to this API using two types of identities:
  • An AWS Identity and Access Management (IAM) principal (role or user)
  • A user in our own OpenID Connect (OIDC) provider
    • Requires authentication to our OIDC provider
    • setup: Authenticate users for your cluster from an OpenID Connect identity provider - Amazon EKS
    • We can associate one OIDC identity provider to our cluster.
    • Kubernetes doesn't provide an OIDC identity provider. We can use an existing public OIDC identity provider, or we can run our own identity provider.
    • The issuer URL of the OIDC identity provider must be publicly accessible, so that Amazon EKS can discover the signing keys. Amazon EKS doesn't support OIDC identity providers with self-signed certificates.
    • Before we can associate an OIDC identity provider with our cluster, we need the following information from our provider:
      • Issuer URL - The URL of the OIDC identity provider that allows the API server to discover public signing keys for verifying tokens. The URL must begin with https:// and should correspond to the iss claim in the provider's OIDC ID tokens. In accordance with the OIDC standard, path components are allowed but query parameters are not. Typically the URL consists of only a host name, like https://server.example.org or https://example.com. This URL should point to the level below .well-known/openid-configuration and must be publicly accessible over the internet.
      • Client ID (also known as audience) - The ID for the client application that makes authentication requests to the OIDC identity provider.
We can't disable IAM authentication to our cluster, because it's still required for joining nodes to a cluster. OIDC authentication is optional. Both can be enabled on cluster at the same time. 


IAM OIDC identity providers are entities in IAM that describe an external identity provider (IdP) service that supports the OpenID Connect (OIDC) standard, such as Google or Salesforce. You use an IAM OIDC identity provider when you want to establish trust between an OIDC-compatible IdP and your AWS account. This is useful when creating a mobile app or web application that requires access to AWS resources, but you don't want to create custom sign-in code or manage your own user identities. 

You can create and manage an IAM OIDC identity provider using the:AWS Management Console, the AWS Command Line Interface, the Tools for Windows PowerShell, or the IAM API.

After you create an IAM OIDC identity provider, you must create one or more IAM roles. A role is an identity in AWS that doesn't have its own credentials (as a user does). But in this context, a role is dynamically assigned to a federated user that is authenticated by your organization's IdP. The role permits your organization's IdP to request temporary security credentials for access to AWS. The policies assigned to the role determine what the federated users are allowed to do in AWS. 

 


Grant Kubernetes workloads access to AWS


 A workload is an application running in one or more Kubernetes pods.

A Kubernetes service account provides an identity for processes that run in a Pod.

If your Pod needs access to AWS services, you can map the service account to an IAM identity to grant that access.

Granting IAM permissions to workloads on Amazon Elastic Kubernetes Service clusters


Amazon EKS provides two ways to grant IAM permissions to workloads that run in Amazon EKS clusters:
  • IAM roles for service accounts (IRSA)
    • Allows pods to directly use IAM Roles (no need to inject into pods IAM User access credentials anymore)
    • We define the trust relationship between an IAM role and Kubernetes service account (that's a type of account in Kubernetes) in the role's trust policy.
    • Each EKS cluster has an OpenID Connect (OIDC) issuer URL associated with it. 
    • To use/enable IRSA a unique OpenID Connect provider needs to be created for each EKS cluster in IAM. 
  • EKS Pod Identities


In 2014, AWS Identity and Access Management added support for federated identities using OpenID Connect (OIDC). This feature allows you to authenticate AWS API calls with supported identity providers and receive a valid OIDC JSON web token (JWT). You can pass this token to the AWS STS AssumeRoleWithWebIdentity API operation and receive IAM temporary role credentials. You can use these credentials to interact with any AWS service, including Amazon S3 and DynamoDB.

Each JWT token is signed by a signing key pair. The keys are served on the OIDC provider managed by Amazon EKS and the private key rotates every 7 days. Amazon EKS keeps the public keys until they expire. If you connect external OIDC clients, be aware that you need to refresh the signing keys before the public key expires. 

Kubernetes has long used service accounts as its own internal identity system. Pods can authenticate with the Kubernetes API server using an auto-mounted token (which was a non-OIDC JWT) that only the Kubernetes API server could validate. These legacy service account tokens don't expire, and rotating the signing key is a difficult process. In Kubernetes version 1.12, support was added for a new ProjectedServiceAccountToken feature. This feature is an OIDC JSON web token that also contains the service account identity and supports a configurable audience.

Amazon EKS hosts a public OIDC discovery endpoint for each cluster that contains the signing keys for the ProjectedServiceAccountToken JSON web tokens so external systems, such as IAM, can validate and accept the OIDC tokens that are issued by Kubernetes.

OIDC federation access allows you to assume IAM roles via the Secure Token Service (STS), enabling authentication with an OIDC provider, receiving a JSON Web Token (JWT), which in turn can be used to assume an IAM role. Kubernetes, on the other hand, can issue so-called projected service account tokens, which happen to be valid OIDC JWTs for pods. Our setup equips each pod with a cryptographically-signed token that can be verified by STS against the OIDC provider of your choice to establish the pod’s identity.

new credential provider ”sts:AssumeRoleWithWebIdentity”


IRSA authentication
EKS OIDC IdP-signed JWT gets auto-mounted on each pod which uses service account.
AWS SDK sends AssumeRoleWithWebIdentity request containing the desired role and JWT.
STS uses IAM IdP associated to EKS OIDC IdP in order to verify identity of the pod.  


To use/enable IRSA:
  • 1) a unique OpenID Connect provider needs to be created for each EKS cluster in IAM. [Create an IAM OIDC provider for your cluster - Amazon EKS]
    • To use IAM roles for service accounts, an IAM OIDC provider must exist for your cluster's OIDC issuer URL.
    • If your cluster supports IAM roles for service accounts, it has an OpenID Connect (OIDC) issuer URL associated with it. 
    • You can view this URL in the Amazon EKS console, or you can use the following AWS CLI command to retrieve it.
$ aws eks describe-cluster --name cluster_name --query "cluster.identity.oidc.issuer" --output text

The expected output is as follows:

https://oidc.eks.<region-code>.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E

Any OIDC provider implementation needs to have a public OIDC issuer URL (see Issuer URL in OpenID Connect Discovery should be a working URL? - Stack Overflow). So for each cluster we'll have one implementation of OIDC provider.


Your Identity Provider’s Discovery Endpoint contains important configuration information. The OIDC discovery endpoint will always end with /.well-known/openid-configuration as described in the 
OpenID Provider Configuration Request documentation.

You can confirm that the discovery endpoint is correct by entering it in a browser window. If there is a JSON object with metadata about the connection returned, the endpoint is correct.


Obtain the thumbprint for an OpenID Connect identity provider - AWS Identity and Access Management
  • 2) Configure a Kubernetes service account to assume an IAM role
  • 3) Configure Pods to use a Kubernetes service account 
  • 4) Use a supported AWS SDK 

Just like we can create OIDC Identity Provider in IAM for representing an external, 3rd party OIDC Provider so we can allow access to AWS for a user authenticated with that 3rd party OIDC Provider, we can also create OIDC Identity Provider in IAM for representing an internal, EKS OIDC Provider which is available for each cluster (each cluster has its own provider). When EKS cluster is created, its OIDC Provider is also created with two pieces of data available:
  • OIDC Provider issuer
    • has its url which is used for discovery - see the screenshot above
  • OIDC Provider server TLS certificate
    • This certificate protects the url above (OIDC Provider issuer url) and is used for clients to verify the identity of OIDC Provider server
    • TLS certificate is necessary for establishing secure communication with the OIDC provider.

In Terraform, this certificate can be obtained like here:

data "tls_certificate" "example" {
  url = aws_eks_cluster.example.identity[0].oidc[0].issuer
}

To create  OIDC Identity Provider (IdP) in IAM for this cluster-specific OIDC Provider we can use aws_iam_openid_connect_provider :


resource "aws_iam_openid_connect_provider" "example" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.example.certificates[0].sha1_fingerprint]
  url             = aws_eks_cluster.example.identity[0].oidc[0].issuer
}

This resource requires few pieces of information:
  • url  - Describes which OIDC IdP this resource represents. 
    • The URL of the identity provider. Corresponds to the iss claim.
  • thumbprint_list - Describes how will clients communicate with OIDC IdP (servers). HTTP communication goes through TLS secure channel so we need to know the identity of certificates to use.
    •  A list of server certificate thumbprints for the OpenID Connect (OIDC) identity provider's server certificate(s).
    • When we create an OpenID Connect (OIDC) identity provider in IAM, IAM requires the thumbprint for the top intermediate certificate authority (CA) that signed the certificate used by the external identity provider (IdP). The thumbprint is a signature for the CA's certificate that was used to issue the certificate for the OIDC-compatible IdP. When we create an IAM OIDC identity provider, we are trusting identities authenticated by that IdP to have access to our AWS account. By using the CA's certificate thumbprint, we trust any certificate issued by that CA with the same DNS name as the one registered. This eliminates the need to update trusts in each account when we renew the IdP's signing certificate.
  • client_id_list - Describes which clients can use this OIDC IdP
    • A list of client IDs (also known as audiences). When a mobile or web app registers with an OpenID Connect provider, they establish a value that identifies the application. (This is the value that's sent as the client_id parameter on OAuth requests.)

AWS Security Token Service (STS)

  • Web service that enables you to request temporary, limited-privilege credentials for users
  • Available as a global service
  • All AWS STS requests go to a single endpoint at https://sts.amazonaws.com
  • Supports the following actions (requests):
    • AssumeRole
      • Returns a set of temporary security credentials that you can use to access AWS resources. These temporary credentials consist of an access key ID, a secret access key, and a security token. For example, user can authenticate via company's SSO and on AWS sign-on page can get these credentials that can be copied to ~/.aws/credentials under a profile and then this profile is used when accessing AWS.
    • AssumeRoleWithSAML
    • AssumeRoleWithWebIdentity
      • Issues a role session (temporary session)
      • Returns a set of temporary security credentials for users who have been authenticated in a mobile or web application with a web identity provider. Example providers include the OAuth 2.0 providers Login with Amazon and Facebook, or any OpenID Connect-compatible identity provider such as Google or Amazon Cognito federated identities.
      • Calling AssumeRoleWithWebIdentity does not require the use of AWS security credentials. Therefore, you can distribute an application (for example, on mobile devices) that requests temporary security credentials without including long-term AWS credentials in the application. You also don't need to deploy server-based proxy services that use long-term AWS credentials. Instead, the identity of the caller is validated by using a token from the web identity provider. 
      • The temporary security credentials returned by this API consist of an access key ID, a secret access key, and a security token. Applications can use these temporary security credentials to sign calls to AWS service API operations.
      • For example, user can authenticate via company's SSO and on AWS sign-on page can get these credentials that can be copied to ~/.aws/credentials under a profile and then this profile is used when accessing AWS.
      • By default, the temporary security credentials created by AssumeRoleWithWebIdentity last for one hour. However, you can use the optional DurationSeconds parameter to specify the duration of your session. You can provide a value from 900 seconds (15 minutes) up to the maximum session duration setting for the role. This setting can have a value from 1 hour to 12 hours.
      • Required parameters: 
        • RoleArn - The Amazon Resource Name (ARN) of the role that the caller is assuming.
        • RoleSessionName - An identifier for the assumed role session. Typically, you pass the name or identifier that is associated with the user who is using your application. That way, the temporary security credentials that your application will use are associated with that user. This session name is included as part of the ARN and assumed role ID in the AssumedRoleUser response element.
        • WebIdentityToken - The OAuth 2.0 access token or OpenID Connect ID token that is provided by the identity provider. Your application must get this token by authenticating the user who is using your application with a web identity provider before the application makes an AssumeRoleWithWebIdentity call. Timestamps in the token must be formatted as either an integer or a long integer. Only tokens with RSA algorithms (RS256) are supported. 
    • DecodeAuthorizationMessage
    • GetAccessKeyInfo
    • GetCallerIdentity
    • GetFederationToken
    • GetSessionToken


Example: Create an IAM role and associate it with a Kubernetes service account


Our custom service account that we have in cluster, my-service-account requires permission to e.g. launch EC2 instances. We need to assign certain IAM role to this service account (IRSA). 

We've created OIDC IdP in IAM for OIDC IdP associated with our cluster: 

arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

Prepare the policies for the role that the IdP-authenticated user will assume. As with any role, a role for a service account includes two policies:
  • trust policy that specifies who can assume the role
  • permissions policy that specifies the AWS actions and resources that the role owner is allowed or denied access to
Trust Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:default:my-service-account",
                    "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}

Principal here is OIDC session principal which is a role session principal, see https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html#sts-session-principals.

This allows anyone who's authenticated with this OIDC IdP and who has my-service-account as the subject (user) and sts.amazonaws.com as the audience (client) in the WebIdentityToken (sent as the parameter of this request)  to be able to assume the role that has this policy attached to it.

Our service account authenticates with cluster's OIDC provider (IdP) and from it gets the token. This is Identity Token mentioned in OpenID Connect (OIDC) | My Public Notepad and it contains sub (subject identity, service account in our case) and aud (audience - client - who'll be using this token; STS in our case) claims.

It then sends AssumeRoleWithWebIdentity request with this token and role it requires (e.g. role for creating EC2 instances) to STS. STS (Client) then uses this token against EKS cluster IdP to identify user (service account) and finally to grant it a role. 

Troubleshooting



"error": "fetching instance types using ec2.DescribeInstanceTypes,
WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403

References:


Friday, 7 June 2024

OpenID Connect (OIDC)


OpenID Connect (OIDC)

  • Authentication protocol
    • Authentication is a secure process of establishing and communicating that the person operating an application or browser is who they claim to be.
  • Provides a secure and verifiable answer to the question “What is the identity of the person currently using the browser or mobile app that is connected?”
  • Based on the OAuth 2.0
    • OAuth 2.0, is a framework, specified by the IETF in RFCs 6749 and 6750 (published in 2012) designed to support the development of authentication and authorization protocols. It provides a variety of standardized message flows based on JSON and HTTP; OpenID Connect uses these to provide Identity services.
  • Simplifies:
    • user identity verification
      • based on the authentication performed by an Authorization Server
    • obtaining user profile information 
      • in an interoperable and REST-like manner
  • Specification extendable to support optional features like:
    • encryption of identity data
    • discovery of OpenID Providers
    • session logout
  • Benefits for developers:
    • Easy, reliable, secure
    • Removes the responsibility of setting, storing, and managing passwords - they are stored with OpenID providers
    • There are already system-level APIs built into the Android operating system to provide OIDC services
    • OIDC can also accessed by interacting with the built-in system browser on mobile and desktop platforms; a variety of libraries are under construction to simplify this process.
    • OIDC uses standard JSON Web Token (JWT) data structures when signatures are required. This makes OpenID Connect dramatically easier to implement, and in practice has resulted in much better interoperability.

Entities in the oidc system

  • OpenID Provider (OP)
    • Entity that has implemented the OpenID Connect and OAuth 2.0 protocols
    • Sometimes can be referred to by the role it plays, such as:
      • Identity provider (IDP, IdP) - IdentityServer
      • Security token service
      • Authorization server
    • Leading IdPs are currently large cloud services providers, such as Auth0, GitHub, GitLab, Google and Microsoft
  • Identity Token
    • The outcome of an authentication process
    • After successful authentication, OP returns it to the Client
    • It can contain additional identity data but at a bare minimum it contains the following claims:
      • iss -Issuer Identifier for the Issuer of the response. The iss value is a case-sensitive URL using the https scheme that contains scheme, host, and optionally, port number and path components and no query or fragment components.
      • sub - Subject Identifier. Identifier for the user at the issuer. A locally unique and never reassigned identifier within the Issuer for the End-User, which is intended to be consumed by the Client, e.g., 24400320 or AItOawmwtWwcT0k51BayewNvutrJUqsvl6qs7A4.
      • aud - Audience(s) that this ID Token is intended for. It MUST contain the OAuth 2.0 client_id of the Relying Party as an audience value. It MAY also contain identifiers for other audiences. In the general case, the aud value is an array of case-sensitive strings. In the common special case when there is one audience, the aud value MAY be a single case-sensitive string.
      • exp - Expiration time on or after which the ID Token MUST NOT be accepted by the RP when performing authentication with the OP. The processing of this parameter requires that the current date/time MUST be before the expiration date/time listed in the value. Implementers MAY provide for some small leeway, usually no more than a few minutes, to account for clock skew. Its value is a JSON [RFC8259] number representing the number of seconds from 1970-01-01T00:00:00Z as measured in UTC until the date/time.
      • iat - Time at which the JWT was issued. Its value is a JSON number representing the number of seconds from 1970-01-01T00:00:00Z as measured in UTC until the date/time
      • The full list of claims returned within the token: https://openid.net/specs/openid-connect-core-1_0.html#IDToken
  • Access Token
    • After successful authentication, OP usually returns it to the Client
  • User
    • person that is using a registered client to access resources
  • Client
    • also known as audiences
    • software that requests tokens for:
      • authenticating a user
      • accessing a resource (also often called a relying party or RP)
    • must be registered with the OP
      • ClientID is used to identify a client app to IdP servers e.g. for Google OAuth this is in form 1234567890-abcdefghijklmnopqrstuvwxyz.apps.googleusercontent.com
      • When a mobile or web app registers with an OpenID Connect provider, they establish a value that identifies the application. (This is the value that's sent as the client_id parameter on OAuth requests.)
    • can be web, mobile, desktop application
  • Relying Party (RP)
    • resource that user wants to access
    • an application or website that outsources its user authentication function to an IDP
    • application (software) that requires end-user authentication or wants to get access to the user's account. 
    • It needs to get permission from the user before it can get access to the user's account 
      • OpenID Connect identifies a set of personal attributes that can be exchanged between Identity Providers and the apps that use them and includes an approval step (aka authorization) so that users can consent (or deny) the sharing of this information.
    • OIDC Relying Party is also called just a 'client' in OAuth terminology.  (source: OIDC Relying Party)

Approval step (scope authorization) dialog looks like this:

Source: Uploading to Dropbox from Google Drive - Stack Overflow


Source: OpenID Connect  |  Authentication  |  Google for Developers


 OpenID Connect protocol steps

  • User navigates to a website or web application (RP) via a browser
  • User clicks sign-in and types their username and password
  • RP (Client) sends a (authorisation) request to the OpenID Provider (OP)
  • OP authenticates the User and obtains authorization
  • OP responds with an Identity Token and usually an Access Token
  • RP can send a request with the Access Token to the User device
  • UserInfo Endpoint returns Claims about the User

source: OpenID Connect 1.0 - Orange Developer



Source: OpenID Connect Overview: OIDC Flow | OneLogin Developers



Here is a more detailed flow diagram:

Source: OpenID Connect (OIDC) | Cloud Sundial





Source: Plan a single sign-on deployment - Microsoft Entra ID | Microsoft Learn



Resources:


Tuesday, 4 June 2024

Introduction to Amazon GuardDuty with Terraform

 


Amazon GuardDuty:
  • Intelligent threat detection service
  • Continuously monitors, profiles and analyses events across AWS accounts and resources for potential threats 
  • Exposes threats quickly using:
    • Anomaly detection
    • Machine learning (ML)
    • Behavioral modeling
    • Threat intelligence feeds from AWS and leading third parties
  • Helps protecting AWS accounts, workloads, and data from threats

To protect your AWS accounts and resources it is analysing tens of billions of events per minute across multiple AWS data sources, such as:
  • AWS CloudTrail event logs
  • Amazon Virtual Private Cloud (Amazon VPC) Flow Logs
  • DNS query logs
  • Amazon Simple Storage Service (Amazon S3) data plane events
  • Amazon Elastic Kubernetes Service (Amazon EKS) audit logs
  • Amazon Relational Database Service (Amazon RDS) login events 





Enabling Amazon GuardDuty via Terraform


To enable Amazon GuardDuty monitoring and feedback reporting we need to use aws_guardduty_detector resource.

main.tf:

resource "aws_guardduty_detector" "this" {
  enable = true
}


$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_guardduty_detector.this will be created
  + resource "aws_guardduty_detector" "this" {
      + account_id                   = (known after apply)
      + arn                          = (known after apply)
      + enable                       = true
      + finding_publishing_frequency = (known after apply)
      + id                           = (known after apply)
      + tags_all                     = (known after apply)

      + datasources {
          + kubernetes {
              + audit_logs {
                  + enable = (known after apply)
                }
            }

          + malware_protection {
              + scan_ec2_instance_with_findings {
                  + ebs_volumes {
                      + enable = (known after apply)
                    }
                }
            }

          + s3_logs {
              + enable = (known after apply)
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_guardduty_detector.this: Creating...
aws_guardduty_detector.this: Creation complete after 1s [id=e6c7f038a9682cf6ff6bb514c110a66f]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.


If GuardDuty Detector already exists for the current account (e.g. it was added manually, in AWS Console), terraform apply might error out:

aws_guardduty_detector.this: Creating...
│ Error: creating GuardDuty Detector: BadRequestException: The request is rejected because a detector already exists for the current account.
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "6e9a9855-a801-4f98-ab5e-8c4461a307fe"
│   },
│   Message_: "The request is rejected because a detector already exists for the current account.",
│   Type: "InvalidInputException"
│ }
│ 
│   with aws_guardduty_detector.this,
│   on main.tf line 6, in resource "aws_guardduty_detector" "this":
│    6: resource "aws_guardduty_detector" "this" {

To fix this we can list all detectors for the given profile:

% aws guardduty list-detectors --profile my-profile
{
    "DetectorIds": [
        "81cc2fd56fb7a205e9bd39d16ecb1dd2"
    ]
}

...and we need to import this resource to Terraform state. For Terraform versions before v1.5.0 this can be only done via CLI command:

% terraform import aws_guardduty_detector.this 81cc2fd56fb7a205e9bd39d16ecb1dd2
Acquiring state lock. This may take a few moments...
...                          

aws_guardduty_detector.this: Importing from ID "81cc2fd56fb7a205e9bd39d16ecb1dd2"...
aws_guardduty_detector.this: Import prepared!
  Prepared aws_guardduty_detector for import
aws_guardduty_detector.this: Refreshing state... [id= 81cc2fd56fb7a205e9bd39d16ecb1dd2]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

Releasing state lock. This may take a few moments...

The recommended way for Terraform v1.5.0+ is to use import block:

import {
   to = aws_guardduty_detector.this
   id = 81cc2fd56fb7a205e9bd39d16ecb1dd2
}


Once we've provisioned the resource, let's check in AWS Console which protection plans are now enabled:

S3 Protection




From AWS Console side info panel:
S3 protection enables Amazon GuardDuty to monitor object-level API operations to identify potential security risks for data within your S3 buckets.

GuardDuty monitors CloudTrail management events involving your S3 resources, which include bucket-level API operations such as ListBuckets, DeleteBuckets, and PutBucketReplication. This capability is a core part of the service and cannot be disabled. S3 protection is an expansion of existing GuardDuty CloudTrail monitoring capability that allows CloudTrail data events, such as ListObjects, DeleteObject, and PutObject, to be monitored in addition to CloudTrail management events.

This feature is enabled by default for new accounts but can be optionally disabled or enabled at any time from this page. You can also enable S3 protection for all member accounts in your organization from this page. To disable S3 protection for a member account, use the Actions menu on the Accounts page.

GuardDuty highly recommends that you enable this feature to help ensure that your S3 resources are fully monitored.


EKS Protection





From AWS Console side info panel:
EKS Protection in Amazon GuardDuty provides threat detection coverage to help you protect Amazon Elastic Kubernetes Service (Amazon EKS) clusters within your AWS environment. EKS Protection includes configuration for EKS Audit Log Monitoring and EKS Runtime Monitoring.

EKS Audit Log Monitoring – Helps you detect potentially suspicious activities in EKS clusters within Amazon Elastic Kubernetes Service (Amazon EKS), using Kubernetes audit logs.
Kubernetes audit logs capture sequential actions within your Amazon EKS cluster, including activities from users, applications using the Kubernetes API, and the control plane.

EKS Runtime Monitoring – Provides runtime threat detection for Amazon EKS nodes and containers within your AWS environment.
With the support of an Amazon EKS add-on GuardDuty security agent, EKS Runtime Monitoring monitors and processes runtime events for your Amazon EKS clusters to identify potential security threats in your environment.

If you configure your accounts with EKS Audit Log Monitoring and EKS Runtime Monitoring together, you can achieve optimal EKS Protection both at the cluster control plane level and down to the individual pod or container operating system level.


Runtime Monitoring


It is still disabled. To enable it via Terraform we need to use aws_guardduty_detector_feature resource.



From AWS Console side info panel:
Runtime Monitoring helps GuardDuty monitor and analyze the operating system-level events to help you detect potential threats in specific AWS workloads within your AWS environment. Runtime Monitoring was previously only available for Amazon Elastic Kubernetes Service (Amazon EKS) resources, but GuardDuty has now expanded the Runtime Monitoring feature to provide threat detection for the Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Compute Cloud (Amazon EC2) resources.

Presently, the Amazon EC2 instance support is available as a preview release and is subject to change.

The console support for EKS Runtime Monitoring is now available within Runtime Monitoring. To use the console support, you must migrate your existing EKS Runtime Monitoring configuration to Runtime Monitoring.


Malware Protection for EC2




From AWS Console side info panel:
Malware Protection helps you detect the potential presence of malware in Amazon EC2 instances and container workloads in your account. It scans the Amazon EBS volumes attached to these Amazon EC2 instances or container workloads. Malware Protection offers two types of scan – GuardDuty-initiated malware scan and On-demand malware scan. 

GuardDuty-initiated malware scan 

Enable GuardDuty-initiated malware scan to initiate an agentless scan of the Amazon EBS volumes attached to the Amazon EC2 instances and container workloads, automatically whenever GuardDuty generates any of the Findings that invoke GuardDuty-initiated malware scan. If GuardDuty-initiated malware scan is enabled for the first time in your account, it is included in the 30-day free trial period.

If you want to skip certain resources from the automatic scanning process, you can add the tags associated with those Amazon EC2 resources to the exclusion list. If you want to ensure that certain resources are scanned, add the tags associated with those Amazon EC2 resources to the inclusion list. At any point in time, you can choose to follow either inclusion list or exclusion list.

Malware Protection also provides an option to retain the scanned snapshots of your Amazon EBS volumes. By default, the snapshots retention setting is turned off. If you want to retain the scanned snapshots in your AWS account, ensure to turn on this setting.

Both scan options and snapshots retention setting must be configured before the scan gets initiated.

On-demand malware scan

To initiate an on-demand malware scan on the Amazon EBS volumes attached to your Amazon EC2 instance, provide the Amazon Resource Name (ARN) associated with your Amazon EC2 instance. Initiating an on-demand malware scan doesn't require any configuration. If you're a delegated GuardDuty administrator account who is initiating an on-demand malware scan on behalf of one of the member accounts, ensure that the GuardDuty member account has GuardDuty activated. If you're initiating an On-demand malware scan and the Malware Protection service-linked role (SLR) is not available in your account, GuardDuty will create it automatically for your account.

Because On-demand malware scan is independent of GuardDuty-initiated malware scan, you can initiate an on-demand malware scan even when you have turned on GuardDuty-initiated malware scan. On-demand malware scan doesn't have a 30-day free trial period. For each on-demand malware scan, the usage cost applies to the total Amazon EBS volume scanned for each malware scan. For more information, see Amazon GuardDuty pricing. For information about the cost of creating the Amazon EBS volume snapshots and their retention, see Amazon EBS pricing.

RDS Protection




From AWS Console side info panel:
GuardDuty RDS Protection helps you protect Amazon Aurora databases in your account. Enable the RDS Protection feature to analyze and profile RDS login activity for potential access threats to your supported databases. RDS Protection doesn't require additional infrastructure; it is designed so as not to affect the performance of your database instances.

When you enable the RDS Protection feature, GuardDuty automatically starts to monitor login events for your databases directly from the Amazon Aurora service. When RDS Protection is not enabled, GuardDuty neither ingests RDS login activity nor detects anomalous or suspicious login behavior. You can enable or disable the RDS Protection feature for any account at any time.

Lambda Protection




From AWS Console side info panel:
Lambda Protection helps you identify potential security threats when your AWS Lambda functions gets invoked. After your enable Lambda Protection, GuardDuty will start monitoring Lambda network activity logs associated with the Lambda function in your AWS account. When your Lambda function gets invoked and GuardDuty identifies suspicious network traffic that indicates the presence of a potentially malicious piece of code in your Lambda function, GuardDuty generates a finding.

Disabling GuardDuty with Terraform


resource "aws_guardduty_detector" "this" {
  enable = false
}

In AWS Console we'll still be able to see past findings but monitoring will be stopped and no new findings will be added:



References:


Friday, 27 May 2022

How to create PKA key pair using AWS

To SSH to our EC2 instance we need to create a Public Key Authentication (PKA) key pair which consist of public and private key. Public key is stored on EC2 AMI (in ~/.ssh/authorized_keys) and this happens on the first boot. Private key needs to be present on the machine where from we want to establish SSH connection. Its path is passed to SSH connect command.
 
We can create key pair in multiple ways:

 

Public key needs to be imported to EC2 instance. One way is via Terraform, by using aws_key_pair and passing its id attribute value as the value of key_name attribute of aws_instance.

 

How to create PKA key pair using AWS Management Console

 
Log in to AWS Management Console and in left hand list find Key Pairs item in Network & Security group:
 
 
 
At the beginning we have no key pairs created so we click on Create key pair button:
 

 
 
This opens a dialog where we can choose the key pair name, encryption type and private key file format:
 

 
When we click on Create key pair button, private key file (named key-pair--ec2--my-app.pem in this example) gets downloaded to our computer automatically and we can see that new key pair is now listed:
 


 
 
If you want the same key pair to work in multiple AWS regions, make sure public key is applied to each region.
 

How to password-protect the private key file

 
 To password-protect downloaded pem file we can use:

$ ssh-keygen -p -f key-pair--ec2--my-app.pem 
 
If file is readable by anyone, this operation will fail with:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0664 for 'key-pair--ec2--my-app.pem' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Failed to load key key-pair--ec2--my-app.pem: bad permissions


File was indeed readable by everyone:

$ ls -la key-pair--ec2--my-app.pem
-rw-rw-r-- 1 bojan bojan 1678 May 27 11:34 key-pair--ec2--my-app.pem

 
To rectify the error above, we need to assign read permissions only to the file owner:
 
$ sudo chmod 400 key-pair--ec2--my-app.pem 
 
$ ls -la key-pair--ec2--my-app.pem
-r-------- 1 bojan bojan 1678 May 27 11:34 key-pair--ec2--my-app.pem


We can now set the password on the file:

$ sudo ssh-keygen -p -f key-pair--ec2--my-app.pem
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

Next time this file is used by ssh, you'll be prompted to enter the password.
 
 

How to create public key from the private key 



It is not possible to download (or see) public key in EC2 Key pairs dashboard (list, as seen on the screenshot above). But it is possible to generate it from the private key (.pem file):

$ ssh-keygen -y -f key-pair--ec2--my-app.pem > key-pair--ec2--my-app.pub
Enter passphrase:
 
$ cat key-pair--ec2--my-app.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtkH9hzk...0a+UPwPy
hD


How to create PKA key pair using AWS CLI

 
The following command creates the key pair, automatically uploads it to AWS EC2 (it gets listed in AWS Management Console, among EC2 key pairs) and saves private key on the local machine:
 
$ aws ec2 create-key-pair \
--key-name key-pair--ec2--my-app \
--query 'KeyMaterial' \
--output text > key-pair--ec2--my-app.pem

 
--query "KeyMaterial" prints the private key material to the output
--output text > my-key-pair.pem saves the private key material in a file with the specified extension. The extension can be either .pem or .ppk
 
Additional arguments:
 
--key-type: rsa (default) or ed25519
--key-format: pem (default) or ppk 


 

---

Both approaches shown above provision EC2 key pairs manually. 

If we want to use Terraform the best way is to use 3rd party (e.g. OpenSSH) to create key pairs locally and then use aws_key_pair resource in TF configuration. It is NOT possible importing manually provisioned key pairs into TF state without recreating them. (For more details see Importing infrastructure in Terraform | My Public Notepad)


How to list/find key pairs?


Use ec2 describe-key-pairs. To list all key pairs:

$ aws ec2 describe-key-pairs


To list details for specific key pair:
     
$ aws ec2 describe-key-pairs --key-names key-pair--ec2--bojan-temp
{
    "KeyPairs": [
        {
            "KeyPairId": "key-0483bced858d885ba",
            "KeyFingerprint": "c2:18:8e:93:ee:52:f9:13:bb:05:9d:94:0c:52:af:9b:ff:6b:d5:3f",
            "KeyName": "key-pair--ec2--bojan-temp",
            "KeyType": "rsa",
            "Tags": []
        }
    ]
}



How to remove specified key pair?


 
$ aws ec2 delete-key-pair --key-name key_pair_name



---