Showing posts with label GitHub. Show all posts
Showing posts with label GitHub. Show all posts

Friday, 27 June 2025

GitHub Workflows and AWS




GitHub workflow can communicate with our AWS resources, directly (via AWS CLI commands) or indirectly (via e.g. Terraform AWS provider).

Before running AWS CLI commands, deploying AWS infrastructure with Terraform, or interacting with AWS services in any way we need to include a step which configures AWS credentials. It ensures that the workflow runner is authenticated with AWS and knows which region to target.

This step should contain configure-aws-credentials action provided by AWS. This action sets up the necessary environment variables so that AWS CLI commands and SDKs can authenticate with AWS services.

aws-region input sets the default AWS region to us-east-2 (Ohio). All AWS commands run in later steps will use this region unless overridden.

We can use either IAM user or OIDC (temp token) authentication.

IAM User Authentication


If using IAM user authentication, we can store user's credentials in a dedicated GitHub secrets:

env:
    AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    AWS_REGION: us-east-2

// Define this step before steps which are accessing AWS:

- name: Configure AWS Credentials
     uses: aws-actions/configure-aws-credentials@v2
     with:
        aws-region: ${{ env.AWS_REGION }}

 OpenID Connect (OIDC) Authentication


In this authentication, configure-aws-credentials GitHub Action uses GitHub's OpenID Connect (OIDC) for secure authentication with AWS. It leverages the OIDC token provided by GitHub to request temporary AWS credentials from AWS STS, eliminating the need to store long-lived AWS access keys in GitHub Secrets. 

Note that we now need to grant the workflow run a permissions for write access to the id-token:
id-token: write allows the workflow to request and use OpenID Connect (OIDC) tokens. The write level is required for actions that need to generate or use OIDC tokens to authenticate with external systems. Granting id-token: write is essential for workflows that use OIDC-based authentication, such as securely assuming AWS IAM roles via GitHub Actions. This enables secure, short-lived authentication to AWS and other cloud providers. This permission is a security best practice for modern CI/CD workflows that use OIDC to authenticate with cloud providers, reducing the need for static secrets.


env:
    AWS_REGION: us-east-2

permissions:
  id-token: write # aws-actions/configure-aws-credentials (OIDC)

...
- name: Configure AWS Credentials
    uses: aws-actions/configure-aws-credentials@v4
    with:
        role-to-assume: arn:aws:iam::123456789012:role/github-actions-role
        role-session-name: my-app
        aws-region:  ${{ env.AWS_REGION }}



Here's how it works: 
  1. GitHub OIDC Provider: GitHub acts as an OIDC provider, issuing signed JWTs (JSON Web Tokens) to workflows that request them.
  2. configure-aws-credentials Action: This action, when invoked in a GitHub Actions workflow, receives the JWT from the OIDC provider.
  3. AWS STS Request: The action then uses the JWT to request temporary security credentials from AWS Security Token Service (STS).
  4. Credential Injection: AWS STS returns temporary credentials (access key ID, secret access key, and session token) which the action injects as environment variables into the workflow's execution environment.
  5. AWS SDKs and CLI: AWS SDKs and the AWS CLI automatically detect and use these environment variables for authenticating with AWS services.

Benefits of using OIDC with configure-aws-credentials:
  • Enhanced Security: Eliminates the need to store long-lived AWS access keys, reducing the risk of compromise.
  • Simplified Credential Management: Automatic retrieval and injection of temporary credentials, simplifying workflow setup and maintenance.
  • Improved Auditing: Provides better traceability of actions performed within AWS, as the identity is linked to the GitHub user or organization. 

Before using the action:
  • Configure an OpenID Connect provider in AWS: We need to establish an OIDC trust relationship between GitHub and our AWS account.
  • Create an IAM role in AWS: Define the permissions for the role that the configure-aws-credentials action will assume.
  • Set up the GitHub workflow: Configure the configure-aws-credentials action with the appropriate parameters, such as the AWS region and the IAM role to assume. 

In an OpenID Connect (OIDC) authentication scenario, the aws-actions/configure-aws-credentials action creates the following environment variables when assuming a role with temporary credentials: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. These variables are used by the AWS SDK and CLI to interact with AWS resources. 

Here's a breakdown:
  • AWS_ACCESS_KEY_ID: This environment variable stores the access key ID of the temporary credentials. 
  • AWS_SECRET_ACCESS_KEY: This environment variable stores the secret access key of the temporary credentials. 
  • AWS_SESSION_TOKEN: This environment variable stores the session token associated with the temporary credentials, which is required for operations with AWS Security Token Service (STS). 

These environment variables are populated by the action after successful authentication with the OIDC provider and assuming the specified IAM role. The action retrieves the temporary credentials from AWS and makes them available to subsequent steps in the workflow. 


Once AWS authentication is done and this env variables are created, the next steps in the workflow can access our AWS resources, e.g. read secrets from AWS Secrets Manager:

- name: Read secrets from AWS Secrets Manager into environment variables
    uses: aws-actions/aws-secretsmanager-get-secrets@v2
    with:
        secret-ids: |
            my-secret
        parse-json-secrets: true

- name: deploy
    run: |
        echo $AWS_ACCESS_KEY_ID
        echo $AWS_SECRET_ACCESS_KEY
    env:
        MY_KEY: ${{ env.MY_SECRET_MY_KEY }}

This example assumes that in AWS secret my-secret we have a key MY_KEY, set to the secret value we want to fetch and use.

Friday, 28 February 2025

Introduction to GitHub CLI

 


git CLI client can perform some usual tasks with the repo like creating it, cloning, commiting and pushing updates etc but some of the common actions are specific to git repository provider like GitHub or GitLab and can't be done with git client only. Such actions are creating a pull request, checking assigned issues, review requests, adding a comment to pull request etc. This article is a short intro into GitHub's CLI client, gh.

Installation


To install it on Mac:

% brew install gh 

No admin (sudo) permissions are required. 

To verify installation:

% gh
Work seamlessly with GitHub from the command line.

USAGE
  gh <command> <subcommand> [flags]

CORE COMMANDS
  auth:        Authenticate gh and git with GitHub
  browse:      Open repositories, issues, pull requests, and more in the browser
  codespace:   Connect to and manage codespaces
  gist:        Manage gists
  issue:       Manage issues
  org:         Manage organizations
  pr:          Manage pull requests
  project:     Work with GitHub Projects.
  release:     Manage releases
  repo:        Manage repositories

GITHUB ACTIONS COMMANDS
  cache:       Manage GitHub Actions caches
  run:         View details about workflow runs
  workflow:    View details about GitHub Actions workflows

ALIAS COMMANDS
  co:          Alias for "pr checkout"

ADDITIONAL COMMANDS
  alias:       Create command shortcuts
  api:         Make an authenticated GitHub API request
  attestation: Work with artifact attestations
  completion:  Generate shell completion scripts
  config:      Manage configuration for gh
  extension:   Manage gh extensions
  gpg-key:     Manage GPG keys
  label:       Manage labels
  ruleset:     View info about repo rulesets
  search:      Search for repositories, issues, and pull requests
  secret:      Manage GitHub secrets
  ssh-key:     Manage SSH keys
  status:      Print information about relevant issues, pull requests, and notifications across repositories
  variable:    Manage GitHub Actions variables

HELP TOPICS
  actions:     Learn about working with GitHub Actions
  environment: Environment variables that can be used with gh
  exit-codes:  Exit codes used by gh
  formatting:  Formatting options for JSON data exported from gh
  mintty:      Information about using gh with MinTTY
  reference:   A comprehensive reference of all gh commands

FLAGS
  --help      Show help for command
  --version   Show gh version

EXAMPLES
  $ gh issue create
  $ gh repo clone cli/cli
  $ gh pr checkout 321

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual
  Learn about exit codes using `gh help exit-codes`


Setting up authentication with GitHub


In GitHub website, to to Settings >> Developer Settings >> Personal access tokens and create a token with desired permissions. Copy its value and add the following environment variable to your personal settings of the terminal you use:

export GH_TOKEN=<github_personal_access_token_value>

% vi ~/.zshrc
% source ~/.zshrc

% vi ~/.bash_profile
% source ~/.bash_profile

To verify it, execute this in each terminal:

% echo $GH_TOKEN 

The output should be <github_personal_access_token_value>.


Checking the status across multiple repositories 

To check the status of the GitHub account repositories

% gh status
Assigned Issues
...
Assigned Pull Requests
...
Review Requests
...
Mentions
...
Repository Activity
...


Working with Pull Requests

Here are some common commands used to manage pull requests but make sure you are in the directory which is a GitHub repository otherwise gh will emit the error like:

failed to run git: fatal: not a git repository (or any of the parent directories): .git


To view all pull requests in a repository:

% gh pr list

It lists all open pull requests in the current repository. Use --state closed or --state all to see merged/closed PRs.

To view a specific pull request:

% gh pr view <PR_NUMBER>

or

% gh pr view <PR_URL>

It displays details of a specific PR, including description, status, and mergeability.

To open a pull request in the browser:

% gh pr view <PR_NUMBER> --web

It opens the PR page in your default web browser.

To show PRs created by you:

% gh pr list --author @me

To show PRs assigned to you:

% gh pr list --assignee @me


Working with GitHub Workflows


The main command for working with workflows is - workflow. Let's see the list of its subcommands:


% gh workflow                               
List, view, and run workflows in GitHub Actions.

USAGE
  gh workflow <command> [flags]

AVAILABLE COMMANDS
  disable:     Disable a workflow
  enable:      Enable a workflow
  list:        List workflows
  run:         Run a workflow by creating a workflow_dispatch event
  view:        View the summary of a workflow

FLAGS
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

INHERITED FLAGS
  --help   Show help for command


To show all workflows and their IDs in the current repository:

gh workflow list

NAME                      STATE   ID       
my-workflow               active  131118881
...

To see the details of some workflow:

% gh workflow view my-workflow
my-workflow - my-workflow.yaml
ID: 131118881

Total runs 2
Recent runs
   TITLE        WORKFLOW     BRANCH                    EVENT              ID         
✓  my-workflow  my-workflow  project/app-upgrade/test  workflow_dispatch  13543363356
X  my-workflow  my-workflow  project/app-upgrade/test  workflow_dispatch  13158297800

To see more runs for this workflow, try: gh run list --workflow my-workflow.yaml
To see the YAML for this workflow, try: gh workflow view my-workflow.yaml --yaml

To run a workflow by creating a workflow_dispatch event:

% gh workflow run <workflow.yml> --ref <branch_name>

After triggering the workflow, you can check the status with:

% gh run list

Or check the logs of a specific run:

% gh run watch

If your workflow requires inputs, pass them using --json:

% gh workflow run build.yml --ref main --json '{"environment":"staging"}'

To see more details of a specific run:

% gh run view <run_id>

To cancel a running workflow:

% gh run cancel <run_id>


Working with GitHub Workflow Runs



% gh run 
List, view, and watch recent workflow runs from GitHub Actions.

USAGE
  gh run <command> [flags]

AVAILABLE COMMANDS
  cancel:      Cancel a workflow run
  delete:      Delete a workflow run
  download:    Download artifacts generated by a workflow run
  list:        List recent workflow runs
  rerun:       Rerun a run
  view:        View a summary of a workflow run
  watch:       Watch a run until it completes, showing its progress

FLAGS
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

INHERITED FLAGS
  --help   Show help for command



% gh run list --help 
List recent workflow runs.

Note that providing the `workflow_name` to the `-w` flag will not fetch disabled workflows.
Also pass the `-a` flag to fetch disabled workflow runs using the `workflow_name` and the `-w` flag.

For more information about output formatting flags, see `gh help formatting`.

USAGE
  gh run list [flags]

ALIASES
  gh run ls

FLAGS
  -a, --all               Include disabled workflows
  -b, --branch string     Filter runs by branch
  -c, --commit SHA        Filter runs by the SHA of the commit
      --created date      Filter runs by the date it was created
  -e, --event event       Filter runs by which event triggered the run
  -q, --jq expression     Filter JSON output using a jq expression
      --json fields       Output JSON with the specified fields
  -L, --limit int         Maximum number of runs to fetch (default 20)
  -s, --status string     Filter runs by status: {queued|completed|in_progress|requested|waiting|pending|action_required|cancelled|failure|neutral|skipped|stale|startup_failure|success|timed_out}
  -t, --template string   Format JSON output using a Go template; see "gh help formatting"
  -u, --user string       Filter runs by user who triggered the run
  -w, --workflow string   Filter runs by workflow

INHERITED FLAGS
      --help                     Show help for command
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

JSON FIELDS
  attempt, conclusion, createdAt, databaseId, displayTitle, event, headBranch,
  headSha, name, number, startedAt, status, updatedAt, url, workflowDatabaseId,
  workflowName

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual
  Learn about exit codes using `gh help exit-codes`

bojan@admins-MacBook-Pro.local /Users/bojan/repos/CheckpointGG/infra-elastic [project/elastic-upgrade/test]
% gh run list --json --help
Unknown JSON field: "--help"
Available fields:
  attempt
  conclusion
  createdAt
  databaseId
  displayTitle
  event
  headBranch
  headSha
  name
  number
  startedAt
  status
  updatedAt
  url
  workflowDatabaseId
  workflowName

---

Friday, 23 April 2021

How to link Google Colab with GitHub

Create a GitHub repository in which you want to keep Colaboratory notebooks.
Go to https://colab.research.google.com/github/ and add your GitHub account.



GitHub website window will open asking you to accept Colab's access to your GitHub account.
Once accepted that window will close and back in https://colab.research.google.com/github/ a popup window above will contain a list of all repositories in connected GitHub account. Select the one you created. As it's currently empty, there are no notebooks in there, it will show "No results".

If you create a notebook in Colab (https://colab.research.google.com/) and want to add it to that GitHub repo, go to Colab's tab in the browser, File >> Save a copy in GitHub. 




After this, this notebook will appear in the list of notebooks available to be loaded into Colab: