Saturday 29 October 2022

ec2-user on Amazon Linux 2

I'm running an EC2 instance based on Amazon Linux 2 AMI:

$ cat /etc/os-release 
NAME="Amazon Linux"
ID_LIKE="centos rhel fedora"
PRETTY_NAME="Amazon Linux 2"

SSH connection is created for ec2-user:

$ whoami

This user can perform a sudo command and I wanted to learn why. Let's check sudoers:

$ sudo cat /etc/sudoers
## Sudoers allows particular users to run various commands as
## the root user, without needing the root password.
## Examples are provided at the bottom of the file for collections
## of related commands, which can then be delegated out to particular
## users or groups.
## This file must be edited with the 'visudo' command.

## Host Aliases
## Groups of machines. You may prefer to use hostnames (perhaps using 
## wildcards for entire domains) or IP addresses instead.
# Host_Alias     FILESERVERS = fs1, fs2
# Host_Alias     MAILSERVERS = smtp, smtp2

## User Aliases
## These aren't often necessary, as you can use regular groups
## (ie, from files, LDAP, NIS, etc) in this file - just use %groupname 
## rather than USERALIAS
# User_Alias ADMINS = jsmith, mikem

## Command Aliases
## These are groups of related commands...

## Networking
# Cmnd_Alias NETWORKING = /sbin/route, /sbin/ifconfig, /bin/ping, /sbin/dhclient, /usr/bin/net, /sbin/iptables, /usr/bin/rfcomm, /usr/bin/wvdial, /sbin/iwconfig, /sbin/mii-tool

## Installation and management of software
# Cmnd_Alias SOFTWARE = /bin/rpm, /usr/bin/up2date, /usr/bin/yum

## Services
# Cmnd_Alias SERVICES = /sbin/service, /sbin/chkconfig, /usr/bin/systemctl start, /usr/bin/systemctl stop, /usr/bin/systemctl reload, /usr/bin/systemctl restart, /usr/bin/systemctl status, /usr/bin/systemctl enable, /usr/bin/systemctl disable

## Updating the locate database
# Cmnd_Alias LOCATE = /usr/bin/updatedb

## Storage
# Cmnd_Alias STORAGE = /sbin/fdisk, /sbin/sfdisk, /sbin/parted, /sbin/partprobe, /bin/mount, /bin/umount

## Delegating permissions
# Cmnd_Alias DELEGATING = /usr/sbin/visudo, /bin/chown, /bin/chmod, /bin/chgrp 

## Processes
# Cmnd_Alias PROCESSES = /bin/nice, /bin/kill, /usr/bin/kill, /usr/bin/killall

## Drivers
# Cmnd_Alias DRIVERS = /sbin/modprobe

# Defaults specification

# Refuse to run if unable to disable echo on the tty.
Defaults   !visiblepw

# Preserving HOME has security implications since many programs
# use it when searching for configuration files. Note that HOME
# is already set when the the env_reset option is enabled, so
# this option is only effective for configurations where either
# env_reset is disabled or HOME is present in the env_keep list.
Defaults    always_set_home
Defaults    match_group_by_gid

# Prior to version 1.8.15, groups listed in sudoers that were not
# found in the system group database were passed to the group
# plugin, if any. Starting with 1.8.15, only groups of the form
# %:group are resolved via the group plugin by default.
# We enable always_query_group_plugin to restore old behavior.
# Disable this option for new behavior.
Defaults    always_query_group_plugin

Defaults    env_reset

# Adding HOME to env_keep may enable a user to run unrestricted
# commands via sudo.
# Defaults   env_keep += "HOME"

Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin

## Next comes the main part: which users can run what software on 
## which machines (the sudoers file can be shared between multiple
## systems).
## Syntax:
## The COMMANDS section may have other options added to it.
## Allow root to run any commands anywhere 
root ALL=(ALL) ALL

## Allows members of the 'sys' group to run networking, software, 
## service management apps and more.

## Allows people in group wheel to run all commands
%wheel ALL=(ALL) ALL

## Same thing without a password

## Allows members of the users group to mount and unmount the 
## cdrom as root
# %users  ALL=/sbin/mount /mnt/cdrom, /sbin/umount /mnt/cdrom

## Allows members of the users group to shutdown this system
# %users  localhost=/sbin/shutdown -h now

## Read drop-in files from /etc/sudoers.d (the # here does not mean a comment)
#includedir /etc/sudoers.d

Let's check what's in this included directory:

$ sudo ls /etc/sudoers.d/

$ sudo cat  /etc/sudoers.d/90-cloud-init-users
# Created by cloud-init v. 19.3-45.amzn2 on Sun, 23 Oct 2022 10:28:25 +0000

# User rules for ec2-user

The last line means the following: ec2-user on any host can run any command as any user.

Syntax: User <space> OnHost = (Runas-User:Group) <space> Commands
“who where = (as_whom) what”.

Example: root ALL = (ALL:ALL) ALL

If we have some other user that we want to perform some command (e.g. restart NGINX service) that requires root rights we'd need to have in sudoers file a line like this:

my_user ALL = (root) NOPASSWD: systemctl restart nginx

IMPORTANT: Don't use just any editor (vi, vim, nano, ...) to edit sudoers file. If changes contain syntax or formatting errors and they get saved, the next time sudo command is executed, it would fail with message similar to:

$ sudo cat /etc/sudoers.d/90-cloud-init-users
>>> /etc/sudoers.d/90-cloud-init-users: syntax error near line 11 <<<
sudo: parse error in /etc/sudoers.d/90-cloud-init-users near line 11
sudo: no valid sudoers sources found, quitting
sudo: unable to initialize policy plugin

(If this happens on EC2 instance you need to terminate (not just stop it!) and launch a new instance, with fresh, new, EBS volume.)

Always use visudo to edit sudoers file. visudo needs to be run with root privileges otherwise the permissions error is issued:

$ visudo
visudo: /etc/sudoers: Permission denied

To open non-default sudoers file use -f option:
$ sudo visudo -f /etc/sudoers.d/90-cloud-init-users

If we try to save invalid content, visudo will give us 3 options (press ENTER after What now? to see them):

$ sudo visudo -f /etc/sudoers.d/90-cloud-init-users
>>> /etc/sudoers.d/90-cloud-init-users: syntax error near line 9 <<<
What now?
Options are:
  (e)dit sudoers file again
  e(x)it without saving changes to sudoers file
  (Q)uit and save changes to sudoers file (DANGER!)

What now?



Friday 28 October 2022

Configuring vi on Ubuntu

With fresh Ubuntu 22.04.1 LTS (Jammy Jellyfish) installation my vi editor in INSERT mode behaved like this:
  • Arrow Up/Down/Left/Right prints characters A, B, C, D
  • Backspace key is not deleting previous characters
My ~/.vimrc file contained:
$ set nocompatible
$ set backspace=2
To remedy this behaviour I had to change this file to:

set nocompatible
set backspace=indent,eol,start

To reload this configuration file and the changes can take effect in the current terminal:
$ source  ~/.vimrc

Wednesday 19 October 2022

How to fix Ansible error "Failed to connect to the host via ssh: Unable to negotiate with"

On my Ubuntu 22.04 box I had a case where Packer was executing Ansible template and Gathering Facts task was failing with the following error:
amazon-ebs: TASK [Gathering Facts] *********************************************************
==> amazon-ebs: failed to handshake
amazon-ebs: fatal: [default]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Unable to negotiate with port 37203: no matching host key type found. Their offer: ssh-rsa", "unreachable": true}

The solution is to allow authentication with RSA keys by making sure the following lines are present in /etc/ssh/ssh_config:
PubkeyAcceptedAlgorithms ssh-rsa
HostkeyAlgorithms ssh-rsa



Friday 14 October 2022

Symbolic Links (symlinks)


linux - Find out symbolic link target via command line - Server Fault

What are they?

  • files that contain a reference to another file or directory on the same system
  • like shortcuts on Windows OS

What is their purpose?

  • to avoid copying the same binary (usually a library) at multiple locations but simply creating a symlink where file is required to be
  • various clients might require the same file but with name in different format so instead of having multiple copies of the same file but with different names we'd have multiple symlink, each with the name that satisfies requirements of each service

How do they work?

  • opening/running the symlink would open/run the target file
  • editing the content of the symlink edits the content of the target file
  • if target file is deleted symlink becomes a dangling symlink
  • if symlink is deleted, target file remains unaffected
  • it is possible to create symlink that refers to another symlink [How can I create a symlink which points to another symlink?]

How to create them?

How to: Linux / UNIX create soft link with ln command

Use ln command:

       ln - make links between files

       ln [OPTION]... [-T] TARGET LINK_NAME   (1st form)
       ln [OPTION]... TARGET                  (2nd form)
       ln [OPTION]... TARGET... DIRECTORY     (3rd form)
       ln [OPTION]... -t DIRECTORY TARGET...  (4th form)

       In the 1st form, create a link to TARGET with the name LINK_NAME.  In the 2nd form, create a link to TARGET in the current directory.  In the 3rd and 4th forms, create links to each TARGET in DIRECTORY.  Create hard links by default, symbolic links with --symbolic.  By default, each destination (name of new link) should not already exist.  When creating hard links,  each  TARGET  must  exist.
       Symbolic links can hold arbitrary text; if later resolved, a relative link is interpreted in relation to its parent directory.

       Mandatory arguments to long options are mandatory for short options too.

              make a backup of each existing destination file

       -b     like --backup but does not accept an argument

       -d, -F, --directory
              allow the superuser to attempt to hard link directories (note: will probably fail due to system restrictions, even for the superuser)

       -f, --force
              remove existing destination files

       -i, --interactive
              prompt whether to remove destinations

       -L, --logical
              dereference TARGETs that are symbolic links

       -n, --no-dereference
              treat LINK_NAME as a normal file if it is a symbolic link to a directory

       -P, --physical
              make hard links directly to symbolic links

       -r, --relative
              create symbolic links relative to link location

       -s, --symbolic
              make symbolic links instead of hard links

       -S, --suffix=SUFFIX
              override the usual backup suffix

       -t, --target-directory=DIRECTORY
              specify the DIRECTORY in which to create the links

       -T, --no-target-directory
              treat LINK_NAME as a normal file always

       -v, --verbose
              print name of each linked file

       --help display this help and exit

              output version information and exit

       The  backup  suffix is '~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX.  The version control method may be selected via the --backup option or through the VERSION_CONTROL environment variable.
       Here are the values:

       none, off
              never make backups (even if --backup is given)

       numbered, t
              make numbered backups

       existing, nil
              numbered if numbered backups exist, simple otherwise

       simple, never
              always make simple backups

       Using -s ignores -L and -P.  Otherwise, the last option specified controls behavior when a TARGET is a symbolic link, defaulting to -P.


$ sudo ln -s /usr/local/go/bin/go /usr/local/bin/go

Creating a symlink from one folder to another with different names?

Types of symlinks:

(!) Important: at the time the symlink is being used and resolved, target path (in ls command) is understood as a relative path to the parent directory of the symlink (when it doesn't start with /).

$ pwd
$ ln -s foo/bar.txt bar.txt
$ readlink -f /home/beau/bar.txt


$ cd foo
$ ln -s foo/bar.txt ../bar.txt

How to list all symbolic links in the current directory?

$ find -type l

[man find]: If no paths are given, the current directory is used.
[How to list all symbolic links in a directory]

# save symlinks and their targets (relative to ./path/to/dir/) in csv file which will also be pushed to S3
cd ./path/to/dir/
find . -type l -ls | awk '{print $11,",",$13}' > symlinks.csv

How do I tell if a folder is actually a symlink and how do I fix it if it's broken?

Here are some ways that can be used to verify symlink:

$ stat ./data-vol/content/app/74.0.1365.76 
  File: ./data-vol/content/app/74.0.1365.76 -> data-vol/content/app/win/x86/74.0.1365.76
  Size: 45              Blocks: 0          IO Block: 4096   symbolic link
Device: fd01h/64769d    Inode: 26479224    Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2019-07-12 17:17:09.278071996 +0100
Modify: 2019-07-12 17:17:08.666073171 +0100
Change: 2019-07-12 17:17:08.666073171 +0100
 Birth: -

$ stat -L ./data-vol/content/app/74.0.1365.76 
stat: cannot stat './data-vol/content/app/74.0.1365.76': No such file or directory

$ file -L ./data-vol/content/app/74.0.1365.76 
./data-vol/content/app/74.0.1365.76: cannot open `./data-vol/content/app/74.0.1365.76' (No such file or directory)

$ ls ./data-vol/content/app/74.0.1365.76 

$ ll ./data-vol/content/app/74.0.1365.76 
lrwxrwxrwx 1 root root 45 Jul 12 17:17 ./data-vol/content/app/74.0.1365.76 -> data-vol/content/app/win/x86/74.0.1365.76

How to see full symlink path

$ readlink -f symlinkName

Hard links

How to create hardlink of one file in different directories in linux