Friday 22 March 2019

Creating a minimal Docker image for Go application

Let's create a minimal Docker image which contains an arbitrary Go application. As an example application, we can use "Hello world":

cmd/main.go:

package main
import (
   "fmt"
)
func main() {
   fmt.Println("Hello, world!")
}

To build it and place the executable in bin directory we need to run go build:

$ go build -o bin/hello-world cmd/main.go

To test the executable, let's run it:

$ ./bin/hello-world 
Hello, world!

We want to create a Docker image which, when started, runs this binary. We first have to add Dockerfile - a file which defines how will Docker image be created. Creating a Docker image is like creating a lasagne: we take a base layer and then keep adding new layers on top of each other. Dockerfile specifies what will be the base Docker image (base layer), which application has to reside in it, what is its environment and dependencies that have to be installed and also how will that app be executed (or, what shall be executed when that image is launched).

In our case, we only want to have a single binary in the container and we want it to be launched. For this use case, our Dockerfile can be like this:

go-docker-hello-world/Dockerfile:

FROM scratch
COPY bin/hello-world app/
CMD ["/app/hello-world"]


FROM scratch specifies that empty image (0 bytes!) shall be used as a base layer (or...we can say that there is no base layer).

COPY copies files or directories from source in the host to destination in the container. Working directory on host can be specified via context argument to docker build command. Current directory is used by default and in our case that's go-docker-hello-world. Our binary will be copied here from bin directory on host into the app directory in the container. If destination has to be directory, a slash (/) hast to be added after the destination name. If we didn't add slash, COPY would have copied our binary into the root directory of the container and would have renamed it to app.

CMD contains the name of the executable that has to be run upon container's launch. We need to use an array format (square brackets) as Docker then uses the first argument as the entry point (process that is executed first) and subsequent elements are its arguments. If we used "/app/hello-world" instead of ["/app/hello-world"] Docker would have tried to pass the name of the executable as an argument to /bin/sh but as base image is empty, shell is not present and we'd get an error when running the container:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory": unknown.
Let's create the container:

go-docker-hello-world$ docker build -t helloworld ./
Sending build context to Docker daemon 2.055MB
Step 1/3 : FROM scratch
--->
Step 2/3 : COPY bin/hello-world app/
---> 7c0c34e6ad64
Step 3/3 : CMD ["/app/hello-world"]
---> Running in b3f5695b79c5
Removing intermediate container b3f5695b79c5
---> 171dbd862be1
Successfully built 171dbd862be1
Successfully tagged helloworld:latest


-t applies a tag (name) to the container which can be used later in container managing commands (it is easier to use some descriptive name rather than container ID which is just an array of numbers).

./ specifies the context (the current working directory) for commands in the Dockerfile.

Let's verify that it appears in the list of images:

$ docker images
REPOSITORY   TAG    IMAGE ID       CREATED       SIZE
helloworld latest 171dbd862be1  42 minutes ago   2MB


Let's inspect it to verify that entry point is indeed our application:

$ docker inspect 171dbd862be1
[
    {
        "Id": "sha256:171dbd862be107306bcad870587f8961c00566b946a4d2717ccbf3863492ca2c",
        "RepoTags": [
            "helloworld:latest"
        ],
        "RepoDigests": [],
        "Parent": "sha256:7c0c34e6ad64538ff493910efd6046043b6fa28e78015be6333fcd2e880122d4",
        "Comment": "",
        "Created": "2019-03-22T16:15:30.7049579Z",
        "Container": "b3f5695b79c5add5e86af2ea02b893bd5ed35381221cca1fbf84dd6ea401b69e",
        "ContainerConfig": {
            "Hostname": "b3f5695b79c5",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"/app/hello-world\"]"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:7c0c34e6ad64538ff493910efd6046043b6fa28e78015be6333fcd2e880122d4",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "DockerVersion": "18.09.3",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/app/hello-world"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:7c0c34e6ad64538ff493910efd6046043b6fa28e78015be6333fcd2e880122d4",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 1997502,
        "VirtualSize": 1997502,
        "GraphDriver": {
            "Data": {
                "MergedDir": "/var/lib/docker/overlay2/86d02e448ac1c650f65d6eb30b21eeea2f13f176918ccd6af3440c0d89336b19/merged",
                "UpperDir": "/var/lib/docker/overlay2/86d02e448ac1c650f65d6eb30b21eeea2f13f176918ccd6af3440c0d89336b19/diff",
                "WorkDir": "/var/lib/docker/overlay2/86d02e448ac1c650f65d6eb30b21eeea2f13f176918ccd6af3440c0d89336b19/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:60fdb797c60194a24fa8135f6a1dbe2ed03172037ff5e63eedfc372c2a92964d"
            ]
        },
        "Metadata": {
            "LastTagTime": "2019-03-22T16:15:30.835954727Z"
        }
    }
]

Finally, let's run the container:

$ docker run  helloworld
Hello, world!


When I built once natively, on Ubuntu, a similar, small app from scratch, I got the following error when I ran its container:

ERROR: for my_app  Cannot start service carl: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"my_app\": executable file not found in $PATH": unknown

The problem seemed to be related to app being dynamically linked to some of shared libraries on my dev Linux machine so when binary was copied over to empty (scratch) Docker container, binary could not find them so threw such error. I assume this was the reason for such error as solution was to explicitly disable dynamic linking when building my app:

$ CGO_ENABLED=0 go build cmd/main.go 



Indeed, using CGO_ENABLED flag makes a difference. 

Default is dynamic linking:

$ go build cmd/main.go 
$ file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, not stripped

Static linking has to be explicitly set:

$ CGO_ENABLED=0  go build cmd/main.go 
$ file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped




Useful Linux commands


 

Conventions


# - requires given linux commands to be executed with root privileges either directly as a root user or by use of sudo command

$ - requires given linux commands to be executed as a regular non-privileged user

How to change the prompt in terminal?
 
user@host:~/path/to/current-dir$ export PS1='$ '
$

Useful Command Line Utility Tools


To print last N commands form Terminal:

$ history N

Using the history command - Media Temple

The output is command history in form:

 2000  sudo apt install ipython ipython-notebook
 2001  sudo apt list --installed | grep python3-pip
 2002  sudo apt list --installed | grep python3-dev
...

To copy only commands (without command number) keep CTRL+ALT pressed and select desired text columns and rows with left mouse click button.

To shorten some long command: How to Create and Use Alias Command in Linux

To list current aliases:

$ alias
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias l='ls -CF'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'

Operating System

How to find what's the version of the installed OS:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic


User management


Linux User Management | My Public Notepad

File and Directory Ownership and Permissions



Working with directories


To list the content of the directory:

ls

To list directories in the current directory, names only, and each name in its own line:

$ ls -1d *

-1     list one file per line
-d     list directories themselves, not their contents
*        list all items in the current directory (they might be filtered by other flags, like -d)

To list all files (including hidden) in the current directory:

ls -a

To list files in some specific directory use:

$ ls target_directory

Example:
$ ls /usr/local/go/

To list directories and files in form of a tree install tree package:

$ sudo apt install tree

...and use it as e.g.:

$ tree -I *node*

-I = ignores directories that match given pattern

To diff two directories use:

$ diff -r dir1 dir2 

This shows which files are only in dir1 and those only in dir2 and also the changes of the files present in both directories if any. If any file does not end with a newline character, this will be reported as well:

$ diff -r dir1 dir2
diff -r dir1/test.txt dir2/test.txt
1c1
< First line in dir1/test.txt
\ No newline at end of file
---
> First line in dir2/test.txt
\ No newline at end of file
Binary files /dir1/test.bin and /dir2/test.bin differ
Only in /dir1/subdir1/: subsubdir1
Only in /dir2/subdir1/: file.zip
---

What does 1c1 in diff tool mean?
1c1 indicates that line 1 in the first file was c hanged somehow to produce line 1 in the second file.
They probably differ in whitespace (perhaps trailing spaces, or Unix versus Windows line endings?).

man diff
Difference between two directories in Linux [closed]

To find the location of some directory starting from the root directory (/) use:

$ find / -type d -name dir_name

What does the suffix .d (in directory name) mean in Linux? - Server Fault

Time and Date


To convert Unix time to human readable, use:

$ date -d @1583944041
Wed 11 Mar 16:27:21 GMT 2020


Working with files in Linux | My Public Notepad

Package management


apt (Advanced Packaging Tool) - It is not a command itself but a package which contains set of tools which manage installation and removal of other packages.

apt-get

apt-get update - downloads the package lists from the repositories and "updates" them to get information on the newest versions of packages and their dependencies. It will do this for all repositories and PPAs.

http://askubuntu.com/questions/222348/what-does-sudo-apt-get-update-do

apt-cche


add-apt-repository - adds a repository to the list of repositories


To apply latest security updates on Ubuntu:

sudo apt-get update

sudo apt-get -y upgrade


Difference Between apt and apt-get Explained

Should I use apt or apt-get?

You might be thinking if you should use apt or apt-get. And as a regular Linux user, my answer is to go with apt.

apt is the command that is being recommended by the Linux distributions. It provides the necessary option to manage the packages. Most important of all, it is easier to use with its fewer but easy to remember options.

I see no reason to stick with apt-get unless you are going to do specific operations that utilize more features of apt-get.



How to add repository to Apt sources?


For application named app:

$ sudo install -m 0755 -d /etc/apt/keyrings

$ sudo curl -fsSL https://download.app.com/linux/debian/gpg -o /etc/apt/keyrings/app.asc

$ sudo chmod a+r /etc/apt/keyrings/app.asc

# Add the repository to Apt sources:
$ echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/app.asc] https://download.app.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/app.list > /dev/null


debian - what is the function of /etc/apt/sources.list.d? - Stack Overflow


-----------------------------------------------------------------------------------------------------------------------


To download a file into some specific directory:

cd /dest_dir
wget https://example.com/archive_file.tar.gz


To install a package/software in Ubuntu, it is usually enough to copy it to /usr/local directory.
To move dir1 to some other location e.g. /usr/local use:

mv new_app /usr/local


Linux Directories

Unix Filesystem Hierarchy | My Public Notepad

Installing Software on Linux | My Public Notepad


Working with Archive files


Tar Format


To unpack the .tar.gz in the current directory use:

$ tar -zxvf archive_file.tar.gz

-x = extract
-f (--file) = use archive file; this flag has to be the last in the list of flags and to be followed by the archive file name
-v (--verbose) = verbose output
-z (--gzip, --gunzip, --ungzip)  = filter the archive through gzip

To unpack only the specific directory from the archive use:

$ tar -zxvf archive_file.tar.gz dir_name

To unpack archive to the specific directory:

$ tar -zxvf archive_file.tar -C path/to/dest_dir 

-C (--directory) stands for "Change to directory"

Example:

$ tar -xzf go1.12.1.linux-amd64.tar -C /usr/local

$ sudo tar -xzvf Postman-linux-x64-7.5.0.tar.gz -C /opt

Why compressed directories cannot be extracted in /opt?

To unpack multiple rar files first install unrar:

$ sudo apt-get install rar unrar

then go to the directory where all x.party.rar files are and execute:

$ unrar x -e file.part1.rar


ZIP format


Zip one subdirectory ( e.g. ./modules/foo) while traversing it recursively (-r), but exclude (-x) some its subdirectories and files (on MacOS we need to put paths with wildcards under quotes):

% zip -r module-foo.zip \
./modules/foo \
-x "./modules/foo/.terraform/*" \
-x ./modules/foo/.terraform.lock.hcl

Unzipping into the specified directory:

$ unzip bookmarks.zip -d bookmarks-dev-413

$ unzip bazel-1.2.1-dist.zip -d bazel bazel-1.2.1-dist

Unzip files in particular directory or folder under Linux or UNIX - nixCraft


Networking

Introduction to Linux Networking | My Public Notepad

Hardware management


To verify if you're running a 64-bit system use GNU Core Utility: uname.

Swap Memory Management


To check virtual memory statistics (the latest reading) use:

$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0 138816 9157820 1739872 11514740    0    1  1559   366  102   64 26  7 66  0  0

or, to see readings every 1 second:

$ vmstat 1

To disable and then reenable swap:

$ sudo swapoff -a
$ sudo swapon -a


performance - How to empty swap if there is free RAM? - Ask Ubuntu

Displays/Monitors Management


$ xrandr --listmonitors
Monitors: 2
 0: +*HDMI-1 2560/677x1440/381+0+0  HDMI-1
 1: +eDP-1 1920/344x1080/193+112+1440  eDP-1

To see details for the current display:

$ xrandr --current
Screen 0: minimum 320 x 200, current 1920 x 1080, maximum 8192 x 8192
eDP-1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 344mm x 193mm
   1920x1080     60.03*+  60.01    59.97    59.96    59.93    48.02  
   1680x1050     59.95    59.88  
   1600x1024     60.17  
   1400x1050     59.98  
   1600x900      59.99    59.94    59.95    59.82  
   ...
   360x202       59.51    59.13  
   320x180       59.84    59.32  
HDMI-1 disconnected (normal left inverted right x axis y axis)
DP-1 disconnected (normal left inverted right x axis y axis)
HDMI-2 disconnected (normal left inverted right x axis y axis)
DP-2 disconnected (normal left inverted right x axis y axis)
HDMI-3 disconnected (normal left inverted right x axis y axis)

To list all video devices (e.g. webcams):

$ ls /dev/video*

To check the number of processors:

$ grep -c ^processor /proc/cpuinfo | sed 's/^0$/1/'
4

To see details for all processors:

$ cat /proc/cpuinfo

To list CPU architecture information, use lscpu. E.g.

$  lscpu | grep "Byte Order"
Byte Order:          Little Endian


SSH


How to generate SSH key pair on Ubuntu
How to test SSH key password on Ubuntu

---

To get base64 encoding of a string:

$ echo -n my_string | base64

or

$ printf my_string | base64

---
TBC...


Getting Information about Hardware


How to get the GPU info?

lshw = list hardware
-C = class

$ sudo lshw -C display
  *-display UNCLAIMED       
       description: 3D controller
       product: GM107GLM [Quadro M620 Mobile]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a2
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress bus_master cap_list
       configuration: latency=0
       resources: memory:ee000000-eeffffff memory:d0000000-dfffffff memory:e0000000-e1ffffff ioport:e000(size=128) memory:ef000000-ef07ffff
  *-display
       description: VGA compatible controller
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       version: 04
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
       configuration: driver=i915 latency=0

       resources: irq:130 memory:ed000000-edffffff memory:c0000000-cfffffff ioport:f000(size=64) memory:c0000-dffff


NVidia and Intel in same Laptop: which card is used?

$ lspci -k | grep -EA2 'VGA|3D'  
00:02.0 VGA compatible controller: Intel Corporation Device 591b (rev 04)
Subsystem: Dell Device 07a9
Kernel driver in use: i915
--
01:00.0 3D controller: NVIDIA Corporation GM107GLM [Quadro M620 Mobile] (rev a2)
Subsystem: Dell GM107GLM [Quadro M620 Mobile]
Kernel modules: nvidiafb, nouveau


GNU Bash Shell Commands

 
To determine the default shell for the current user:

$ echo "$SHELL"
/bin/bash

or print the name of the current terminal process:

$ echo $0
bash



Robert Muth: Better Bash Scripting in 15 Minutes

Bash script should start with shebang e.g.

#!/bin/bash


---
Escaping and preserving special characters in bash

Bash wildcards: ? (question mark) and * (asterisk) | LinuxG.net
What is the meaning of a question mark in bash variable parameter expansion as in ${var?}? - Stack Overflow
shell - Expansion of variable inside single quotes in a command in Bash - Stack Overflow
linux command line find escape question mark - Server Fault

---

$_


What does 'cd $_' mean?

$_ expands to the last argument to the previous simple command* or to previous command if it had no arguments. Typical use:

mkdir dirA && cd $_


---

To see all Bash commands, execute:

$ help

---

set


$ help set 
set: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
    Set or unset values of shell options and positional parameters.
    
    Change the value of shell attributes and positional parameters, or
    display the names and values of shell variables.
    
    Options:
      -a  Mark variables which are modified or created for export.
      -b  Notify of job termination immediately.
      -e  Exit immediately if a command exits with a non-zero status.
      -f  Disable file name generation (globbing).
      -h  Remember the location of commands as they are looked up.
      -k  All assignment arguments are placed in the environment for a
          command, not just those that precede the command name.
      -m  Job control is enabled.
      -n  Read commands but do not execute them.
      -o option-name
          Set the variable corresponding to option-name:
              allexport    same as -a
              braceexpand  same as -B
              emacs        use an emacs-style line editing interface
              errexit      same as -e
              errtrace     same as -E
              functrace    same as -T
              hashall      same as -h
              histexpand   same as -H
              history      enable command history
              ignoreeof    the shell will not exit upon reading EOF
              interactive-comments
                           allow comments to appear in interactive commands
              keyword      same as -k
              monitor      same as -m
              noclobber    same as -C
              noexec       same as -n
              noglob       same as -f
              nolog        currently accepted but ignored
              notify       same as -b
              nounset      same as -u
              onecmd       same as -t
              physical     same as -P
              pipefail     the return value of a pipeline is the status of
                           the last command to exit with a non-zero status,
                           or zero if no command exited with a non-zero status
              posix        change the behaviour of bash where the default
                           operation differs from the Posix standard to
                           match the standard
              privileged   same as -p
              verbose      same as -v
              vi           use a vi-style line editing interface
              xtrace       same as -x
      -p  Turned on whenever the real and effective user ids do not match.
          Disables processing of the $ENV file and importing of shell
          functions.  Turning this option off causes the effective uid and
          gid to be set to the real uid and gid.
      -t  Exit after reading and executing one command.
      -u  Treat unset variables as an error when substituting.
      -v  Print shell input lines as they are read.
      -x  Print commands and their arguments as they are executed.
      -B  the shell will perform brace expansion
      -C  If set, disallow existing regular files to be overwritten
          by redirection of output.
      -E  If set, the ERR trap is inherited by shell functions.
      -H  Enable ! style history substitution.  This flag is on
          by default when the shell is interactive.
      -P  If set, do not resolve symbolic links when executing commands
          such as cd which change the current directory.
      -T  If set, the DEBUG and RETURN traps are inherited by shell functions.
      --  Assign any remaining arguments to the positional parameters.
          If there are no remaining arguments, the positional parameters
          are unset.
      -   Assign any remaining arguments to the positional parameters.
          The -x and -v options are turned off.
    
    Using + rather than - causes these flags to be turned off.  The
    flags can also be used upon invocation of the shell.  The current
    set of flags may be found in $-.  The remaining n ARGs are positional
    parameters and are assigned, in order, to $1, $2, .. $n.  If no
    ARGs are given, all shell variables are printed.
    
    Exit Status:
    Returns success unless an invalid option is given.



set -x 
  • enables a mode of the shell where all executed commands are printed to the terminal.
  • typically used for debugging
---

What is the difference between [] and [[]]?

Bash Brackets Quick Reference

Test if a command outputs an empty string

if [[ $(ls -A) ]]; then
    echo "there are files"
else
    echo "no files found"
fi

---
Create a directory if this does not exist:

DATA_DIR=data-vol
if [ -d ${DATA_DIR} ]; then
   echo Directory ${DATA_DIR} exists.
else 
   echo Directory ${DATA_DIR} does not exist.
   echo Creating directory ${DATA_DIR} ...
   mkdir ${DATA_DIR}
fi

or, as one-liner:

[ -d path/to/mydir ] || mkdir -p path/to/mydir

(-p option instructs mkdir to create all intermediate parent directories to mydir)

---

Semicolon in conditional structures

The semicolon is needed only when the end of line is missing:

if [ "a" == "a" ] ; then echo "true" ; fi

Without semicolons, you get Syntax error.

---

How to capture exit code of the application most recently executed in Terminal?

$ echo $?

It can also be used in a bash script, e.g.:

ginkgo -r

if [[ $? != 0 ]]; then
   echo "Unit tests failed. Terminating build process..."
   exit 1
fi

---

Variables


$ dependencies=(build-essential cmake pkg-config libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libavresample-dev python3-dev libtbb2 libtbb-dev libtiff-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev libgtk-3-dev libcanberra-gtk3-module libatlas-base-dev gfortran wget unzip)

We can print the array:

$ echo ${dependencies[@]}

build-essential cmake pkg-config libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libavresample-dev python3-dev libtbb2 libtbb-dev libtiff-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev libgtk-3-dev libcanberra-gtk3-module libatlas-base-dev gfortran wget unzip

We can print elements with specific indexes in the array:

$ echo ${dependencies[0]}
$ echo ${dependencies[1]}


---

Thursday 21 March 2019

Golang - Miscellaneous Topics & Useful Links

Go CLI commands

go get

-u is used frequently as it makes get not just to check out missing packages but also to look for updates to existing packages 

Coding Style


https://github.com/golang/go/wiki/CodeReviewComments
https://github.com/Unknwon/go-code-convention/blob/master/en-US/naming_rules.md
https://rakyll.org/style-packages/
Golang - Code organisation with structs, variables and interfaces
Special Packages and Directories in Go

Go naming conventions for const
The standard library uses camel-case, so I advise you do that as well. The first letter is uppercase or lowercase depending on whether you want to export the constant.

Inner/package functions should not be calling panic on error but should return an error (together with a value) and thus allow caller to decide how they want to handle errors and if they want to panic or not.

Function should detect an error and return as soon as possible. The last statement in function body should be returning a valid value and nil as an error.

How to organize Go project?


Go Project Layout
golang-standards/project-layout

How to order packages in import?


Keep them in alphabetical order, with a blank line between:

  • the standard library
  • other libraries 
  • project-specific imports

gofmt: organize imports like eclipse does

Initialization


init functions in Go



Arrays


12 Practical Array Examples in GoLang Go Programming Language


File System


Check if file or directory exists in Golang
go/fileutil/fileutil.go - buildbot - Git at Google


Labels


Labels in Go - golangspec - Medium

Error Handling


Error handling and Go
Custom Errors
Return nil or custom error in Go
When should I use panic vs log.fatalln() in golang?
Part 32: Panic and Recover
Go by Example: Panic

    if err != nil {
        panic(err)
    }

Constants


https://blog.golang.org/constants
Constants in Go established during init

Types


integers



Strings


Printing " (double quote) in GoLang
Use:
  • escape character: fmt.Println("\"") or
  • raw strings: fmt.Println(`"`)
Go by Example: Number Parsing

Enums


Ultimate Visual Guide to Go Enums
Command stringer
Simple enumeration in Golang using Stringer
Assign a string representation to an enum/const (forum thread)
[Golang] Enum and String Representation
Enums in Go

Structs


Go by Example: Structs
Anonymous fields in structs - like object composition
The empty struct

Return pointer to local struct
Go performs pointer escape analysis. If the pointer escapes the local function, the object is allocated on the heap. If it doesn't escape the local function, the compiler is free to allocate it on the stack.

Note that, unlike in C, it's perfectly OK to return the address of a local variable; the storage associated with the variable survives after the function returns.[1]

When possible, the Go compilers will allocate variables that are local to a function in that function’s stack frame. However, if the compiler cannot prove that the variable is not referenced after the function returns, then the compiler must allocate the variable on the garbage-collected heap to avoid dangling pointer errors.[1]

Can you “pin” an object in memory with Go?
An object on which you keep a reference won't move. There is no handle or indirection, and the address you get is permanent. Go's GC is not a compacting GC. It will never move live objects around.

Local variable can escape the local function scope in two cases:
  • variable’s address is returned
  • its address is assigned to a variable in an outer scope


When to return a local instance of struct and when its address?


There are two usual reasons to return a pointer:

  • if we want methods of the struct to generally modify the struct in place (versus needing to create or copy to a new struct when we want to make modifications)
  • if the struct is rather large, a pointer is preferred for efficiency


When to return a pointer?
Constructor returning a pointer

Immutability


Does Go have immutable data structures?

Arrays


GO explicit array initialization
Keyed items in golang array initialization

Slices


https://golang.org/ref/spec#Slices
https://gobyexample.com/slices
https://blog.golang.org/go-slices-usage-and-internals

Channels


Unbuffered 


Example in SO quuestion

Buffered 


A Tour of Go - Channels 
BUFFERED CHANNELS IN GO: TIPS & TRICKS
Buffered Channels In Go — What Are They Good For?
Buffered Channels
Go by Example: Channel Buffering
Different ways to pass channels as arguments in function in go (golang)
https://www.reddit.com/r/golang/comments/7ep1un/golang_channel_is_really_fifo/

Interfaces


Interfaces in Go


Control Flow


for / range


Is there a way to iterate over a range of integers in Golang?
for range [5]int{} {...}

Switch

https://gobyexample.com/switch
https://tour.golang.org/flowcontrol/9
https://github.com/golang/go/wiki/Switch
https://yourbasic.org/golang/switch-statement/


defer

A Tour of Go - defer

panic


When should I use panic vs log.fatalln() in golang?


Functions


Go function type, reusable function signatures
type myFunctionType = func(a, b string) string

Arguments


Verifying arguments



Methods


Methods on structs
You will be able to add methods for a type only if the type is defined in the same package.
Don't Get Bitten by Pointer vs Non-Pointer Method Receivers in Golang
Anatomy of methods in Go

Lambdas inside a function

Why doesn't Go allow nested function declarations (functions inside functions)?

OOP - Encapsulation & import


Exported identifiers in Go
What does an underscore in front of an import statement mean in Golang?

dot-import

When you import package prefixed with dot (e.g.: import( . "fmt")), it would be imported in current namespace, so you can omit "fmt" prefix before calling methods. BUT: Don't ever do that; the only time it's useful is in very rare cases involving test files, everywhere else it's a really bad idea.

Concurrency


Go Concurrency Patterns: Pipelines and cancellation
Go Language Patterns - Semaphores
Go’s Extended Concurrency: Semaphores (Part 1)
Essential Go - Limiting concurrency with a semaphore


Templates



To get curly brackets like

data-my-attribute="{true}"

or

data-my-attribute="{false}"

in HTML, we need to use the following format in template:

data-my-attribute="{ {{- toBoolString .DataMyAttributeValue -}} }"

Unit & Integration Testing



First, something not related specifically to Go but to general software engineering:
TDD and BDD Differences"BDD is just TDD with different words"
Why do some software development companies prefer to use TDD instead of BDD?
How can I do test setup using the testing package in Go
Nice example how to inject unit test checker in an internal method: compare Do() and do() here.
Verbose output prints the names of unit test functions: go test -v
Exploring the landscape of Go testing frameworks
Go With Go(Lang): Features and Testing Frameworks

BDD:
  • ginkgo [Ginkgo] - very active development
  • goconvey
  • goblin - seems that its development has stopped; the last commit in repo was 5 months ago

5 simple tips and tricks for writing unit tests in #golang
Integration Tests in Go
Integration Test With Database in Golang
Separating unit tests and integration tests in Go
Learn Go by writing tests: Structs, methods, interfaces & table driven tests
Testing with golden files in Go
BDD Testing with Ginkgo and GoMock
Clean Go Testing with Ginkgo
done channel 
Using Ginkgo with Gomock
golang/mock
Gomega
Filesystem impact testing
Testing Techniques - Google I/O 2014

Go & DataBases


Scanners and Valuers with Go
How to write unit tests for database calls

Design Patterns in Go


The factory method pattern in Go
Factory patterns in Go (Golang)
A Class Factory in Golang (Google Go)

Go in VSCode

Debugging Go code using VS Code

Misc



Things I Wish Someone Had Told Me About Golang

Random


Package rand
Generating random numbers and strings in Go
https://flaviocopes.com/go-random/
Go by Example: Random Numbers
package uuid

Environment Variables

Pass environment variables from docker to my GoLang.

Configuration Files


Best practices for Configuration file in your code
Manage config in Golang to get variables from file and env variables

Useful packages


For unit tests: https://github.com/Pallinder/go-randomdata

Go Modules


Inspired by: Getting started with Go modules

To initialize Go modules in to your application you can run:

$ go mod init /path/to/directory

Example shows that we need to include the path:

root@bc7429fce4a8:~/TestApp# go mod init 
go: cannot determine module path for source directory /root/TestApp (outside GOPATH, no import comments)

root@bc7429fce4a8:~/TestApp# ls
main.go

root@bc7429fce4a8:~/TestApp# go mod init /root/TestApp
go: creating new go.mod: module /root/TestApp

root@bc7429fce4a8:~/TestApp# ls
go.mod main.go

root@bc7429fce4a8:~/TestApp# cat go.mod

module /root/TestApp

# go run main.go 
go: finding github.com/Pallinder/go-randomdata v1.1.0
go: downloading github.com/Pallinder/go-randomdata v1.1.0
Running the TestApp
Backstump

root@bc7429fce4a8:~/TestApp# go run main.go 
Running the TestApp
Frightbrown

# cat go.mod 
module /root/TestApp

require github.com/Pallinder/go-randomdata v1.1.0 // indirect

root@bc7429fce4a8:~/TestApp# cat go.sum
github.com/Pallinder/go-randomdata v1.1.0 h1:gUubB1IEUliFmzjqjhf+bgkg1o6uoFIkRsP3VrhEcx8=
github.com/Pallinder/go-randomdata v1.1.0/go.mod h1:yHmJgulpD2Nfrm0cR9tI/+oAgRqCQQixsA8HyRZfV9Y=

If we want to change version of the 3rd party package (go-randomdata in our case) we just need to manually edit the go.mod and set there desired version. go run (or go build) would then apply that version and hashes in go.sum would also change.

TBC...

Go in Use


JFrog CLI
"JFrog CLI runs on any modern OS that fully supports the Go programming language."