Friday, 22 March 2019

Creating a minimal Docker image for Go application

Let's create a minimal Docker image which contains an arbitrary Go application. As an example application, we can use "Hello world":

cmd/main.go:

package main
import (
   "fmt"
)
func main() {
   fmt.Println("Hello, world!")
}

To build it and place the executable in bin directory we need to run go build:

$ go build -o bin/hello-world cmd/main.go

To test the executable, let's run it:

$ ./bin/hello-world 
Hello, world!

We want to create a Docker image which, when started, runs this binary. We first have to add Dockerfile - a file which defines how will Docker image be created. Creating a Docker image is like creating a lasagne: we take a base layer and then keep adding new layers on top of each other. Dockerfile specifies what will be the base Docker image (base layer), which application has to reside in it, what is its environment and dependencies that have to be installed and also how will that app be executed (or, what shall be executed when that image is launched).

In our case, we only want to have a single binary in the container and we want it to be launched. For this use case, our Dockerfile can be like this:

go-docker-hello-world/Dockerfile:

FROM scratch
COPY bin/hello-world app/
CMD ["/app/hello-world"]


FROM scratch specifies that empty image (0 bytes!) shall be used as a base layer (or...we can say that there is no base layer).

COPY copies files or directories from source in the host to destination in the container. Working directory on host can be specified via context argument to docker build command. Current directory is used by default and in our case that's go-docker-hello-world. Our binary will be copied here from bin directory on host into the app directory in the container. If destination has to be directory, a slash (/) hast to be added after the destination name. If we didn't add slash, COPY would have copied our binary into the root directory of the container and would have renamed it to app.

CMD contains the name of the executable that has to be run upon container's launch. We need to use an array format (square brackets) as Docker then uses the first argument as the entry point (process that is executed first) and subsequent elements are its arguments. If we used "/app/hello-world" instead of ["/app/hello-world"] Docker would have tried to pass the name of the executable as an argument to /bin/sh but as base image is empty, shell is not present and we'd get an error when running the container:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory": unknown.
Let's create the container:

go-docker-hello-world$ docker build -t helloworld ./
Sending build context to Docker daemon 2.055MB
Step 1/3 : FROM scratch
--->
Step 2/3 : COPY bin/hello-world app/
---> 7c0c34e6ad64
Step 3/3 : CMD ["/app/hello-world"]
---> Running in b3f5695b79c5
Removing intermediate container b3f5695b79c5
---> 171dbd862be1
Successfully built 171dbd862be1
Successfully tagged helloworld:latest


-t applies a tag (name) to the container which can be used later in container managing commands (it is easier to use some descriptive name rather than container ID which is just an array of numbers).

./ specifies the context (the current working directory) for commands in the Dockerfile.

Let's verify that it appears in the list of images:

$ docker images
REPOSITORY   TAG    IMAGE ID       CREATED       SIZE
helloworld latest 171dbd862be1  42 minutes ago   2MB


Let's inspect it to verify that entry point is indeed our application:

$ docker inspect 171dbd862be1
[
    {
        "Id": "sha256:171dbd862be107306bcad870587f8961c00566b946a4d2717ccbf3863492ca2c",
        "RepoTags": [
            "helloworld:latest"
        ],
        "RepoDigests": [],
        "Parent": "sha256:7c0c34e6ad64538ff493910efd6046043b6fa28e78015be6333fcd2e880122d4",
        "Comment": "",
        "Created": "2019-03-22T16:15:30.7049579Z",
        "Container": "b3f5695b79c5add5e86af2ea02b893bd5ed35381221cca1fbf84dd6ea401b69e",
        "ContainerConfig": {
            "Hostname": "b3f5695b79c5",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"/app/hello-world\"]"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:7c0c34e6ad64538ff493910efd6046043b6fa28e78015be6333fcd2e880122d4",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "DockerVersion": "18.09.3",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/app/hello-world"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:7c0c34e6ad64538ff493910efd6046043b6fa28e78015be6333fcd2e880122d4",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 1997502,
        "VirtualSize": 1997502,
        "GraphDriver": {
            "Data": {
                "MergedDir": "/var/lib/docker/overlay2/86d02e448ac1c650f65d6eb30b21eeea2f13f176918ccd6af3440c0d89336b19/merged",
                "UpperDir": "/var/lib/docker/overlay2/86d02e448ac1c650f65d6eb30b21eeea2f13f176918ccd6af3440c0d89336b19/diff",
                "WorkDir": "/var/lib/docker/overlay2/86d02e448ac1c650f65d6eb30b21eeea2f13f176918ccd6af3440c0d89336b19/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:60fdb797c60194a24fa8135f6a1dbe2ed03172037ff5e63eedfc372c2a92964d"
            ]
        },
        "Metadata": {
            "LastTagTime": "2019-03-22T16:15:30.835954727Z"
        }
    }
]

Finally, let's run the container:

$ docker run  helloworld
Hello, world!


When I built once natively, on Ubuntu, a similar, small app from scratch, I got the following error when I ran its container:

ERROR: for my_app  Cannot start service carl: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"my_app\": executable file not found in $PATH": unknown

The problem seemed to be related to app being dynamically linked to some of shared libraries on my dev Linux machine so when binary was copied over to empty (scratch) Docker container, binary could not find them so threw such error. I assume this was the reason for such error as solution was to explicitly disable dynamic linking when building my app:

$ CGO_ENABLED=0 go build cmd/main.go 



Indeed, using CGO_ENABLED flag makes a difference. 

Default is dynamic linking:

$ go build cmd/main.go 
$ file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, not stripped

Static linking has to be explicitly set:

$ CGO_ENABLED=0  go build cmd/main.go 
$ file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped




Useful Linux commands

User management


To request security privileges of superuser (root):

$ sudo 


/usr/local/etc/sudoers

To log in and run the current shell as root use:

user@computer:~$ sudo -i
root@computer:~# whoami
root
root@computer:~# exit

logout
user@computer:~$

.profile file 


There is one global profile file (executed when anyone logs in):

/etc/profile

There are three user-specific bash profile files (executed when current/specific user logs in):

~/.profile
~/.bash_profile
~/.bashrc

If ~/.profile doesn't exist, just create it.

This is the comment at the beginning of ~/.profile:

# ~/.profile: executed by the command interpreter for login shells.
# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login exists.
# see /usr/share/doc/bash/examples/startup-files for examples.
# the files are located in the bash-doc package.
# the default umask is set in /etc/profile; for setting the umask
# for ssh logins, install and configure the libpam-umask package.
#umask 022

To add en environment variable during the session of a particular user (and also make them available in terminal) append the desired var name and its value to ~/.profile file. Example:

export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

We'd need to restart the terminal in order to get these changes visible but to make terminal fetch them in the current session, we can update the current shell session with:

source ~/.profile


To add a new or modify existing environment variable permanently (for non-root user) we need to change ~/.bashrc:

Example of ~/.bashrc snippet:

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
...
export GOPATH=$HOME/dev/go
export PATH=$PATH:$GOPATH/bin


To do the same for root user, open /etc/environment:

$ sudo gedit /etc/environment

...and add desired path:

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
GOPATH="/home/bojan/dev/go"

How to logout current user from the Terminal?


$ gnome-session-quit

or

$ gnome-session-quit --no-prompt

to suppress showing Logout confirmation dialog.

File & Directory Ownership

To see permissions (r= read, w=write, x=execute) and ownership (user:group) over some file or directory use:

$ ls -la

To change ownership (e.g. from root to current user) of directory and all its content use chown (change owner):

$ sudo chown -R $USER directory

or

$ sudo chown -R username:group directory


Change folder permissions and ownership


Permissions


Chmod permissions (flags) explained
os.MkDir and os.MkDirAll permission value?

To allow only file user to read and write (but not to execute):

$ chmod 600 filename

Working with directories


To list the content of the directory:

ls

To list all files (including hidden) in the current directory:

ls -a

To list files in some specific directory use:

$ ls target_directory

Example:
$ ls /usr/local/go/

To list directories and files in form of a tree install tree package:

$ sudo apt install tree

...and use it as e.g.:

$ tree -I *node*

-I = ignores directories that match given pattern

To diff two directories use:

$ diff -r dir1 dir2 

This shows which files are only in dir1 and those only in dir2 and also the changes of the files present in both directories if any. If any file does not end with a newline character, this will be reported as well:

$ diff -r dir1 dir2
diff -r dir1/test.txt dir2/test.txt
1c1
< First line in dir1/test.txt
\ No newline at end of file
---
> First line in dir2/test.txt
\ No newline at end of file
Binary files /dir1/test.bin and /dir2/test.bin differ
Only in /dir1/subdir1/: subsubdir1
Only in /dir2/subdir1/: file.zip
---

What does 1c1 in diff tool mean?
1c1 indicates that line 1 in the first file was c hanged somehow to produce line 1 in the second file.
They probably differ in whitespace (perhaps trailing spaces, or Unix versus Windows line endings?).


Difference between two directories in Linux [closed]




Working with files

Creating a file


To create a file use touch:

$ touch filename

Ending file with new line character


[No newline at end of file]

It is a good style to always put the newline as a last character if it is allowed by the file format.

Unix historically had a convention of all human-readable text files ending in a newline. Reasons:
Practically, because many Unix tools require or expect it for proper display.
Philosophically, because each line in a text file terminates with an "end-of-line" character--the last line shouldn't be any exception.

Getting the information about a file


To get the number of lines (well, newline characters) in the file:

$ wc -l myfile.txt
23 myfile.txt

(This is why it's important to follow the convention and end each line with newline character.)

To see the last couple of lines in the file use command tail:

$ tail myfile

To find various hash sums of a file:

$ md5sum file_name
$ sha1sum file_name
$ sha256sum file_name

Copying files


cp - copy


File viewing and editing

To simply view the content of some file, use cat:

$ cat filename

To edit some file, you can use vi editor. Example:

$ vi ~/.profile 

gedit can also be used as graphic editor:

sudo gedit ~/.profile


Searching for text across files


How do I find all files containing specific text on Linux?

man grep

$ grep -rnw '/path/to/somewhere/' -e 'pattern'

-r or -R = recursive,
-n = line number
-w = match the whole word.
-l (lower-case L) = just give the file name of matching files
--include=\*.{c,h} =  search through those files which have .c or .h extensions
--exclude=*.o = exclude searching all the files ending with .o extension
--exclude-dir={dir1,dir2,*.dst} = exclude a particular directory(ies)
-e PATTERN = string pattern to be searched
-i = ignore the case

Example:

$ grep -r /var/lib/go/src/ -e "CodeDecode"
/var/lib/go/src/encoding/json/bench_test.go:func BenchmarkCodeDecoder(b *testing.B) {


Example:

$ find ./go/src/pkg -type f -name "*.go" | xargs egrep '^type.*(er|or) interface {'

xargs manual - xargs builds and executes command lines from standard input
egrep manual - egrep prints lines matching a pattern

Comparing Files


How to ignore line endings when comparing files?

$ diff --strip-trailing-cr file1 file2


How to detect file ends in newline?


Working with executable files


Running a command prefixed by the time command will tell us how long our code took to execute.

$ time myapp
real 0m13.761s
user 0m0.262s
sys 0m0.039s


Symbolic links


What are they?

  • files that contain a reference to another file or directory on the same system
  • like shortcuts on Windows OS


What is their purpose?

  • to avoid copying the same binary (usually a library) at multiple locations but simply creating a symlink where file is required to be
  • various clients might require the same file but with name in different format so instead of having multiple copies of the same file but with different names we'd have multiple symlink, each with the name that satisfies requirements of each service

How do they work?

  • opening/running the symlink would open/run the target file
  • editing the content of the symlink edits the content of the target file
  • if target file is deleted symlink becomes a dangling symlink
  • if symlink is deleted, target file remains unaffected
  • it is possible to create symlink that refers to another symlink [How can I create a symlink which points to another symlink?]


How to create them?

How to: Linux / UNIX create soft link with ln command

Use ln command:

NAME
       ln - make links between files

SYNOPSIS
       ln [OPTION]... [-T] TARGET LINK_NAME   (1st form)
       ln [OPTION]... TARGET                  (2nd form)
       ln [OPTION]... TARGET... DIRECTORY     (3rd form)
       ln [OPTION]... -t DIRECTORY TARGET...  (4th form)

DESCRIPTION
       In the 1st form, create a link to TARGET with the name LINK_NAME.  In the 2nd form, create a link to TARGET in the current directory.  In the 3rd and 4th forms, create links to each TARGET in DIRECTORY.  Create hard links by default, symbolic links with --symbolic.  By default, each destination (name of new link) should not already exist.  When creating hard links,  each  TARGET  must  exist.
       Symbolic links can hold arbitrary text; if later resolved, a relative link is interpreted in relation to its parent directory.

       Mandatory arguments to long options are mandatory for short options too.

       --backup[=CONTROL]
              make a backup of each existing destination file

       -b     like --backup but does not accept an argument

       -d, -F, --directory
              allow the superuser to attempt to hard link directories (note: will probably fail due to system restrictions, even for the superuser)

       -f, --force
              remove existing destination files

       -i, --interactive
              prompt whether to remove destinations

       -L, --logical
              dereference TARGETs that are symbolic links

       -n, --no-dereference
              treat LINK_NAME as a normal file if it is a symbolic link to a directory

       -P, --physical
              make hard links directly to symbolic links

       -r, --relative
              create symbolic links relative to link location

       -s, --symbolic
              make symbolic links instead of hard links

       -S, --suffix=SUFFIX
              override the usual backup suffix

       -t, --target-directory=DIRECTORY
              specify the DIRECTORY in which to create the links

       -T, --no-target-directory
              treat LINK_NAME as a normal file always

       -v, --verbose
              print name of each linked file

       --help display this help and exit

       --version
              output version information and exit

       The  backup  suffix is '~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX.  The version control method may be selected via the --backup option or through the VERSION_CONTROL environment variable.
       Here are the values:

       none, off
              never make backups (even if --backup is given)

       numbered, t
              make numbered backups

       existing, nil
              numbered if numbered backups exist, simple otherwise

       simple, never
              always make simple backups

       Using -s ignores -L and -P.  Otherwise, the last option specified controls behavior when a TARGET is a symbolic link, defaulting to -P.


Example:

$ sudo ln -s /usr/local/go/bin/go /usr/local/bin/go

Creating a symlink from one folder to another with different names?


Types of symlinks:


$ pwd
/home/beau
$ ln -s foo/bar.txt bar.txt
$ readlink -f /home/beau/bar.txt
/home/beau/foo/bar.txt

Or:

$ cd foo
$ ln -s foo/bar.txt ../bar.txt


How to list all symbolic links in the current directory?

$ find -type l

[man find]: If no paths are given, the current directory is used.
[How to list all symbolic links in a directory]

How do I tell if a folder is actually a symlink and how do I fix it if it's broken?

Here are some ways that can be used to verify symlink:

$ stat ./data-vol/content/app/74.0.1365.76 
  File: ./data-vol/content/app/74.0.1365.76 -> data-vol/content/app/win/x86/74.0.1365.76
  Size: 45              Blocks: 0          IO Block: 4096   symbolic link
Device: fd01h/64769d    Inode: 26479224    Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2019-07-12 17:17:09.278071996 +0100
Modify: 2019-07-12 17:17:08.666073171 +0100
Change: 2019-07-12 17:17:08.666073171 +0100
 Birth: -


$ stat -L ./data-vol/content/app/74.0.1365.76 
stat: cannot stat './data-vol/content/app/74.0.1365.76': No such file or directory

$ file -L ./data-vol/content/app/74.0.1365.76 
./data-vol/content/app/74.0.1365.76: cannot open `./data-vol/content/app/74.0.1365.76' (No such file or directory)

$ ls ./data-vol/content/app/74.0.1365.76 
./data-vol/content/app/74.0.1365.76

$ ll ./data-vol/content/app/74.0.1365.76 
lrwxrwxrwx 1 root root 45 Jul 12 17:17 ./data-vol/content/app/74.0.1365.76 -> data-vol/content/app/win/x86/74.0.1365.76

How to see full symlink path

$ readlink -f symlinkName

Hard links


How to create hardlink of one file in different directories in linux



Working with Environment Variables

To list all environment variables and their values use:

$ env

To set environment variables for the single command:

Example:

$ env GOOS=linux GOARCH=amd64 go build cmd/main.go

From the executable's point of view, the same would have been achieved without using env:

$ GOOS=linux GOARCH=amd64 go build cmd/main.go

To set environment variables for the current terminal session:

$ export GOPATH=/mnt/c/dev/go

export is a bash builtin. export key=value is extended syntax and should not be used in portable scripts (i.e. #! /bin/sh)

What's the difference between set, export and env and when should I use each?
What is the difference between set, env, declare and export when setting a variable in a Linux shell?


Package management


apt (Advanced Packaging Tool) - It is not a command itself but a package which contains set of tools which manage installation and removal of other packages.

apt-get

apt-get update - downloads the package lists from the repositories and "updates" them to get information on the newest versions of packages and their dependencies. It will do this for all repositories and PPAs.

http://askubuntu.com/questions/222348/what-does-sudo-apt-get-update-do

apt-cche


add-apt-repository - adds a repository to the list of repositories


To apply latest security updates on Ubuntu:

sudo apt-get update

sudo apt-get -y upgrade


Difference Between apt and apt-get Explained

Should I use apt or apt-get?

You might be thinking if you should use apt or apt-get. And as a regular Linux user, my answer is to go with apt.

apt is the command that is being recommended by the Linux distributions. It provides the necessary option to manage the packages. Most important of all, it is easier to use with its fewer but easy to remember options.

I see no reason to stick with apt-get unless you are going to do specific operations that utilize more features of apt-get.


-----------------------------------------------------------------------------------------------------------------------


To download a file into some specific directory:

cd /dest_dir
wget https://example.com/archive_file.tar.gz


To install a package/software in Ubuntu, it is usually enough to copy it to /usr/local directory.
To move dir1 to some other location e.g. /usr/local use:

mv new_app /usr/local


A word on Linux directories

[source]

/usr/src is meant for source code for the binaries that come with your system's installation. For example, it could contain the source code for your kernel, tools such as ls, passwd, cp, mv etc, which are all installed BY YOUR DISTRIBUTION. If you upgrade your OS from source, all the source code would go here, when you rebuild your system. You DON'T want to put any software that you install BY YOURSELF in here, because they may get overwritten when you upgrade your system. In general, files that are in /, /usr/bin, /usr/sbin, /bin, /sbin etc. have their source code in /usr/src.

The /usr/local directory tree is meant to be used for software that you install by yourself, without using the distribution CD(s). For example, /usr/local/bin, /usr/local/sbin are for the binaries that are installed by you separately, /usr/local/etc is for config files etc. Thus /usr/local/src is for source files that you yourself downloaded.

Example: go (binary distribution) gets installed in /usr/local/go.

If you upgrade your system, files under the /usr tree get overwritten, such as /usr/bin, /usr/sbin, /usr/src etc. However, anything under /usr/local will not be touched. This is why all the software you installed separately should go to /usr/local tree.



Installing Software


How to install software distributed via .deb files?  


$ sudo dpkg -i /path/to/deb/file 
$ sudo apt-get install -f

The latter is necessary in order to fix broken packages (install eventual missing dependencies).

How to install a deb file, by dpkg -i or by apt?



Working with Archive files


To unpack the .tar.gz in the current directory use:

tar -xvf archive_file.tar.gz

-x = extract
-f (--file) = use archive file
-c (--verbose) = verbose output
-z (--gzip, --gunzip, --ungzip)  = filter the archive through gzip

To unpack the .tar.gz in the specific output directory use:

tar zxvf archive_file.tar.gz output_directory

Or:

$ tar -C path/to/dest_dir -xzf archive_file.tar

-C (--directory) stands for "Change to directory"

Example:
$ tar -C /usr/local -xzf go1.12.1.linux-amd64.tar


Hardware management

To verify if you're running a 64-bit system:

uname -m 

x86_64 is the output in case of 64-bit system.

SSH


How to test password for private key?
If id_rsa and id_rsa.pub is a keypair, we can execute (after cd ~/.ssh/):

$ ssh-keygen -y -f id_rsa

...which will prompt us to enter the password. If correct, this will output the public key.

-y = This option will read a private OpenSSH format file and print an OpenSSH public key to stdout.
-f filename = Specifies the filename of the key file.

Another example:

$ ssh-keygen -t rsa -C "your_email@example.com"

-t = Specifies the type of key to create.  The possible values are “dsa”, “ecdsa”, “ed25519”, or “rsa”.
-C comment = Provides a new comment.

To copy the contents of the id_rsa.pub file to clipboard:

$ xclip -sel clip < ~/.ssh/id_rsa.pub


---

To get base64 encoding of a string:

$ echo -n my_string | base64

or

$ printf my_string | base64

---
TBC...



Thursday, 21 March 2019

Golang - Miscellaneous Topics & Useful Links

Go CLI commands

go get

-u is used frequently as it makes get not just to check out missing packages but also to look for updates to existing packages 

Coding Style


https://github.com/golang/go/wiki/CodeReviewComments
https://github.com/Unknwon/go-code-convention/blob/master/en-US/naming_rules.md
https://rakyll.org/style-packages/
Golang - Code organisation with structs, variables and interfaces
Special Packages and Directories in Go

Go naming conventions for const
The standard library uses camel-case, so I advise you do that as well. The first letter is uppercase or lowercase depending on whether you want to export the constant.

Inner/package functions should not be calling panic on error but should return an error (together with a value) and thus allow caller to decide how they want to handle errors and if they want to panic or not.

Function should detect an error and return as soon as possible. The last statement in function body should be returning a valid value and nil as an error.

How to organize Go project?


Go Project Layout
golang-standards/project-layout

How to order packages in import?


Keep them in alphabetical order, with a blank line between:

  • the standard library
  • other libraries 
  • project-specific imports

gofmt: organize imports like eclipse does

Initialization


init functions in Go


Error Handling


Error handling and Go
Custom Errors
Return nil or custom error in Go
When should I use panic vs log.fatalln() in golang?
Part 32: Panic and Recover
Go by Example: Panic

    if err != nil {
        panic(err)
    }

Constants


https://blog.golang.org/constants
Constants in Go established during init

Types


integers



Strings


Printing " (double quote) in GoLang
Use:
  • escape character: fmt.Println("\"") or
  • raw strings: fmt.Println(`"`)
Go by Example: Number Parsing

Enums


Ultimate Visual Guide to Go Enums
Command stringer
Simple enumeration in Golang using Stringer
Assign a string representation to an enum/const (forum thread)
[Golang] Enum and String Representation
Enums in Go

Structs


Go by Example: Structs
Anonymous fields in structs - like object composition
The empty struct

Return pointer to local struct
Go performs pointer escape analysis. If the pointer escapes the local function, the object is allocated on the heap. If it doesn't escape the local function, the compiler is free to allocate it on the stack.

Note that, unlike in C, it's perfectly OK to return the address of a local variable; the storage associated with the variable survives after the function returns.[1]

When possible, the Go compilers will allocate variables that are local to a function in that function’s stack frame. However, if the compiler cannot prove that the variable is not referenced after the function returns, then the compiler must allocate the variable on the garbage-collected heap to avoid dangling pointer errors.[1]

Can you “pin” an object in memory with Go?
An object on which you keep a reference won't move. There is no handle or indirection, and the address you get is permanent. Go's GC is not a compacting GC. It will never move live objects around.

Local variable can escape the local function scope in two cases:
  • variable’s address is returned
  • its address is assigned to a variable in an outer scope


When to return a local instance of struct and when its address?


There are two usual reasons to return a pointer:

  • if we want methods of the struct to generally modify the struct in place (versus needing to create or copy to a new struct when we want to make modifications)
  • if the struct is rather large, a pointer is preferred for efficiency


When to return a pointer?
Constructor returning a pointer

Immutability


Does Go have immutable data structures?

Arrays


GO explicit array initialization
Keyed items in golang array initialization

Slices


https://golang.org/ref/spec#Slices
https://gobyexample.com/slices
https://blog.golang.org/go-slices-usage-and-internals

Channels


Unbuffered 


Example in SO quuestion

Buffered 


A Tour of Go - Channels 
BUFFERED CHANNELS IN GO: TIPS & TRICKS
Buffered Channels In Go — What Are They Good For?
Buffered Channels
Go by Example: Channel Buffering
Different ways to pass channels as arguments in function in go (golang)
https://www.reddit.com/r/golang/comments/7ep1un/golang_channel_is_really_fifo/

Interfaces


Interfaces in Go


Control Flow


for / range


Is there a way to iterate over a range of integers in Golang?
for range [5]int{} {...}

Switch

https://gobyexample.com/switch
https://tour.golang.org/flowcontrol/9
https://github.com/golang/go/wiki/Switch
https://yourbasic.org/golang/switch-statement/


defer

A Tour of Go - defer

panic


When should I use panic vs log.fatalln() in golang?


Functions


Go function type, reusable function signatures
type myFunctionType = func(a, b string) string

Arguments


Verifying arguments



Methods


Methods on structs
You will be able to add methods for a type only if the type is defined in the same package.
Don't Get Bitten by Pointer vs Non-Pointer Method Receivers in Golang
Anatomy of methods in Go

Lambdas inside a function

Why doesn't Go allow nested function declarations (functions inside functions)?

OOP - Encapsulation & import


Exported identifiers in Go
What does an underscore in front of an import statement mean in Golang?

dot-import

When you import package prefixed with dot (e.g.: import( . "fmt")), it would be imported in current namespace, so you can omit "fmt" prefix before calling methods. BUT: Don't ever do that; the only time it's useful is in very rare cases involving test files, everywhere else it's a really bad idea.

Concurrency


Go Concurrency Patterns: Pipelines and cancellation
Go Language Patterns - Semaphores
Go’s Extended Concurrency: Semaphores (Part 1)
Essential Go - Limiting concurrency with a semaphore

Unit & Integration Testing



First, something not related specifically to Go but to general software engineering:
TDD and BDD Differences"BDD is just TDD with different words"
Why do some software development companies prefer to use TDD instead of BDD?
How can I do test setup using the testing package in Go
Nice example how to inject unit test checker in an internal method: compare Do() and do() here.
Verbose output prints the names of unit test functions: go test -v
Exploring the landscape of Go testing frameworks
Go With Go(Lang): Features and Testing Frameworks

BDD:
  • ginkgo [Ginkgo] - very active development
  • goconvey
  • goblin - seems that its development has stopped; the last commit in repo was 5 months ago

5 simple tips and tricks for writing unit tests in #golang
Integration Tests in Go
Integration Test With Database in Golang
Separating unit tests and integration tests in Go
Learn Go by writing tests: Structs, methods, interfaces & table driven tests
Testing with golden files in Go
BDD Testing with Ginkgo and GoMock
Clean Go Testing with Ginkgo
done channel 
Using Ginkgo with Gomock
golang/mock
Gomega
Filesystem impact testing
Testing Techniques - Google I/O 2014

Go & DataBases


Scanners and Valuers with Go
How to write unit tests for database calls

Design Patterns in Go


The factory method pattern in Go
Factory patterns in Go (Golang)
A Class Factory in Golang (Google Go)

Go in VSCode

Debugging Go code using VS Code

Misc



Things I Wish Someone Had Told Me About Golang

Random


Package rand
Generating random numbers and strings in Go
https://flaviocopes.com/go-random/
Go by Example: Random Numbers
package uuid

Environment Variables

Pass environment variables from docker to my GoLang.

Configuration Files


Best practices for Configuration file in your code
Manage config in Golang to get variables from file and env variables

Useful packages


For unit tests: https://github.com/Pallinder/go-randomdata

Go Modules


Inspired by: Getting started with Go modules

To initialize Go modules in to your application you can run:

$ go mod init /path/to/directory

Example shows that we need to include the path:

root@bc7429fce4a8:~/TestApp# go mod init 
go: cannot determine module path for source directory /root/TestApp (outside GOPATH, no import comments)

root@bc7429fce4a8:~/TestApp# ls
main.go

root@bc7429fce4a8:~/TestApp# go mod init /root/TestApp
go: creating new go.mod: module /root/TestApp

root@bc7429fce4a8:~/TestApp# ls
go.mod main.go

root@bc7429fce4a8:~/TestApp# cat go.mod

module /root/TestApp

# go run main.go 
go: finding github.com/Pallinder/go-randomdata v1.1.0
go: downloading github.com/Pallinder/go-randomdata v1.1.0
Running the TestApp
Backstump

root@bc7429fce4a8:~/TestApp# go run main.go 
Running the TestApp
Frightbrown

# cat go.mod 
module /root/TestApp

require github.com/Pallinder/go-randomdata v1.1.0 // indirect

root@bc7429fce4a8:~/TestApp# cat go.sum
github.com/Pallinder/go-randomdata v1.1.0 h1:gUubB1IEUliFmzjqjhf+bgkg1o6uoFIkRsP3VrhEcx8=
github.com/Pallinder/go-randomdata v1.1.0/go.mod h1:yHmJgulpD2Nfrm0cR9tI/+oAgRqCQQixsA8HyRZfV9Y=

If we want to change version of the 3rd party package (go-randomdata in our case) we just need to manually edit the go.mod and set there desired version. go run (or go build) would then apply that version and hashes in go.sum would also change.

TBC...