Wednesday 12 June 2019

Introduction to Go language

Here are some notes on Go language features, packages and tools.

How to Write Go Code


Installation


Upon installing Go on Ubuntu, its source code is placed at the following path: /var/lib/go/src/

Environment Variables


GOPROXY


How to Use JFrog Artifactory and GoCenter Together to Build Go Apps


GOSUMDB


Command go
The GOSUMDB environment variable identifies the name of checksum database to use and optionally its public key and URL.

sumdb.go
How To Use GoCenter with Golang 1.13

From version 1.13 of Golang, go get performs authentication of modules through an auditable checksum database that Google maintains at sum.golang.org. Installing version 1.13 sets this checksum DB URL by default in the GOSUMDB environment variable.




For the build tools to produce an executable, the function main must be declared, and it
becomes the entry point for the program.


Packages, import and go get


Function main is located in a package called main.
If your main function doesn’t exist in package main, the build tools won’t produce an executable.

Every code file in Go belongs to a package, and main.go is no exception.
Packages define a unit of compiled code.
Their names help provide a level of indirection to the identifiers that are declared inside of them, just like a namespace. This makes it possible to distinguish identifiers that are declared with exactly the same name in the different packages you import.

All code files in a folder must use the same package name, and it’s common practice to name the package after the folder. As stated before, a package defines a unit of compiled code, and each unit of code represents a package.

Each code file will contain the keyword package at the top with a name for the package. e.g. Each code file in the "search" folder will contain "search" for the package name.
Imports are just that: they import code and give you access to identifiers such as types, functions, constants, and interfaces.

Imported packages can be grouped as:

import (
   "log"
   "os"
    _ "github.com/goinaction/code/chapter2/sample/matchers"
   "github.com/goinaction/code/chapter2/sample/search"
)
Packages from standard library don't need to have path specified.
Packages from the local workspace have to have their relative path specified (relative to the workspace).
When you import code from the standard library, you only need to reference the name of the package, unlike when you import code from outside of the standard library. The compiler will always look for the packages you import at the locations referenced by the GOROOT and GOPATH environment variables.

When importing 3rd party package (the one which is not part of the standard library), then import statement in the code is not enough, if we run the app we'll get an error similar to this:

$ go run ./cmd/main.go 
internal/pkg/jsondemo/jsondemo.go:7:2: cannot find package "github.com/nsf/jsondiff" in any of:
        /var/lib/go/src/github.com/nsf/jsondiff (from $GOROOT)
        /home/bojan/dev/go/src/github.com/nsf/jsondiff (from $GOPATH)

We need to install that package on the local machine with:

$ go get <package_uri>

e.g.

$ go get -u github.com/nsf/jsondiff


See also: Removing packages installed with go get

import the matchers package and use the blank identifier (_) before listing out the import path.This is a technique in Go to allow initialization from a package to occur, even if you
don’t directly use any identifiers from the package. To make your programs more
readable, the Go compiler won’t let you declare a package to be imported if it’s not
used. The blank identifier allows the compiler to accept the import and call any init
functions that can be found in the different code files within that package.

Is there an efficient way to share structure between golang packages?


$ go help get
usage: go get [-d] [-m] [-u] [-v] [-insecure] [build flags] [packages]

Get resolves and adds dependencies to the current development module
and then builds and installs them.

The first step is to resolve which dependencies to add.

For each named package or package pattern, get must decide which version of
the corresponding module to use. By default, get chooses the latest tagged
release version, such as v0.4.5 or v1.2.3. If there are no tagged release
versions, get chooses the latest tagged prerelease version, such as
v0.0.1-pre1. If there are no tagged versions at all, get chooses the latest
known commit.

This default version selection can be overridden by adding an @version
suffix to the package argument, as in 'go get golang.org/x/text@v0.3.0'.
For modules stored in source control repositories, the version suffix can
also be a commit hash, branch identifier, or other syntax known to the
source control system, as in 'go get golang.org/x/text@master'.
The version suffix @latest explicitly requests the default behavior
described above.

If a module under consideration is already a dependency of the current
development module, then get will update the required version.
Specifying a version earlier than the current required version is valid and
downgrades the dependency. The version suffix @none indicates that the
dependency should be removed entirely, downgrading or removing modules
depending on it as needed.

Although get defaults to using the latest version of the module containing
a named package, it does not use the latest version of that module's
dependencies. Instead it prefers to use the specific dependency versions
requested by that module. For example, if the latest A requires module
B v1.2.3, while B v1.2.4 and v1.3.1 are also available, then 'go get A'
will use the latest A but then use B v1.2.3, as requested by A. (If there
are competing requirements for a particular module, then 'go get' resolves
those requirements by taking the maximum requested version.)

The -u flag instructs get to update dependencies to use newer minor or
patch releases when available. Continuing the previous example,
'go get -u A' will use the latest A with B v1.3.1 (not B v1.2.3).

The -u=patch flag (not -u patch) instructs get to update dependencies
to use newer patch releases when available. Continuing the previous example,
'go get -u=patch A' will use the latest A with B v1.2.4 (not B v1.2.3).

In general, adding a new dependency may require upgrading
existing dependencies to keep a working build, and 'go get' does
this automatically. Similarly, downgrading one dependency may
require downgrading other dependencies, and 'go get' does
this automatically as well.

The -m flag instructs get to stop here, after resolving, upgrading,
and downgrading modules and updating go.mod. When using -m,
each specified package path must be a module path as well,
not the import path of a package below the module root.

The -insecure flag permits fetching from repositories and resolving
custom domains using insecure schemes such as HTTP. Use with caution.

The second step is to download (if needed), build, and install
the named packages.

If an argument names a module but not a package (because there is no
Go source code in the module's root directory), then the install step
is skipped for that argument, instead of causing a build failure.
For example 'go get golang.org/x/perf' succeeds even though there
is no code corresponding to that import path.

Note that package patterns are allowed and are expanded after resolving
the module versions. For example, 'go get golang.org/x/perf/cmd/...'
adds the latest golang.org/x/perf and then installs the commands in that
latest version.

The -d flag instructs get to download the source code needed to build
the named packages, including downloading necessary dependencies,
but not to build and install them.

With no package arguments, 'go get' applies to the main module,
and to the Go package in the current directory, if any. In particular,
'go get -u' and 'go get -u=patch' update all the dependencies of the
main module. With no package arguments and also without -u,
'go get' is not much more than 'go install', and 'go get -d' not much
more than 'go list'.

For more about modules, see 'go help modules'.

For more about specifying packages, see 'go help packages'.

This text describes the behavior of get using modules to manage source
code and dependencies. If instead the go command is running in GOPATH
mode, the details of get's flags and effects change, as does 'go help get'.
See 'go help modules' and 'go help gopath-get'.

See also: go build, go install, go clean, go mod.


Go Modules

The Go Blog - Using Go Modules
The Go Blog - Migrating to Go Modules

Go modules were introduced in order to prevent having to change import path (of dependencies )on breaking changes.

A module is a collection of Go packages stored in a file tree with a go.mod file at its root.

The go.mod file contains:
  •  module directive which defines module’s path which is also the import path used for the root directory (e.g. this path is the base path upon which paths to other packages in the same project are built)
  • go directive which declares the expected version of the Go language used to compile the code within the module.
  • its dependency requirements, which are the other modules needed for a successful build

To use Go modules, project directory has to be outside $GOPATH/src otherwise the go command still runs in the old GOPATH mode, even if a go.mod is found.

Packages in subdirectories have import paths consisting of the module path plus the path to the subdirectory. For example, if we created a subdirectory world, we would not need to (nor want to) run go mod init there. The package would automatically be recognized as part of the example.com/hello module, with import path example.com/hello/world.

The go command resolves imports by using the specific dependency module versions listed in go.mod.

Only direct dependencies are recorded in the go.mod file.

Running go command again will not repeat module detection and downloading, since the go.mod is now up-to-date and the downloaded modules are cached locally (in $GOPATH/pkg/mod).

command go list -m all lists the current module and all its dependencies. In the go list output, the current module, also known as the main module, is always the first line, followed by dependencies sorted by module path.

---

Steps I did in order to migrate my app (which didn't use any package manager before) to modules:

• moved project's directory from $GOPATH/src/example.com/myapp/ to ./wherever/myapp/

• verified that go build fails with error: 
cannot find package "example.com/myapp/internal/pkg/mypkg1" in any of:
/var/lib/go/src/example.com/myapp/internal/pkg/mypkg1 (from $GOROOT)
/home/bojan/dev/go/src/example.com/myapp/internal/pkg/mypkg1 (from $GOPATH)

• created go.mod by executing: 

$ go mod init example.com/myapp

Another example:

$ go mod init github.com/BojanKomazec/go-demo

It's content was at that point:

$ cat go.mod 
module example.com/myapp

go 1.12 

• added to go.mod the list of all dependencies (this list does not contain their dependencies) and created go.sum:

$ go mod tidy
...
go: finding github.com/onsi/gomega v1.7.0
go: finding github.com/onsi/ginkgo v1.10.2
go: finding github.com/joho/godotenv v1.3.0
go: finding github.com/jmoiron/sqlx v1.2.0
go: finding github.com/lib/pq v1.2.0
go: finding github.com/onsi/ginkgo/extensions/table latest
go: finding github.com/google/uuid v1.1.1
go: downloading github.com/joho/godotenv v1.3.0
go: extracting github.com/joho/godotenv v1.3.0
...
go: downloading github.com/golang/protobuf v1.3.1
go: extracting github.com/golang/protobuf v1.3.1
go: extracting golang.org/x/net v0.0.0-20190603091049-60506f45cf65 // <--  pseudo-version, which is the go command's version syntax for a specific untagged commit.
go: downloading golang.org/x/text v0.3.2
go: extracting golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: extracting golang.org/x/text v0.3.2

go mod tidy added module requirements for all the packages transitively imported by packages in your module and built a go.sum with checksums for each library at a specific version.

After this go.mod got require section:

$ cat go.mod 
module example.com/myapp

go 1.12

require (
        github.com/google/uuid v1.1.1
        github.com/jmoiron/sqlx v1.2.0
        github.com/joho/godotenv v1.3.0
        github.com/lib/pq v1.2.0
        github.com/onsi/ginkgo v1.10.2
        github.com/onsi/gomega v1.7.0
        google.golang.org/appengine v1.6.5 // indirect
)

go.sum contains the expected cryptographic hashes of the content of specific module versions and is used to ensure that future downloads of these modules retrieve the same bits as the first download:

$ cat go.sum 
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/go-sql-driver/mysql v1.4.0 h1:7LxgVwFb2hIQtMm87NdgAVfXjnt4OePseqT1tKx+opk=
github.com/go-sql-driver/mysql v1.4.0/go.mod ...

• I verified that import paths for the internal packages were based on the path assigned to module directive in go.mod. E.g. if we have project structure as:

cmd/main.go
internal/pkg/mypkg1
internal/pkg/mypkg1.go
internal/pkg/mypkg1_test.go

Then to import package mypkg1 in main.go we need to use:

import "example.com/myapp/internal/pkg/mypkg1"


When go mod tidy adds a requirement, it adds the latest version of the module. If your GOPATH included an older version of a dependency that subsequently published a breaking change, you may see errors in go mod tidy, go build, or go test.

After this my Go project was again building successfully.


SWITCHING TO GO MODULES
Switching to Go Modules - bye GOPATH and Godep
Anatomy of Modules in Go
Organize local code in packages using Go modules
Accessing local packages within a go module (go 1.11)

---


call to the Run function that belongs to the search package:
search.Run("president")

The log package provides support for logging messages to the stdout , stderr , or even
custom devices. The sync package provides support for synchronizing goroutines,

var matchers = make(map[string]Matcher)

This variable is located outside the scope of any function and so is considered a
package-level variable. The variable is declared using the keyword var and is declared
as a map of Matcher type values with a key of type string .

The name of the variable matchers starts with a lowercase letter.

In Go, identifiers are either exported or unexported from a package.
  • An exported identifier can be directly accessed by code in other packages when the respective package is imported. These identifiers start with a capital letter.
  • Unexported identifiers start with a lowercase letter and can’t be directly accessed by code in other packages. But just because an identifier is unexported, it doesn’t mean other packages can’t indirectly access these identifiers. As an example, a function can return a value of an unexported type and this value is accessible by any calling function, even if the calling function has been declared in a different package.

This variable declaration also contains an initialization of the variable via the
assignment operator and a special built-in function called make .

A map is a reference type that you’re required to make in Go. If you don’t make the
map first and assign it to your variable, you’ll receive errors when you try to use the map
variable. This is because the zero value for a map variable is nil .

In Go, all variables are initialized to their zero value. For numeric types, that value
is 0 ; for strings it’s an empty string; for Booleans it’s false ; and for pointers, the zero
value is nil . When it comes to reference types, there are underlying data structures
that are initialized to their zero values. But variables declared as a reference type set to
their zero value will return the value of nil.

Go programs can be structured to handle the launching and synchronization of goroutines that run concurrently


the short variable declaration operator ( := ). This operator is
used to both declare and initialize variables at the same time. The type of each value
being returned is used by the compiler to determine the type for each variable,
respectively. The short variable declaration operator is just a shortcut to streamline
your code and make the code more readable. The variable it declares is no different
than any other variable you may declare when using the keyword var .

results := make(chan *Result)

  • = assigns the right to the left
  • :=  creates a new variable named the left, and assigns it the value of the item on the right

Short Variable Declaration Operator(:=) in Go - GeeksforGeeks


we use the built-in function make to create an unbuffered channel. We use
the short variable declaration operator to declare and initialize the channel variable
with the call to make . A good rule of thumb when declaring variables is to use the key-
word var when declaring variables that will be initialized to their zero value, and to
use the short variable declaration operator when you’re providing extra initialization
or making a function call.

Channels are also a reference type in Go like maps and slices, but channels imple-
ment a queue of typed values that are used to communicate data between goroutines.
Channels provide inherent synchronization mechanisms to make communication
safe.


The next two lines of code are used later to prevent the program from terminating
before all the search processing is complete.

// Setup a wait group so we can process all the feeds.
var waitGroup sync.WaitGroup
// Set the number of goroutines we need to wait for while
// they process the individual feeds.
waitGroup.Add(len(feeds))


In Go, once the main function returns, the program terminates. Any goroutines that
were launched and are still running at this time will also be terminated by the Go run-
time. When you write concurrent programs, it’s best to cleanly terminate any gorou-
tines that were launched prior to letting the main function return. Writing programs
that can cleanly start and shut down helps reduce bugs and prevents resources from
corruption.

Our program is using a WaitGroup from the sync package to track all the goroutines we’re going to launch. A WaitGroup is a great way to track when a goroutine is
finished performing its work. A WaitGroup is a counting semaphore, and we’ll use it to
count off goroutines as they finish their work.

On line 23 we declare a variable of type WaitGroup from the sync package. Then
on line 27 we set the value of the WaitGroup variable to match the number of gorou-
tines we’re going to launch. As you’ll soon see, we’ll process each feed concurrently
with its own goroutine. As each goroutine completes its work, it will decrement the

count of the WaitGroup variable, and once the variable gets to zero, we’ll know all the
work is done.


for _, item := range items {
   ...
}

range (keyword)

  • can be used with arrays, strings, slices, maps, and channels

for range

  • used to iterate over the slice of items
  • When used to iterate over a slice, we get two values back on each iteration:
    • index position of the element we’re iterating over
    • a copy of the value in that element

blank identifier (_)

  • used as a substitution for the variable that would be assigned to the index value for the range call. When you have a function that returns multiple values, and you don’t have a need for one, you can use the blank identifier to ignore those values. In our case with this range, we won’t be using the index value, so the blank identifier allows us to ignore it.


matcher, exists := matchers[feed.Type]
if !exists {
   matcher = matchers["default"]
}

we check the map for a key that matches the feed type. When looking up a key in a map, you have two options: you can assign a single variable or two variables for the lookup call. The first
variable is always the value returned for the key lookup, and the second value, if specified, is a Boolean flag that reports whether the key exists or not. When a key doesn’t
exist, the map will return the zero value for the type of value being stored in the map.
When the key does exist, the map will return a copy of the value for that key.


---

Variables


Variable names can't start with number.
Variable names can't be the same as names of imported packages.


---

Constants


  • cannot be declared using the := syntax

const Pi = 3.14

Tour of Go - Constants
The Go Blog - Constants

"Hello, 世界" is untyped string constant.
It remains an untyped string constant even when given a name:

const hello = "Hello, 世界"

hello is also an untyped string constant.

An untyped constant is just a value, one not yet given a defined type that would force it to obey the strict rules that prevent combining differently typed values.


typed string constant is one that's been given a type:

const typedHello string = "Hello, 世界"


---

Logical Operators


Operands in logical expressions are evaluated lazily - only if needed:

A && B <==> If A then B else FALSE
A || B <==> If A then TRUE else B


---

Data Types

How to find a type of an object in Go?

import "reflect"
tst := "string"
fmt.Println(reflect.TypeOf(tst))


A Tour of Go - Type switches

switch v := i.(type) {
case T:
    // here v has type T
case S:
    // here v has type S
default:
    // no match; here v has the same type as i
}

Type assertions

x.(T)

Type casting (Type conversions)

string(x)


---
type is reserved word in Go and can't be used as e.g. struct field name (Go linter issues error: syntax error: unexpected type, expecting field name or embedded type)

What (exactly) does the type keyword do in go?
---

Where to declare custom types?

Declare (custom) type just before the place you need it. That does not have to be at the beginning of the file, but e.g. your custom struct can be declared just before the set of functions that are using it. Example: https://golang.org/src/net/http/client.go

Golang - Code organisation with structs, variables and interfaces

All the files are based on features, and it is best to use a proximity principle approach, where you can find in the same file the definition of what you are using.
Generally, those features are grouped in one file per package, except for large ones, where one package is composed of many files (net, net/http)

If you want to separate anything, separate the source (xxx.go) from the tests/benchmarks (xxx_test.go)


iota is predeclared untyped integer identifier which resets to 0 at each constant declaration and its value is incremented by 1 for each constant withing that declaration.

type Day int

const (
 MONDAY Day = 1 + iota
 TUESDAY
 WEDNESDAY
 THURSDAY
 FRIDAY
 SATURDAY
 SUNDAY
)

var days = [...]string {
 "MONDAY",
 "TUESDAY",
 "WEDNESDAY",
 "THURSDAY",
 "FRIDAY",
 "SATURDAY",
 "SUNDAY",
}

func (day Day) String() string {
 return days[day - 1]
}

Type of all variables above is Day. But be careful, the type of first variable in the const block applies to subsequent variables only if no values are explicitly assigned to them.

Take  this example:

type architecture string
const (
mac   architecture = "mac"
win64              = "win64"
win86              = "win86"
)

Only mac is of type architecture. win64 and win86 are still of type string! Solution:

const (
mac    architecture = "mac"
win64  architecture = "win64"
win86  architecture = "win86"
)

Go does not provide off-the-shelf way to find the count of enum values but here is one trick:

const (
Sunday = iota
Monday
Tuesday
Wednesday
Thursday
Friday
Partyday
numberOfDays  // this constant is not exported
)

We could have used string as the underlying type:

type Day string
const (
   MONDAY Day = "MONDAY"
   TUESDAY = "TUESDAY"
   WEDNESDAY = "WEDNESDAY"
   THURSDAY = "THURSDAY"
   FRIDAY = "FRIDAY"
   SATURDAY = "SATURDAY"
   SUNDAY = "SUNDAY"
)


for _, day := range []Day{MONDAY, TUESDAY} {
   // cast enumerator to string
   s := string(day) }
---

4 iota enum examples
Ultimate Visual Guide to Go Enums and iota
Create an enum from a group of related constants in Go


Structures


type S struct {
   X int
}

// create an instance of S
var s1 S

// create an instance of S and get a pointer to it
s2 := &S{123} 
x1 := (*s2).X 
x2 := s2.X // a shorter form

// another way to get a pointer:
s3 := new(S)

// create an instance and initialize it
s4 := S {X: 456}
s5 := S {789}


If some function has to return a struct variable:

return S{1}

Bypassing Golang's lack of constructors

Empty structure


The empty struct 

type S struct{}
var s S
fmt.Println(unsafe.Sizeof(s)) // 0

chan struct{} construct is used for signalling between goroutines.

---
Returning nil for a struct? - If function returns (MyStruct, error) which value for MyStruct to return in case of error?

Don't be too concerned about sending back an empty/blank struct, as it's generally good practice to first check the error before proceeding to do anything with the value of the struct. So it's value is generally irrelevant until the error has been checked.

If function returns a pointer and error - (*MyStruct, error), we can then return nil (but have to cast it to pointer type):

func New(arg1 T1) (*MyStruct, error) {
if arg1 == nil {
return (*MyStruct)(nil), errors.New("arg1 argument is nil")
}
 ... 
 return (*MyStruct)(nil), nil
 ...
}

Convert nil interface to pointer of something in Golang?

---


Structs + Methods = "Classes in Go"


Go Best Practices: Pointer or value receivers?

If you want to change the state of the receiver in a method, manipulating the value of it, use a pointer receiver. It’s not possible with a value receiver, which copies by value. Any modification to a value receiver is local to that copy.


When to use free-standing methods and when to attach them to receivers?



Golang receiver vs function argument - Gregory Trubetskoy

Composite Literals


Composite literals

Golang: What the heck is a composite literal?

In:

myArray := [5]int{3, 3, 3, 3, 3}

[5]int{3, 3, 3, 3, 3} is composite literal, that evaluates to an array of size 5, with the values 3, 3, 3, 3, 3[3, 3, 3, 3, 3].

Composite literals in Go

Literals in source code allow to specify fixed values like numbers, strings or booleans. Go allows using literals also to create compound types (arrays, maps, slices or structs).

type Thing struct {
    name       string
    generation int
    model      string
}
thing := Thing{“Raspberry Pi”, 2, “B”}
// or using explicit field names
thing = Thing{name: “Raspberry Pi”, generation: 2, model: “B”}
---

Channels

The Go Programming Language Specification - The Go Programming Language - Channel Types
Curious Channels

Go by Example: Closing Channels
How to Gracefully Close Channels
Range and Close
idiomatic goroutine termination and error handling

To pass a channel as a function argument:

func foo(c chan struct{}) {
   ...
}

...
c := make(chan struct{}, 10)
foo(c)
...

Should channel be passed by reference (as a pointer)?
No. See Are channels passed by reference implicitly

The reference types in Go are slice, map and channel. When passing these, you're making a copy of the reference. Strings are also implemented as a reference type, though they're immutable.

Thread-safe way to funnel data from multiple go routines - Stack Overflow

Channels are completely thread safe. They are the official way to communicate between goroutines. It is safe to read from or write to the same channel from multiple goroutines.

Channels can only be closed once and nothing can be sent after the channel is closed.

multithreading - How to safely interact with channels in goroutines in Golang - Stack Overflow
multithreading - Is this Go code thread safe or do I need a mutex? - Stack Overflow
Thread-safe way to funnel data from multiple go routines - Stack Overflow
Synchronizing states using Mutex vs Channel in Go - MindOrks - Medium

Unbuffered Channels 


errChannel := make(chan error)

Capacity of unbuffered channel (e.g. cap(errChannel)) is always 0.
Length of unbuffered channel (e.g. len(errChannel)) is always 0.

Writing to unbuffered channel blocks until someone reads from it. If goroutine wants to write to unbuffered channel, this will block until someone tries to read from that channel (maybe in main thread).

Reading from unbuffered channel blocks until someone writes to it.

go - Golang unbuffered channel size? - Stack Overflow

Buffered Channels


errChannel := make(chan error)

Capacity of unbuffered channel tells how many elements can channel store. Example cap(errChannel) is always 1.

Length of unbuffered channel tells how many elements are currently present in the channle Example len(errChannel) can be 0 or 1.

Writing does not block as long as channel is not full.
Reading blocks as long as channel is empty.


Buffered Channels and Worker Pools - golangbot.com

How to Gracefully Close Channels - Go 101 (an online book for Go programming language + Golang knowledge base)
Go by Example: Non-Blocking Channel Operations
Checking if a channel has a ready-to-read value, using Go - Stack Overflow
Anatomy of Channels in Go - Concurrency in Go - RunGo - Medium


select 


In Go, does a break statement break from a switch/select? - Stack Overflow

---


The empty interface


interface{}

  • interface type that specifies zero methods
  • may hold values of any type
  • used by code that handles values of unknown type

var i interface{}
i = 42
i = "hello"

A Tour of Go - The empty interface



package main

import (
"fmt"
)

func main() {
var i interface{}
var j interface{} = 2
var k *interface{}
i = &j
k = &j
fmt.Println(i)
        
        // invalid indirect of i (type interface {})
// fmt.Println(*i) 

fmt.Println(*k)

}

Output:

0x40c130
2


Array


  • collection of elements of a single type
  • a fixed-length data type that contains a contiguous block of elements of the same type


var intArray [5]int
intArray[0] = 10
var intArray = [5]int {10, 20, 30}
var intArray = [5]int {0:10, 2:30, 4:50}
intArray := [5]int {10, 20, 30, 40, 50}
intArray := [...]int {10, 20, 30, 40, 50}

Create array of array literal in Golang

a := [][]int{{1,2,3},{4,5,6}}



Slice

  • a segment of dynamic arrays that can grow and shrink
  • indexable and have a length
  • used as a dynamic array - when we want to use an array but don't know its size in advance
  • can be created with built-in function make. make has one mandatory (type) and two optional arguments (length and capacity)
Make distinction between nil slice:

var numbers []int

...and empty slice:

numbers := make([]int, 0)

which is the same as using empty slice literal:

numbers := []int{}

Both have zero length and capacity but empty slice has allocated underlying zero-length array.




Go: Empty slice vs. nil slice | Programming.Guide
arrays - Correct way to initialize empty slice - Stack Overflow

If slice will be JSON encoded then prefer using empty slice (as it would be encoded into JSON empty array) otherwise prefer nil slice (which would be JSON encoded into null).

To add elements to slice:

numbers = append(numbers, 1)
numbers = append(numbers, 2)
fmt.Println(len(numbers)) // == 2

Note this:

slc := make([]string, len(in)) // creates a slice which already contains 3 elements (empty strings)

slc := make([]string, 0, len(in)) // creates a slice which contains 0 but has capacity to hold 3 elements (3 strings)


Creating a slice with make
Go Slices: usage and internals
The Go Blog - Arrays, slices (and strings): The mechanics of 'append'


Declare slice or make slice?

var s []int
  • simple declaration
  • crates so called "nil slice"
  • does not allocate memory
  • s points to nil
  • should not be a return value of an API which returns slice (an empty slice should be returned)
  • marshaling the nil slice (var s []int) will produce null

s  := make([]int, 0)
  • creates so called "empty slice"
  • allocates memory
  • s points to memory to a slice with 0 elements
  • should be returned if an API needs to return a slice with 0 elements
  • marshalling the empty slice will produce the expected []


Convert slice of string to slice of pointer to string

Convert slice of type A to slice of pointers to type A

This:

for _, v := range a {
b = append(b, &v)
}

and this:

for i := 0; i < len(a); i++ {
b = append(b, &a[i])
}

are not the same. In the latter, you are taking the address of element a[i] in the array (which is what you want). In the former, you are taking the address of the iteration variable v, which, as you discovered, does not change during the loop execution (it gets allocated once, it's in a completely new location in memory, and gets overwritten at each iteration).

BK: We can still use range:

 for i := range a {
b = append(b, &a[i])
}

---
How to declare a slice of slices (2-D slice)?


slice literal is written as []type{<value 1>, <value 2>, ... }.

A slice of ints would be []int{1,2,3}
A slice of int slices would be [][]int{{1,2,3}, {4,5,6}}

var s2d [][]string = [][]string{{"a", "b", "c"}, {"d", "e", "f"}}

Try it in The Go Playground

---

Maps

Note that it is not possible to get the address (use & operator or get the pointer) of the map element.

myMap["keyName"] is called map index expression.

Why do I get a “cannot assign” error when setting value to a struct as a value in a map? 



Interfaces

Struct implements interface if it implements all its methods.


pinger/pinger.go:

package pinger

type (
   PingerConfig struct {
      timeout int
   }

   Pinger interface {
      Ping(ip string) error
   }

   // New is a Pinger factory method
   // New func(pingerConfig PingerConfig) (Pinger, error)
)

superPinger/pinger.go:

package superpinger

import (
   "example.com/myproduct/myproject/internal/pkg/pinger"


type superPinger struct {
id     string
}

// New function creates an instance of PostgreSQL client
func New(config pinger.PingerConfig) (pinger.Pinger, error) {
        superPinger := superPinger(config)
        err := superPinger.Init(...)
return superPinger, err
}


// Ping function verifies remote endpoint accessibility
func (superPinger superPinger) Ping(ip string) error {
err := doPing(ip)
return err
}

main.go:

import (
   "example.com/myproduct/myproject/internal/pkg/pinger"
   "example.com/myproduct/myproject/internal/pkg/superpinger"
)

...
pingerConfig := pinger.NewPingerConfig(...)
pinger, err := superpinger.New(pingerConfig)
pinger.Ping("8.8.8.8")
...


How to pass an interface to function: by value or pointer?

What is correct:

func CalculateSHA(h hash.Hash, file *os.File)
or
func CalculateSHA(h *hash.Hash, file *os.File) ([]byte, error) {

(hash.Hash is an interface)


Interfaces and Pointers to Interfaces
you almost never need a pointer to an interface, since interfaces are just pointers themselves
If I was doing a code review and saw that you were passing the address of an interface value to a function, it would raise a flag. An interface value is designed to be copied. An interface value is a reference type, just like slices, maps, channels and function variables. 
Have the pointer inside the interface if possible, rather than as a pointer to interface.
can't use pointers to interface types?

>     func Foo(r io.Reader)
>
> Why is this? Won't this copy the thing?
interface{} is just 2 words, copying it is very cheap.
interface copy does not copy the stored object, interface assignment can  
Function calls in Go (like most other languages) is 'pass by value'. The values passed to the function are copied on to the stack for the function to access. In languages without complex values type (i.e. structs) these values are either primitive values (ints, floats etc) or they are references(pointers) to objects.  
Interface values in Go are the size of two pointers, one pointer to type information and the other is a pointer to the value currently stored in the interface value. eg. an io.Reader that you've assigned an *os.File to will contain that *os.File pointer and a pointer to the *os.File type information.  
Passing interface values to a function copies the interface value.
It generally doesn't make sense to declare a function that takes a pointer to an interface, e.g. *io.Writer. It's not very useful - io.Writer itself might *contain* a pointer (as in your p2) example and all is fine. 

Go Data Structures: Interfaces


Generics (Templates)


They do NOT exist in Go!

That's why you'll need to write many functions from scratch...like here:
What is the correct way to find the min between two integers in Go?


Polymorphism

Part 28: Polymorphism - OOP in Go:
A variable of type interface can hold any value which implements the interface. This property of interfaces is used to achieve polymorphism in Go.

---

For Loop


for i :=0; i<10; i++ {
   fmt.Println("i =", i)
}


Switch

There is one important difference between switch statement in Go and in other languages (like C, C++, C#, JS) and that is a keyword fallthrough which has to be used if we want to use same case handler for multiple cases:


v := 1
switch v {
case 1:
fallthrough
case 2:
fmt.Println("1 or 2")
        break 
default:
fmt.Println("default")
}

// output: 1 or 2

Without fallthrough in case 1, this snippet would not print anything.


Error Creation and Handling


3 simple ways to create an error

If function returns type error, it can return nil if there is no error.


Functions in Go | My Public Notepad

Working with File system in Go | My Public Notepad

Working with strings in Go | My Public Notepad

Encryption



Errors 


Concurrency & Parallelism


How can my Go program keep all the CPU cores busy?

How to find the number of CPUs?

logicalCPUMaxCount := runtime.NumCPU()

---


Packages

If we have:

import "mypackage"

then we need to use package name when calling any its exported member:

mypackage.Foo()

It is possible to assign an alias name to the imported package. Example:

import b64 "encoding/base64"
...
es := b64.StdEncoding.EncodeToString([]byte(data))


Internal Packages


https://notes.shichao.io/gopl/ch10/#internal-packages

How to import local packages in go?

Go starting path for import is $HOME/go/src.


encoding/base64


Go by Example: Base64 Encoding



encoding/json


Parsing JSON in Golang

the Species attribute in our Bird struct will map to the species, or Species or sPeCiEs JSON property.

Go by Example: JSON

When defining a structure into which we want to unmarshal some JSON, we have to declare struct members as exported (their names have to start with capital letter). This is required so other package (json) can use reflection and access these fields.

JSON and dealing with unexported fields

The json library does not have the power to view fields using reflect unless they are exported. A package can only view the unexported fields of types within its own package.


It's fundamentally limited by the reflect package that any package can't set unexported fields in types
defined in other packages (without using unsafe.)

---
How to unmarshal an escaped JSON string in Go?
strconv.Unquote()
---

Converting Go struct to JSON

---
Custom MarshalJSON in GOLang - ashleyd.ws
Bad Go: Adventures with JSON marshalling - Phil Pearl - Medium


http


Making HTTP Requests in Golang - Abu Ashraf Masnun - Medium
---

io (input/output)

io


WARNING: Second io.Copy will not copy any data from file f to sha1 hasher! This is because reader (passed as second argument to io.Copy) advances to EOF during the first io.Copy call and stays there.

f, err := os.Open(path)
defer f.Close()

sha256Hash := sha256.New()
io.Copy(sha256Hash, f)
...
sha1Hash := sha1.New()
io.Copy(sha1Hash, f)

Fix: before calling io.Copy for the second time, we need to move reader pointer to the beginning of the file:

f.Seek(0, os.SEEK_SET)

or, in the latest Go version:

f.Seek(0, io.SeekStart)

---

log 


Why should I use log.Println instead of fmt.Println?

log.Println writes to standard error by default. To change it to stdout, use:

log.SetOutput(os.Stdout)

To set it back to stderr:

log.SetOutput(os.Stderr)


---

os (Operating System)


os.Create() truncates file if file already exists.

Truncate a file Truncate a file to a specific length.Shrink or extend the size of each FILE to the specified size.
How to empty (“truncate”) a file on linux that already exists and is protected in someway?
Word for “truncate to size zero”
Truncating a file means to eliminate all the content.In general, "truncating a file" means truncating it to a specific length, which may or may not be zero; the size defaults to zero if not specified. 

To get current working directory use:
os.Getwd()

How to create nested directories using Mkdir in Golang?

To create a single directory:

os.Mkdir

To create a folder path:

os.MkdirAll(folderPath, os.ModePerm)

Creating a relative symbolic link through the os package
os.Symlink(target, symlink)

Golang : Create and resolve(read) symbolic links

os.IsExist(err) vs os.IsNotExist(err)


---

path


Rather than using + to create paths, use:

import "path/filepath"
path := filepath.Join(someRootPath, someSubPath)

The above uses the correct separators automatically on each platform for you.


sync


WaitGroup

  • a structure which monitors current number of goroutines being executed
  • has 3 functions defined on it:
    • Add(delta int) - add a number of goroutines that are about to be created
    • Done() - decrements counter by 1
    • Wait()
      • blocks current thread until counter drops to 0 (all goroutines have returned)
      • introduces synchronisation points between goroutines
Application will panic if counter goes below 0 at any time:

panic: sync: negative WaitGroup counter

Add(1) has to be executed before Done() => call to Add should be done before launching the go routine containing the call to Done for  the call to Add after launching the go routine may not execute until the go routine completes, which will generate a runtime panic as the call to Done inside the routine makes the waitgroup negative.

Add(1) should not be executed from goroutine but before scheduling it. (?) [Is it safe to call WaitGroup.Add concurrently from multiple go routines?]

defer wg.Done() should be the first line in goroutine. 

Very basic concurrency for beginners in Go


template


There are two flavours:


go - Quoted string in templates - Stack Overflow

Templates - Go Web Examples
Simply output go html template execution to strings (Example)

Unit Testing

Package testing
Proper package naming for testing with the Go language
How to write benchmarks in Go
Go Benchmarks

go test command executes any function which is in this form:

func TestXxx(*testing.T)

go test -bench executes benchmarks (and also other tests unless -run flag specifies what has to be run) which must be in form:

func BenchmarkXxx(*testing.B)

To suppress running other (regular) unit tests, set the regex value of the -run parameter to something that does not match the name of any unit test:

$ go test -bench=. -run=NONE 

To  show how many allocation operations were performed per invocation and how much memory was allocated during each invocation, add -benchmem flag:

$ go test -bench=. -benchmem -run=NONE

or add to the beginning of the benchmark function:

b.ReportAllocs()

Test memory consumption
How can i limit/gate memory usage in go unit test
Practical Go Benchmarks

Example of -benchmem:
How much memory do golang maps reserve?

---

Profiling

dotGo 2019 - Daniel Martí - Optimizing Go code without a blindfold presentation shows how to use the following tools:

  • benchmark tests (-bench, -cpuprofile)
  • pprof
  • benchcmp
  • benchstat (from perf suite)
  • perflock
  • go compiler flags ( -gcflags)
  • GOSSAFUNC


You can try running benchmark tests already provided within the go source code available on your dev machine upon installing go:

/var/lib/go/src/encoding/json$ go test -bench=CodeDecoder
goos: linux
goarch: amd64
pkg: encoding/json
BenchmarkCodeDecoder-4         200   10837270 ns/op  179.06 MB/s
PASS
ok   encoding/json 3.332s

Results vary on each run. Variance quantifies how far are measurements from the mean value in average.
Variance Definition

Better benchcmp: https://godoc.org/golang.org/x/perf/cmd/benchstat
It is part of perf - Go performance measurement, storage, and analysis tools.
To install perf:

$ go get -u golang.org/x/perf/cmd/...

benchstat should now be available in terminal:

$ benchstat --help
usage: benchstat [options] old.txt [new.txt] [more.txt ...]
options:
  -alpha α
    consider change significant if p < α (default 0.05)
  -csv
    print results in CSV form
  -delta-test test
    significance test to apply to delta: utest, ttest, or none (default "utest")
  -geomean
    print the geometric mean of each file
  -html
    print results as an HTML table
  -norange
    suppress range columns (CSV only)
  -sort order
    sort by order: [-]delta, [-]name, none (default "none")
  -split labels
    split benchmarks by labels (default "pkg,goos,goarch")

This is the output of benchmark test:

/var/lib/go/src/encoding/json$ go test -bench=CodeDecoder -count=8
goos: linux
goarch: amd64
pkg: encoding/json
BenchmarkCodeDecoder-4         100   12851072 ns/op 151.00 MB/s
BenchmarkCodeDecoder-4         100   12327673 ns/op 157.41 MB/s
BenchmarkCodeDecoder-4         100   12171500 ns/op 159.43 MB/s
BenchmarkCodeDecoder-4         100   12609445 ns/op 153.89 MB/s
BenchmarkCodeDecoder-4         100   12769039 ns/op 151.97 MB/s
BenchmarkCodeDecoder-4         100   12903776 ns/op 150.38 MB/s
BenchmarkCodeDecoder-4         100   12563360 ns/op 154.45 MB/s
BenchmarkCodeDecoder-4         100   12523548 ns/op 154.95 MB/s
PASS
ok  encoding/json 20.097s

To use benchstat we need to run it twice and save results in files on disk:

$ cd ~
$ mkdir tmp
$ cd tmp
$ go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > old.txt
$ go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > new.txt
$ benchstat old.txt
name           time/op
CodeDecoder-4   11.1ms ± 4%

name           speed
CodeDecoder-4  175MB/s ± 4%

Variance is 4% which is still quite high if we want to optimize some algorithm in e.g. range 5-10%. 

Higher variance can be the consequence of the current/random CPU load/spikes caused by various apps running on the computer (Electron-based apps and GIFs are resource hungry and can throttle CPU). The first thing we have to make sure before running benchmarks is that test computer is idle. Benchmark demand 100% of CPU. If we close browsers, Slack, etc...variance can go down to 1%.

Nevertheless, if we increase number of benchmark loops:

$ go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=20

...CPU burns, laptop throttles, fans turn on and speed deteriorates. CPU speed (frequency) goes down as it can't keep up with overheating due to limited fan speed.

Solution for this is perflock. To install it and run the daemon:

$ go get github.com/aclements/perflock/cmd/perflock
$ cd $GOPATH/src/github.com/aclements/perflock
$ ./install.bash
Installing /home/user/dev/go/bin/perflock to /usr/bin
[sudo] password for user: 
Installing init script for Upstart
Installing service for systemd
Starting perflock daemon (using systemd)

Let's explore perflock CLI:

$ perflock --help
Usage of perflock:

  perflock [flags] command...
  perflock -list
  perflock -daemon

  -daemon
    start perflock daemon
  -governor percent
    set CPU frequency to percent between the min and max
    while running command, or "none" for no adjustment (default 90%)
  -list
    print current and pending commands
  -shared
    acquire lock in shared mode (default: exclusive mode)
  -socket path
    connect to socket path (default "/var/run/perflock.socket")

---
$ perflock -governor=70% go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8
goos: linux
goarch: amd64
pkg: encoding/json
BenchmarkCodeDecoder-4         100   13234601 ns/op 146.62 MB/s
BenchmarkCodeDecoder-4         100   13389018 ns/op 144.93 MB/s
BenchmarkCodeDecoder-4         100   13170943 ns/op 147.33 MB/s
BenchmarkCodeDecoder-4         100   13384738 ns/op 144.98 MB/s
BenchmarkCodeDecoder-4         100   13105316 ns/op 148.07 MB/s
BenchmarkCodeDecoder-4         100   13210405 ns/op 146.89 MB/s
BenchmarkCodeDecoder-4         100   13094757 ns/op 148.19 MB/s
BenchmarkCodeDecoder-4         100   13965663 ns/op 138.95 MB/s
PASS
ok  encoding/json 21.430s

$ perflock -governor=70% go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > old.txt

$ perflock -governor=70% go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > new.txt

$ benchstat old.txt new.txt
name           old time/op   new time/op   delta
CodeDecoder-4   13.4ms ± 5%   13.2ms ± 5%   ~     (p=0.645 n=8+8)

name           old speed     new speed     delta
CodeDecoder-4  145MB/s ± 5%  147MB/s ± 5%   ~     (p=0.645 n=8+8)

Larger variance => need larger N

Tools for CPU load: pprof, perflock
benchstat works for all benchmarks e.g. for measuring network latency or disk I/O
benchstat to compare statistics
perflock to avoid noise.

Compiler options

To get a full list of compiler options use:

$ go tool compile -help
usage: compile [options] file.go...
  -% debug non-static initializers
  -+ compiling runtime
  -B disable bounds checking
  -C disable printing of columns in error messages
  -D path
    set relative path for local imports
  -E debug symbol export
  -I directory
    add directory to import search path
  -K debug missing line numbers
  -L show full file names in error messages
  -N disable optimizations
  -S print assembly listing
  -V print version and exit
  -W debug parse tree after type checking
  -allabis
    generate ABI wrappers for all symbols (for bootstrap)
  -asmhdr file
    write assembly header to file
  -bench file
    append benchmark times to file
  -blockprofile file
    write block profile to file
  -buildid id
    record id as the build id in the export metadata
  -c int
    concurrency during compilation, 1 means no concurrency (default 1)
  -complete
    compiling complete package (no C or assembly)
  -cpuprofile file
    write cpu profile to file
  -d list
    print debug information about items in list; try -d help
  -dwarf
    generate DWARF symbols (default true)
  -dwarflocationlists
    add location lists to DWARF in optimized mode (default true)
  -dynlink
    support references to Go symbols defined in other shared libraries
  -e no limit on number of errors reported
  -gendwarfinl int
    generate DWARF inline info records (default 2)
  -goversion string
    required version of the runtime
  -h halt on error
  -importcfg file
    read import configuration from file
  -importmap definition
    add definition of the form source=actual to import map
  -installsuffix suffix
    set pkg directory suffix
  -j debug runtime-initialized variables
  -l disable inlining
  -lang string
    release to compile for
  -linkobj file
    write linker-specific object to file
  -live
    debug liveness analysis

  -m print optimization decisions
(e.g. escape analysis decisions)

  -memprofile file
    write memory profile to file
  -memprofilerate rate
    set runtime.MemProfileRate to rate
  -msan
    build code compatible with C/C++ memory sanitizer
  -mutexprofile file
    write mutex profile to file
  -nolocalimports
    reject local (relative) imports
  -o file
    write output to file
  -p path
    set expected package import path
  -pack
    write to file.a instead of file.o
  -r debug generated wrappers
  -race
    enable race detector
  -s warn about composite literals that can be simplified
  -shared
    generate code that can be linked into a shared library
  -std
    compiling standard library
  -symabis file
    read symbol ABIs from file
  -traceprofile file
    write an execution trace to file
  -trimpath prefix
    remove prefix from recorded source file paths
  -v increase debug verbosity
  -w debug type checking
  -wb
    enable write barrier (default true)
---

Repeat the flag value in order to increase the output verbosity. 
Use any of these two forms of passing flag values:

~/tmp$ go build -gcflags '-m -m' /var/lib/go/src/io

or

~/tmp$ go build -gcflags='-m -m' /var/lib/go/src/io
# io
/var/lib/go/src/io/pipe.go:73:27: (*pipe).CloseRead.func1 capturing by value: p (addr=false assign=false width=8)
/var/lib/go/src/io/pipe.go:112:27: (*pipe).CloseWrite.func1 capturing by value: p (addr=false assign=false width=8)
/var/lib/go/src/io/io.go:289:6: cannot inline WriteString: function too complex: cost 136 exceeds budget 80
/var/lib/go/src/io/io.go:304:6: cannot inline ReadAtLeast: unhandled op FOR
/var/lib/go/src/io/io.go:328:6: can inline ReadFull as: func(Reader, []byte) (int, error) { return ReadAtLeast(r, buf, len(buf)) }
/var/lib/go/src/io/io.go:430:6: can inline LimitReader as: func(Reader, int64) Reader { return &LimitedReader literal }
/var/lib/go/src/io/io.go:380:6: cannot inline copyBuffer: unhandled op FOR
/var/lib/go/src/io/io.go:363:6: can inline Copy as: func(Writer, Reader) (int64, error) { return copyBuffer(dst, src, nil) }
/var/lib/go/src/io/io.go:339:6: cannot inline CopyN: function too complex: cost 100 exceeds budget 80
/var/lib/go/src/io/io.go:340:38: inlining call to LimitReader func(Reader, int64) Reader { return &LimitedReader literal }
...

Example when to use -m and that gcflags can be also passed to go run (not just go build): Why does a pointer to a local variable escape to the heap?

$ go run -gcflags='-m -m' esc.go


bce = bounds check elimination; this is a runtime check that you're accessing slice within its bounds

~/tmp$ go build -gcflags=-d=ssa/check_bce/debug=1 /var/lib/go/src/io
# io
/var/lib/go/src/io/io.go:310:23: Found IsSliceInBounds
/var/lib/go/src/io/io.go:404:27: Found IsSliceInBounds
/var/lib/go/src/io/io.go:537:27: Found IsSliceInBounds
/var/lib/go/src/io/multi.go:30:18: Found IsInBounds
/var/lib/go/src/io/multi.go:31:27: Found IsSliceInBounds
/var/lib/go/src/io/multi.go:106:15: Found IsSliceInBounds
/var/lib/go/src/io/pipe.go:90:9: Found IsSliceInBounds

~/tmp$ go build -gcflags=-d=ssa/prove/debug=1 /var/lib/go/src/io
# io
/var/lib/go/src/io/io.go:404:27: Proved Geq64
/var/lib/go/src/io/io.go:446:8: Proved IsSliceInBounds
/var/lib/go/src/io/io.go:446:8: Proved Geq64
/var/lib/go/src/io/io.go:473:8: Proved IsSliceInBounds
/var/lib/go/src/io/io.go:507:8: Proved IsSliceInBounds
/var/lib/go/src/io/io.go:537:27: Proved Geq64
/var/lib/go/src/io/multi.go:21:27: Proved IsInBounds
/var/lib/go/src/io/multi.go:26:23: Proved IsInBounds
/var/lib/go/src/io/multi.go:61:10: Induction variable: limits [0,?), increment 1
/var/lib/go/src/io/multi.go:77:3: Induction variable: limits [0,?), increment 1
/var/lib/go/src/io/multi.go:105:17: Induction variable: limits [0,?), increment 1

~/tmp$ go build -gcflags=-d=ssa/prove/debug=2 /var/lib/go/src/io
# io
/var/lib/go/src/io/io.go:404:27: Proved Geq64 (v147)
/var/lib/go/src/io/io.go:446:8: Proved IsSliceInBounds (v46)
/var/lib/go/src/io/io.go:446:8: Proved Geq64 (v42)
/var/lib/go/src/io/io.go:473:8: Proved IsSliceInBounds (v49)
/var/lib/go/src/io/io.go:507:8: Proved IsSliceInBounds (v60)
/var/lib/go/src/io/io.go:537:27: Proved Geq64 (v35)
/var/lib/go/src/io/multi.go:21:27: Proved IsInBounds (v34)
/var/lib/go/src/io/multi.go:26:23: Proved IsInBounds (v72)
/var/lib/go/src/io/multi.go:59:14: x+d >= w; x:v24 b6 delta:1 w:0 d:signed
/var/lib/go/src/io/multi.go:61:10: Induction variable: limits [0,v15), increment 1 (v24)
/var/lib/go/src/io/multi.go:76:14: x+d >= w; x:v25 b6 delta:1 w:0 d:signed
/var/lib/go/src/io/multi.go:77:3: Induction variable: limits [0,v16), increment 1 (v25)
/var/lib/go/src/io/multi.go:104:14: x+d >= w; x:v30 b6 delta:1 w:0 d:signed
/var/lib/go/src/io/multi.go:105:17: Induction variable: limits [0,v8), increment 1 (v30)

Same benchmark can give different results upon code change simply for different compiled code organization (e.g. code alignments...).

To see how is compiler compiling specific function use:

$ GOSSAFUNC=pattern go build

Example:

~/tmp$ echo $'package p\n\nfunc HelloWorld(){\n\tprintln(\"Hello, world!\")\n}\n' > p.go

$ GOSSAFUNC=HelloWorld go build
# _/home/bojan/tmp
dumped SSA to ./ssa.html

$ chromium ./ssa.html 
---




Profile your golang benchmark with pprof
High Performance Go Workshop
An Overview of Go's Tooling


CPU Profiling


With the current version of Go it is possible to perform CPU profiling only for one package at a time.

$ go test \
-run=NONE \
-bench=. \
-cpuprofile=cpu.out \
"./internal/pkg/package_name/"

Example:

$ go test \
-run=NONE \
-bench=. \
-cpuprofile=cpu.out \
"./internal/pkg/datatypesdemo/"

This will create file cpu.out in the current/root directory.

To open cpu.out and enter interactive mode in pprof tool:

$ go tool pprof cpu.out

To check how much time it takes to execute parts of function that were called in benchmarks:

(pprof) list <function_name>

Example:

$ go tool pprof cpu.out
File: datatypesdemo.test
Type: cpu
Time: Jul 5, 2019 at 4:50pm (BST)
Duration: 1.70s, Total samples = 1.94s (113.98%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) list copyList
Total: 1.94s
ROUTINE ======================== github.com/BojanKomazec/go-demo/internal/pkg/datatypesdemo.copyList in /home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/internal/pkg/datatypesdemo/datatypesdemo.go
     210ms      1.31s (flat, cum) 67.53% of Total
         .          .     61:func copyList(in []string) []string {
         .          .     62:   var out []string
      80ms       80ms     63:   for _, s := range in {
     130ms      1.23s     64:           out = append(out, s)
         .          .     65:   }
         .          .     66:   return out
         .          .     67:}
(pprof)

To exit, type exit:

(pprof) exit

benchcmp tool automatically calculates differences (in %) in performances before and after code changes:

$ cd ../go/src/github.com/BojanKomazec/go-demo
$ go test -run=NONE -bench=. ./... > old.txt
$ go test -run=NONE -bench=. ./... > new.txt
$ benchcmp old.txt new.txt
benchmark                       old ns/op     new ns/op     delta
BenchmarkCopyList1_100x16-4     1400          1404          +0.29%
BenchmarkCopyList2_100x16-4     467           468           +0.21%

benchmark                       old allocs     new allocs     delta
BenchmarkCopyList1_100x16-4     8              8              +0.00%
BenchmarkCopyList2_100x16-4     1              1              +0.00%

benchmark                       old bytes     new bytes     delta
BenchmarkCopyList1_100x16-4     4080          4080          +0.00%
BenchmarkCopyList2_100x16-4     1792          1792          +0.00%

Go CLI



$ go
Go is a tool for managing Go source code.

Usage:

        go <command> [arguments]

The commands are:

        bug         start a bug report
        build       compile packages and dependencies
        clean       remove object files and cached files
        doc         show documentation for package or symbol
        env         print Go environment information
        fix         update packages to use new APIs
        fmt         gofmt (reformat) package sources
        generate    generate Go files by processing source
        get         download and install packages and dependencies
        install     compile and install packages and dependencies
        list        list packages or modules
        mod         module maintenance
        run         compile and run Go program
        test        test packages
        tool        run specified go tool
        version     print Go version
        vet         report likely mistakes in packages

Use "go help <command>" for more information about a command.

Additional help topics:

        buildmode   build modes
        c           calling between Go and C
        cache       build and test caching
        environment environment variables
        filetype    file types
        go.mod      the go.mod file
        gopath      GOPATH environment variable
        gopath-get  legacy GOPATH go get
        goproxy     module proxy protocol
        importpath  import path syntax
        modules     modules, module versions, and more
        module-get  module-aware go get
        packages    package lists and patterns
        testflag    testing flags
        testfunc    testing functions

Use "go help <topic>" for more information about that topic.


go build


$ go help build
usage: go build [-o output] [-i] [build flags] [packages]

Build compiles the packages named by the import paths,
along with their dependencies, but it does not install the results.

If the arguments to build are a list of .go files, build treats
them as a list of source files specifying a single package.

When compiling a single main package, build writes
the resulting executable to an output file named after
the first source file ('go build ed.go rx.go' writes 'ed' or 'ed.exe') or the source code directory ('go build unix/sam' writes 'sam' or 'sam.exe'). The '.exe' suffix is added when writing a Windows executable.

When compiling multiple packages or a single non-main package,
build compiles the packages but discards the resulting object,
serving only as a check that the packages can be built.

When compiling packages, build ignores files that end in '_test.go'.

The -o flag, only allowed when compiling a single package,
forces build to write the resulting executable or object
to the named output file, instead of the default behavior described
in the last two paragraphs.

The -i flag installs the packages that are dependencies of the target.

The build flags are shared by the build, clean, get, install, list, run, and test commands:

        -a
                force rebuilding of packages that are already up-to-date.
        -n
                print the commands but do not run them.
        -p n
                the number of programs, such as build commands or
                test binaries, that can be run in parallel.
                The default is the number of CPUs available.
        -race
                enable data race detection.
                Supported only on linux/amd64, freebsd/amd64, darwin/amd64 and windows/amd64.
        -msan
                enable interoperation with memory sanitizer.
                Supported only on linux/amd64, linux/arm64
                and only with Clang/LLVM as the host C compiler.
        -v
                print the names of packages as they are compiled.
        -work
                print the name of the temporary work directory and
                do not delete it when exiting.
        -x
                print the commands.

        -asmflags '[pattern=]arg list'
                arguments to pass on each go tool asm invocation.
        -buildmode mode
                build mode to use. See 'go help buildmode' for more.
        -compiler name
                name of compiler to use, as in runtime.Compiler (gccgo or gc).
        -gccgoflags '[pattern=]arg list'
                arguments to pass on each gccgo compiler/linker invocation.
        -gcflags '[pattern=]arg list'
                arguments to pass on each go tool compile invocation.
        -installsuffix suffix
                a suffix to use in the name of the package installation directory,
                in order to keep output separate from default builds.
                If using the -race flag, the install suffix is automatically set to race
                or, if set explicitly, has _race appended to it. Likewise for the -msan
                flag. Using a -buildmode option that requires non-default compile flags
                has a similar effect.
        -ldflags '[pattern=]arg list'
                arguments to pass on each go tool link invocation.
        -linkshared
                link against shared libraries previously created with
                -buildmode=shared.
        -mod mode
                module download mode to use: readonly or vendor.
                See 'go help modules' for more.
        -pkgdir dir
                install and load all packages from dir instead of the usual locations.
                For example, when building with a non-standard configuration,
                use -pkgdir to keep generated packages in a separate location.
        -tags 'tag list'
                a space-separated list of build tags to consider satisfied during the
                build. For more information about build tags, see the description of
                build constraints in the documentation for the go/build package.
        -toolexec 'cmd args'
                a program to use to invoke toolchain programs like vet and asm.
                For example, instead of running asm, the go command will run
                'cmd args /path/to/asm <arguments for asm>'.

The -asmflags, -gccgoflags, -gcflags, and -ldflags flags accept a
space-separated list of arguments to pass to an underlying tool
during the build. To embed spaces in an element in the list, surround it with either single or double quotes. The argument list may be receded by a package pattern and an equal sign, which restricts the use of that argument list to the building of packages matching that pattern (see 'go help packages' for a description of package patterns). Without a pattern, the argument list applies only to the packages named on the command line. The flags may be repeated
with different patterns in order to specify different arguments for
different sets of packages. If a package matches patterns given in
multiple flags, the latest match on the command line wins.
For example, 'go build -gcflags=-S fmt' prints the disassembly
only for package fmt, while 'go build -gcflags=all=-S fmt'
prints the disassembly for fmt and all its dependencies.

For more about specifying packages, see 'go help packages'.
For more about where packages and binaries are installed,
run 'go help gopath'.
For more about calling between Go and C/C++, run 'go help c'.

Note: Build adheres to certain conventions such as those described
by 'go help gopath'. Not all projects can follow these conventions,
however. Installations that have their own conventions or that use
a separate software build system may choose to use lower-level
invocations such as 'go tool compile' and 'go tool link' to avoid
some of the overheads and design decisions of the build tool.

See also: go install, go get, go clean.

Note that list of options goes BEFORE list of source code files.

$ go build -o build/wiki wiki.go 

To keep the flag package simple, the first non-flag is treated as the first command line argument, regardless of what might come after, so all flags must come before all regular arguments.

The go tool uses the flag package and so is subjected to the same limitations for it's built-in subcommands. Therefore the "output" flag must be set before the arguments (which contain the input).

example:
go build foo -o bar -> flags: {} args: [foo, -o, bar]
go build -o bar foo -> flags: {o: bar} args: [foo] 

[(How to) Give a name to the binary using go build]

go get



$ go help get
usage: go get [-d] [-f] [-t] [-u] [-v] [-fix] [-insecure] [build flags] [packages]

Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'.

The -d flag instructs get to stop after downloading the packages; that is, it instructs get not to install the packages.

The -f flag, valid only when -u is set, forces get -u not to verify that each package has been checked out from the source control repository implied by its import path. This can be useful if the source is a local fork of the original.

The -fix flag instructs get to run the fix tool on the downloaded packages before resolving dependencies or building the code.

The -insecure flag permits fetching from repositories and resolving
custom domains using insecure schemes such as HTTP. Use with caution.

The -t flag instructs get to also download the packages required to build the tests for the specified packages.

The -u flag instructs get to use the network to update the named packages and their dependencies. By default, get uses the network to check out missing packages but does not use it to look for updates to existing packages.

The -v flag enables verbose progress and debug output.

Get also accepts build flags to control the installation. See 'go help build'.

When checking out a new package, get creates the target directory
GOPATH/src/<import-path>. If the GOPATH contains multiple entries,
get uses the first one. For more details see: 'go help gopath'.

When checking out or updating a package, get looks for a branch or tag that matches the locally installed version of Go. The most important rule is that if the local installation is running version "go1", get searches for a branch or tag named "go1". If no such version exists it retrieves the default branch of the package.

When go get checks out or updates a Git repository,
it also updates any git submodules referenced by the repository.

Get never checks out or updates code stored in vendor directories.

For more about specifying packages, see 'go help packages'.

For more about how 'go get' finds source code to download, see 'go help importpath'.

This text describes the behavior of get when using GOPATH
to manage source code and dependencies. If instead the go command is running in module-aware mode, the details of get's flags and effects change, as does 'go help get'. See 'go help modules' and 'go help module-get'.

See also: go build, go install, go clean.

Example:

$ go get -d -v ./...

What do three dots “./…” mean in Go command line invocations?

./ tells to start from the current folder, ... tells to go down recursively.

Example:

$ go get -u golang.org/x/perf/cmd/...

$ go get github.com/aclements/perflock/cmd/perflock


go env


This is the best way to check which $GOPATH and $GOROOT values are used by any local Go tool.
Mind that these values can be different for current user and for root:

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/my_user_name/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/my_user_name/dev/go"
GOPROXY=""
GORACE=""
GOROOT="/var/lib/go"
GOTMPDIR=""
GOTOOLDIR="/var/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build480098893=/tmp/go-build -gno-record-gcc-switches"

For root:

$ sudo go env
...
GOPATH="/home/my_user_name/go"
...



go install



$ go help install
usage: go install [-i] [build flags] [packages]

Install compiles and installs the packages named by the import paths.

The -i flag installs the dependencies of the named packages as well.

For more about the build flags, see 'go help build'.
For more about specifying packages, see 'go help packages'.

See also: go build, go get, go clean.

It:

  • places the executable file in $GOPATH/bin
  • caches all non-main packages (which app imports) in $GOPATH/pkg
    • Cache is used in the next compilation unless it changes in the meantime

Go naming convention

What are conventions for filenames in Go?

File names that begin with "." or "_" are ignored by the go tool
Files with the suffix _test.go are only compiled and run by the go test tool.
Files with os and architecture specific suffixes automatically follow those same constraints, e.g. name_linux.go will only build on linux, name_amd64.go will only build on amd64. This is the same as having a //+build amd64 line at the top of the file

filenames are generally all lowercase in case, both for consistency and for systems with case-insensitive filesystems

regular file names are lower case, short, and without any sort of underscore or space. Generally, file names follow the same convention as package names.

there is no convention but _ suffixes may have special semantics in the future so I recommend to avoid them

Naming a file dns_windows.go will cause it to be included only when building the package for Windows; similarly, math_386.s will be included only when building the package for 32-bit x86.

File name convention for compound words?

Source file names are usually kept short, so in your case I would tend to call it find.go in package weightedunion <--  the idea is to make better use of packages when you are tempted to name it long


What is the best naming convention of keeping source files in a programming language?

Go follows a convention where source files are all lower case with underscore separating multiple words. Example:

https://github.com/kubernetes/client-go/tree/master/discovery/cached/diskGolang


folder structure and filename conventions under clean architecture

Interfaces in Go have a er-suffix (io.Reader, fmt.Stringer, etc).

The filenames are not per se named after a single type as often the package contains several types (see your first question).

Mixed case filenames are cute until your filesystem does not distinguish cases. Keep it simple, lowercase only.

Style guideline for Go packages


Go code Documentation & Comments


Godoc: documenting Go code

To document a type, variable, constant, function, or even a package, write a regular comment directly preceding its declaration, with no intervening blank line. Godoc will then present that comment as text alongside the item it documents.

Comment is a complete sentence that begins with the name of the element it describes.

The "note marker" needs to follow this syntax [source]:

MARKER(uid): note body

Then godoc -notes="MARKER" includes this note into the doc.

If we have in the code:

// TODO(id_manager): Add id range checking... 

And then run godoc comments parser as:


godoc -note="TODO" .

We'll see in the output:

PACKAGE DOCUMENTATION

package my_package
   ...
TODOS

   Add id range checking... 


Installing godoc 


On Ubuntu:

$ sudo apt install golang-golang-x-tools

Difference between godoc and go doc

godoc:

user@host:~/.../go/src/github.com/user/app$ godoc ./internal/pkg/pgclient/ 
PACKAGE DOCUMENTATION

package pgclient
    import "./internal/pkg/pgclient/"


FUNCTIONS

func New(params dbclient.ConnParams) (dbclient.DbClient, error)
    New function creates an instance of PostgreSQL client

go doc:

user@host:~/.../go/src/github.com/user/app$ go doc ./internal/pkg/pgclient/ 
package pgclient // import "github.com/user/app/internal/pkg/pgclient"

func New(params dbclient.ConnParams) (dbclient.DbClient, error)

---

Go & Docker


Deploying a Go Application in a Docker Container

Go & PostgreSQL


Go comes with embedded database/sql package.
The standard PostgreSQL DB driver for Go is lib/pq.

package pq

How to use PostgreSQL

Querying for a single record using Go's database/sql package

QueryRow() is a method provided by the DB type and is used to execute an SQL query that is expected to return a single row.


Useful packages


https://github.com/go-ozzo/ozzo-validation

CLI flags and arguments:



General purpose extensions to database/sql:
package sqlx - helper package that sits on top of database/sql and adds additional helpers, to help with binding results to Go
Illustrated guide to SQLX

Postgres DB driver:
https://godoc.org/github.com/jackc/pgx

Logging:

HTTP Client:

networking - Access HTTP response as string in Go - Stack Overflow

HTTP Server:


Golang: HTTP Serve static files correctly - DEV Community



Resources:


"Go in Action" by William Kennedy
Effective Go
jbuberel/go-patterns
Golang examples and solutions from different packages of the standard library. - golangprograms.com
#Go @ siongui.github.io
GoLang Tutorials
Common Mistakes
I Spy Code - Blog - golang (Programming puzzles)

1 comment:

Anonymous said...

very nicely explained a go language. you can also refer to this blog https://www.gowitek.com/golang/blog/go-programming-language for more details.