When importing 3rd party package (the one which is not part of the standard library), then import statement in the code is not enough, if we run the app we'll get an error similar to this:
$ go run ./cmd/main.go
internal/pkg/jsondemo/jsondemo.go:7:2: cannot find package "github.com/nsf/jsondiff" in any of:
/var/lib/go/src/github.com/nsf/jsondiff (from $GOROOT)
/home/bojan/dev/go/src/github.com/nsf/jsondiff (from $GOPATH)
We need to install that package on the local machine with:
$ go get <package_uri>
e.g.
$ go get -u github.com/nsf/jsondiff
See also:
Removing packages installed with go get
import the matchers package and use the blank identifier (_) before listing out the import path.This is a technique in Go to allow initialization from a package to occur, even if you
don’t directly use any identifiers from the package. To make your programs more
readable, the Go compiler won’t let you declare a package to be imported if it’s not
used. The blank identifier allows the compiler to accept the import and call any init
functions that can be found in the different code files within that package.
Is there an efficient way to share structure between golang packages?
$ go help get
usage: go get [-d] [-m] [-u] [-v] [-insecure] [build flags] [packages]
Get resolves and adds dependencies to the current development module
and then builds and installs them.
The first step is to resolve which dependencies to add.
For each named package or package pattern, get must decide which version of
the corresponding module to use. By default, get chooses the latest tagged
release version, such as v0.4.5 or v1.2.3. If there are no tagged release
versions, get chooses the latest tagged prerelease version, such as
v0.0.1-pre1. If there are no tagged versions at all, get chooses the latest
known commit.
This default version selection can be overridden by adding an @version
suffix to the package argument, as in 'go get golang.org/x/text@v0.3.0'.
For modules stored in source control repositories, the version suffix can
also be a commit hash, branch identifier, or other syntax known to the
source control system, as in 'go get golang.org/x/text@master'.
The version suffix @latest explicitly requests the default behavior
described above.
If a module under consideration is already a dependency of the current
development module, then get will update the required version.
Specifying a version earlier than the current required version is valid and
downgrades the dependency. The version suffix @none indicates that the
dependency should be removed entirely, downgrading or removing modules
depending on it as needed.
Although get defaults to using the latest version of the module containing
a named package, it does not use the latest version of that module's
dependencies. Instead it prefers to use the specific dependency versions
requested by that module. For example, if the latest A requires module
B v1.2.3, while B v1.2.4 and v1.3.1 are also available, then 'go get A'
will use the latest A but then use B v1.2.3, as requested by A. (If there
are competing requirements for a particular module, then 'go get' resolves
those requirements by taking the maximum requested version.)
The -u flag instructs get to update dependencies to use newer minor or
patch releases when available. Continuing the previous example,
'go get -u A' will use the latest A with B v1.3.1 (not B v1.2.3).
The -u=patch flag (not -u patch) instructs get to update dependencies
to use newer patch releases when available. Continuing the previous example,
'go get -u=patch A' will use the latest A with B v1.2.4 (not B v1.2.3).
In general, adding a new dependency may require upgrading
existing dependencies to keep a working build, and 'go get' does
this automatically. Similarly, downgrading one dependency may
require downgrading other dependencies, and 'go get' does
this automatically as well.
The -m flag instructs get to stop here, after resolving, upgrading,
and downgrading modules and updating go.mod. When using -m,
each specified package path must be a module path as well,
not the import path of a package below the module root.
The -insecure flag permits fetching from repositories and resolving
custom domains using insecure schemes such as HTTP. Use with caution.
The second step is to download (if needed), build, and install
the named packages.
If an argument names a module but not a package (because there is no
Go source code in the module's root directory), then the install step
is skipped for that argument, instead of causing a build failure.
For example 'go get golang.org/x/perf' succeeds even though there
is no code corresponding to that import path.
Note that package patterns are allowed and are expanded after resolving
the module versions. For example, 'go get golang.org/x/perf/cmd/...'
adds the latest golang.org/x/perf and then installs the commands in that
latest version.
The -d flag instructs get to download the source code needed to build
the named packages, including downloading necessary dependencies,
but not to build and install them.
With no package arguments, 'go get' applies to the main module,
and to the Go package in the current directory, if any. In particular,
'go get -u' and 'go get -u=patch' update all the dependencies of the
main module. With no package arguments and also without -u,
'go get' is not much more than 'go install', and 'go get -d' not much
more than 'go list'.
For more about modules, see 'go help modules'.
For more about specifying packages, see 'go help packages'.
This text describes the behavior of get using modules to manage source
code and dependencies. If instead the go command is running in GOPATH
mode, the details of get's flags and effects change, as does 'go help get'.
See 'go help modules' and 'go help gopath-get'.
See also: go build, go install, go clean, go mod.
Go Modules
The Go Blog - Using Go Modules
The Go Blog - Migrating to Go Modules
Go modules were introduced in order to prevent having to change import path (of dependencies )on breaking changes.
A module is a collection of Go packages stored in a file tree with a
go.mod file at its root.
The
go.mod file contains:
- module directive which defines module’s path which is also the import path used for the root directory (e.g. this path is the base path upon which paths to other packages in the same project are built)
- go directive which declares the expected version of the Go language used to compile the code within the module.
- its dependency requirements, which are the other modules needed for a successful build
To use Go modules, project directory has to be outside
$GOPATH/src otherwise the go command still runs in the old
GOPATH mode, even if a
go.mod is found.
Packages in subdirectories have import paths consisting of the module path plus the path to the subdirectory. For example, if we created a subdirectory world, we would not need to (nor want to) run go mod init there. The package would automatically be recognized as part of the example.com/hello module, with import path example.com/hello/world.
The go command resolves imports by using the specific dependency module versions listed in
go.mod.
Only direct dependencies are recorded in the
go.mod file.
Running go command again will not repeat module detection and downloading, since the
go.mod is now up-to-date and the downloaded modules are cached locally (in
$GOPATH/pkg/mod).
command
go list -m all lists the current module and all its dependencies. In the
go list output, the
current module, also known as the
main module, is always the first line, followed by dependencies sorted by module path.
---
Steps I did in order to migrate my app (which didn't use any package manager before) to modules:
• moved project's directory from
$GOPATH/src/example.com/myapp/ to
./wherever/myapp/
• verified that go build fails with error:
cannot find package "example.com/myapp/internal/pkg/mypkg1" in any of:
/var/lib/go/src/example.com/myapp/internal/pkg/mypkg1 (from $GOROOT)
/home/bojan/dev/go/src/example.com/myapp/internal/pkg/mypkg1 (from $GOPATH)
• created go.mod by executing:
$ go mod init example.com/myapp
Another example:
$ go mod init github.com/BojanKomazec/go-demo
It's content was at that point:
$ cat go.mod
module example.com/myapp
go 1.12
• added to go.mod the list of all dependencies (this list does not contain their dependencies) and created go.sum:
$ go mod tidy
...
go: finding github.com/onsi/gomega v1.7.0
go: finding github.com/onsi/ginkgo v1.10.2
go: finding github.com/joho/godotenv v1.3.0
go: finding github.com/jmoiron/sqlx v1.2.0
go: finding github.com/lib/pq v1.2.0
go: finding github.com/onsi/ginkgo/extensions/table latest
go: finding github.com/google/uuid v1.1.1
go: downloading github.com/joho/godotenv v1.3.0
go: extracting github.com/joho/godotenv v1.3.0
...
go: downloading github.com/golang/protobuf v1.3.1
go: extracting github.com/golang/protobuf v1.3.1
go: extracting golang.org/x/net v0.0.0-20190603091049-60506f45cf65 // <-- pseudo-version, which is the go command's version syntax for a specific untagged commit.
go: downloading golang.org/x/text v0.3.2
go: extracting golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: extracting golang.org/x/text v0.3.2
go mod tidy added module requirements for all the packages transitively imported by packages in your module and built a go.sum with checksums for each library at a specific version.
After this go.mod got require section:
$ cat go.mod
module example.com/myapp
go 1.12
require (
github.com/google/uuid v1.1.1
github.com/jmoiron/sqlx v1.2.0
github.com/joho/godotenv v1.3.0
github.com/lib/pq v1.2.0
github.com/onsi/ginkgo v1.10.2
github.com/onsi/gomega v1.7.0
google.golang.org/appengine v1.6.5 // indirect
)
go.sum contains the expected cryptographic hashes of the content of specific module versions and is used to ensure that future downloads of these modules retrieve the same bits as the first download:
$ cat go.sum
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/go-sql-driver/mysql v1.4.0 h1:7LxgVwFb2hIQtMm87NdgAVfXjnt4OePseqT1tKx+opk=
github.com/go-sql-driver/mysql v1.4.0/go.mod ...
• I verified that import paths for the internal packages were based on the path assigned to module directive in go.mod. E.g. if we have project structure as:
cmd/main.go
internal/pkg/mypkg1
internal/pkg/mypkg1.go
internal/pkg/mypkg1_test.go
Then to import package mypkg1 in main.go we need to use:
import "example.com/myapp/internal/pkg/mypkg1"
When go mod tidy adds a requirement, it adds the latest version of the module. If your GOPATH included an older version of a dependency that subsequently published a breaking change, you may see errors in go mod tidy, go build, or go test.
After this my Go project was again building successfully.
SWITCHING TO GO MODULES
Switching to Go Modules - bye GOPATH and Godep
Anatomy of Modules in Go
Organize local code in packages using Go modules
Accessing local packages within a go module (go 1.11)---
call to the Run function that belongs to the search package:
search.Run("president")
The
log package provides support for logging messages to the stdout , stderr , or even
custom devices. The
sync package provides support for synchronizing goroutines,
var matchers = make(map[string]Matcher)
This variable is located outside the scope of any function and so is considered a
package-level variable. The variable is declared using the keyword
var and is declared
as a map of Matcher type values with a key of type string .
The name of the variable matchers starts with a lowercase letter.
In Go, identifiers are either exported or unexported from a package.
- An exported identifier can be directly accessed by code in other packages when the respective package is imported. These identifiers start with a capital letter.
- Unexported identifiers start with a lowercase letter and can’t be directly accessed by code in other packages. But just because an identifier is unexported, it doesn’t mean other packages can’t indirectly access these identifiers. As an example, a function can return a value of an unexported type and this value is accessible by any calling function, even if the calling function has been declared in a different package.
This variable declaration also contains an initialization of the variable via the
assignment operator and a special built-in function called
make .
A map is a reference type that you’re required to make in Go. If you don’t make the
map first and assign it to your variable, you’ll receive errors when you try to use the map
variable. This is because the zero value for a map variable is nil .
In Go, all variables are initialized to their zero value. For numeric types, that value
is 0 ; for strings it’s an empty string; for Booleans it’s false ; and for pointers, the zero
value is nil . When it comes to reference types, there are underlying data structures
that are initialized to their zero values. But variables declared as a reference type set to
their zero value will return the value of nil.
Go programs can be structured to handle the launching and synchronization of goroutines that run concurrently
the short variable declaration operator ( := ). This operator is
used to both declare and initialize variables at the same time. The type of each value
being returned is used by the compiler to determine the type for each variable,
respectively. The short variable declaration operator is just a shortcut to streamline
your code and make the code more readable. The variable it declares is no different
than any other variable you may declare when using the keyword var .
results := make(chan *Result)
- = assigns the right to the left
- := creates a new variable named the left, and assigns it the value of the item on the right
Short Variable Declaration Operator(:=) in Go - GeeksforGeeks
we use the built-in function
make to create an unbuffered channel. We use
the short variable declaration operator to declare and initialize the channel variable
with the call to make . A good rule of thumb when declaring variables is to use the key-
word var when declaring variables that will be initialized to their zero value, and to
use the short variable declaration operator when you’re providing extra initialization
or making a function call.
Channels are also a reference type in Go like maps and slices, but channels imple-
ment a queue of typed values that are used to communicate data between goroutines.
Channels provide inherent synchronization mechanisms to make communication
safe.
The next two lines of code are used later to prevent the program from terminating
before all the search processing is complete.
// Setup a wait group so we can process all the feeds.
var waitGroup sync.WaitGroup
// Set the number of goroutines we need to wait for while
// they process the individual feeds.
waitGroup.Add(len(feeds))
In Go, once the
main function returns, the program terminates. Any goroutines that
were launched and are still running at this time will also be terminated by the Go run-
time. When you write concurrent programs, it’s best to cleanly terminate any gorou-
tines that were launched prior to letting the main function return. Writing programs
that can cleanly start and shut down helps reduce bugs and prevents resources from
corruption.
Our program is using a
WaitGroup from the
sync package to track all the goroutines we’re going to launch. A WaitGroup is a great way to track when a goroutine is
finished performing its work. A WaitGroup is a counting semaphore, and we’ll use it to
count off goroutines as they finish their work.
On line 23 we declare a variable of type WaitGroup from the sync package. Then
on line 27 we set the value of the WaitGroup variable to match the number of gorou-
tines we’re going to launch. As you’ll soon see, we’ll process each feed concurrently
with its own goroutine. As each goroutine completes its work, it will decrement the
count of the WaitGroup variable, and once the variable gets to zero, we’ll know all the
work is done.
for _, item := range items {
...
}
range (keyword)
- can be used with arrays, strings, slices, maps, and channels
for range
- used to iterate over the slice of items
- When used to iterate over a slice, we get two values back on each iteration:
- index position of the element we’re iterating over
- a copy of the value in that element
blank identifier (_)
- used as a substitution for the variable that would be assigned to the index value for the range call. When you have a function that returns multiple values, and you don’t have a need for one, you can use the blank identifier to ignore those values. In our case with this range, we won’t be using the index value, so the blank identifier allows us to ignore it.
matcher, exists := matchers[feed.Type]
if !exists {
matcher = matchers["default"]
}
we check the map for a key that matches the feed type. When looking up a key in a map, you have two options: you can assign a single variable or two variables for the lookup call. The first
variable is always the value returned for the key lookup, and the second value, if specified, is a Boolean flag that reports whether the key exists or not. When a key doesn’t
exist, the map will return the zero value for the type of value being stored in the map.
When the key does exist, the map will return a copy of the value for that key.
---
Variables
Variable names can't start with number.
Variable names can't be the same as names of imported packages.
---
Constants
- cannot be declared using the := syntax
const Pi = 3.14
Tour of Go - Constants
The Go Blog - Constants
"Hello, 世界" is
untyped string constant.
It remains an untyped string constant even when given a name:
const hello = "Hello, 世界"
hello is also an untyped string constant.
An untyped constant is just a value, one not yet given a defined type that would force it to obey the strict rules that prevent combining differently typed values.
typed string constant is one that's been given a type:
const typedHello string = "Hello, 世界"
---
Logical Operators
Operands in logical expressions are evaluated lazily - only if needed:
A && B <==> If A then B else FALSE
A || B <==> If A then TRUE else B
---
Data Types
How to find a type of an object in Go?
import "reflect"
tst := "string"
fmt.Println(reflect.TypeOf(tst))
A Tour of Go - Type switches
switch v := i.(type) {
case T:
// here v has type T
case S:
// here v has type S
default:
// no match; here v has the same type as i
}
Type assertions
x.(T)
Type casting (Type conversions)
string(x)
---
type is reserved word in Go and can't be used as e.g. struct field name (Go linter issues error:
syntax error: unexpected type, expecting field name or embedded type)
What (exactly) does the type keyword do in go?
---
Where to declare custom types?
Declare (custom) type just before the place you need it. That does not have to be at the beginning of the file, but e.g. your custom struct can be declared just before the set of functions that are using it. Example:
https://golang.org/src/net/http/client.go
Golang - Code organisation with structs, variables and interfaces
All the files are based on features, and it is best to use a proximity principle approach, where you can find in the same file the definition of what you are using.
Generally, those features are grouped in one file per package, except for large ones, where one package is composed of many files (net, net/http)
If you want to separate anything, separate the source (xxx.go) from the tests/benchmarks (xxx_test.go)
iota is predeclared untyped integer identifier which resets to 0 at each constant declaration and its value is incremented by 1 for each constant withing that declaration.
type Day int
const (
MONDAY Day = 1 + iota
TUESDAY
WEDNESDAY
THURSDAY
FRIDAY
SATURDAY
SUNDAY
)
The empty interface
interface{}
- interface type that specifies zero methods
- may hold values of any type
- used by code that handles values of unknown type
var i interface{}
i = 42
i = "hello"
A Tour of Go - The empty interface
package main
import (
"fmt"
)
func main() {
var i interface{}
var j interface{} = 2
var k *interface{}
i = &j
k = &j
fmt.Println(i)
// invalid indirect of i (type interface {})
// fmt.Println(*i)
fmt.Println(*k)
}
Output:
0x40c130
2
Array
- collection of elements of a single type
- a fixed-length data type that contains a contiguous block of elements of the same type
var intArray [5]int
intArray[0] = 10
var intArray = [5]int {10, 20, 30}
var intArray = [5]int {0:10, 2:30, 4:50}
intArray := [5]int {10, 20, 30, 40, 50}
intArray := [...]int {10, 20, 30, 40, 50}
Create array of array literal in Golang
a := [][]int{{1,2,3},{4,5,6}}
Slice
- a segment of dynamic arrays that can grow and shrink
- indexable and have a length
- used as a dynamic array - when we want to use an array but don't know its size in advance
- can be created with built-in function make. make has one mandatory (type) and two optional arguments (length and capacity)
Make distinction between
nil slice:
var numbers []int
...and
empty slice:
numbers := make([]int, 0)
which is the same as using empty slice literal:
numbers := []int{}
Both have zero length and capacity but empty slice has allocated underlying zero-length array.
Go: Empty slice vs. nil slice | Programming.Guide
arrays - Correct way to initialize empty slice - Stack Overflow
If slice will be JSON encoded then prefer using empty slice (as it would be encoded into JSON empty array) otherwise prefer nil slice (which would be JSON encoded into
null).
To add elements to slice:
numbers = append(numbers, 1)
numbers = append(numbers, 2)
fmt.Println(len(numbers)) // == 2
Note this:
slc := make([]string, len(in)) // creates a slice which already contains 3 elements (empty strings)
slc := make([]string, 0, len(in)) // creates a slice which contains 0 but has capacity to hold 3 elements (3 strings)
Creating a slice with make
Go Slices: usage and internals
The Go Blog - Arrays, slices (and strings): The mechanics of 'append'
Declare slice or make slice?
var s []int
- simple declaration
- crates so called "nil slice"
- does not allocate memory
- s points to nil
- should not be a return value of an API which returns slice (an empty slice should be returned)
- marshaling the nil slice (var s []int) will produce null
s := make([]int, 0)
- creates so called "empty slice"
- allocates memory
- s points to memory to a slice with 0 elements
- should be returned if an API needs to return a slice with 0 elements
- marshalling the empty slice will produce the expected []
Convert slice of string to slice of pointer to string
Convert slice of type A to slice of pointers to type A
This:
for _, v := range a {
b = append(b, &v)
}
and this:
for i := 0; i < len(a); i++ {
b = append(b, &a[i])
}
are not the same. In the latter, you are taking the address of element a[i] in the array (which is what you want). In the former, you are taking the address of the iteration variable v, which, as you discovered, does not change during the loop execution (it gets allocated once, it's in a completely new location in memory, and gets overwritten at each iteration).
BK: We can still use range:
for i := range a {
b = append(b, &a[i])
}
---
How to declare a slice of slices (2-D slice)?
slice literal is written as
[]type{<value 1>, <value 2>, ... }.
A slice of ints would be
[]int{1,2,3}
A slice of int slices would be
[][]int{{1,2,3}, {4,5,6}}
var s2d [][]string = [][]string{{"a", "b", "c"}, {"d", "e", "f"}}
Try it in The Go Playground
---
Maps
Note that it is not possible to get the address (use & operator or get the pointer) of the map element.
myMap["keyName"] is called
map index expression.
Why do I get a “cannot assign” error when setting value to a struct as a value in a map?
Interfaces
Struct implements interface if it implements all its methods.
pinger/pinger.go:
package pinger
type (
PingerConfig struct {
timeout int
}
Pinger interface {
Ping(ip string) error
}
// New is a Pinger factory method
// New func(pingerConfig PingerConfig) (Pinger, error)
)
superPinger/pinger.go:
package superpinger
import (
"example.com/myproduct/myproject/internal/pkg/pinger"
)
type superPinger struct {
id string
}
// New function creates an instance of PostgreSQL client
func New(config pinger.PingerConfig) (pinger.Pinger, error) {
superPinger := superPinger(config)
err := superPinger.Init(...)
return superPinger, err
}
// Ping function verifies remote endpoint accessibility
func (superPinger superPinger) Ping(ip string) error {
err := doPing(ip)
return err
}
main.go:
import (
"example.com/myproduct/myproject/internal/pkg/pinger"
"example.com/myproduct/myproject/internal/pkg/superpinger"
)
...
pingerConfig := pinger.NewPingerConfig(...)
pinger, err := superpinger.New(pingerConfig)
pinger.Ping("8.8.8.8")
...
How to pass an interface to function: by value or pointer?
What is correct:
func CalculateSHA(h hash.Hash, file *os.File)
or
func CalculateSHA(h *hash.Hash, file *os.File) ([]byte, error) {
(hash.Hash is an interface)
Interfaces and Pointers to Interfaces
you almost never need a pointer to an interface, since interfaces are just pointers themselves
If I was doing a code review and saw that you were passing the address of an interface value to a function, it would raise a flag. An interface value is designed to be copied. An interface value is a reference type, just like slices, maps, channels and function variables.
Have the pointer inside the interface if possible, rather than as a pointer to interface.
can't use pointers to interface types?
> func Foo(r io.Reader)
>
> Why is this? Won't this copy the thing?
interface{} is just 2 words, copying it is very cheap.
interface copy does not copy the stored object, interface assignment can
Function calls in Go (like most other languages) is 'pass by value'. The values passed to the function are copied on to the stack for the function to access. In languages without complex values type (i.e. structs) these values are either primitive values (ints, floats etc) or they are references(pointers) to objects.
Interface values in Go are the size of two pointers, one pointer to type information and the other is a pointer to the value currently stored in the interface value. eg. an io.Reader that you've assigned an *os.File to will contain that *os.File pointer and a pointer to the *os.File type information.
Passing interface values to a function copies the interface value.
It generally doesn't make sense to declare a function that takes a pointer to an interface, e.g. *io.Writer. It's not very useful - io.Writer itself might *contain* a pointer (as in your p2) example and all is fine.
Go Data Structures: Interfaces
Generics (Templates)
They do
NOT exist in Go!
That's why you'll need to write many functions from scratch...like here:
What is the correct way to find the min between two integers in Go?
Polymorphism
Part 28: Polymorphism - OOP in Go:
A variable of type interface can hold any value which implements the interface. This property of interfaces is used to achieve polymorphism in Go.
---
For Loop
for i :=0; i<10; i++ {
fmt.Println("i =", i)
}
Switch
There is one important difference between switch statement in Go and in other languages (like C, C++, C#, JS) and that is a keyword
fallthrough which has to be used if we want to use same case handler for multiple cases:
v := 1
switch v {
case 1:
fallthrough
case 2:
fmt.Println("1 or 2")
break
default:
fmt.Println("default")
}
// output: 1 or 2
Without
fallthrough in case 1, this snippet would not print anything.
Error Creation and Handling
3 simple ways to create an error
If function returns type
error, it can return
nil if there is no error.
Encryption
Concurrency & Parallelism
How can my Go program keep all the CPU cores busy?
How to find the number of CPUs?
logicalCPUMaxCount := runtime.NumCPU()
---
Packages
If we have:
import "mypackage"
then we need to use package name when calling any its exported member:
mypackage.Foo()
It is possible to assign an alias name to the imported package. Example:
import b64 "encoding/base64"
...
es := b64.StdEncoding.EncodeToString([]byte(data))
Internal Packages
https://notes.shichao.io/gopl/ch10/#internal-packages
How to import local packages in go?
Go starting path for import is
$HOME/go/src.
encoding/base64
Go by Example: Base64 Encoding
encoding/json
Parsing JSON in Golang
the Species attribute in our Bird struct will map to the species, or Species or sPeCiEs JSON property.
Go by Example: JSON
When defining a structure into which we want to unmarshal some JSON, we have to declare struct members as exported (their names have to start with capital letter). This is required so other package (json) can use reflection and access these fields.
JSON and dealing with unexported fields
The json library does not have the power to view fields using reflect unless they are exported. A package can only view the unexported fields of types within its own package.
It's fundamentally limited by the reflect package that any package can't set unexported fields in types
defined in other packages (without using unsafe.)
---
How to unmarshal an escaped JSON string in Go?
strconv.Unquote()
---
Converting Go struct to JSON
---
Custom MarshalJSON in GOLang - ashleyd.ws
Bad Go: Adventures with JSON marshalling - Phil Pearl - Medium
http
Making HTTP Requests in Golang - Abu Ashraf Masnun - Medium
---
io (input/output)
io
WARNING: Second
io.Copy will not copy any data from file f to sha1 hasher! This is because reader (passed as second argument to
io.Copy) advances to
EOF during the first
io.Copy call and stays there.
f, err := os.Open(path)
defer f.Close()
sha256Hash := sha256.New()
io.Copy(sha256Hash, f)
...
sha1Hash := sha1.New()
io.Copy(sha1Hash, f)
Fix: before calling
io.Copy for the second time, we need to move reader pointer to the beginning of the file:
f.Seek(0, os.SEEK_SET)
or, in the latest Go version:
f.Seek(0, io.SeekStart)
---
log
Why should I use log.Println instead of fmt.Println?
log.Println writes to standard error by default. To change it to stdout, use:
log.SetOutput(os.Stdout)
To set it back to stderr:
log.SetOutput(os.Stderr)
---
os (Operating System)
os.Create() truncates file if file already exists.
Truncate a file Truncate a file to a specific length.Shrink or extend the size of each FILE to the specified size.
How to empty (“truncate”) a file on linux that already exists and is protected in someway?
Word for “truncate to size zero”
Truncating a file means to eliminate all the content.In general, "truncating a file" means truncating it to a specific length, which may or may not be zero; the size defaults to zero if not specified.
To get current working directory use:
os.Getwd()
How to create nested directories using Mkdir in Golang?
To create a single directory:
os.Mkdir
To create a folder path:
os.MkdirAll(folderPath, os.ModePerm)
Creating a relative symbolic link through the os package
os.Symlink(target, symlink)
Golang : Create and resolve(read) symbolic links
os.IsExist(err) vs os.IsNotExist(err)
---
path
Rather than using + to create paths, use:
import "path/filepath"
path := filepath.Join(someRootPath, someSubPath)
The above uses the correct separators automatically on each platform for you.
sync
WaitGroup
- a structure which monitors current number of goroutines being executed
- has 3 functions defined on it:
- Add(delta int) - add a number of goroutines that are about to be created
- Done() - decrements counter by 1
- Wait()
- blocks current thread until counter drops to 0 (all goroutines have returned)
- introduces synchronisation points between goroutines
Application will panic if counter goes below 0 at any time:
panic: sync: negative WaitGroup counter
Add(1) has to be executed before Done() => call to Add should be done before launching the go routine containing the call to Done for the call to Add after launching the go routine may not execute until the go routine completes, which will generate a runtime panic as the call to Done inside the routine makes the waitgroup negative.
defer wg.Done() should be the first line in goroutine.
Very basic concurrency for beginners in Go
template
There are two flavours:
- html/template
- text/template
go - Quoted string in templates - Stack Overflow
Templates - Go Web Examples
Simply output go html template execution to strings (Example)
Unit Testing
Package testing
Proper package naming for testing with the Go language
How to write benchmarks in Go
Go Benchmarks
go test command executes any function which is in this form:
func TestXxx(*testing.T)
go test -bench executes benchmarks (and also other tests unless
-run flag specifies what has to be run) which must be in form:
func BenchmarkXxx(*testing.B)
To suppress running other (regular) unit tests, set the regex value of the -run parameter to something that does not match the name of any unit test:
$ go test -bench=. -run=NONE
To show how many allocation operations were performed per invocation and how much memory was allocated during each invocation, add -benchmem flag:
$ go test -bench=. -benchmem -run=NONE
or add to the beginning of the benchmark function:
b.ReportAllocs()
Test memory consumption
How can i limit/gate memory usage in go unit test
Practical Go Benchmarks
Example of -benchmem:
How much memory do golang maps reserve?
---
Profiling
dotGo 2019 - Daniel Martà - Optimizing Go code without a blindfold presentation shows how to use the following tools:
- benchmark tests (-bench, -cpuprofile)
- pprof
- benchcmp
- benchstat (from perf suite)
- perflock
- go compiler flags ( -gcflags)
- GOSSAFUNC
You can try running benchmark tests already provided within the go source code available on your dev machine upon installing go:
/var/lib/go/src/encoding/json$ go test -bench=CodeDecoder
goos: linux
goarch: amd64
pkg: encoding/json
BenchmarkCodeDecoder-4 200 10837270 ns/op 179.06 MB/s
PASS
ok encoding/json 3.332s
Results vary on each run. Variance quantifies how far are measurements from the mean value in average.
Variance Definition
Better benchcmp: https://godoc.org/golang.org/x/perf/cmd/benchstat
It is part of
perf - Go performance measurement, storage, and analysis tools.
To install perf:
$ go get -u golang.org/x/perf/cmd/...
benchstat should now be available in terminal:
$ benchstat --help
usage: benchstat [options] old.txt [new.txt] [more.txt ...]
options:
-alpha α
consider change significant if p < α (default 0.05)
-csv
print results in CSV form
-delta-test test
significance test to apply to delta: utest, ttest, or none (default "utest")
-geomean
print the geometric mean of each file
-html
print results as an HTML table
-norange
suppress range columns (CSV only)
-sort order
sort by order: [-]delta, [-]name, none (default "none")
-split labels
split benchmarks by labels (default "pkg,goos,goarch")
This is the output of benchmark test:
/var/lib/go/src/encoding/json$ go test -bench=CodeDecoder -count=8
goos: linux
goarch: amd64
pkg: encoding/json
BenchmarkCodeDecoder-4 100 12851072 ns/op 151.00 MB/s
BenchmarkCodeDecoder-4 100 12327673 ns/op 157.41 MB/s
BenchmarkCodeDecoder-4 100 12171500 ns/op 159.43 MB/s
BenchmarkCodeDecoder-4 100 12609445 ns/op 153.89 MB/s
BenchmarkCodeDecoder-4 100 12769039 ns/op 151.97 MB/s
BenchmarkCodeDecoder-4 100 12903776 ns/op 150.38 MB/s
BenchmarkCodeDecoder-4 100 12563360 ns/op 154.45 MB/s
BenchmarkCodeDecoder-4 100 12523548 ns/op 154.95 MB/s
PASS
ok encoding/json 20.097s
To use benchstat we need to run it twice and save results in files on disk:
$ cd ~
$ mkdir tmp
$ cd tmp
$ go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > old.txt
$ go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > new.txt
$ benchstat old.txt
name time/op
CodeDecoder-4 11.1ms ± 4%
name speed
CodeDecoder-4 175MB/s ± 4%
Variance is 4% which is still quite high if we want to optimize some algorithm in e.g. range 5-10%.
Higher variance can be the consequence of the current/random CPU load/spikes caused by various apps running on the computer (Electron-based apps and GIFs are resource hungry and can throttle CPU). The first thing we have to make sure before running benchmarks is that test computer is idle. Benchmark demand 100% of CPU. If we close browsers, Slack, etc...variance can go down to 1%.
Nevertheless, if we increase number of benchmark loops:
$ go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=20
...CPU burns, laptop throttles, fans turn on and speed deteriorates. CPU speed (frequency) goes down as it can't keep up with overheating due to limited fan speed.
Solution for this is
perflock. To install it and run the daemon:
$ go get github.com/aclements/perflock/cmd/perflock
$ cd $GOPATH/src/github.com/aclements/perflock
$ ./install.bash
Installing /home/user/dev/go/bin/perflock to /usr/bin
[sudo] password for user:
Installing init script for Upstart
Installing service for systemd
Starting perflock daemon (using systemd)
Let's explore perflock CLI:
$ perflock --help
Usage of perflock:
perflock [flags] command...
perflock -list
perflock -daemon
-daemon
start perflock daemon
-governor percent
set CPU frequency to percent between the min and max
while running command, or "none" for no adjustment (default 90%)
-list
print current and pending commands
-shared
acquire lock in shared mode (default: exclusive mode)
-socket path
connect to socket path (default "/var/run/perflock.socket")
---
$ perflock -governor=70% go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8
goos: linux
goarch: amd64
pkg: encoding/json
BenchmarkCodeDecoder-4 100 13234601 ns/op 146.62 MB/s
BenchmarkCodeDecoder-4 100 13389018 ns/op 144.93 MB/s
BenchmarkCodeDecoder-4 100 13170943 ns/op 147.33 MB/s
BenchmarkCodeDecoder-4 100 13384738 ns/op 144.98 MB/s
BenchmarkCodeDecoder-4 100 13105316 ns/op 148.07 MB/s
BenchmarkCodeDecoder-4 100 13210405 ns/op 146.89 MB/s
BenchmarkCodeDecoder-4 100 13094757 ns/op 148.19 MB/s
BenchmarkCodeDecoder-4 100 13965663 ns/op 138.95 MB/s
PASS
ok encoding/json 21.430s
$ perflock -governor=70% go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > old.txt
$ perflock -governor=70% go test -bench=CodeDecoder /var/lib/go/src/encoding/json -count=8 > new.txt
$ benchstat old.txt new.txt
name old time/op new time/op delta
CodeDecoder-4 13.4ms ± 5% 13.2ms ± 5% ~ (p=0.645 n=8+8)
name old speed new speed delta
CodeDecoder-4 145MB/s ± 5% 147MB/s ± 5% ~ (p=0.645 n=8+8)
Larger variance => need larger N
Tools for CPU load: pprof, perflock
benchstat works for all benchmarks e.g. for measuring network latency or disk I/O
benchstat to compare statistics
perflock to avoid noise.
Compiler options
To get a full list of compiler options use:
$ go tool compile -help
usage: compile [options] file.go...
-% debug non-static initializers
-+ compiling runtime
-B disable bounds checking
-C disable printing of columns in error messages
-D path
set relative path for local imports
-E debug symbol export
-I directory
add directory to import search path
-K debug missing line numbers
-L show full file names in error messages
-N disable optimizations
-S print assembly listing
-V print version and exit
-W debug parse tree after type checking
-allabis
generate ABI wrappers for all symbols (for bootstrap)
-asmhdr file
write assembly header to file
-bench file
append benchmark times to file
-blockprofile file
write block profile to file
-buildid id
record id as the build id in the export metadata
-c int
concurrency during compilation, 1 means no concurrency (default 1)
-complete
compiling complete package (no C or assembly)
-cpuprofile file
write cpu profile to file
-d list
print debug information about items in list; try -d help
-dwarf
generate DWARF symbols (default true)
-dwarflocationlists
add location lists to DWARF in optimized mode (default true)
-dynlink
support references to Go symbols defined in other shared libraries
-e no limit on number of errors reported
-gendwarfinl int
generate DWARF inline info records (default 2)
-goversion string
required version of the runtime
-h halt on error
-importcfg file
read import configuration from file
-importmap definition
add definition of the form source=actual to import map
-installsuffix suffix
set pkg directory suffix
-j debug runtime-initialized variables
-l disable inlining
-lang string
release to compile for
-linkobj file
write linker-specific object to file
-live
debug liveness analysis
-m print optimization decisions
(e.g. escape analysis decisions)
-memprofile file
write memory profile to file
-memprofilerate rate
set runtime.MemProfileRate to rate
-msan
build code compatible with C/C++ memory sanitizer
-mutexprofile file
write mutex profile to file
-nolocalimports
reject local (relative) imports
-o file
write output to file
-p path
set expected package import path
-pack
write to file.a instead of file.o
-r debug generated wrappers
-race
enable race detector
-s warn about composite literals that can be simplified
-shared
generate code that can be linked into a shared library
-std
compiling standard library
-symabis file
read symbol ABIs from file
-traceprofile file
write an execution trace to file
-trimpath prefix
remove prefix from recorded source file paths
-v increase debug verbosity
-w debug type checking
-wb
enable write barrier (default true)
---
Repeat the flag value in order to increase the output verbosity.
Use any of these two forms of passing flag values:
~/tmp$ go build -gcflags '-m -m' /var/lib/go/src/io
or
~/tmp$ go build -gcflags='-m -m' /var/lib/go/src/io
# io
/var/lib/go/src/io/pipe.go:73:27: (*pipe).CloseRead.func1 capturing by value: p (addr=false assign=false width=8)
/var/lib/go/src/io/pipe.go:112:27: (*pipe).CloseWrite.func1 capturing by value: p (addr=false assign=false width=8)
/var/lib/go/src/io/io.go:289:6: cannot inline WriteString: function too complex: cost 136 exceeds budget 80
/var/lib/go/src/io/io.go:304:6: cannot inline ReadAtLeast: unhandled op FOR
/var/lib/go/src/io/io.go:328:6: can inline ReadFull as: func(Reader, []byte) (int, error) { return ReadAtLeast(r, buf, len(buf)) }
/var/lib/go/src/io/io.go:430:6: can inline LimitReader as: func(Reader, int64) Reader { return &LimitedReader literal }
/var/lib/go/src/io/io.go:380:6: cannot inline copyBuffer: unhandled op FOR
/var/lib/go/src/io/io.go:363:6: can inline Copy as: func(Writer, Reader) (int64, error) { return copyBuffer(dst, src, nil) }
/var/lib/go/src/io/io.go:339:6: cannot inline CopyN: function too complex: cost 100 exceeds budget 80
/var/lib/go/src/io/io.go:340:38: inlining call to LimitReader func(Reader, int64) Reader { return &LimitedReader literal }
...
$ go run -gcflags='-m -m' esc.go
bce = bounds check elimination; this is a runtime check that you're accessing slice within its bounds
~/tmp$ go build -gcflags=-d=ssa/check_bce/debug=1 /var/lib/go/src/io
# io
/var/lib/go/src/io/io.go:310:23: Found IsSliceInBounds
/var/lib/go/src/io/io.go:404:27: Found IsSliceInBounds
/var/lib/go/src/io/io.go:537:27: Found IsSliceInBounds
/var/lib/go/src/io/multi.go:30:18: Found IsInBounds
/var/lib/go/src/io/multi.go:31:27: Found IsSliceInBounds
/var/lib/go/src/io/multi.go:106:15: Found IsSliceInBounds
/var/lib/go/src/io/pipe.go:90:9: Found IsSliceInBounds
~/tmp$ go build -gcflags=-d=ssa/prove/debug=1 /var/lib/go/src/io
# io
/var/lib/go/src/io/io.go:404:27: Proved Geq64
/var/lib/go/src/io/io.go:446:8: Proved IsSliceInBounds
/var/lib/go/src/io/io.go:446:8: Proved Geq64
/var/lib/go/src/io/io.go:473:8: Proved IsSliceInBounds
/var/lib/go/src/io/io.go:507:8: Proved IsSliceInBounds
/var/lib/go/src/io/io.go:537:27: Proved Geq64
/var/lib/go/src/io/multi.go:21:27: Proved IsInBounds
/var/lib/go/src/io/multi.go:26:23: Proved IsInBounds
/var/lib/go/src/io/multi.go:61:10: Induction variable: limits [0,?), increment 1
/var/lib/go/src/io/multi.go:77:3: Induction variable: limits [0,?), increment 1
/var/lib/go/src/io/multi.go:105:17: Induction variable: limits [0,?), increment 1
~/tmp$ go build -gcflags=-d=ssa/prove/debug=2 /var/lib/go/src/io
# io
/var/lib/go/src/io/io.go:404:27: Proved Geq64 (v147)
/var/lib/go/src/io/io.go:446:8: Proved IsSliceInBounds (v46)
/var/lib/go/src/io/io.go:446:8: Proved Geq64 (v42)
/var/lib/go/src/io/io.go:473:8: Proved IsSliceInBounds (v49)
/var/lib/go/src/io/io.go:507:8: Proved IsSliceInBounds (v60)
/var/lib/go/src/io/io.go:537:27: Proved Geq64 (v35)
/var/lib/go/src/io/multi.go:21:27: Proved IsInBounds (v34)
/var/lib/go/src/io/multi.go:26:23: Proved IsInBounds (v72)
/var/lib/go/src/io/multi.go:59:14: x+d >= w; x:v24 b6 delta:1 w:0 d:signed
/var/lib/go/src/io/multi.go:61:10: Induction variable: limits [0,v15), increment 1 (v24)
/var/lib/go/src/io/multi.go:76:14: x+d >= w; x:v25 b6 delta:1 w:0 d:signed
/var/lib/go/src/io/multi.go:77:3: Induction variable: limits [0,v16), increment 1 (v25)
/var/lib/go/src/io/multi.go:104:14: x+d >= w; x:v30 b6 delta:1 w:0 d:signed
/var/lib/go/src/io/multi.go:105:17: Induction variable: limits [0,v8), increment 1 (v30)
Same benchmark can give different results upon code change simply for different compiled code organization (e.g. code alignments...).
To see how is compiler compiling specific function use:
$ GOSSAFUNC=pattern go build
Example:
~/tmp$ echo $'package p\n\nfunc HelloWorld(){\n\tprintln(\"Hello, world!\")\n}\n' > p.go
$ GOSSAFUNC=HelloWorld go build
# _/home/bojan/tmp
dumped SSA to ./ssa.html
$ chromium ./ssa.html
---
Profile your golang benchmark with pprof
High Performance Go Workshop
An Overview of Go's Tooling
CPU Profiling
With the current version of Go it is possible to perform CPU profiling only for one package at a time.
$ go test \
-run=NONE \
-bench=. \
-cpuprofile=cpu.out \
"./internal/pkg/package_name/"
Example:
$ go test \
-run=NONE \
-bench=. \
-cpuprofile=cpu.out \
"./internal/pkg/datatypesdemo/"
This will create file cpu.out in the current/root directory.
To open cpu.out and enter interactive mode in
pprof tool:
$ go tool pprof cpu.out
To check how much time it takes to execute parts of function
that were called in benchmarks:
(pprof) list <function_name>
Example:
$ go tool pprof cpu.out
File: datatypesdemo.test
Type: cpu
Time: Jul 5, 2019 at 4:50pm (BST)
Duration: 1.70s, Total samples = 1.94s (113.98%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) list copyList
Total: 1.94s
ROUTINE ======================== github.com/BojanKomazec/go-demo/internal/pkg/datatypesdemo.copyList in /home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/internal/pkg/datatypesdemo/datatypesdemo.go
210ms 1.31s (flat, cum) 67.53% of Total
. . 61:func copyList(in []string) []string {
. . 62: var out []string
80ms 80ms 63: for _, s := range in {
130ms 1.23s 64: out = append(out, s)
. . 65: }
. . 66: return out
. . 67:}
(pprof)
To exit, type exit:
(pprof) exit
benchcmp tool automatically calculates differences (in %) in performances before and after code changes:
$ cd ../go/src/github.com/BojanKomazec/go-demo
$ go test -run=NONE -bench=. ./... > old.txt
$ go test -run=NONE -bench=. ./... > new.txt
$ benchcmp old.txt new.txt
benchmark old ns/op new ns/op delta
BenchmarkCopyList1_100x16-4 1400 1404 +0.29%
BenchmarkCopyList2_100x16-4 467 468 +0.21%
benchmark old allocs new allocs delta
BenchmarkCopyList1_100x16-4 8 8 +0.00%
BenchmarkCopyList2_100x16-4 1 1 +0.00%
benchmark old bytes new bytes delta
BenchmarkCopyList1_100x16-4 4080 4080 +0.00%
BenchmarkCopyList2_100x16-4 1792 1792 +0.00%
Go CLI
$ go
Go is a tool for managing Go source code.
Usage:
go <command> [arguments]
The commands are:
bug start a bug report
build compile packages and dependencies
clean remove object files and cached files
doc show documentation for package or symbol
env print Go environment information
fix update packages to use new APIs
fmt gofmt (reformat) package sources
generate generate Go files by processing source
get download and install packages and dependencies
install compile and install packages and dependencies
list list packages or modules
mod module maintenance
run compile and run Go program
test test packages
tool run specified go tool
version print Go version
vet report likely mistakes in packages
Use "go help <command>" for more information about a command.
Additional help topics:
buildmode build modes
c calling between Go and C
cache build and test caching
environment environment variables
filetype file types
go.mod the go.mod file
gopath GOPATH environment variable
gopath-get legacy GOPATH go get
goproxy module proxy protocol
importpath import path syntax
modules modules, module versions, and more
module-get module-aware go get
packages package lists and patterns
testflag testing flags
testfunc testing functions
Use "go help <topic>" for more information about that topic.
go build
$ go help build
usage: go build [-o output] [-i] [build flags] [packages]
Build compiles the packages named by the import paths,
along with their dependencies, but it does not install the results.
If the arguments to build are a list of .go files, build treats
them as a list of source files specifying a single package.
When compiling a single main package, build writes
the resulting executable to an output file named after
the first source file ('go build ed.go rx.go' writes 'ed' or 'ed.exe') or the source code directory ('go build unix/sam' writes 'sam' or 'sam.exe'). The '.exe' suffix is added when writing a Windows executable.
When compiling multiple packages or a single non-main package,
build compiles the packages but discards the resulting object,
serving only as a check that the packages can be built.
When compiling packages, build ignores files that end in '_test.go'.
The -o flag, only allowed when compiling a single package,
forces build to write the resulting executable or object
to the named output file, instead of the default behavior described
in the last two paragraphs.
The -i flag installs the packages that are dependencies of the target.
The build flags are shared by the build, clean, get, install, list, run, and test commands:
-a
force rebuilding of packages that are already up-to-date.
-n
print the commands but do not run them.
-p n
the number of programs, such as build commands or
test binaries, that can be run in parallel.
The default is the number of CPUs available.
-race
enable data race detection.
Supported only on linux/amd64, freebsd/amd64, darwin/amd64 and windows/amd64.
-msan
enable interoperation with memory sanitizer.
Supported only on linux/amd64, linux/arm64
and only with Clang/LLVM as the host C compiler.
-v
print the names of packages as they are compiled.
-work
print the name of the temporary work directory and
do not delete it when exiting.
-x
print the commands.
-asmflags '[pattern=]arg list'
arguments to pass on each go tool asm invocation.
-buildmode mode
build mode to use. See 'go help buildmode' for more.
-compiler name
name of compiler to use, as in runtime.Compiler (gccgo or gc).
-gccgoflags '[pattern=]arg list'
arguments to pass on each gccgo compiler/linker invocation.
-gcflags '[pattern=]arg list'
arguments to pass on each go tool compile invocation.
-installsuffix suffix
a suffix to use in the name of the package installation directory,
in order to keep output separate from default builds.
If using the -race flag, the install suffix is automatically set to race
or, if set explicitly, has _race appended to it. Likewise for the -msan
flag. Using a -buildmode option that requires non-default compile flags
has a similar effect.
-ldflags '[pattern=]arg list'
arguments to pass on each go tool link invocation.
-linkshared
link against shared libraries previously created with
-buildmode=shared.
-mod mode
module download mode to use: readonly or vendor.
See 'go help modules' for more.
-pkgdir dir
install and load all packages from dir instead of the usual locations.
For example, when building with a non-standard configuration,
use -pkgdir to keep generated packages in a separate location.
-tags 'tag list'
a space-separated list of build tags to consider satisfied during the
build. For more information about build tags, see the description of
build constraints in the documentation for the go/build package.
-toolexec 'cmd args'
a program to use to invoke toolchain programs like vet and asm.
For example, instead of running asm, the go command will run
'cmd args /path/to/asm <arguments for asm>'.
The -asmflags, -gccgoflags, -gcflags, and -ldflags flags accept a
space-separated list of arguments to pass to an underlying tool
during the build. To embed spaces in an element in the list, surround it with either single or double quotes. The argument list may be receded by a package pattern and an equal sign, which restricts the use of that argument list to the building of packages matching that pattern (see 'go help packages' for a description of package patterns). Without a pattern, the argument list applies only to the packages named on the command line. The flags may be repeated
with different patterns in order to specify different arguments for
different sets of packages. If a package matches patterns given in
multiple flags, the latest match on the command line wins.
For example, 'go build -gcflags=-S fmt' prints the disassembly
only for package fmt, while 'go build -gcflags=all=-S fmt'
prints the disassembly for fmt and all its dependencies.
For more about specifying packages, see 'go help packages'.
For more about where packages and binaries are installed,
run 'go help gopath'.
For more about calling between Go and C/C++, run 'go help c'.
Note: Build adheres to certain conventions such as those described
by 'go help gopath'. Not all projects can follow these conventions,
however. Installations that have their own conventions or that use
a separate software build system may choose to use lower-level
invocations such as 'go tool compile' and 'go tool link' to avoid
some of the overheads and design decisions of the build tool.
See also: go install, go get, go clean.
Note that list of options goes BEFORE list of source code files.
$ go build -o build/wiki wiki.go
To keep the flag package simple, the first non-flag is treated as the first command line argument, regardless of what might come after, so all flags must come before all regular arguments.
The go tool uses the flag package and so is subjected to the same limitations for it's built-in subcommands. Therefore the "output" flag must be set before the arguments (which contain the input).
example:
go build foo -o bar -> flags: {} args: [foo, -o, bar]
go build -o bar foo -> flags: {o: bar} args: [foo]
[
(How to) Give a name to the binary using go build]
go get
$ go help get
usage: go get [-d] [-f] [-t] [-u] [-v] [-fix] [-insecure] [build flags] [packages]
Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'.
The -d flag instructs get to stop after downloading the packages; that is, it instructs get not to install the packages.
The -f flag, valid only when -u is set, forces get -u not to verify that each package has been checked out from the source control repository implied by its import path. This can be useful if the source is a local fork of the original.
The -fix flag instructs get to run the fix tool on the downloaded packages before resolving dependencies or building the code.
The -insecure flag permits fetching from repositories and resolving
custom domains using insecure schemes such as HTTP. Use with caution.
The -t flag instructs get to also download the packages required to build the tests for the specified packages.
The -u flag instructs get to use the network to update the named packages and their dependencies. By default, get uses the network to check out missing packages but does not use it to look for updates to existing packages.
The -v flag enables verbose progress and debug output.
Get also accepts build flags to control the installation. See 'go help build'.
When checking out a new package, get creates the target directory
GOPATH/src/<import-path>. If the GOPATH contains multiple entries,
get uses the first one. For more details see: 'go help gopath'.
When checking out or updating a package, get looks for a branch or tag that matches the locally installed version of Go. The most important rule is that if the local installation is running version "go1", get searches for a branch or tag named "go1". If no such version exists it retrieves the default branch of the package.
When go get checks out or updates a Git repository,
it also updates any git submodules referenced by the repository.
Get never checks out or updates code stored in vendor directories.
For more about specifying packages, see 'go help packages'.
For more about how 'go get' finds source code to download, see 'go help importpath'.
This text describes the behavior of get when using GOPATH
to manage source code and dependencies. If instead the go command is running in module-aware mode, the details of get's flags and effects change, as does 'go help get'. See 'go help modules' and 'go help module-get'.
See also: go build, go install, go clean.
Example:
$ go get -d -v ./...
What do three dots “./…” mean in Go command line invocations?
./ tells to start from the current folder,
... tells to go down recursively.
Example:
$ go get -u golang.org/x/perf/cmd/...
$ go get github.com/aclements/perflock/cmd/perflock
go env
This is the best way to check which $GOPATH and $GOROOT values are used by any local Go tool.
Mind that these values can be different for current user and for root:
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/my_user_name/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/my_user_name/dev/go"
GOPROXY=""
GORACE=""
GOROOT="/var/lib/go"
GOTMPDIR=""
GOTOOLDIR="/var/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build480098893=/tmp/go-build -gno-record-gcc-switches"
For root:
$ sudo go env
...
GOPATH="/home/my_user_name/go"
...
go install
$ go help install
usage: go install [-i] [build flags] [packages]
Install compiles and installs the packages named by the import paths.
The -i flag installs the dependencies of the named packages as well.
For more about the build flags, see 'go help build'.
For more about specifying packages, see 'go help packages'.
See also: go build, go get, go clean.
It:
- places the executable file in $GOPATH/bin
- caches all non-main packages (which app imports) in $GOPATH/pkg
- Cache is used in the next compilation unless it changes in the meantime
Go naming convention
What are conventions for filenames in Go?
File names that begin with "." or "_" are ignored by the go tool
Files with the suffix _test.go are only compiled and run by the go test tool.
Files with os and architecture specific suffixes automatically follow those same constraints, e.g. name_linux.go will only build on linux, name_amd64.go will only build on amd64. This is the same as having a //+build amd64 line at the top of the file
filenames are generally all lowercase in case, both for consistency and for systems with case-insensitive filesystems
regular file names are lower case, short, and without any sort of underscore or space. Generally, file names follow the same convention as package names.
there is no convention but _ suffixes may have special semantics in the future so I recommend to avoid them
Naming a file dns_windows.go will cause it to be included only when building the package for Windows; similarly, math_386.s will be included only when building the package for 32-bit x86.
File name convention for compound words?
Source file names are usually kept short, so in your case I would tend to call it find.go in package weightedunion <-- the idea is to make better use of packages when you are tempted to name it long
What is the best naming convention of keeping source files in a programming language?
Go follows a convention where source files are all lower case with underscore separating multiple words. Example:
https://github.com/kubernetes/client-go/tree/master/discovery/cached/diskGolang
folder structure and filename conventions under clean architecture
Interfaces in Go have a er-suffix (io.Reader, fmt.Stringer, etc).
The filenames are not per se named after a single type as often the package contains several types (see your first question).
Mixed case filenames are cute until your filesystem does not distinguish cases. Keep it simple, lowercase only.
Style guideline for Go packages
Go code Documentation & Comments
Godoc: documenting Go code
To document a type, variable, constant, function, or even a package, write a regular comment directly preceding its declaration, with no intervening blank line. Godoc will then present that comment as text alongside the item it documents.
Comment is a complete sentence that begins with the name of the element it describes.
The "note marker" needs to follow this syntax [
source]:
MARKER(uid): note body
Then godoc -notes="MARKER" includes this note into the doc.
If we have in the code:
// TODO(id_manager): Add id range checking...
And then run godoc comments parser as:
godoc -note="TODO" .
We'll see in the output:
PACKAGE DOCUMENTATION
package my_package
...
TODOS
Add id range checking...
Installing godoc
On Ubuntu:
$ sudo apt install golang-golang-x-tools
Difference between godoc and go doc
godoc:
user@host:~/.../go/src/github.com/user/app$ godoc ./internal/pkg/pgclient/
PACKAGE DOCUMENTATION
package pgclient
import "./internal/pkg/pgclient/"
FUNCTIONS
func New(params dbclient.ConnParams) (dbclient.DbClient, error)
New function creates an instance of PostgreSQL client
go doc:
user@host:~/.../go/src/github.com/user/app$ go doc ./internal/pkg/pgclient/
package pgclient // import "github.com/user/app/internal/pkg/pgclient"
func New(params dbclient.ConnParams) (dbclient.DbClient, error)
---
Go & Docker
Deploying a Go Application in a Docker Container
Go & PostgreSQL
Go comes with embedded
database/sql package.
The standard PostgreSQL DB driver for Go is lib/pq.
package pq
How to use PostgreSQL
Querying for a single record using Go's database/sql package
QueryRow() is a method provided by the DB type and is used to execute an SQL query that is expected to return a single row.
Useful packages
networking - Access HTTP response as string in Go - Stack Overflow
HTTP Server:
Golang: HTTP Serve static files correctly - DEV Community
Resources:
"Go in Action" by William Kennedy
Effective Go
jbuberel/go-patterns
Golang examples and solutions from different packages of the standard library. - golangprograms.com
#Go @ siongui.github.io
GoLang Tutorials
Common Mistakes
I Spy Code - Blog - golang (Programming puzzles)