Sunday, 28 August 2016

Git Tips and Tricks


Git is a (free) version control system. It allows:
  • saving code snapshots (commits)
  • working with alternative code versions (branches)
  • moving between branches and commits (checkout)
  • easily rolling back to older code snapshots or develop new features without breaking production code

Repository is a bucket which contains a code. 

GitHub is a platform for creating, storing and managing Git repositories in the cloud
  • allows multiple developers to work simultaneously on the centrally stored code base
  • keeps the code safe in case code is lost on dev machines for any reason
  • provides tools for automating code and repository management via GitHub Actions

Switching repositories and branches


Cloning a remote repository:

$ git clone https://github.com/OpenVPN/openvpn-build.git



After cloning desired repository it is always good to run the following command and fetch/clone all its submodules:
$ git submodule update --init

Cloning a remote repository with preserving UNIX-style line endings:

$ git clone https://github.com/OpenVPN/openvpn-build.git --config core.autocrlf=false



Getting a list of all local branches:

$ git branch



Deleting local branch:

$ git branch -d my_branch
Deleted branch my_branch (was 6d5921868b).



Getting a list of all remote branches:

$ git branch -r



Getting a list of all branches at some specific path (on Windows):

> git branch -r | find "origin/private/bojan"



Example output:

origin/HEAD -> origin/master
origin/master
origin/release/2.3



Deleting remote branch:

git push origin --delete my_branch



To update local list of remote branches:

$ git remote update origin --prune



To list all local and remote branches:

$ git branch -a




Checking out the remote branch (for the first time):

$ git checkout -b branch_name remote_name/branch_name


or shorter (on newer Git versions):

$ git checkout --track remote_name/branch_name



Example:

$ git checkout -b release/2.3 origin/release/2.3
Branch release/2.3 set up to track remote branch release/2.3 from origin.
Switched to a new branch 'release/2.3'



Or:

$ git fetch
...
* [new branch] ModuleA/Project2/JIRA-123 -> origin/ModuleA/Project2/JIRA-123
...

$ git checkout ModuleA/Project2/JIRA-123
Checking out files: 100% (1485/1485), done.
Branch ModuleA/Project2/JIRA-123 set up to track remote branch ModuleA/Project2/JIRA-123 from origin.
Switched to a new branch 'ModuleA/Project2/JIRA-123'


If you get the following error:

$ git checkout ModuleA/Project2/JIRA-123
error: Your local changes to the following files would be overwritten by checkout:
Module2/SomeDir/SomeFile.cpp
Please commit your changes or stash them before you switch branches.
Aborting


...you can force checking out with -f switch:

$ git checkout -f ModuleA/Project2/JIRA-123


To create new local branch:

$ git checkout -b issue-007
Switched to a new branch "issue-007"



The command above is a compact version for a group of these two subsequent git commands:

$ git branch issue-007
$ git checkout issue-007



To create a new branch based on a commit from another branch:

$ git branch branch_name <commit_sha1>

To create that branch and check it out:

$ git checkout -b branch_name <commit_sha1>

To rename local branch:

$ git branch -m old_name new_name

To rename current local branch:

$ git branch -m new_name

To push new and delete old branch name to remote:
 
$ git push origin :old_name new_name

Switching to another local branch:

$ git checkout branch_name



Example:

$ git checkout master
Your branch is behind 'origin/master' by 179 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
Switched to branch 'master'



To checkout commit specified with its hash:

$ git checkout 3e6dc47cde0e45bb3198ac4618367afc57ddf6a3
$ git checkout 3e6dc47cde0e45bb3198ac4618367afc57ddf6a3
Checking out files: 100% (5111/5111), done.
M features/feature1
M framework/module2
Note: checking out '3e6dc47cde0e45bb3198ac4618367afc57ddf6a3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at 3e6dc47cde... Fixed EULA BK brand;



If local branch is behind the remote by 1 or more commits, it can be updated:

$ git pull



Note that this takes local repository in the state of detached head. HEAD is symbolic name for the currently checked out commit and in this state it points directly to a commit instead of pointing to a branch (e.g. refs/heads/master).

To fix detached head (and delete all local changes) do the following:

$ git branch temp
$ git checkout
$ git branch -d temp



In order to switch from one to another branch we have to have all staged changes either committed or stashed. If we have only non-staged changes we can switch to a new branch with no other actions necessary - all non-staged changes will remain and apply to the new branch. [moving changed files to another branch for check-in]

To stash local changes do the following:

$ git stash
Saved working directory and index state WIP on my_branch: 6d94b73b53 JIRA-123 Fixed memory leak
HEAD is now at 6d94b73b53 JIRA-123 Fixed memory leak




Sometimes we have a bunch of files modified but want to stash only some of them (and possibly discard the other). For example, we have modified many files but want to stash only those in solution1/project1:

~/dev/projects (branchA)
$ git status
On branch branchA
Your branch is up-to-date with 'origin/branchA'.
Changes not staged for commit:
(use "git add
..." to update what will be committed)
(use "git checkout --
..." to discard changes in working directory)
(commit or discard the untracked or modified content in submodules)
...
modified: tools/gadget/file1
modified: solution1/project1/file2
modified: solution1/project1/file3
modified: solution1/project1/file4
modified: web/src/file5
...



Since Git version 2.13 it is possible to use git stash command and specify which files shall be stashed (note that path to each file has to be specified so includes all directories within the root repo directory otherwise you'll get error message like error: pathspec 'A/b' did not match any file(s) known to git. Did you forget to 'git add'?):

~/dev/projects (branchA)
$ git stash push solution1/project1/file2 solution1/project1/file3 solution1/project1/file4
Saved working directory and index state WIP on branchA: 64807ad229 some_commit_comment




We can verify that new stash is on the top of the list of all stashes on the current branch:

$ git stash list
stash@{0}: WIP on branchA: 64807ad229 some_commit_comment
stash@{1}: WIP on my_branch: 6d94b73b53 JIRA-123 Fixed memory leak




To list all files from the latest stash:

$ git stash show
solution1/project1/file2 | 41 ++++++++++++++++++++++
solution1/project1/file3 | 5 +--
solution1/project1/file4 | 3 +-
3 files changed, 45 insertions(+), 4 deletions(-)



To show diff code from the latest stash:

$ git stash show -p

If single stash is saved, it can be applied with this:

$ git stash apply



If you were on some branch (e.g. master) and made some changes but now want to move them to a new branch and revert working version of master to the state before your changes, do the following:


$ git stash
$ git checkout -b my_new_branch
$ git stash pop




git stash - Git: Create a branch from unstaged/uncommitted changes on master - Stack Overflow
 
 
 
If default branch is renamed (e.g. master is renamed to trunk) via GitHub web page you need to update the local clone to reflect that change:

$ git branch -m master trunk
$ git fetch origin
$ git branch -u origin/trunk trunk
$ git remote set-head origin -a



Checking the current status of the local repository


Checking out the status of the current local repository:

$ git status



Checking the name of the remote:

$ git remote
origin



To see the tracking state of all remote branches (and so to verify upstream branches):


C:\dev\jsis\plugin-notifyicon>git remote show origin
Enter passphrase for key '/c/Users/user/.ssh/id_rsa':
* remote origin
Fetch URL: git@git.example.com:product1/my_module1.git
Push URL: git@git.example.com:product1/my_module1.git
HEAD branch: master
Remote branches:
feature/Project1-1412_Silent_install_fix tracked
master tracked
refs/remotes/origin/Project1--2315_Click_on_title_opens_URL_in_browser stale (use 'git remote prune' to remove)
refs/remotes/origin/Project1--1202_Add_version_info stale (use 'git remote prune' to remove)
Local branch configured for 'git pull':
master merges with remote master
Local refs configured for 'git push':
feature/Project1-1412_Silent_install_fix pushes to feature/ABC-1412_Silent_install_fix (up to date)
master pushes to master (local out of date)



Getting SHA-1 of the last commit on the current branch:

$ git rev-parse HEAD
e39616c0cc351c027cff762008b2da77ee1cc6cc



Getting short version of the hash:

$ git rev-parse --short HEAD
e39616c0cc



To see what has changed in the commit with id 'commit_hash':

$ git show commit_hash



To see which files have been modified and how in last N commits in the specific directory:

$ git log option -N path_to_directory


path_to_directory can be relative path from the current directory.
option can be:
-p - full patch
--stat - numbers of changed lines
--numstat - like --stat but machine-readable
--name-status shows the file name and status: Added (A), Copied (C), Deleted (D), Modified (M), Renamed (R).
--name-only - just the filenames

To display commit history, branches, merges...all in a single graph with a one line comments:

$ git log --oneline --abbrev-commit --all --graph

Expand the previous command so it shows also branch and tag labels:

$ git log --oneline --abbrev-commit --all --graph --decorate --color

Use --first-parent to get simpler graph. This can also be used to identify the parent branch (the branch from which the current branch was derived):
 
$ git log --first-parent
 
List all commits on the current branch assuming it has been branched off some known branch ('trunk'):

$ git log --no-pager --no-merges trunk..

--no-pager prevents git pagination.

List all commits made by specific author:

$ git log --author="name@example.com"


git log takes zero or more commits as arguments, showing the history leading up to that commit. When no argument is given, HEAD is assumed. [source]


To show graph of commits and merges between two branches use:

$ git log --graph --oneline current_branch other_branch

Adding changed file (already tracked but modified) or new file to the commit:

$ git add my_file



Very often there are tracked but modified and also untracked files in the local repo and we want to add only already tracked files. To do that:

$ git add -u

This -u means -update as we are updating already added files.

The above commands also apply for directories.

In order to stage all files (both untracked and modified but not staged files) we can use:

$ git add .


To unstage a single file:


$ git reset HEAD dir/file.ext



It is possible to use wildcards to denote all files from some directory:

$ git add dir_1/file_1 dir_1/file_2 dir_1/dir_2/*.*



Add files from dir_1 which have same name but different extension:

$ git add dir_1/file_1.*



If we want to delete some file, we can delete it in usual way and then execute git add:

$ rm my_file
$ git add my_file



This can be done with a single git command:

$ git rm add my_file



Modified file can be unstaged (in the working directory), staged (indexed/cached) and commited (HEAD points to its last commit).

If we want to remove staged file from the commit index (to "unstage" it) but at the same time to keep changes in the file:

$ git reset some_file



We can also use:

$ git reset HEAD some_file


...to unstage single file or:

$ git reset HEAD some_directory


...to unstage all modified files from the specified directory.


If we want to revert unstaged (unindexed) changes to the last committed revision of file we can simply check out the file:

$ git checkout some_file



To do this for all files:

$ git checkout .


This will affect only files in the current directory so if you want to do this for all files in the repository, first move to the repository's root directory.

To remove all untracked files:

$ git clean -f



To check what will be removed with 'git clean' (dry run):

$ git clean -n



To remove all untracked files and directories:

$ git clean -fd


or use -fdx to remove ignored files as well.

To discard changes in ALL modified but not staged files:

$ git checkout -- .



Working with submodules


Git - Submodules

Let's say that ProjectA resides in Repository1 and is dependent (uses) ProjectB which resides in Repository2. ProjectB might be shared among multiple projects and its development is independent from development of ProjectA.

When we checkout ProjectA, we would like also to pull the latest ProjectB. Submodules provide a way for this. Repository2 will be pulled in a subdirectory in Repository1.

How to check if current repository have any submodules?

$ git submodule
f97bae646f5341216752c4c19950e4b4c15bf616 dir1/dir2/submodule1 (remotes/origin/HEAD)
6e1db0f970c11ec571cf17516d5d8a4b93bd66ce dir3/submodule2 (remotes/origin/HEAD)
3ac8aba9e57c273e249724f8458622e51dbea233 dir4/submodule3 (remotes/origin/HEAD)




What happens if you pull repository but not its submodules (you don't update submodules)? Git can detect that there are new commits in submodules:

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
(use "git add
..." to update what will be committed)
(use "git checkout --
..." to discard changes in working directory)

modified: dir1/dir2/submodule1 (new commits)
modified: dir3/submodule2 (new commits)
modified: dir4/submodule3 (new commits)

no changes added to commit (use "git add" and/or "git commit -a")




How to clone repository and its submodules?

$ git clone --recursive ssh://git@mydomain.com:8082/my_repo.git



How to update submodules?

$ git submodule update --recursive --remote
Submodule path 'dir1/dir2/submodule1': checked out '6e1db0f970c21ec571cf17516d5d8d4b93bd66ce'
Submodule path 'dir3/submodule2': checked out '3ac8aba9e57c273e244724f845862ee51dbea233'
Submodule path 'dir4/submodule3': checked out '8d58faf56d1a3f5ab50da24ef66c624be9678ba6'




To revert them to the latest locally checked out version:

$ git submodule update
Submodule path 'dir1/dir2/submodule1': checked out 'd5011032541d2fb81e484c78ff8306633a219c66'
Submodule path 'dir3/submodule2': checked out 'b15ab11cd92e1f88ed2b334d0f01a8130ddae28a'
Submodule path 'dir4/submodule3': checked out '3ed4ac1d8dc8bd9862d9e5be2a18f22713899b85'




After every pull, checkout or merge you must initialize and update submodules.

$ git submodule update --init --recursive

url property for each submodule in .gitmodules can contain relative or absolute paths to repositories (.git files) on the local machine OR repository URLs. In the former case we need to manually clone/fetch submodule repositories to local machine and also in such location that matches relative path. Using URL is better option:

.gitmodules example:

$ cat .gitmodules
[submodule "moduleA"]
path = moduleA
url = git@example.com:project1/subproject1/moduleA.git
branch = master
[submodule "moduleB"]
path = moduleB
url = git@example.com:project2/moduleB.git
branch = master

You might need to execute this sequence:

$ git submodule sync
Synchronising submodule url for 'moduleA'
Synchronising submodule url for 'moduleB'
$ git submodule update --init --recursive
Submodule path 'moduleA': checked out '03dcfffa13389708746375698171e01ccf070f15'
Submodule path 'moduleB': checked out 'e0eae012ea59d58656968d5f8a3797479deb2b2f'

From git submodule update vs git submodule sync:

git submodule sync updates the metadata about a submodule to reflect changes in the submodule URL. It re-synchronizes the information in .git/config with the information in .gitmodules.

git submodule update updates the contents of the submodules. It is effectively running a "git fetch" and "git checkout" in each of your submodules.

Git creates a cache for submodules. If you ever get access rights error on a submodule repo on which you do have access rights like here:

$ git submodule update --init
Cloning into 'C:/dev/project/submodule-project'...
fatal: '.:dir/submodule-project-repo-name.git' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

...first check .gitmodules where you might have url of given submodule which is different from the url in the error message reported by git submodule command. This is because Git caches submodule url in .git/config:

/c/dev/project/.git (GIT_DIR!)
$ cat config
[core]
        repositoryformatversion = 0
        filemode = false
        bare = false
        logallrefupdates = true
        symlinks = false
        ignorecase = true
[remote "origin"]
        url = git@...xxx....git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master
[submodule "submodule-project"]
        active = true
        url = .:dir/submodule-project-repo-name.git

Edit this config file and apply correct url for submodule (in form git@xxx.com:path/to/submodule-project-repo-name.git).

How effectively delete a git submodule.

How to add a new submodule?


$ git submodule add git@git.example.com:projectXYZ/my-project.git my-project/

After this there's gonna be a new:

  • subdirectory named my-project 
  • a new entry in .gitmodules


$ cat .gitmodules 
...
[submodule "my-project"]
        path = my-project
        url = git@git.example.com:projectXYZ/my-project.git

.gitmodules has to be commited:

$ git status
On branch submodule-add-demo
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   .gitmodules


`--name` option doesn't work with `git submodule add` command - Stack Overflow


To rename submodule directory:

$ git mv old new
$ git status
On branch feature/test
Your branch is up-to-date with 'origin/feature/test'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   .gitmodules
        renamed:    old -> new

$ cat .gitmodules 
[submodule "myproj"]
        path = new
        url = git@git.example.com:path/to/myproj.git


Creating new commits


Saving changes into the repository (creating a new commit containing changes in all staged files):

$ git commit


After this, there will be no staged files.

Git will prompt you to enter the commit message (in vi editor; ESC to exit editing mode, :wq to save changes and quit). Commit message can be specified when committing:


$ git commit -m "Code updated"



Staging, committing and specifying message can be done in one go:

$ git commit -a -m "Code updated"



If commit has not been pushed to remote yet, you can still change commit message:

$ git commit --amend



If commit has been pushed to remote and is the last commit pushed, it is still possible to amend commit message but then it has to be pushed again as here:

$ git push --force-with-lease <remote> <branch>

To add some file to the previous commit:

$ git add the_left_out_file
$ git commit --amend --no-edit

To see the latest commit:

$ git show




Pushing local commits to remote


If local branch is ahead of remote by 1 or more commits, they can be published (pushed to remote):

$ git push



In the previous command we didn't specify remote branch which means we assumed it's specified in the git configuration. If default remote branch has not been set specified there we might get message like this:


$ git push
warning: push.default is unset; its implicit value has changed in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the traditional behaviour, use:

git config --global push.default matching

To squelch this message and adopt the new behaviour now, use:

git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

Since Git 2.0, Git defaults to the more conservative 'simple'
behaviour, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)



We can either set push.default or simply specify remote and branch within pull command:


$ git push origin master



To set push.default use:

$ git config --global push.default matching



Verification:


$ git config --list --global
user.email=user.name@example.com
user.name=User Name
push.default=matching

After publishing local commits, local branch is up-to-date with remote.

It is possible to update another branch without checking it out:

user@my_computer ~/Documents/my_dev/my_projects/ProjectA (some_branch)
$ git pull origin master



The code above will update master branch although we're on some_branch.

If you've created local branch and want to push it to the remote (e.g. origin) prior to setting upstream branch, the following error occurs:

$ git push origin
fatal: The current branch my_new_branch has no upstream branch.
To push the current branch and set the remote as upstream, use

git push --set-upstream origin my_new_branch



Do what Git suggests:


$ git push --set-upstream origin my_new_branch



(!) NOTE: --set-upstream is not longer supported. Use --set-upstream-to:


$ git branch --set-upstream-to origin/master




Working with local changes


Modified file can be unstaged (in the working directory), staged (indexed/cached) and commited (HEAD points to its last commit). It is possible to show differences between any of these three states of the file.

To view all differences between unstaged (working) and staged (cached/indexed) version of the file:

$ git diff my_file



To increase the context (number of lines printed around the line with the difference) use -U:

$ git diff -U10 my_file



To view all differences between staged (cached/indexed) and last committed (HEAD) version of the file:

$ git diff --cached my_file


or:

$ git diff --staged my_file



To view all differences between unstaged (working) and last committed (HEAD) version of the file:

$ git diff HEAD my_file



To list all staged (cached/indexed) files:

git diff --name-only --cached



To view all differences in all modified but unstaged and staged files in the current directory:

$ git diff .



Once you perform commit locally, you can't use just git diff to see the changes which will be pushed to the remote.
To see all differences in all files which were committed locally and same files on the remote:

$ git diff origin/my_branch HEAD



If you've made changes locally, haven't committed them yet and want to pull the remote, you first need to dismiss these changes:

$ git reset --hard
$ git pull
 
If you've created some commits locally and haven't pushed them to remote your local branch will be ahead by N commits:

$ git status
On branch branch_a
Your branch is ahead of 'origin/branch_a' by 84 commits.
(use "git push" to publish your local commits)
Untracked files:
(use "git add ..." to include in what will be committed)

A/a.cpp
A/B
A/c.h

nothing added to commit but untracked files present (use "git add" to track)



If you want to dismiss these commits and pull the remote (even if remote got some new commits), you need to specify the remote branch when doing the hard reset:

$ git reset --hard origin/branch_a
HEAD is now at 0cdeba9a9a Implemented new Service skeleton. Fixed typo.
$ git pull
 
 
If you've pushed to remote some commits but now want to reverse this, first reset the local branch to the desired commit, then force push it to remote:

$ git reset --hard 9b3309c2e3f8bf4f8d5175aeeac32c0ee0534e04
$ git push --force

Example:

% git log                       
commit d2d2a60e90533c2c5e452d3ff6533db89094d58d (HEAD -> main, origin/main, origin/HEAD)
...
commit 07359315a288cc5ffea4695653f58b5cbb6b47e6
...
commit d7f2174877116e8c0e7a574b1190f2ec24b88358
...
commit e9b5a7c8603793cf8ff5f667da2b2a7cad8f4191

% git reset --hard 07359315a288cc5ffea4695653f58b5cbb6b47e6
HEAD is now at 0735931 

% git log
commit 07359315a288cc5ffea4695653f58b5cbb6b47e6 (HEAD -> main)
...
commit e9b5a7c8603793cf8ff5f667da2b2a7cad8f4191

% git status
On branch main
Your branch is behind 'origin/main' by 2 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

nothing to commit, working tree clean

% git push -f
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:UserName/myrepo.git
 + d2d2a60...0735931 main -> main (forced update)

 
If local and remote diverged and you want to overwrite remote history with local:
 
$ git push --force
 

One example when local and remote can diverge is the following:
  • a feature branch is created from trunk, joint commit is now e.g. t1
  • commits e.g. f11, f12 and f13 are added to the branch and branch is pushed to remote so remote has t1-f11-f12-f13
  • in the meantime, other devs have merged their features into trunk so trunk now has e.g. t1-t2-t3 where t2 and t3 are merge commits
  • before merging back to trunk, we want to rebase our feature branch onto trunk; this means that our local feature branch will now be: t1-t2-t3-f11-f12-f13 while on the remote t1-f11-f12-f13. NOTE that during rebase, commit IDs of f11, f12 and f13 change locally (Merge vs Rebase: Part 3 - What is a rebase?)
  • Git status now shows that local and remote diverged: 
$ git status
On branch feature/JIRA-1234
Your branch and 'origin/feature/JIRA-1234' have diverged,
and have X and Y different commits each, respectively.
  (use "git pull" to merge the remote branch into yours) 
  • Although git status suggests pulling remote into local, we don't want to do this as our local is source of truth and we want to rewrite the history on remote. This is where we use git push --force
  • If we try just to push, we'll get an error: 
$ git push
To git.example.com:workspace/projectA.git
 ! [rejected]        feature/JIRA-1234 -> feature/
JIRA-1234 (non-fast-forward)
error: failed to push some refs to 'git.example.com:workspace/projectA.git'

hint: Updates were rejected because the tip of your current branch is behind its remote counterpart. 
hint: Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
  • $ git push --force executes successfully

 
To get some submodule to some particular commit:


$ cd module_xyz
$ git reset --hard 6028a4a42446db118e615658382dc588caf71766
...
$ cd ..
$ git add module_xyz    
$ git commit -m "Submodule module_xyz set to particular commit"



Another (cleaner) way to reset submodules is:

$ git submodule deinit -f .

Cleared directory 'A/B/module1'
Submodule 'A/B/module1' (https://git.bojan.com/pro/module1.git) unregistered for path 'A/B/module1'
...

$ git submodule update --init
Submodule 'A/B/module1' (https://git.bojan.com/pro/module1.git) registered for path 'A/B/module1'
...
remote: Compressing objects: 100% (34305/34305), done.
remote: Total 141668 (delta 107055), reused 141648 (delta 107045)
Receiving objects: 100% (141668/141668), 122.02 MiB | 140.00 KiB/s, done.
Resolving deltas: 100% (107055/107055), done.
From https://git.bojan.com/pro/module1.git
* [new branch] branch1 -> origin/branch1
...



To diff two branches:

$ git diff branchA..branchB

To only list files that are different:

$ git diff --name-status branchA..branchB

To diff current branch with some other:

$ git diff ..other


Managing what shall be under revision control


It is often necessary to prevent adding under git control files with certain name and/or extension (e.g. temporary, backup, log, intermediate files etc...). Ignoring this files is set in a file called .gitignore which is located in repository's root directory. This file is like any other and can be updated and pushed to remote. To see its content we can use:

$ cat ../../.gitignore
## Ignore Visual Studio temporary files, build results

# User-specific files
*.suo
*.user
*.userosscache
*.sln.docstates

# cUser-specific files (MonoDevelop/Xamarin Studio)
*.userprefs

# Build results
BUILDS/
BUILDSENG/
COMPILE/
log/
[Rr]elease/
[Rr]elease [Ll]ib/
[Rr]elease [Ss]tatic/
[Rr]elease [Ss]tatic [Ll]ib/
[Dd]ebug/
[Dd]ebug [Ll]ib/
[Dd]ebug [Ss]tatic/
[Dd]ebug [Ss]tatic [Ll]ib/
...
...

*.dll    #Exclude all dlls
!foo.dll #Except for foo.dll
...



To see which files are ignored by git:


$ git status --ignored



Merging


To merge branch other_branch into current one use:

$ git merge other_branch

To merge all commits from the original branch from the point of splitting up to some commit (on the original branch) into the feature branch, it is enough to state the commit id (commit hash):

$ git merge 49aec175bad17cd00733f748a2416cb46bd1706a



In this case Git will choose which merging algorithm to use. If branches did not diverge (tip of the current branch is in the chain of commits on the other branch) Git will use Fast-forward merge: it will simply move the HEAD index to the top of the other branch without creating new commits.

If branches diverged, fast-forwarding is not possible so Git uses 3-way merge. Manual solution of conflicts is necessary if different branches contain changes made on the same lines in file(s).

We can force creation of a new commit (a merge commit) even if fast-forwarding is possible:

$ git merge --no-ff other_branch



This is a preferred approach as it will keep history of merges (via merge commits).

If there are no conflicts, merge will perform git commit as well (git might open default text editor so you can confirm or edit default commit message which is "Merge...").

It is possible to merge into current branch a single commit from some other branch. This is called "cherry-picking". First you have to fetch the branch where desired commit resides and you have to know the hash of that commit you want to "cherry-pick".


$ git fetch origin some_branch
...
$ git cherry-pick commit_hash



Cherry-picking is a merge operation. Git might not be able to merge automatically some files. Such files will be listed as "Unmerged" in git status output:

$ git status
On branch my_branch
You are currently cherry-picking commit 6d91a73b53.
(fix conflicts and run "git cherry-pick --continue")
(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

modified: fileA
modified: fileB

Unmerged paths:
(use "git add ..." to mark resolution)

both modified: fileC
both modified: fileD

Untracked files:
(use "git add ..." to include in what will be committed)

fileE



We have to resolve conflicts in fileC and fileD.

Where there are merges, there are conflicts. And you don't want to scroll up and down the conflicted file in order to look for and compare lines which are different on remote and local copy. If on Windows, download and install WinMerge tool. During the installation opt in to add WinMerge's path to the system paths (Path environment variable). Once WinMerge is installed, set it as a default merge tool for Git - open C:\Users\USER\.gitconfig (we're applying this rule for all USER's repositories) and copy the following section into it:

[mergetool]
prompt = false
keepBackup = false
keepTemporaries = false
[merge]
tool = winmerge
[mergetool "winmerge"]
name = WinMerge
trustExitCode = true
cmd = "/c/Program\\ Files\\ \\(x86\\)/WinMerge/WinMergeU.exe" -u -e -dl \"Local\" -dr \"Remote\" $LOCAL $REMOTE $MERGED
[diff]
tool = winmerge
[difftool "winmerge"]
name = WinMerge
trustExitCode = true
cmd = "/c/Program\\ Files\\ \\(x86\\)/WinMerge/WinMergeU.exe" -u -e $LOCAL $REMOTE



To verify that changes have been properly applied, list local git config:

$ git config --list --global
...
mergetool.prompt=false
mergetool.keepbackup=false
mergetool.keeptemporaries=false
merge.tool=winmerge
mergetool.winmerge.name=WinMerge
mergetool.winmerge.trustexitcode=true
mergetool.winmerge.cmd=/c/Program\ Files\ \(x86\)/WinMerge/WinMergeU.exe -u -e -dl "Local" -dr "Remote" $LOCAL $REMOTE $MERGED
diff.tool=winmerge
difftool.winmerge.name=WinMerge
difftool.winmerge.trustexitcode=true
difftool.winmerge.cmd=/c/Program\ Files\ \(x86\)/WinMerge/WinMergeU.exe -u -e $LOCAL $REMOTE
...



To test launching WinMerge from the command line, we can diff current HEAD and its parent:

$ git difftool HEAD HEAD~1



This shall open Local file (working copy) on the Left-hand side pane and Remote on the Right-side pane of the WinMerge. (Remember it as Local-Left / Remote-Right)

In order to resolve conflicts in particular file, we can launch WinMerge as a merge tool and provide a file name (with its relative path):

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git mergetool fileC
Merging:
fileC

Normal merge conflict for 'fileC':
{local}: modified file
{remote}: modified file



Local file version (working copy) opens in the Left pane and Remote file version opens in a Right pane. Change file in Left pane and save changes.

After we've resolve conflicts in fileC, Git adds that file to the list of indexed files (ready to be committed):

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git status
On branch my_branch
You are currently cherry-picking commit 6d91a73b53.
(fix conflicts and run "git cherry-pick --continue")
(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

modified: fileA
modified: fileB
modified: fileC

Unmerged paths:
(use "git add ..." to mark resolution)

both modified: fileD

Untracked files:
(use "git add ..." to include in what will be committed)

fileE



Once all conflicted files are resolved, git status notifies us that we can continue cherry-picking:

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git status
On branch my_branch
You are currently cherry-picking commit 6d91a73b53.
(all conflicts fixed: run "git cherry-pick --continue")
(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

modified: fileA
modified: fileB
modified: fileC
modified: fileD

Untracked files:
(use "git add ..." to include in what will be committed)

fileE



We can now complete cherry-picking:

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git cherry-pick --continue
[my_branch a9c57b587e] JIRA-1234 Fix memory leak
Author: user_name_2
Date: Thu Mar 2 16:13:51 2017 +0100
8 files changed, 79 insertions(+), 12 deletions(-)



To resolve conflicts by simply choosing theirs or ours version of the file:

$ git checkout --theirs path_to_file
$ git checkout --ours path_to_file



To merge all files manually:

$ git merge --no-commit --no-ff merge_branch



--no-commit: merge takes place but git pretends the merge failed and does not autocommit, to give the user a chance to inspect and further tweak the merge result before committing
--no-ff: the fast forwarding is disabled, which will happen if there are no conflicts

Rebasing


Beside merging, rebasing is another way to incorporate changes from one into another branch. In case of rebasing, all commits on the current branch made after the splitting from the trunk are cut off, the latest trunk state is brought to the current branch and cut off part is attached on the top of the "trunk" part. There is no "merge commit" like in the merge.

To rebase current (e.g. feature) branch onto the latest development on the main trunk (e.g. develop branch) first make sure you've updated both branches and that current branch is the feature branch, then execute:

$ git rebase dev



Git configuration


Git keeps configuration at three levels: System, Global and Local. Local overrides Global and Global overrides System.

System


Configuration at this level applies across all repositories of all users at the local system. It is located in file /etc/gitconfig.

To list configuration that applies at this level (all repositories, all users):

$ git config --list --system



Global


Global config file contains configuration which applies for all repositories of the currently logged user on the local machine. Its content, if only email and name are set (and are set to be same across all repositories), looks similar to this:


$ cat ~/.gitconfig
[user]
email = you@example.com
name = Your Name


To list configuration that applies at this level (all repositories of current user):

$ git config --list --global
user.email=john.johnson@example.com
user.name=John Johnson


To check which user's email address is applied to the current repository:

$ git config --get user.email
user.name@example.com


To set global email and username:

$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"


*** Please tell me who you are.

To set your account's default identity.

$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"


Omit --global to set the identity only in this repository.
For system/local scope use --system/--local args.

Local


Local config file contains configuration which applies only for the repository it belongs to. Its content can be similar to this:


$ cat .git/config
[core]
   repositoryformatversion = 0
   filemode = true
   bare = false
   logallrefupdates = true
[remote "origin"]
   url = https://repo_domain_name/YourName/your_repo.git
   fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
   remote = origin
   merge = refs/heads/master



To list configuration that applies at this level (local repository, for current user):

$ git config --list --local
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://repo_domain_name/YourName/your_repo.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master



Configuration set for a particular repository (in its local config file) overrides configuration from global config file.

If you try to commit but have not set email and username, the following error occurs:


$ git commit

*** Please tell me who you are.

Run

git config --global user.email "you@example.com"
git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'YourName@YourName-YourComputerName.(none)')

To remove some configuration e.g. user.email from a local config:

$ git config --local --unset user.email

To edit e.g. global configuration file (this can also be used to remove/delete any config):

$ git config --global --edit

If we made a commit locally but then want to change the author (name and email address), we can fix the identity used for this commit with:

$ git commit --amend --reset-author

We can amend the author of the last local commit which has not yet been pushed with:

$ git commit --amend --author="Name Surname "

How to change the author of a commit - Software Development 
(Before everything I set locally correct user.email and then used "$ git commit --amend --reset-author" instead of git commit --amend --author="Author ")
 
To change the author of the last commit after it was pushed to remote:
 
$ git commit --amend --author="Name Surname <user@example.com>"
$ git push origin main --force
 

How to set URL of the remote


If you clone remote repository, the URL you use will be set as value if remote.origin.url. That can be either HTTPS or SSH-based URL. If you used HTTPS-based URL to clone the repository but now want to switch to SSH-based URL, you can do the following:

Check what are current remotes (I am using GitHub repo hosting in the example):

$ git remote -v
origin https://github.com/YourName/repo.git (fetch)
origin https://github.com/YourName/repo.git (push)



Set new URL (Changing a remote's URL):

$ git remote set-url origin git@github.com:YourName/repo.git



Verify change:

$ git remote -v
origin git@github.com:YourName/repo.git (fetch)
origin git@github.com:YourName/repo.git (push)



How to set up SSH authentication with Git server


Create SSH key pair:

$ ssh-keygen -t rsa -b 4096 -C john.johnson@example.com



Run local SSH agent:

$ eval "$(ssh-agent -s)"



Add identity:

$ ssh-add ~/.ssh/id_rsa
Identity added: /home/user_name/.ssh/id_rsa (/home/user_name/.ssh/id_rsa)



Check if have xclip installed. If not, install it:

$ sudo apt-get install xclip



Copy public key to clipboard:

$ xclip -sel clip < ~/.ssh/id_rsa.pub


..and paste it to designated place in the remote Git hosting system. (In case of GitHub, Settings-->SSH and GPG keys-->Add SSH key)

Working with tags


To list all local tags:

$ git tag


To list all remote tags:

$ git ls-remote --tags origin


To pull all tags from remote:

$ git pull --prune --tags


To add a tag to the current commit:

$ git tag -a alpha-1-release -m "MyProduct Alpha 1 release"

If -m argument is omitted, git will open a terminal text editor (vi/vim...) and prompt you to add a tag message there.


To add a tag to the commit specified by its hash:

$ git tag -a alpha-1-release 4fceb02 -m "MyProduct Alpha 1 release"


To push tags to remote:

$ git push --tags

or

$ git push --tags origin master

To push a single tag:

$ git push origin <tag-name> 

If some tags already exist on remote, git will issue an error but new tags will nevertheless be pushed to remote:
 
$ git push --tags origin master
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 175 bytes | 175.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0)
remote: GCV check in progress...
To git.company.com:mainproduct/product.git
 * [new tag]         v8.1.0-rc1 -> v8.1.0-rc1
 ! [rejected]        v6.11.2 -> v6.11.2 (already exists)
error: failed to push some refs to 'git@git.company.com:mainproduct/product.git'
hint: Updates were rejected because the tag already exists in the remote.



To list all annotated tags with their messages:

$ git tag -n
v0.0.1          my-product v0.0.1


To see tags and hashes of matching commits:

$ git show-ref --tags
05dab0b04b74bba928e97fd1305130e59692141e refs/tags/my_tag_1
47acb8f214a5a426bd68759b19154897a1e4a3d2 refs/tags/my_tag_2
6d24c915fab9a6509f1f9e7915a55b5d8de3a4da refs/tags/my_tag_3



To rename tag:

$ git tag new old
$ git tag -d old
$ git push origin :refs/tags/old
$ git push --tags


Note that colon (:) is used (it instructs Git to remove tag from remote).


To create a new branch from a specific tag:

$ git checkout -b my_branch tag_name


To see all commits of some particular contributor:

$ git log --author=Bojan


To update (e.g. to reassign it to some other commit) the existing tag (locally):

$ git tag -f -a v1.1.0 129d7e3e45f62ce5af35e254c407b8dbbe9494e5 -m "Releasing v1.1.0"
Updated tag 'v1.1.0' (was 4d2eefa)

We also need to force push it to remote:

$ git push origin v1.1.0
To git.company.com:project/app.git
 ! [rejected]        v1.1.0 -> v1.1.0 (already exists)
error: failed to push some refs to 'git@git.company.com:project/app.git'
hint: Updates were rejected because the tag already exists in the remote.

$ git push -f origin v1.1.0
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 169 bytes | 169.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0)
remote: GCV check in progress...
To git.company.com:project/app.git
 + 4d2eefa...55b25e1 v1.1.0 -> v1.1.0 (forced update)



Forking


How to synchronize forked repository with the original one?

Configuring a remote for a fork
Example:
$ git remote -v
origin https://github.com/BojanKomazec/jetson-inference.git (fetch)
origin https://github.com/BojanKomazec/jetson-inference.git (push)

$ git remote add upstream git@github.com:dusty-nv/jetson-inference.git

$ git remote -v
origin https://github.com/BojanKomazec/jetson-inference.git (fetch)
origin https://github.com/BojanKomazec/jetson-inference.git (push)
upstream git@github.com:dusty-nv/jetson-inference.git (fetch)
upstream git@github.com:dusty-nv/jetson-inference.git (push)

Syncing a fork

Versioning Systems


Pull request vs Merge request

Debugging Git Commands


$ GIT_TRACE=true \
GIT_CURL_VERBOSE=true \
GIT_SSH_COMMAND="ssh -vvv" \
git clone https://git.example.com/path/to/repo.git

...helped me identifying why git clone hanged on my Ubuntu. In the output I found that the git server was returning HTTP error 401 (Unathorized):

15:53:37.750801 http.c:623              <= Recv header: HTTP/2 401


Bash scripts cheat sheet



Bash is:
  • Unix shell (command-line interpreter, command processor). It can process commands that user types in text window (terminal) or commands written in a file (shell script).
  • command language 
This article contains some my notes about writing shell scripts.

Here are some useful utility functions (source):


yell() { echo "$0: $*" >&2; }
die() { yell "$*"; exit 111; }
try() { "$@" || die "cannot $*"; }
asuser() { sudo su - "$1" -c "${*:2}"; }


  • yell: print the script name and all arguments to stderr
    • $0 is the path to the script
    • $* are all arguments
    • >&2 means > redirect stdout to & pipe 2. pipe 1 would be stdout itself.
  • die does the same as yell, but exits with a non-0 exit status, which means “fail”.
  • try uses the || (boolean OR), which only evaluates the right side if the left one failed.
  • $@ is all arguments again, but different.



set -e 

At the top of the script this will cause it to exit upon any command which errors.

 

set -x

Setting the -x tells Bash to print out the statements as they are being executed. It can be very useful as a logging facility and for debugging when you need to know which statements were execute and in what order. [from Bash set -x to print statements as they are executed]

It is possible to combine two above commands into a single one:

set -ex

Getting the current directory:


$ pwd


To find some executable:


$ where perl
C:\cygwin64\bin\perl.exe



$ which perl
/usr/bin/perl


Viewing the whole content of the file:


$ cat /bin/man2html


Viewing only the first line of the file:

$ head -n 1 file.txt


Viewing only the last line of the file:

$ tail -n 1 file.txt


Editing file:

$ vi /bin/man2html


Initially, vi opens in command mode.

i - to enter edit (insert) mode
ESC - to exit edit mode
:wq - write and quit
:q! - quit without saving changes


Logging



It is a good practice to create a log file for each command/process/script/application that we run. We append both stdout and stderr into file:

my_command >> /tmp/my_command.log 2>&1


----
 

Resources:

 

How to compile OpenVpn for Windows from source code

OpenVpn suggests cross-compilation - compiling Windows executables with Unix build toolchain. This can be accomplished either by using Linux/Unix or Cygwin on Windows.

This article demonstrates compiling OpenVpn with Cygwin on Windows.

The first step is to get build scripts from https://github.com/OpenVPN/openvpn-build.

When cloning OpenVpn repositories to Windows machine, make sure Git client for Windows does not automatically convert Unix-style line endings (LF) in source files into Windows-style line endings (CRLF). If this happens UNIX tools that run in Cygwin will report an error complaining about extra CR characters. So, when cloning, use the following command:


$ git clone https://github.com/OpenVPN/openvpn-build.git --config core.autocrlf=false



Required Cygwin packages are listed here: https://github.com/OpenVPN/openvpn-build/tree/master/generic

Package name Cygwin installer path
diffutils All --> Utils
m4All --> Interpreters
makeAll -> Devel
mingw64-i686-binutilsAll -> Devel
mingw64-i686-gcc-coreAll -> Devel
mingw64-i686-headersAll -> Devel
mingw64-i686-pthreadsAll -> Devel
mingw64-i686-runtimeAll -> Devel
mingw64-x86_64-binutilsAll -> Devel
mingw64-x86_64-gcc-coreAll -> Devel
mingw64-x86_64-headersAll -> Devel
mingw64-x86_64-pthreadsAll -> Devel
mingw64-x86_64-runtimeAll -> Devel
patchAll -> Devel
patchutilsAll -> Devel
perlAll --> Interpreters
unzipAll --> Archive
wgetAll --> Web


Download Perl script man2html and save it as a file with no extensions in /bin directory (C:\cygwin64\bin). Script URL is: http://cpansearch.perl.org/src/EHOOD/man2html3.0.1/man2html. If build is run with no having this script, the following error is reported at the output:


checking for man2html... no
configure: error: man2html is required for win32
FATAL: Configure pkcs11-helper



Make sure that man2html's shebang contains correct path to perl. In order to find perl's path in Cygwin we can do the following:


$ which perl
/usr/bin/perl



In my case shebang

#!/usr/local/bin/perl


has to be replaced with:

#!/usr/bin/perl



In order to build x64 version of OpenVpn go to ../OpenVpn/openvpn-build/generic and execute:

$ IMAGEROOT=`pwd`/image-win64 CHOST=x86_64-w64-mingw32 CBUILD=i686-pc-cygwin ./build




A bit about Cygwin

Checking whether some particular package has been installed (e.g. ca-certificates):

$ cygcheck -c ca-certificates
Cygwin Package Information
Package Version Status
ca-certificates 2.9-1 OK



http://stackoverflow.com/questions/9224298/how-do-i-fix-certificate-errors-when-running-wget-on-an-https-url-in-cygwin

How to create and submit Git patch

Some open-source projects prefer submitting Git patch files to pull requests.

OpenVpn: https://community.openvpn.net/openvpn/wiki/DeveloperDocumentation
Linux kernel: https://www.kernel.org/doc/Documentation/SubmittingPatches

https://shkspr.mobi/blog/2014/04/submitting-trivial-linux-kernel-patches/
https://kernelnewbies.org/FirstKernelPatch
https://www.kernel.org/doc/Documentation/SubmittingPatches
https://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html
https://git-scm.com/docs/git-format-patch

http://who-t.blogspot.co.uk/2009/12/on-commit-messages.html

Tuesday, 23 August 2016

Why is async void bad?

Methods returning void are not awaitable. await operator can be applied only to methods which are returning Task or Task

Let's assume that method Bar is async void. Consequences of  having method Foo calling method Bar are the following:

  • This is a fire and forget model: Foo can't get information when Bar terminates as it can't await on async void method. Foo continues to run before Bar call is completed.
  • If Bar throws an exception, Foo can't catch it as there is no Task object to carry the exception object.
For these reasons async void methods shall be avoided. But this rule has a single exception; top-level events (e.g. GUI events) of void type can be async,

Links:


https://channel9.msdn.com/Series/Three-Essential-Tips-for-Async/Tip-1-Async-void-is-for-top-level-event-handlers-only

http://blog.stephencleary.com/2014/02/synchronous-and-asynchronous-delegate.html


http://stackoverflow.com/questions/15522900/how-to-safely-call-an-async-method-in-c-sharp-without-await

http://stackoverflow.com/questions/23285753/how-to-await-on-async-delegate

http://stackoverflow.com/questions/20624667/how-do-you-implement-an-async-action-delegate-method

Friday, 8 July 2016

Logic in getters and setters

Setters:
  • used to set a (new) value of a property
  • expected to be fast
  • value can be validated (e.g. null check, range check etc...); failed validation CAN throw exception
  • event which publishes information that property's value has been changed can be fired; event handlers are not under publisher's control and they CAN throw exception


Getters:
  • return value of the property
  • expected to be fast
  • expected to be idempotent (i.e. no destructive actions should be performed there)
  • expected to cause no side effects on the object (no changing values)
  • MUST NOT throw exception (otherwise would break principle of least surprise)
  • use memoisation (lazy computation/evaluation)



References:
Property Design (MSDN)
http://stackoverflow.com/questions/495864/logic-in-get-part-of-property-good-practice
http://stackoverflow.com/questions/2923116/is-it-a-good-practice-to-implement-logic-in-properties
http://stackoverflow.com/questions/1488472/best-practices-throwing-exceptions-from-properties