Sunday, 28 August 2016

Git cheat sheet


Switching repositories and branches


Cloning a remote repository:

$ git clone https://github.com/OpenVPN/openvpn-build.git



Cloning a remote repository with preserving UNIX-style line endings:

$ git clone https://github.com/OpenVPN/openvpn-build.git --config core.autocrlf=false



Getting a list of all local branches:

$ git branch



Deleting local branch:

$ git branch -d my_branch
Deleted branch my_branch (was 6d5921868b).



Getting a list of all remote branches:

$ git branch -r



Example output:

origin/HEAD -> origin/master
origin/master
origin/release/2.3



Checking out the remote branch (for the first time):

$ git checkout -b branch_name remote_name/branch_name



Example:

$ git checkout -b release/2.3 origin/release/2.3
Branch release/2.3 set up to track remote branch release/2.3 from origin.
Switched to a new branch 'release/2.3'



Or:

$ git fetch
...
* [new branch] ModuleA/Project2/JIRA-123 -> origin/ModuleA/Project2/JIRA-123
...

$ git checkout ModuleA/Project2/JIRA-123
Checking out files: 100% (1485/1485), done.
Branch ModuleA/Project2/JIRA-123 set up to track remote branch ModuleA/Project2/JIRA-123 from origin.
Switched to a new branch 'ModuleA/Project2/JIRA-123'


If you get the following error:

$ git checkout ModuleA/Project2/JIRA-123
error: Your local changes to the following files would be overwritten by checkout:
Module2/SomeDir/SomeFile.cpp
Please commit your changes or stash them before you switch branches.
Aborting


...you can force checking out with -f switch:

$ git checkout -f ModuleA/Project2/JIRA-123


To create new local branch:

$ git checkout -b issue-007
Switched to a new branch "issue-007"



This is the compact version for:

$ git branch issue-007
$ git checkout issue-007



Switching to another local branch:

$ git checkout branch_name



Example:

$ git checkout master
Your branch is behind 'origin/master' by 179 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
Switched to branch 'master'



To checkout commit specified with its hash:

$ git checkout 3e6dc47cde0e45bb3198ac4618367afc57ddf6a3
$ git checkout 3e6dc47cde0e45bb3198ac4618367afc57ddf6a3
Checking out files: 100% (5111/5111), done.
M features/feature1
M framework/module2
Note: checking out '3e6dc47cde0e45bb3198ac4618367afc57ddf6a3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at 3e6dc47cde... Fixed EULA BK brand;



If local branch is behind the remote by 1 or more commits, it can be updated:

$ git pull



Note that this takes local repository in the state of detached head. HEAD is symbolic name for the currently checked out commit and in this state it points directly to a commit instead of pointing to a branch (e.g. refs/heads/master).

To fix detached head (and delete all local changes) do the following:

$ git branch temp
$ git checkout
$ git branch -d temp



In order to switch from one to another branch we have to have all changes either committed or stashed. To stash local changes do the following:

$ git stash
Saved working directory and index state WIP on my_branch: 6d94b73b53 JIRA-123 Fixed memory leak
HEAD is now at 6d94b73b53 JIRA-123 Fixed memory leak



To list all stashes on the current branch:

$ git stash list
stash@{0}: WIP on my_branch: 6d94b73b53 JIRA-123 Fixed memory leak



If single stash is saved, it can be applied with this:

$ git stash apply



If you were on some branch (e.g. master) and made some changes but now want to move them to a new branch and revert working version of master to the state before your changes, do the following:


$ git stash
$ git checkout -b my_new_branch
$ git stash pop


Checking the current status of the local repository


Checking out the status of the current local repository:

$ git status



Checking the name of the remote:

$ git remote
origin





Getting SHA-1 of the last commit on the current branch:

$ git rev-parse HEAD
e39616c0cc351c027cff762008b2da77ee1cc6cc



Getting short version of the hash:

$ git rev-parse --short HEAD
e39616c0cc



To see what has changed in the last commit:

$ git show commit_hash



To see which files have been modified and how in last N commits in the specific directory:

$ git log option -N path_to_directory


path_to_directory can be relative path from the current directory.
option can be:
-p - full patch
--stat - numbers of changed lines
--numstat - like --stat but machine-readable
--name-status shows the file name and status: Added (A), Copied (C), Deleted (D), Modified (M), Renamed (R).
--name-only - just the filenames

To display commit history, branches, merges...all in a single graph with a one line comments use this very cool command:

$ git log --oneline --abbrev-commit --all --graph



Expand the previous command so it shows also branch and tag labels:

$ git log --oneline --abbrev-commit --all --graph --decorate --color



Adding changed file (already tracked but modified) or new file to the commit:

$ git add my_file



The same applies for directories.

In order to stage all files (both newly added files and also old modified) we can use:

$ git add .



It is possible to use wildcards to denote all files from some directory:

$ git add dir_1/file_1 dir_1/file_2 dir_1/dir_2/*.*



Add files from dir_1 which have same name but different extension:

$ git add dir_1/file_1.*



If we want to delete some file, we can delete it in usual way and then execute git add:

$ rm my_file
$ git add my_file



This can be done with a single git command:

$ git rm add my_file



Modified file can be unstaged (in the working directory), staged (indexed/cached) and commited (HEAD points to its last commit).

If we want to remove staged file from the commit index (to "unstage" it) but at the same time to keep changes in the file:

$ git reset some_file



If we want to revert unstaged (unindexed) changes to the last committed revision of file we can simply check out the file:

$ git checkout some_file



Working with submodules


Let's say that ProjectA resides in Repository1 and is dependent (uses) ProjectB which resides in Repository2. ProjectB might be shared among multiple projects and its development is independent from development of ProjectA.

When we checkout ProjectA, we would like also to pull the latest ProjectB. Submodules provide a way for this. Repository2 will be pulled in a subdirectory in Repository1.

How to check if current repository have any submodules?

$ git submodule
f97bae646f5341216752c4c19950e4b4c15bf616 dir1/dir2/submodule1 (remotes/origin/HEAD)
6e1db0f970c11ec571cf17516d5d8a4b93bd66ce dir3/submodule2 (remotes/origin/HEAD)
3ac8aba9e57c273e249724f8458622e51dbea233 dir4/submodule3 (remotes/origin/HEAD)



What happens if you pull repository but not its submodules (you don't update submodules)? Git can detect that there are new commits in submodules:

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git checkout -- ..." to discard changes in working directory)

modified: dir1/dir2/submodule1 (new commits)
modified: dir3/submodule2 (new commits)
modified: dir4/submodule3 (new commits)

no changes added to commit (use "git add" and/or "git commit -a")



How to clone repository and its submodules?

$ git clone --recursive ssh://git@mydomain.com:8082/my_repo.git



How to update submodules?

$ git submodule update --recursive --remote
Submodule path 'dir1/dir2/submodule1': checked out '6e1db0f970c21ec571cf17516d5d8d4b93bd66ce'
Submodule path 'dir3/submodule2': checked out '3ac8aba9e57c273e244724f845862ee51dbea233'
Submodule path 'dir4/submodule3': checked out '8d58faf56d1a3f5ab50da24ef66c624be9678ba6'



To revert them to the latest locally checked out version:

$ git submodule update
Submodule path 'dir1/dir2/submodule1': checked out 'd5011032541d2fb81e484c78ff8306633a219c66'
Submodule path 'dir3/submodule2': checked out 'b15ab11cd92e1f88ed2b334d0f01a8130ddae28a'
Submodule path 'dir4/submodule3': checked out '3ed4ac1d8dc8bd9862d9e5be2a18f22713899b85'



Creating new commits


Saving changes into the repository (creating a new commit containing changes in all staged files):

$ git commit


After this, there will be no staged files.

Git will prompt you to enter the commit message (in vi editor; ESC to exit editing mode, :wq to save changes and quit). Commit message can be specified when committing:


$ git commit -m "Code updated"



Staging, committing and specifying message can be done in one go:

$ git commit -a -m "Code updated"



If commit has not been pushed to remote yet, you can still change commit message:

$ git commit --amend



Pushing local commits to remote


If local branch is ahead of remote by 1 or more commits, they can be published (pushed to remote):

$ git push



In the previous command we didn't specify remote branch which means we assumed it's specified in the git configuration. If default remote branch has not been set specified there we might get message like this:


$ git push
warning: push.default is unset; its implicit value has changed in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the traditional behaviour, use:

git config --global push.default matching

To squelch this message and adopt the new behaviour now, use:

git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

Since Git 2.0, Git defaults to the more conservative 'simple'
behaviour, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)



We can either set push.default or simply specify remote and branch within pull command:


$ git push origin master



To set push.default use:

$ git config --global push.default matching



Verification:


$ git config --list --global
user.email=user.name@example.com
user.name=User Name
push.default=matching




After publishing local commits, local branch is up-to-date with remote.

It is possible to update another branch without checking it out:

user@my_computer ~/Documents/my_dev/my_projects/ProjectA (some_branch)
$ git pull origin master



The code above will update master branch although we're on some_branch.

If you've created local branch and want to push it to the remote (e.g. origin) prior to setting upstream branch, the following error occurs:

$ git push origin
fatal: The current branch my_new_branch has no upstream branch.
To push the current branch and set the remote as upstream, use

git push --set-upstream origin my_new_branch



Do what Git suggests:


$ git push --set-upstream origin my_new_branch




Working with local changes


Modified file can be unstaged (in the working directory), staged (indexed/cached) and commited (HEAD points to its last commit). It is possible to show differences between any of these three states of the file.

To view all differences between unstaged (working) and staged (cached/indexed) version of the file:

$ git diff my_file



To view all differences between staged (cached/indexed) and last committed (HEAD) version of the file:

$ git diff --cached my_file



To view all differences between unstaged (working) and last committed (HEAD) version of the file:

$ git diff HEAD my_file



To list all staged (cached/indexed) files:

git diff --name-only --cached



To view all differences in all modified but unstaged and staged files in the current directory:

$ git diff .



If you've created some commits locally and haven't pushed them to remote your local branch will be ahead by N commits:

$ git status
On branch branch_a
Your branch is ahead of 'origin/branch_a' by 84 commits.
(use "git push" to publish your local commits)
Untracked files:
(use "git add ..." to include in what will be committed)

A/a.cpp
A/B
A/c.h

nothing added to commit but untracked files present (use "git add" to track)



If you want to dismiss these commits and pull the remote, you can do the following:


$ git reset --hard origin/branch_a
HEAD is now at 0cdeba9a9a Implemented new Service skeleton. Fixed typo.



To get some submodule to some particular commit:


$ cd module_xyz
$ git reset --hard 6028a4a42446db118e615658382dc588caf71766
...
$ cd ..
$ git add module_xyz
$ git commit -m "Submodule module_xyz set to particular commit"



Managing what shall be under revision control


It is often necessary to prevent adding under git control files with certain name and/or extension (e.g. temporary, backup, log, intermediate files etc...). Ignoring this files is set in a file called .gitignore which is located in repository's root directory. This file is like any other and can be updated and pushed to remote. To see its content we can use:

$ cat ../../.gitignore
## Ignore Visual Studio temporary files, build results

# User-specific files
*.suo
*.user
*.userosscache
*.sln.docstates

# cUser-specific files (MonoDevelop/Xamarin Studio)
*.userprefs

# Build results
BUILDS/
BUILDSENG/
COMPILE/
log/
[Rr]elease/
[Rr]elease [Ll]ib/
[Rr]elease [Ss]tatic/
[Rr]elease [Ss]tatic [Ll]ib/
[Dd]ebug/
[Dd]ebug [Ll]ib/
[Dd]ebug [Ss]tatic/
[Dd]ebug [Ss]tatic [Ll]ib/
...
...
...



To see which files are ignored by git:


$ git status --ignored



Merging


To merge branch other_branch into current one use:

$ git merge other_branch



In this case Git will choose which merging algorithm to use. If branches did not diverge (tip of the current branch is in the chain of commits on the other branch) Git will use Fast-forward merge: it will simply move the HEAD index to the top of the other branch without creating new commits.

If branches diverged, fast-forwarding is not possible so Git uses 3-way merge. Manual solution of conflicts is necessary if different branches contain changes made on the same lines in file(s).

We can force creation of a new commit (a merge commit) even if fast-forwarding is possible:

$ git merge --no-ff other_branch



This is a preferred approach as it will keep history of merges (via merge commits).

It is possible to merge into current branch a single commit from some other branch. This is called "cherry-picking". First you have to fetch the branch where desired commit resides and you have to know the hash of that commit you want to "cherry-pick".


$ git fetch origin some_branch
...
$ git cherry-pick commit_hash



Cherry-picking is a merge operation. Git might not be able to merge automatically some files. Such files will be listed as "Unmerged" in git status output:

$ git status
On branch my_branch
You are currently cherry-picking commit 6d91a73b53.
(fix conflicts and run "git cherry-pick --continue")
(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

modified: fileA
modified: fileB

Unmerged paths:
(use "git add ..." to mark resolution)

both modified: fileC
both modified: fileD

Untracked files:
(use "git add ..." to include in what will be committed)

fileE



We have to resolve conflicts in fileC and fileD.

Where there are merges, there are conflicts. And you don't want to scroll up and down the conflicted file in order to look for and compare lines which are different on remote and local copy. If on Windows, download and install WinMerge tool. During the installation opt in to add WinMerge's path to the system paths (Path environment variable). Once WinMerge is installed, set it as a default merge tool for Git - open C:\Users\USER\.gitconfig (we're applying this rule for all USER's repositories) and copy the following section into it:

[mergetool]
prompt = false
keepBackup = false
keepTemporaries = false
[merge]
tool = winmerge
[mergetool "winmerge"]
name = WinMerge
trustExitCode = true
cmd = "/c/Program\\ Files\\ \\(x86\\)/WinMerge/WinMergeU.exe" -u -e -dl \"Local\" -dr \"Remote\" $LOCAL $REMOTE $MERGED
[diff]
tool = winmerge
[difftool "winmerge"]
name = WinMerge
trustExitCode = true
cmd = "/c/Program\\ Files\\ \\(x86\\)/WinMerge/WinMergeU.exe" -u -e $LOCAL $REMOTE



To verify that changes have been properly applied, list local git config:

$ git config --list --global
...
mergetool.prompt=false
mergetool.keepbackup=false
mergetool.keeptemporaries=false
merge.tool=winmerge
mergetool.winmerge.name=WinMerge
mergetool.winmerge.trustexitcode=true
mergetool.winmerge.cmd=/c/Program\ Files\ \(x86\)/WinMerge/WinMergeU.exe -u -e -dl "Local" -dr "Remote" $LOCAL $REMOTE $MERGED
diff.tool=winmerge
difftool.winmerge.name=WinMerge
difftool.winmerge.trustexitcode=true
difftool.winmerge.cmd=/c/Program\ Files\ \(x86\)/WinMerge/WinMergeU.exe -u -e $LOCAL $REMOTE
...



To test launching WinMerge from the command line, we can diff current HEAD and its parent:

$ git difftool HEAD HEAD~1



This shall open Local file (working copy) on the Left-hand side pane and Remote on the Right-side pane of the WinMerge. (Remember it as Local-Left / Remote-Right)

In order to resolve conflicts in particular file, we can launch WinMerge as a merge tool and provide a file name (with its relative path):

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git mergetool fileC
Merging:
fileC

Normal merge conflict for 'fileC':
{local}: modified file
{remote}: modified file



Local file version (working copy) opens in the Left pane and Remote file version opens in a Right pane. Change file in Left pane and save changes.

After we've resolve conflicts in fileC, Git adds that file to the list of indexed files (ready to be committed):

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git status
On branch my_branch
You are currently cherry-picking commit 6d91a73b53.
(fix conflicts and run "git cherry-pick --continue")
(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

modified: fileA
modified: fileB
modified: fileC

Unmerged paths:
(use "git add ..." to mark resolution)

both modified: fileD

Untracked files:
(use "git add ..." to include in what will be committed)

fileE



Once all conflicted files are resolved, git status notifies us that we can continue cherry-picking:

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git status
On branch my_branch
You are currently cherry-picking commit 6d91a73b53.
(all conflicts fixed: run "git cherry-pick --continue")
(use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

modified: fileA
modified: fileB
modified: fileC
modified: fileD

Untracked files:
(use "git add ..." to include in what will be committed)

fileE



We can now complete cherry-picking:

USER@Machine ~/dev/project (my_branch|CHERRY-PICKING)
$ git cherry-pick --continue
[my_branch a9c57b587e] JIRA-1234 Fix memory leak
Author: user_name_2
Date: Thu Mar 2 16:13:51 2017 +0100
8 files changed, 79 insertions(+), 12 deletions(-)



Git configuration


Git keeps configuration at three levels: System, Global and Local. Local overrides Global and Global overrides System.

System


Configuration at this level applies across all repositories of all users at the local system. It is located in file /etc/gitconfig.

To list configuration that applies at this level (all repositories, all users):

$ git config --list --system




Global


Global config file contains configuration which applies for all repositories of the currently logged user on the local machine. Its content, if only email and name are set (and are set to be same across all repositories), looks similar to this:


$ cat ~/.gitconfig
[user]
email = you@example.com
name = Your Name



To list configuration that applies at this level (all repositories of current user):

$ git config --list --global
user.email=bojan.komazec@gmail.com
user.name=Bojan Komazec



To set global email and username:


git config --global user.email "you@example.com"
git config --global user.name "Your Name"



Local


Local config file contains configuration which applies only for the repository it belongs to. Its content can be similar to this:


$ cat .git/config
[core]
   repositoryformatversion = 0
   filemode = true
   bare = false
   logallrefupdates = true
[remote "origin"]
   url = https://repo_domain_name/YourName/your_repo.git
   fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
   remote = origin
   merge = refs/heads/master



To list configuration that applies at this level (local repository, for current user):

$ git config --list --local
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://repo_domain_name/YourName/your_repo.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master



Configuration set for a particular repository (in its local config file) overrides configuration from global config file.

If you try to commit but have not set email and username, the following error occurs:


$ git commit

*** Please tell me who you are.

Run

git config --global user.email "you@example.com"
git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'YourName@YourName-YourComputerName.(none)')



How to set URL of the remote


If you clone remote repository, the URL you use will be set as value if remote.origin.url. That can be either HTTPS or SSH-based URL. If you used HTTPS-based URL to clone the repository but now want to switch to SSH-based URL, you can do the following:

Check what are current remotes (I am using GitHub repo hosting in the example):

$ git remote -v
origin https://github.com/YourName/repo.git (fetch)
origin https://github.com/YourName/repo.git (push)



Set new URL:

$ git remote set-url origin git@github.com:YourName/repo.git



Verify change:

$ git remote -v
origin git@github.com:YourName/repo.git (fetch)
origin git@github.com:YourName/repo.git (push)



How to set up SSH authentication with Git server


Create SSH key pair:

$ ssh-keygen -t rsa -b 4096 -C bojan.komazec@gmail.com



Run local SSH agent:

$ eval "$(ssh-agent -s)"



Add identity:

$ ssh-add ~/.ssh/id_rsa
Identity added: /home/user_name/.ssh/id_rsa (/home/user_name/.ssh/id_rsa)



Check if have xclip installed. If not, install it:

$ sudo apt-get install xclip



Copy public key to clipboard:

$ xclip -sel clip < ~/.ssh/id_rsa.pub


..and paste it to designated place in the remote Git hosting system. (In case of GitHub, Settings-->SSH and GPG keys-->Add SSH key)

Working with tags


To list all local tags:

$git tag



To list all remote tags:

$ git ls-remote --tags origin



To pull all tags from remote:

$ git pull --prune --tags



To add a tag to the current commit:

$ git tag -a alpha-1-release -m "MyProduct Alpha 1 release"



To add a tag to the commit specified by its hash:

$ git tag -a alpha-1-release 4fceb02 -m "MyProduct Alpha 1 release"



To push tags to remote:

$ git push --tags origin master



To see tags and hashes of matching commits:

$ git show-ref --tags
05dab0b04b74bba928e97fd1305130e59692141e refs/tags/my_tag_1
47acb8f214a5a426bd68759b19154897a1e4a3d2 refs/tags/my_tag_2
6d24c915fab9a6509f1f9e7915a55b5d8de3a4da refs/tags/my_tag_3



To rename tag:

$ git tag new old
$ git tag -d old
$ git push origin :refs/tags/old
$ git push --tags



Note that colon (:) is used (it instructs Git to remove tag from remote).

To create a new branch from a specific tag:

$ git checkout -b my_branch tag_name



To see all commits of some particular contributor:

$ git log --author=Bojan



Unix bash cheat sheet


Getting the current directory:

$ pwd



To find some executable:


$ where perl
C:\cygwin64\bin\perl.exe




$ which perl
/usr/bin/perl



Viewing the whole content of the file:


$ cat /bin/man2html



Viewing only the first line of the file:

$ head -n 1 file.txt



Viewing only the last line of the file:

$ tail -n 1 file.txt



Editing file:

$ vi /bin/man2html



Initially, vi opens in command mode.

i - to enter edit (insert) mode
ESC - to exit edit mode
:wq - write and quit
:q! - quit without saving changes

How to compile OpenVpn for Windows from source code

OpenVpn suggests cross-compilation - compiling Windows executables with Unix build toolchain. This can be accomplished either by using Linux/Unix or Cygwin on Windows.

This article demonstrates compiling OpenVpn with Cygwin on Windows.

The first step is to get build scripts from https://github.com/OpenVPN/openvpn-build.

When cloning OpenVpn repositories to Windows machine, make sure Git client for Windows does not automatically convert Unix-style line endings (LF) in source files into Windows-style line endings (CRLF). If this happens UNIX tools that run in Cygwin will report an error complaining about extra CR characters. So, when cloning, use the following command:


$ git clone https://github.com/OpenVPN/openvpn-build.git --config core.autocrlf=false



Required Cygwin packages are listed here: https://github.com/OpenVPN/openvpn-build/tree/master/generic

Package name Cygwin installer path
diffutils All --> Utils
m4All --> Interpreters
makeAll -> Devel
mingw64-i686-binutilsAll -> Devel
mingw64-i686-gcc-coreAll -> Devel
mingw64-i686-headersAll -> Devel
mingw64-i686-pthreadsAll -> Devel
mingw64-i686-runtimeAll -> Devel
mingw64-x86_64-binutilsAll -> Devel
mingw64-x86_64-gcc-coreAll -> Devel
mingw64-x86_64-headersAll -> Devel
mingw64-x86_64-pthreadsAll -> Devel
mingw64-x86_64-runtimeAll -> Devel
patchAll -> Devel
patchutilsAll -> Devel
perlAll --> Interpreters
unzipAll --> Archive
wgetAll --> Web


Download Perl script man2html and save it as a file with no extensions in /bin directory (C:\cygwin64\bin). Script URL is: http://cpansearch.perl.org/src/EHOOD/man2html3.0.1/man2html. If build is run with no having this script, the following error is reported at the output:


checking for man2html... no
configure: error: man2html is required for win32
FATAL: Configure pkcs11-helper



Make sure that man2html's shebang contains correct path to perl. In order to find perl's path in Cygwin we can do the following:


$ which perl
/usr/bin/perl



In my case shebang

#!/usr/local/bin/perl


has to be replaced with:

#!/usr/bin/perl



In order to build x64 version of OpenVpn go to ../OpenVpn/openvpn-build/generic and execute:

$ IMAGEROOT=`pwd`/image-win64 CHOST=x86_64-w64-mingw32 CBUILD=i686-pc-cygwin ./build




A bit about Cygwin

Checking whether some particular package has been installed (e.g. ca-certificates):

$ cygcheck -c ca-certificates
Cygwin Package Information
Package Version Status
ca-certificates 2.9-1 OK



http://stackoverflow.com/questions/9224298/how-do-i-fix-certificate-errors-when-running-wget-on-an-https-url-in-cygwin

How to create and submit Git patch

Some open-source projects prefer submitting Git patch files to pull requests.

OpenVpn: https://community.openvpn.net/openvpn/wiki/DeveloperDocumentation
Linux kernel: https://www.kernel.org/doc/Documentation/SubmittingPatches

https://shkspr.mobi/blog/2014/04/submitting-trivial-linux-kernel-patches/
https://kernelnewbies.org/FirstKernelPatch
https://www.kernel.org/doc/Documentation/SubmittingPatches
https://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html
https://git-scm.com/docs/git-format-patch

http://who-t.blogspot.co.uk/2009/12/on-commit-messages.html

Tuesday, 23 August 2016

Why is async void bad?

Methods returning void are not awaitable. await operator can be applied only to methods which are returning Task or Task

Let's assume that method Bar is async void. Consequences of  having method Foo calling method Bar are the following:

  • This is a fire and forget model: Foo can't get information when Bar terminates as it can't await on async void method. Foo continues to run before Bar call is completed.
  • If Bar throws an exception, Foo can't catch it as there is no Task object to carry the exception object.
For these reasons async void methods shall be avoided. But this rule has a single exception; top-level events (e.g. GUI events) of void type can be async,

Links:


https://channel9.msdn.com/Series/Three-Essential-Tips-for-Async/Tip-1-Async-void-is-for-top-level-event-handlers-only

http://blog.stephencleary.com/2014/02/synchronous-and-asynchronous-delegate.html


http://stackoverflow.com/questions/15522900/how-to-safely-call-an-async-method-in-c-sharp-without-await

http://stackoverflow.com/questions/23285753/how-to-await-on-async-delegate

http://stackoverflow.com/questions/20624667/how-do-you-implement-an-async-action-delegate-method

Friday, 8 July 2016

Logic in getters and setters

Setters:
  • used to set a (new) value of a property
  • expected to be fast
  • value can be validated (e.g. null check, range check etc...); failed validation CAN throw exception
  • event which publishes information that property's value has been changed can be fired; event handlers are not under publisher's control and they CAN throw exception


Getters:
  • return value of the property
  • expected to be fast
  • expected to be idempotent (i.e. no destructive actions should be performed there)
  • expected to cause no side effects on the object (no changing values)
  • MUST NOT throw exception (otherwise would break principle of least surprise)
  • use memoisation (lazy computation/evaluation)



References:
Property Design (MSDN)
http://stackoverflow.com/questions/495864/logic-in-get-part-of-property-good-practice
http://stackoverflow.com/questions/2923116/is-it-a-good-practice-to-implement-logic-in-properties
http://stackoverflow.com/questions/1488472/best-practices-throwing-exceptions-from-properties

Monday, 2 May 2016

How to host PHP web application on IIS web server

Although I work as a desktop application developer I talked the other day about PHP language with one of my colleagues who is a web developer. Having at home a Windows machine which comes with IIS web server (IIS10 on Win10), I became curious about what would it take to create a simple PHP web application and host it on IIS. I decided to make a single PHP page which would display the public IP address of the host which sends request.

And here we go. I googled for a simple PHP code which echoes the IP address of the client so all credits for the code below go to the guy who answered the following question on StackOverflow: How to get the client IP address in PHP?

main.php:


Index page in PHP world might have a different standard name and I am sure this code should be rearranged and optimized but I am leaving it as it is as my goal is to make fast proof of concept.

It has been a while since I touched IIS web server last time but I remember I could add a web application under the Default Web Site. Each web application has its own directory at path c:\inetpub\wwwroot so we can save main.php at a new directory: c:\inetpub\wwwroot\WhatIsMyIp. We can then add a new application:



When IIS receives request for PHP page, it has to forward it to PHP engine which would process it. This engine is actually PHP language binding for FastCGI protocol which comes as an executable (php-cgi.exe) within a PHP pack for Windows which can be downloaded from http://windows.php.net/download/. Side note on that page says "If you are using PHP as FastCGI with IIS you should use the Non-Thread Safe (NTS) versions of PHP." so I downloaded the latest version (php-7.0.6-nts-Win32-VC14-x64.zip at the moment...) and unpacked it into arbitrary location e.g. c:\PHP\php-7.0.6-nts-Win32-VC14-x64\.

CGI has to be enabled on IIS and this can be done in Windows Features. If we click on Windows start button and type Turn Windows features on and off, Windows Features window opens. We have to go to Internet Information Services, World Wide Web Services, Application Development Features and select (tick) CGI item:



We can now add a PHP CGI handler to our website in IIS:


We also have to make sure that World Wide Web Publishing Service (W3SVC) is running:


If IIS server and Default Web Site are started, we can test our web application from the local browser:


In order to allow accessing this web site from other devices on the same LAN we have to set Inbound rule on the firewall running on the IIS host. In case of Windows Firewall we can simply enable predefined rule World Wide Web Services (HTTP Traffic-In):


I am using default port for HTTP traffic (TCP port 80) so this rule will allow all incoming packets destined for IIS host and port 80 to reach IIS process - the one which listens to HTTP requests on that port.

If we want to access website hosted on another device within the same LAN we have to know what's its IP address (or hostname if we would set up some DNS resolution within this LAN or on the client device). ipconfig command run on IIS host outputs 192.168.0.3 as its IP.

If we open browser on another device which is on the same LAN and type http://192.168.0.3/WhatIsMyIp we'll get the following:



Obviously, all IP addresses we got are their IP addresses within the same LAN because so far both server and clients were on the same network. All devices behind router have the same public IP address which is the WAN (public) address of the router. This address is usually assigned by the ISP.

The true test would be if we try to load this page from a browser on a device which is not on this network. In order to allow accessing this web site from devices which are not on the same LAN, we have to enable port forwarding for HTTP traffic on the router behind which is host with our IIS server. That could look like this:


If WAN address of this router is e.g. 74.90.112.208 we can call our web app from e.g. mobile device connected to Internet via mobile 3G or 4G interface by typing in the browser: http://74.90.112.208/WhatIsMyIp.


Tuesday, 12 January 2016

Sniffing the traffic on the loopback interface on Windows with RawCap

If you ever had to write a chunk of code for inter-process communication via Internet Protocol (IP) you must have came across using the loopback interfaceIPv4 address 127.0.0.1 is usually the address of this virtual interface and also the address that hostname localhost  resolves to. You would, at least for tests, set one process (server) listening on some port on the localhost while the other process (client) would connect to the server and start their data exchange. 

In order to check the data sent from one end to another, we can use any kind of process output (standard output, log, ...) provided that given process actually shows that data. But what if there is no such output? We have to find a way how to sniff the traffic between these two parties.

RawCap is one of free tools capable of sniffing packets on the loopback interface. It collects packets and stores them in the file in pcap format. This file can be opened in Wireshark.

In order to demonstrate RawCap packet capturing we can use ready-made networking application which can serve both as a server and a client - Ncat. It comes as a part of Nmap for Windows and uses TCP by default.

Once Nmap is installed, open console window in C:\Program Files (x86)\Nmap directory and  type the following:


> ncat -v -4 -l localhost 6789


-v (or --verbose) sets verbose output
-4 instructs ncat to use IPv4 addresses only
-l (or --listen) switch makes Ncat to listen for incoming connections on the provided hostname and port. localhost will be resolved to 127.0.0.1 and I also used an arbitrary port which was free - 6789.

We effectively have server running now but before we run the client it is necessary to start capturing packets. From the directory with RawCap.exe we have to open terminal window with Administrator's privileges and run RawCap.exe (which has to be run as Administrator). Now we just have to follow the instructions. RawCap lists network interfaces and asks which interface shall be monitored. We have to type in the number next to the loopback interface. In the next step we can set the name of the packet dump file or just leave default name which is dumpfile.pcap. At this point we have started sniffing packets on the localhost!


> RawCap.exe
Interfaces:
0. 169.xxx.xxx.xxx Local Area Connection Ethernet
1. 169.xxx.xxx.xxx Local Area Connection* 2 Wireless80211
2. 169.xxx.xxx.xxx Ethernet Ethernet
3. 169.xxx.xxx.xxx Ethernet 2 Ethernet
4. 169.xxx.xxx.xxx Local Area Connection 2 Ethernet
5. 127.0.0.1 Loopback Pseudo-Interface 1 Loopback
6. 192.xxx.xxx.xxx Wireless Network Connection 2 Wireless80211
Select interface to sniff [default '0']: 5
Output path or filename [default 'dumpfile.pcap']:
Sniffing IP : 127.0.0.1
File : dumpfile.pcap
Packets : 0


Let's now start the client. We have to open another terminal window in Nmap's directory and run another instance of Ncat which will try to establish TCP connection with the localhost:6789 endpoint:


> ncat -v 127.0.0.1 6789


Both server and client show the state of their current connections:

Server:

> ncat -v -4 -l localhost 6789
Ncat: Version 7.01 ( https://nmap.org/ncat )
Ncat: Listening on 127.0.0.1:6789
Ncat: Connection from 127.0.0.1.
Ncat: Connection from 127.0.0.1:9323.



Client:

> ncat -v 127.0.0.1 6789
Ncat: Version 7.01 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:6789.



So far, only TCP handshake has taken place in the communication between these two instances of Ncat. We can expect that RawCap has captured at least these 3 packets so far (SYN, SYN-ACK, ACK). Let's now send some user data. By default Ncat works as an echo server which means that it echoes any text message it receives. If we type in the client "hello from client", that string will appear on the server's output. The same happens if we go the other way round and send message from the server to the client. Once we want to stop communication, we can use CTRL-C combination to stop one process. The other will get disconnected and also terminate. We can use the same combination in order to stop RawCap.exe.

At the end of the session we have:

Server:


Client:


RawCap.exe:



RawCap.exe saved all captured packets in file dumpfile.pcap. If we open this file in Wireshark and filter out all packets with TCP port 6789 we can see packets exchanged between our Ncat processes:


If we follow that TCP stream we can see exact text messages sent between client and server:


Friday, 1 January 2016

How to use internal type as a unit test argument

Behavioral tests sometimes require access to types which are internal for the SUT assembly. If those types are used in test methods, test assembly name has to be specified through InternalsVisibleToAttribute in SUT assembly. For example, if MyAppTests is a library containing tests for classes within MyApp we have to do the following:
  • in MyAppTests: add the reference to MyApp so all public types are visible
  • in MyApp: specify MyAppTests as the assembly in which internal types are visible. The following line has to be added to MyApp/Properties/AssemblyInfo.cs:

[assembly: InternalsVisibleTo("MyAppTests")]



This makes internal types visible within unit tests. But if we want to pass such types as arguments to unit tests (via TestCase or TestCaseSource attributes), compiler will complain about inconsistent accessibility.

For example, if we have public class (SUT) and internal enum:


and the following test:


Compiler issues an error:

Error CS0051 Inconsistent accessibility: parameter type 'MyEnum' is less accessible than method 'MyClass_Tests.Foo_Does_This_When_MyEnum_Is_Value1_or_Value2(MyEnum)'

Unit test methods have to be public, otherwise unit test runners will not be able to see them or would report that test fails (Nunit Test Adapter also issues error message: "Method is not public"). Unit tests are public API so types of their arguments also have to be public (visible in any assembly, not just in test library).

The solution for this is to pass arguments to test method as type object and then cast them back to the original internal type:


In this particular case, when internal type is enum, we could gave passed type int instead of object but for any other custom internal type, object is the only solution.


Thread which explains reasons why NUnit fails but does not ignore non-public test methods.