Sunday, 28 August 2016

Git Tips and Tricks


Git is a (free) version control system. It allows:
  • saving code snapshots (commits)
  • working with alternative code versions (branches)
  • moving between branches and commits (checkout)
  • easily rolling back to older code snapshots or develop new features without breaking production code

Repository is a bucket which contains a code. 

GitHub is a platform for creating, storing and managing Git repositories in the cloud
  • allows multiple developers to work simultaneously on the centrally stored code base
  • keeps the code safe in case code is lost on dev machines for any reason
  • provides tools for automating code and repository management via GitHub Actions

Switching repositories

Cloning a remote repository:

$ git clone https://github.com/OpenVPN/openvpn-build.git



After cloning desired repository it is always good to run the following command and fetch/clone all its submodules:
$ git submodule update --init

Cloning a remote repository with preserving UNIX-style line endings:

$ git clone https://github.com/OpenVPN/openvpn-build.git --config core.autocrlf=false


Working with Branches



Getting a list of all local branches:

$ git branch



Deleting local branch:

$ git branch -d my_branch
Deleted branch my_branch (was 6d5921868b).



Getting a list of all remote branches:

$ git branch -r



Getting a list of all branches at some specific path (on Windows):

> git branch -r | find "origin/private/bojan"



Example output:

origin/HEAD -> origin/master
origin/master
origin/release/2.3



Deleting remote branch:

git push origin --delete my_branch



To update local list of remote branches:

$ git remote update origin --prune



To list all local and remote branches:

$ git branch -a




Checking out the remote branch (for the first time):

$ git checkout -b branch_name remote_name/branch_name


or shorter (on newer Git versions):

$ git checkout --track remote_name/branch_name



Example:

$ git checkout -b release/2.3 origin/release/2.3
Branch release/2.3 set up to track remote branch release/2.3 from origin.
Switched to a new branch 'release/2.3'



Or:

$ git fetch
...
* [new branch] ModuleA/Project2/JIRA-123 -> origin/ModuleA/Project2/JIRA-123
...

$ git checkout ModuleA/Project2/JIRA-123
Checking out files: 100% (1485/1485), done.
Branch ModuleA/Project2/JIRA-123 set up to track remote branch ModuleA/Project2/JIRA-123 from origin.
Switched to a new branch 'ModuleA/Project2/JIRA-123'


If you get the following error:

$ git checkout ModuleA/Project2/JIRA-123
error: Your local changes to the following files would be overwritten by checkout:
Module2/SomeDir/SomeFile.cpp
Please commit your changes or stash them before you switch branches.
Aborting


...you can force checking out with -f switch:

$ git checkout -f ModuleA/Project2/JIRA-123


To create new local branch:

$ git checkout -b issue-007
Switched to a new branch "issue-007"



The command above is a compact version for a group of these two subsequent git commands:

$ git branch issue-007
$ git checkout issue-007



To create a new branch based on a commit from the current branch:

$ git branch <commit_sha1>

To create a new branch based on a commit from another branch:

$ git branch branch_name <commit_sha1>

To create that branch and check it out:

$ git checkout -b branch_name <commit_sha1>


Bash scripts cheat sheet



Bash is:
  • Unix shell (command-line interpreter, command processor). It can process commands that user types in text window (terminal) or commands written in a file (shell script).
  • command language 
This article contains some my notes about writing shell scripts.

Here are some useful utility functions (source):


yell() { echo "$0: $*" >&2; }
die() { yell "$*"; exit 111; }
try() { "$@" || die "cannot $*"; }
asuser() { sudo su - "$1" -c "${*:2}"; }


  • yell: print the script name and all arguments to stderr
    • $0 is the path to the script
    • $* are all arguments
    • >&2 means > redirect stdout to & pipe 2. pipe 1 would be stdout itself.
  • die does the same as yell, but exits with a non-0 exit status, which means “fail”.
  • try uses the || (boolean OR), which only evaluates the right side if the left one failed.
  • $@ is all arguments again, but different.



set -e 

At the top of the script this will cause it to exit upon any command which errors.

 

set -x

Setting the -x tells Bash to print out the statements as they are being executed. It can be very useful as a logging facility and for debugging when you need to know which statements were execute and in what order. [from Bash set -x to print statements as they are executed]

It is possible to combine two above commands into a single one:

set -ex

Getting the current directory:


$ pwd


To find some executable:


$ where perl
C:\cygwin64\bin\perl.exe



$ which perl
/usr/bin/perl


Viewing the whole content of the file:


$ cat /bin/man2html


Viewing only the first line of the file:

$ head -n 1 file.txt


Viewing only the last line of the file:

$ tail -n 1 file.txt


Editing file:

$ vi /bin/man2html


Initially, vi opens in command mode.

i - to enter edit (insert) mode
ESC - to exit edit mode
:wq - write and quit
:q! - quit without saving changes


Logging



It is a good practice to create a log file for each command/process/script/application that we run. We append both stdout and stderr into file:

my_command >> /tmp/my_command.log 2>&1


----
 

Resources:

 

How to compile OpenVpn for Windows from source code

OpenVpn suggests cross-compilation - compiling Windows executables with Unix build toolchain. This can be accomplished either by using Linux/Unix or Cygwin on Windows.

This article demonstrates compiling OpenVpn with Cygwin on Windows.

The first step is to get build scripts from https://github.com/OpenVPN/openvpn-build.

When cloning OpenVpn repositories to Windows machine, make sure Git client for Windows does not automatically convert Unix-style line endings (LF) in source files into Windows-style line endings (CRLF). If this happens UNIX tools that run in Cygwin will report an error complaining about extra CR characters. So, when cloning, use the following command:


$ git clone https://github.com/OpenVPN/openvpn-build.git --config core.autocrlf=false



Required Cygwin packages are listed here: https://github.com/OpenVPN/openvpn-build/tree/master/generic

Package name Cygwin installer path
diffutils All --> Utils
m4All --> Interpreters
makeAll -> Devel
mingw64-i686-binutilsAll -> Devel
mingw64-i686-gcc-coreAll -> Devel
mingw64-i686-headersAll -> Devel
mingw64-i686-pthreadsAll -> Devel
mingw64-i686-runtimeAll -> Devel
mingw64-x86_64-binutilsAll -> Devel
mingw64-x86_64-gcc-coreAll -> Devel
mingw64-x86_64-headersAll -> Devel
mingw64-x86_64-pthreadsAll -> Devel
mingw64-x86_64-runtimeAll -> Devel
patchAll -> Devel
patchutilsAll -> Devel
perlAll --> Interpreters
unzipAll --> Archive
wgetAll --> Web


Download Perl script man2html and save it as a file with no extensions in /bin directory (C:\cygwin64\bin). Script URL is: http://cpansearch.perl.org/src/EHOOD/man2html3.0.1/man2html. If build is run with no having this script, the following error is reported at the output:


checking for man2html... no
configure: error: man2html is required for win32
FATAL: Configure pkcs11-helper



Make sure that man2html's shebang contains correct path to perl. In order to find perl's path in Cygwin we can do the following:


$ which perl
/usr/bin/perl



In my case shebang

#!/usr/local/bin/perl


has to be replaced with:

#!/usr/bin/perl



In order to build x64 version of OpenVpn go to ../OpenVpn/openvpn-build/generic and execute:

$ IMAGEROOT=`pwd`/image-win64 CHOST=x86_64-w64-mingw32 CBUILD=i686-pc-cygwin ./build




A bit about Cygwin

Checking whether some particular package has been installed (e.g. ca-certificates):

$ cygcheck -c ca-certificates
Cygwin Package Information
Package Version Status
ca-certificates 2.9-1 OK



http://stackoverflow.com/questions/9224298/how-do-i-fix-certificate-errors-when-running-wget-on-an-https-url-in-cygwin

How to create and submit Git patch

Some open-source projects prefer submitting Git patch files to pull requests.

OpenVpn: https://community.openvpn.net/openvpn/wiki/DeveloperDocumentation
Linux kernel: https://www.kernel.org/doc/Documentation/SubmittingPatches

https://shkspr.mobi/blog/2014/04/submitting-trivial-linux-kernel-patches/
https://kernelnewbies.org/FirstKernelPatch
https://www.kernel.org/doc/Documentation/SubmittingPatches
https://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html
https://git-scm.com/docs/git-format-patch

http://who-t.blogspot.co.uk/2009/12/on-commit-messages.html

Tuesday, 23 August 2016

Why is async void bad?

Methods returning void are not awaitable. await operator can be applied only to methods which are returning Task or Task

Let's assume that method Bar is async void. Consequences of  having method Foo calling method Bar are the following:

  • This is a fire and forget model: Foo can't get information when Bar terminates as it can't await on async void method. Foo continues to run before Bar call is completed.
  • If Bar throws an exception, Foo can't catch it as there is no Task object to carry the exception object.
For these reasons async void methods shall be avoided. But this rule has a single exception; top-level events (e.g. GUI events) of void type can be async,

Links:


https://channel9.msdn.com/Series/Three-Essential-Tips-for-Async/Tip-1-Async-void-is-for-top-level-event-handlers-only

http://blog.stephencleary.com/2014/02/synchronous-and-asynchronous-delegate.html


http://stackoverflow.com/questions/15522900/how-to-safely-call-an-async-method-in-c-sharp-without-await

http://stackoverflow.com/questions/23285753/how-to-await-on-async-delegate

http://stackoverflow.com/questions/20624667/how-do-you-implement-an-async-action-delegate-method

Friday, 8 July 2016

Logic in getters and setters

Setters:
  • used to set a (new) value of a property
  • expected to be fast
  • value can be validated (e.g. null check, range check etc...); failed validation CAN throw exception
  • event which publishes information that property's value has been changed can be fired; event handlers are not under publisher's control and they CAN throw exception


Getters:
  • return value of the property
  • expected to be fast
  • expected to be idempotent (i.e. no destructive actions should be performed there)
  • expected to cause no side effects on the object (no changing values)
  • MUST NOT throw exception (otherwise would break principle of least surprise)
  • use memoisation (lazy computation/evaluation)



References:
Property Design (MSDN)
http://stackoverflow.com/questions/495864/logic-in-get-part-of-property-good-practice
http://stackoverflow.com/questions/2923116/is-it-a-good-practice-to-implement-logic-in-properties
http://stackoverflow.com/questions/1488472/best-practices-throwing-exceptions-from-properties

Monday, 2 May 2016

How to host PHP web application on IIS web server

Although I work as a desktop application developer I talked the other day about PHP language with one of my colleagues who is a web developer. Having at home a Windows machine which comes with IIS web server (IIS10 on Win10), I became curious about what would it take to create a simple PHP web application and host it on IIS. I decided to make a single PHP page which would display the public IP address of the host which sends request.

And here we go. I googled for a simple PHP code which echoes the IP address of the client so all credits for the code below go to the guy who answered the following question on StackOverflow: How to get the client IP address in PHP?

main.php:


Index page in PHP world might have a different standard name and I am sure this code should be rearranged and optimized but I am leaving it as it is as my goal is to make fast proof of concept.

It has been a while since I touched IIS web server last time but I remember I could add a web application under the Default Web Site. Each web application has its own directory at path c:\inetpub\wwwroot so we can save main.php at a new directory: c:\inetpub\wwwroot\WhatIsMyIp. We can then add a new application:



When IIS receives request for PHP page, it has to forward it to PHP engine which would process it. This engine is actually PHP language binding for FastCGI protocol which comes as an executable (php-cgi.exe) within a PHP pack for Windows which can be downloaded from http://windows.php.net/download/. Side note on that page says "If you are using PHP as FastCGI with IIS you should use the Non-Thread Safe (NTS) versions of PHP." so I downloaded the latest version (php-7.0.6-nts-Win32-VC14-x64.zip at the moment...) and unpacked it into arbitrary location e.g. c:\PHP\php-7.0.6-nts-Win32-VC14-x64\.

CGI has to be enabled on IIS and this can be done in Windows Features. If we click on Windows start button and type Turn Windows features on and off, Windows Features window opens. We have to go to Internet Information Services, World Wide Web Services, Application Development Features and select (tick) CGI item:



We can now add a PHP CGI handler to our website in IIS:


We also have to make sure that World Wide Web Publishing Service (W3SVC) is running:


If IIS server and Default Web Site are started, we can test our web application from the local browser:


In order to allow accessing this web site from other devices on the same LAN we have to set Inbound rule on the firewall running on the IIS host. In case of Windows Firewall we can simply enable predefined rule World Wide Web Services (HTTP Traffic-In):


I am using default port for HTTP traffic (TCP port 80) so this rule will allow all incoming packets destined for IIS host and port 80 to reach IIS process - the one which listens to HTTP requests on that port.

If we want to access website hosted on another device within the same LAN we have to know what's its IP address (or hostname if we would set up some DNS resolution within this LAN or on the client device). ipconfig command run on IIS host outputs 192.168.0.3 as its IP.

If we open browser on another device which is on the same LAN and type http://192.168.0.3/WhatIsMyIp we'll get the following:



Obviously, all IP addresses we got are their IP addresses within the same LAN because so far both server and clients were on the same network. All devices behind router have the same public IP address which is the WAN (public) address of the router. This address is usually assigned by the ISP.

The true test would be if we try to load this page from a browser on a device which is not on this network. In order to allow accessing this web site from devices which are not on the same LAN, we have to enable port forwarding for HTTP traffic on the router behind which is host with our IIS server. That could look like this:


If WAN address of this router is e.g. 74.90.112.208 we can call our web app from e.g. mobile device connected to Internet via mobile 3G or 4G interface by typing in the browser: http://74.90.112.208/WhatIsMyIp.


Tuesday, 12 January 2016

Sniffing the traffic on the loopback interface on Windows with RawCap

If you ever had to write a chunk of code for inter-process communication via Internet Protocol (IP) you must have came across using the loopback interfaceIPv4 address 127.0.0.1 is usually the address of this virtual interface and also the address that hostname localhost  resolves to. You would, at least for tests, set one process (server) listening on some port on the localhost while the other process (client) would connect to the server and start their data exchange. 

In order to check the data sent from one end to another, we can use any kind of process output (standard output, log, ...) provided that given process actually shows that data. But what if there is no such output? We have to find a way how to sniff the traffic between these two parties.

RawCap is one of free tools capable of sniffing packets on the loopback interface. It collects packets and stores them in the file in pcap format. This file can be opened in Wireshark.

In order to demonstrate RawCap packet capturing we can use ready-made networking application which can serve both as a server and a client - Ncat. It comes as a part of Nmap for Windows and uses TCP by default.

Once Nmap is installed, open console window in C:\Program Files (x86)\Nmap directory and  type the following:


> ncat -v -4 -l localhost 6789


-v (or --verbose) sets verbose output
-4 instructs ncat to use IPv4 addresses only
-l (or --listen) switch makes Ncat to listen for incoming connections on the provided hostname and port. localhost will be resolved to 127.0.0.1 and I also used an arbitrary port which was free - 6789.

We effectively have server running now but before we run the client it is necessary to start capturing packets. From the directory with RawCap.exe we have to open terminal window with Administrator's privileges and run RawCap.exe (which has to be run as Administrator). Now we just have to follow the instructions. RawCap lists network interfaces and asks which interface shall be monitored. We have to type in the number next to the loopback interface. In the next step we can set the name of the packet dump file or just leave default name which is dumpfile.pcap. At this point we have started sniffing packets on the localhost!


> RawCap.exe
Interfaces:
0. 169.xxx.xxx.xxx Local Area Connection Ethernet
1. 169.xxx.xxx.xxx Local Area Connection* 2 Wireless80211
2. 169.xxx.xxx.xxx Ethernet Ethernet
3. 169.xxx.xxx.xxx Ethernet 2 Ethernet
4. 169.xxx.xxx.xxx Local Area Connection 2 Ethernet
5. 127.0.0.1 Loopback Pseudo-Interface 1 Loopback
6. 192.xxx.xxx.xxx Wireless Network Connection 2 Wireless80211
Select interface to sniff [default '0']: 5
Output path or filename [default 'dumpfile.pcap']:
Sniffing IP : 127.0.0.1
File : dumpfile.pcap
Packets : 0


Let's now start the client. We have to open another terminal window in Nmap's directory and run another instance of Ncat which will try to establish TCP connection with the localhost:6789 endpoint:


> ncat -v 127.0.0.1 6789


Both server and client show the state of their current connections:

Server:

> ncat -v -4 -l localhost 6789
Ncat: Version 7.01 ( https://nmap.org/ncat )
Ncat: Listening on 127.0.0.1:6789
Ncat: Connection from 127.0.0.1.
Ncat: Connection from 127.0.0.1:9323.



Client:

> ncat -v 127.0.0.1 6789
Ncat: Version 7.01 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:6789.



So far, only TCP handshake has taken place in the communication between these two instances of Ncat. We can expect that RawCap has captured at least these 3 packets so far (SYN, SYN-ACK, ACK). Let's now send some user data. By default Ncat works as an echo server which means that it echoes any text message it receives. If we type in the client "hello from client", that string will appear on the server's output. The same happens if we go the other way round and send message from the server to the client. Once we want to stop communication, we can use CTRL-C combination to stop one process. The other will get disconnected and also terminate. We can use the same combination in order to stop RawCap.exe.

At the end of the session we have:

Server:


Client:


RawCap.exe:



RawCap.exe saved all captured packets in file dumpfile.pcap. If we open this file in Wireshark and filter out all packets with TCP port 6789 we can see packets exchanged between our Ncat processes:


If we follow that TCP stream we can see exact text messages sent between client and server:


Friday, 1 January 2016

How to use internal type as a unit test argument

Behavioral tests sometimes require access to types which are internal for the SUT assembly. If those types are used in test methods, test assembly name has to be specified through InternalsVisibleToAttribute in SUT assembly. For example, if MyAppTests is a library containing tests for classes within MyApp we have to do the following:
  • in MyAppTests: add the reference to MyApp so all public types are visible
  • in MyApp: specify MyAppTests as the assembly in which internal types are visible. The following line has to be added to MyApp/Properties/AssemblyInfo.cs:

[assembly: InternalsVisibleTo("MyAppTests")]



This makes internal types visible within unit tests. But if we want to pass such types as arguments to unit tests (via TestCase or TestCaseSource attributes), compiler will complain about inconsistent accessibility.

For example, if we have public class (SUT) and internal enum:


and the following test:


Compiler issues an error:

Error CS0051 Inconsistent accessibility: parameter type 'MyEnum' is less accessible than method 'MyClass_Tests.Foo_Does_This_When_MyEnum_Is_Value1_or_Value2(MyEnum)'

Unit test methods have to be public, otherwise unit test runners will not be able to see them or would report that test fails (Nunit Test Adapter also issues error message: "Method is not public"). Unit tests are public API so types of their arguments also have to be public (visible in any assembly, not just in test library).

The solution for this is to pass arguments to test method as type object and then cast them back to the original internal type:


In this particular case, when internal type is enum, we could gave passed type int instead of object but for any other custom internal type, object is the only solution.


Thread which explains reasons why NUnit fails but does not ignore non-public test methods.