Wednesday 24 August 2011

What is behind file and printer sharing in Microsoft networks

Let's say our IP address is 192.168.0.1 and public folder is put on share on 192.168.0.2 machine.

What happens when we type \\192.168.0.2 in Windows Explorer on the local machine? Windows Explorer is a process and we are issuing a command "List all network shares on \\192.168.0.2". Explorer does some magic and a list of all shared folders appears in its right pane. But what is behind that magic? Under the bonnet a special file sharing protocol is engaged and a whole set of messages is exchanged between workstation (the client; machine that wants to access shared files on a remote host) and a server (remote host that shares resources). We can monitor these packets in the Wireshark:

client <-> server
-> TCP: syn
<- TCP: syn, ack
-> TCP: ack
-> SMB: Negotiate Protocol Request
<- TCP: ack
<- SMB: Negotiate Protocol Response
-> SMB: Session Setup AndX Request, NTLMSSP_NEGOTIATE
<- SMB: Session Setup AndX Response, NTLMSSP_CHALLENGE, Error: STATUS_MORE_PROCESSING_REQUIRED
-> SMB: Session Setup AndX Request, NTLMSSP_AUTH, User: TEST-PC\test_user
<- SMB: Session Setup AndX Response; accept-completed
...
-> SMB: Tree Connect AndX Request, Path: \\192.168.0.2\IPC$
<- SMB: Tree Connect AndX Response, NT Status: STATUS_SUCCESS
-> SMB: NT Create AndX Request, Path: \srvsvc
<- SMB: NT Create AndX Response, FID: 0xabc
-> SMB: Trans2 Request, QUERY_FILE_INFO, FID: 0xabc, Query File Standard Info
<- SMB: Trans2 Response, FID: 0xabc, QUERY_FILE_INFO
-> DCERPC: Bind: call_id:2, 2 context items, 1st SRVSVC v3.0
<- SMB: Write AndX Response, FID: 0x3ba7, 116 bytes
-> SMB: Read AndX Request, FID: 0x3ba7, 1024 bytes at offset 0
<- DCERPC: Bind_ack: call_id: 2 accept max_xmit: 4280 max_recv: 4280
-> SRVSVC: NetShareEnumAll request
<- SMB: Write AndX Response, FID: 0x3ba7, 100 bytes
-> SMB: Read AndX Request, FID: 0x3ba7, 1024 bytes at offset 0
<- SMB: Read AndX Response, FID: 0x3ba7, 1024 bytes
-> SMB: Read AndX Request, FID: 0x3ba7, 2228 bytes at offset 0
<- SRVSVC: NetShareEnumAll response
-> SMB: Close Request, FID: 0x3ba7
<- SMB: Close Response, FID: 0x3ba7
...
-> SMB: Trans2 Request, GET_DFS_REFERRAL, File: \\192.168.0.2\public
<- SMB: Trans2 Response, GET_DFS_REFERRAL, Error: STATUS_NOT_FOUND
-> SMB: Tree Connect AndX Request, Path: \\192.168.0.2\PUBLIC
<- SMB: Tree Connect AndX Response, Error: STATUS_LOGON_FAILURE
...
-> SMB: Trans2 Request, GET_DFS_REFERRAL, File: \\192.168.0.2\public2
<- SMB: Trans2 Response, GET_DFS_REFERRAL, Error: STATUS_NOT_FOUND
-> SMB: Tree Connect AndX Request, Path: \\192.168.0.2\PUBLIC2
<- SMB: Tree Connect AndX Response, Error: STATUS_LOGON_FAILURE
...
-> SMB: Tree Disconnect Request
<- SMB: Tree Disconnect Response
-> SMB: Logoff AndX Request
<- SMB: Logoff AndX Response
-> TCP: ack

We can see that client establishes TCP connection on server's port microsoft-ds (445) and that further communication uses SMB protocol. SMB stands for Server Message Block, an application layer protocol for accessing shared resources on a network. SMB lays on the top of NetBIOS over TCP/IP. SMB is a client-server, request-response protocol that is based on sessions: client establishes connection to the server and then sends SMB requests to browse directories, open/read/write files etc. SMB uses NT Domain authentication to control access to shared resources.

"Session Setup" SMB message includes the user account, a hash function of the encrypted password and logon domain. A domain controller will examine all this information to determine whether the client has permissions to complete this command.

For each shared resource a set of permissions can be defined (which user/group/domain member can open/read/write/browse). SMB protocol authentication has two levels: user and share.

If we type \\192.168.0.2\public in Windows Explorer, SMB authentication considers credentials we used to log on to Windows and permissions defined on share. For example, if our host and 192.168.0.2 are on the same domain and we are logged on to our machine with local credentials, authentication dialog will appear:

------------------------------------------------------------------
Windows Security [x]
------------------------------------------------------------------
Enter Network Password
Enter your password to connect to: 192.168.0.2
------------------------------------------------------------------
User name:
Password:
Domain:
------------------------------------------------------------------
[] Remember my credentials
------------------------------------------------------------------
OK | Cancel
------------------------------------------------------------------

We need to type domain name, domain user name and password.

If both machines are workgroup machines, we need to type a proper credentials defined on 192.168.0.2.

Article "SMB: The Server Message Block Protocol" explains SMB authentication in depth.

Let us go back to Windows Explorer. Does it implement BSD itself? Does it establish connection between client and server machines itself? No. Connection is actually established between the network-related services (applications) running on client and server. Windows Explorer (and any other process which needs network share access support) just uses these services which use BSD to communicate.

So which services are we talking about?

If you open Control Panel\Network and Internet\Network Connections and look the properties of Local Area Connection, you will see it uses:

  • Client for Microsoft Networks - Allows your computer to access resources on a Microsoft Network. The Client for Microsoft Networks component is actually the Workstation service. If you remove this service, the Netlogon and RPC Locator services are also removed. Uninstalling Client for Microsoft Networks disables server message block (SMB) protocol
  • File and Printer Sharing for Microsoft Networks - Allows other computers to access resources on your computer using a Microsoft Network. The File and Printer Sharing for Microsoft Networks component is the equivalent of the Server service

Relevant services in Service Manager are:

Service Display Name: Workstation
Service Name: LanmanWorkstation
Path: C:\Windows\System32\svchost.exe -k NetworkService
Description: Creates and maintains client network connections to remote servers using the SMB protocol. If this service is stopped, these connections will be unavailable. If this service is disabled, any services that explicitly depend on it will fail to start.
Dependants: Computer Browser, Netlogon,...
This service can be configured with NET CONFIG command.

Service Display Name: Server
Service Name: LanmanServer
Path: C:\Windows\system32\svchost.exe -k netsvcs
Description: Supports file, print, and named-pipe sharing over the network for this computer. If this service is stopped, these functions will be unavailable. If this service is disabled, any services that explicitly depend on it will fail to start.
Dependants: Computer Browser,...
This service can be configured with NET CONFIG command.

Service Display Name: Netlogon
Service Name: Netlogon
Path: C:\Windows\system32\lsass.exe
Description: Maintains a secure channel between this computer and the domain controller for authenticating users and services. If this service is stopped, the computer may not authenticate users and services and the domain controller cannot register DNS records. If this service is disabled, any services that explicitly depend on it will fail to start.
Dependants: None

Story about resource sharing is not simple. Windows uses various protocols on the top of SMB. Some of them are:

Remote Administration Protocol (MS-RAP):
NetServerEnum2 obtains a list of all servers (that have put some resources on share) in a network.
NetShareEnum obtains a list of all shared resources on a specific server.

Common Internet File System (CIFS) Browser Protocol (Microsoft Browser Protocol). Local host gets a list of all shared resources on neighbouring computers via it (filter BROWSER packets in Wireshark). It lays on the top of SMB Mailslot Protocol (Mailslot Name: \MAILSLOT\BROWSE). Hosts from range 192.168.0.xxx send messages to Browse Server (e.g. 192.168.0.255). Host Announcement, Domain/Workgroup Announcement, Local Master Announcement, Request Announcement, Browser Election Request, Become Backup Browser, Get Backup List Announcement...are some of its commands.

SMB Mailslot Protocol (MS-MAIL) lays on the top of SMB and is unidirectional interprocess communications (IPC) protocol between a client and server. A mailslot server creates a mailslot, and a mailslot client writes messages to the mailslot created by the server. The server then reads these messages, thus achieving communication between the client and server. Netlogon Remote Protocol uses it (\MAILSLOT\NET\NETLOGON) to locate domain controllers. Common Internet File System (CIFS) Browser Protocol uses it (\MAILSLOT\LANMAN and \MAILSLOT\BROWSE) for inter-machine communication.

Following picture (source URL) displays all Microsoft networking protocols and their dependencies:


One more thing. We can use NET USE command in order to establish SMB session:

net use \\host_name password /USER:DOMAIN_NAME\user_name

NET USE command sends "Session Setup AndX Request" out:

client <-> server
-> TCP: syn
<- TCP: syn, ack
-> TCP: ack
-> SMB: Negotiate Protocol Request
<- TCP: ack
<- SMB: Negotiate Protocol Response
-> SMB: Session Setup AndX Request, NTLMSSP_NEGOTIATE
<- SMB: Session Setup AndX Response, NTLMSSP_CHALLENGE, Error: STATUS_MORE_PROCESSING_REQUIRED
-> SMB: Session Setup AndX Request, NTLMSSP_AUTH, User: TEST-PC\test_user
<- SMB: Session Setup AndX Response; accept-completed
-> SMB: Tree Connect AndX Request, Path: \\192.168.0.2\IPC$
<- SMB: Tree Connect AndX Response, STATUS_SUCCESS
-> TCP: ack

Links and references:

Shared resource (Wikipedia)
Browser service (Wikipedia)
Description of the Microsoft Computer Browser Service
Just what is SMB?
Net use
Net Use Command
Disable NetBIOS and SMB to protect public Web servers
How to disable NetBIOS over TCP/IP?
How to Disable SMB 2.0 on Windows Vista/2008
Comparing TCP and SMB Connections for Windows XP Embedded-based Devices
Network access validation algorithms and examples for Windows Server 2003, Windows XP, and Windows 2000
Server Message Block (SMB) Protocol Specification
File Session Traffic
Copy File (Remote to Local)
NetBIOS (Wikipedia)
NetBIOS over TCP/IP (Wikipedia)
NetBIOS Over TCP/IP (MSDN)
NetBIOS over TCP/IP (TCP/IP Fundamentals for Microsoft Windows)
NetBIOS NULL Sessions: The Good, The Bad, and The Ugly
Connecting to NetBIOS Resources Using DNS Names or IP Addresses
NET command
Windows NET USE / NetBIOS commands
NETBIOS HACKING
Remote Network Penetration via NetBios Hack/Hacking
Service overview and network port requirements for the Windows Server system
Network Connections Concepts (MSDN)
Client for Microsoft Networks
Troubleshooting Unwanted "Access Denied" Messages in Domain-Based Networks
Network Access Validation Algorithm and Example
Microsoft SMB Protocol and CIFS Protocol Overview
member of both Workgroup and Domain

Axis2/C and linkinfo.dll on Windows Server 2008

axis2_http_sender.dll depends on linkinfo.dll (use Dependency Walker to see) and might not load on Windows Server 2008 because w3wp.exe cannot find this dll (use Process Monitor to see). linkinfo.dll is usually stored in %Windir%\system32 (c:\windows\system32) but not on Windows Server 2008. In order to get it there, install Desktop Experience feature.

Links and references:
Desktop Experience Overview
When you do not install the Desktop Experience feature of Windows Server 2008, programs that require programs that are included in Desktop Experience do not run

Axis2/C Windows binary package and OpenSSL

I deployed Axis2C (1.6.0, Windows binary distribution) on IIS 7.0 running on Windows Server 2008 but simple browser-based test (http://localhost/axis2/services) failed - I got HTTP error 500 in W3SVC log:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
2011-08-24 09:02:53 ::1 GET /axis2/services - 80 - ::1 Mozilla/5.0+(compatible;+MSIE+9.0;+Windows+NT+6.0;+Trident/5.0) 500 0 0 421

Page in the browser displayed:

An IIS server error occurred
An error occurred while initilizing Axis2/C.

Axis log file showed:

[error] ..\..\util\src\class_loader.c(167) Loading shared library C:\axis2c/lib/axis2_http_sender.dll Failed. DLERROR IS DLL Load Error 126: The specified module could not be found.

I used Process Explorer to find Axis2C DLLs loaded by IIS working process and here they are:

axiom.dll
axis2_engine.dll
axis2_parser.dll
axutil.dll
guththila.dll
mod_axis2_iis.dll
neethi.dll

Some of these modules failed to load axis2_http_sender.dll. I tried to reload aforementioned URL in order to wake up w3wp.exe and make it load Axis2C modules. Process Monitor was filtering w3wp.exe events and I could see that w3wp failed to find libeay32.dll (QueryOpen would return NAME NOT FOUND for all locations listed in PATH environment variable). This dll is a part of OpenSSL and is not included in original Axi2C binary package. I downloaded OpenSSL and, just for a test, placed libeay32.dll in C:\axis2c\lib, where axis2_http_sender.dll resides. This time Process Monitor showed that ssleay32.dll could not be found. This file, just like libeay32, belongs to OpenSSL. I placed it in C:\axis2c\lib and repeated the test. Voila! I got the list of my services listed in the browser which proved that Axi2C was set properly.

Process Explorer now shows the complete list of Axis2C DLLs loaded by w3wp.exe:

axiom.dll
axis2_engine.dll
axis2_http_receiver.dll
axis2_http_sender.dll
axis2_mod_addr.dll
axis2_mod_log.dll
axis2_parser.dll
axutil.dll
guththila.dll
libeay32.dll
ssleay32.dll
mod_axis2_iis.dll
neethi.dll

Conclusion: Axis2C Windows distribution depends on OpenSSL DLLs which are not included in binary package which can be downloaded from Apache Axis2/C Releases page. OpenSSL must be installed separately. Unfortunately, this is not mentioned in Axis2/C manual.

Note: upon installing OpenSSL on your system, path to its binaries (e.g. c:\openssl\bin) should be added to PATH environment variable. Make sure you restart machine after changing environment variables as IIS gets them only upon restart - it does not lookup their update values!

Note: If OpenSSL support is not required, download Axis2C source and built it with OpenSSL disabled (--enable-openssl=no)

References and links:
Axis2/C built with SSL support. axis2_http_sender.dll fails to load
problems running default http server
axis2/c depending on openSSL
re-distribute openSSL dlls with AXIS2/c and patent issue

Tuesday 23 August 2011

How to set common paths in VS2010

"VC++ Directories change" paragraph in Visual Studio 2010 C++ Project Upgrade Guide says:

VC++ Directories are no longer supported in VS2010 through Tools->Options page. Instead, VS2010 introduces the user settings file (Microsoft.cpp..users.props) to control global settings including Global search path. These files are located at $(USERPROFILE)\appdata\local\microsoft\msbuild\v4.0 directory. Upon migration to VS2010, the custom settings of VC++ Directories from VS2005 or VS2008 are migrated to these user files. These global settings files are imported into all the converted and newly created projects.

Here are the steps to change the settings file through UI:

  • Open up property manager by clicking on View.Property Manager.
  • Expand the project node and then the Configuration|Platform nodes, you will see "Microsoft.cpp..users" file for each Configuration|Platform. These are the files for the global settings, similar to the old tools/Options/VC++ Directories.
  • Multi-Select "Microsoft.cpp..users", right click and bring up the property page window
  • In the property page window, click on "VC++ Directories" (for example) in the left pane, add new paths for the directories such as "Include Directories". separated by semicolons
  • Make sure to save the settings before shutting down Visual Studio.
  • Re-launch Visual Studio and the new settings will be in effect.


Note: If you would like to only change the settings for one project, you can right click on the project and bring up the property page. Change the settings for “VC++ Directories”, these settings will be persisted to the project file.

How to check if some process is running (NSIS)

Use NsProcess plugin. FindProcDLL does not work properly on Windows Server 2008.

!include nsProcess.nsh
...
Section -Main SEC0000
...
App_Running_Check:
   ; is app.exe process running? result is stored in $R0
   ${nsProcess::FindProcess} "app.exe" $R0

   ${If} $R0 == 0
      MessageBox MB_RETRYCANCEL|MB_ICONEXCLAMATION "Please stop App.exe before continuing" /SD IDCANCEL IDRETRY App_Running_Check
      Quit
   ${EndIf}
SectionEnd

NSIS installer and locked library files

Installer tries to replace some existing library file with a new one (with the same name) but file is locked, some process is using it at the moment and installer brings up nasty dialog:

---------------------------
MyApp Setup
---------------------------
Error opening file for writing:
C:\Program Files\MyApp\a.dll
Click Abort to stop the installation, Retry to try again, or Ignore to skip this file.
---------------------------
Abort Retry Ignore
---------------------------

The solution is to recognize that file is locked, store a copy of the new file in some temporary directory, ask user to reboot machine at the end of installation, and move file to the intended location upon reboot.

Simplified code that tries to place some file to its intended path on a target machine could be like this:

SetOutPath "$SMPROGRAMS\MyApp"
SetOverwrite on
File "lib\a.dll"

If a.dll is locked, we get the dialog above. The solution is to use InstallLibmacro from a Library header which sets installer error flag if copy error occurs. Additionally, provided argument tells this macro whether to set reboot flag or not. At the end of installation, we check the state of the reboot flag and if it is set, we notify user that reboot is required:

${If} ${Installed} == "True"
   StrCpy $ALREADY_INSTALLED 1
${EndIf}
...
!insertmacro InstallLib DLL $ALREADY_INSTALLED REBOOT_NOTPROTECTED "lib\a.dll" "$SMPROGRAMS\MyApp\a.dll" $TEMP
...
Function .onInstSuccess
   WriteRegStr HKLM "${REGKEY}" 'Version' "${PRODUCT_VERSION}"

   IfRebootFlag 0 NoReboot
      MessageBox MB_YESNO|MB_ICONEXCLAMATION "A reboot is required to finish the installation. Do you wish to reboot now?" IDNO NoReboot
      Reboot
   NoReboot:
FunctionEnd

NOTE 1: If some DLL is locked and you want to see which process has loaded it, use Process Explorer. Go to Find->Find handle or DLL... and type DLL's name...

NOTE 2: To test installer with locked files, use Easy File Locker, a simple GUI tool which sets Visible/Accessable/Writable/Deletable attributes on file or folder.

Lexical string comparison in NSIS

Standard (built-in) string comparison operators are:
  • case-insensitive: ==, !=, StrCmp
  • case-sensitive: StrCmpS

They are only capable of telling whether two strings are equal or not. They don't provide information on which string is greater or less, if strings differ.

LogicLib plug-in extends this set of comparison operators with following ones:
  • case-insensitive: <, <=, >, >=
  • case-sensitive: S==, S!=

Note that operators with capital letter S are case-sensitive.

Here are some tests:

StrCpy $0 "abc"
StrCpy $1 "abc"
StrCpy $2 "Abc"
StrCpy $3 "bbc"

${If} $0 == $1
   DetailPrint "Equal to $1"
${Else}
   DetailPrint "Not equal to $1"
${EndIf}

${If} $0 == $2
   DetailPrint "Equal to $2"
${Else}
   DetailPrint "Not equal to $2"
${EndIf}

; case sensitive comparison (LogicLib operator)
${If} $0 S== $2
   DetailPrint "Equal to $2"
${Else}
   DetailPrint "Not equal to $2"
${EndIf}

${If} $0 == $3
   DetailPrint "Equal to $3"
${Else}
   DetailPrint "Not equal to $3"
${EndIf}

${If} $0 S< $2
   DetailPrint "Less than $2"
${Else}
   DetailPrint "Greater or equal than $2"
${EndIf}

${If} $0 S< $3
   DetailPrint "Less than $3"
${Else}
   DetailPrint "Greater or equal than $3"
${EndIf}

Output:

Equal to abc.
Equal to Abc.
Not equal to Abc.
Not equal to bbc.
Greater or equal than Abc.
Less than bbc.

Friday 19 August 2011

How to create Windows executable from Python script

After some online research I decided to give PyInstaller a try. It converts Python scripts into executable Windows programs, able to run without requiring a Python installation on a target machine. PyInstaller works under any version of Python from 2.2 up to 2.7 (it still does not support Python 3) so I installed the latest Python 2 release - 2.7.2.

NOTE: Make sure Python installation path (e.g. in C:\Python27) has been added to PATH environmental variable.

PyInstaller requires PyWin32 to be installed for Python versions 2.6+. It is a Python extension for Microsoft Windows which provides access to the Win32 API, COM and Pythonwin environment. If you try to perform initial PyInstaller configuration without it, a following error is reported:

..\pyinstaller-1.5.1>python Configure.py
ERROR: Python 2.6+ on Windows support needs pywin32
Please install http://sourceforge.net/projects/pywin32/




I picked installer for Python 2.7 - the version I will be using. PyInstaller configuration now runs with no problems:

..\pyinstaller-1.5.1>python Configure.py
I: computing EXE_dependencies
I: Finding TCL/TK...
W: library coredll.dll required via ctypes not found
I: Analyzing c:\Python27\DLLs\_tkinter.pyd
I: Adding tcl85.dll dependency of _tkinter.pyd
I: Adding tk85.dll dependency of _tkinter.pyd
I: Skipping KERNEL32.dll dependency of _tkinter.pyd
I: Adding python27.dll dependency of _tkinter.pyd
I: Skipping MSVCR90.dll dependency of _tkinter.pyd
I: Analyzing c:\Python27\DLLs\_hashlib.pyd
I: Skipping KERNEL32.dll dependency of _hashlib.pyd
I: Skipping USER32.dll dependency of _hashlib.pyd
I: Skipping ADVAPI32.dll dependency of _hashlib.pyd
I: Skipping MSVCR90.dll dependency of _hashlib.pyd
I: Analyzing c:\Python27\DLLs\_ctypes.pyd
I: Skipping KERNEL32.dll dependency of _ctypes.pyd
I: Skipping ole32.dll dependency of _ctypes.pyd
I: Skipping OLEAUT32.dll dependency of _ctypes.pyd
I: Skipping MSVCR90.dll dependency of _ctypes.pyd
I: Analyzing c:\Python27\DLLs\select.pyd
I: Skipping WS2_32.dll dependency of select.pyd
I: Skipping MSVCR90.dll dependency of select.pyd
I: Skipping KERNEL32.dll dependency of select.pyd
I: Analyzing c:\Python27\DLLs\unicodedata.pyd
I: Skipping MSVCR90.dll dependency of unicodedata.pyd
I: Skipping KERNEL32.dll dependency of unicodedata.pyd
I: Analyzing c:\Python27\DLLs\bz2.pyd
I: Skipping MSVCR90.dll dependency of bz2.pyd
I: Skipping KERNEL32.dll dependency of bz2.pyd
I: Analyzing c:\Python27\DLLs\tcl85.dll
I: Dependent assemblies of c:\Python27\DLLs\tcl85.dll:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
I: Searching for assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_no
ne...
I: Found manifest C:\Windows\WinSxS\Manifests\x86_microsoft.vc90.crt_1fc8b3b9a1e
18e3b_9.0.21022.8_none_bcb86ed6ac711f91.manifest
I: Searching for file msvcr90.dll
I: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.2102
2.8_none_bcb86ed6ac711f91\msvcr90.dll
I: Searching for file msvcp90.dll
I: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.2102
2.8_none_bcb86ed6ac711f91\msvcp90.dll
I: Searching for file msvcm90.dll
I: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.2102
2.8_none_bcb86ed6ac711f91\msvcm90.dll
I: Adding Microsoft.VC90.CRT.manifest
I: Adding msvcr90.dll
I: Adding msvcp90.dll
I: Adding msvcm90.dll
I: Skipping KERNEL32.dll dependency of tcl85.dll
I: Skipping USER32.dll dependency of tcl85.dll
I: Skipping WS2_32.dll dependency of tcl85.dll
I: Skipping ADVAPI32.dll dependency of tcl85.dll
I: Analyzing c:\Python27\DLLs\tk85.dll
I: Dependent assemblies of c:\Python27\DLLs\tk85.dll:
I: X86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.0.0_none
I: Dependent assemblies of c:\Python27\DLLs\tk85.dll:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
I: Skipping assembly X86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.
0.0_none
I: Skipping KERNEL32.dll dependency of tk85.dll
I: Skipping USER32.dll dependency of tk85.dll
I: Skipping GDI32.dll dependency of tk85.dll
I: Skipping COMDLG32.dll dependency of tk85.dll
I: Skipping SHELL32.dll dependency of tk85.dll
I: Skipping ole32.dll dependency of tk85.dll
I: Skipping OLEAUT32.dll dependency of tk85.dll
I: Skipping COMCTL32.dll dependency of tk85.dll
I: Skipping ADVAPI32.dll dependency of tk85.dll
I: Skipping IMM32.dll dependency of tk85.dll
I: Analyzing C:\Windows\system32\python27.dll
I: Dependent assemblies of C:\Windows\system32\python27.dll:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
I: Skipping KERNEL32.dll dependency of python27.dll
I: Skipping USER32.dll dependency of python27.dll
I: Skipping ADVAPI32.dll dependency of python27.dll
I: Skipping SHELL32.dll dependency of python27.dll
I: Analyzing c:\Python27\python.exe
I: Dependent assemblies of c:\Python27\python.exe:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
I: Skipping KERNEL32.dll dependency of python.exe
I: found TCL/TK version 8.5
I: found TCL/TK version 8.5
I: could not find TCL/TK
I: testing for Zlib...
I: ... Zlib available
I: Testing for ability to set icons, version resources...
I: ... resource update available
I: Testing for Unicode support...
I: ... Unicode available
I: testing for UPX...
I: ...UPX unavailable
I: computing PYZ dependencies...
I: done generating config.dat





Let us try now to create a binary example.exe from a script example.py, which just prints "Hello World" to the console window. It is a two-step process. The first step is generating a spec file. Makespec creates it and places it in a directory named after the script:

..\pyinstaller-1.5.1>python Makespec.py "J:\Users\Bojan\wor
kspace\Test1\src\root\nested\example.py"
wrote ..\pyinstaller-1.5.1\example\example.spec
now run Build.py to build the executable

If we follow instruction given above, making sure we are providing correct (relative) path to spec file, Build creates executable:

..\pyinstaller-1.5.1>python Build.py example\example.spec
I: Dependent assemblies of c:\Python27\python.exe:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
checking Analysis
building Analysis because outAnalysis0.toc non existent
running Analysis outAnalysis0.toc
Analyzing: support\_mountzlib.py
Analyzing: support\useUnicode.py
Analyzing: C:\Users\Bojan\workspace\Test1\src\root\nested\example.py
I: Analyzing c:\Python27\python.exe
I: Dependent assemblies of c:\Python27\python.exe:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
Adding Microsoft.VC90.CRT to dependent assemblies of final executable
I: Searching for assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_no
ne...
I: Found manifest C:\Windows\WinSxS\Manifests\x86_microsoft.vc90.crt_1fc8b3b9a1e
18e3b_9.0.21022.8_none_bcb86ed6ac711f91.manifest
I: Searching for file msvcr90.dll
I: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.2102
2.8_none_bcb86ed6ac711f91\msvcr90.dll
I: Searching for file msvcp90.dll
I: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.2102
2.8_none_bcb86ed6ac711f91\msvcp90.dll
I: Searching for file msvcm90.dll
I: Found file C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.2102
2.8_none_bcb86ed6ac711f91\msvcm90.dll
I: Adding Microsoft.VC90.CRT.manifest
I: Adding msvcr90.dll
I: Adding msvcp90.dll
I: Adding msvcm90.dll
I: Adding python27.dll dependency of python.exe
I: Skipping KERNEL32.dll dependency of python.exe
I: Analyzing C:\Windows\system32\python27.dll
I: Dependent assemblies of C:\Windows\system32\python27.dll:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
I: Skipping KERNEL32.dll dependency of python27.dll
I: Skipping USER32.dll dependency of python27.dll
I: Skipping ADVAPI32.dll dependency of python27.dll
I: Skipping SHELL32.dll dependency of python27.dll
I: Analyzing c:\Python27\DLLs\_hashlib.pyd
I: Skipping KERNEL32.dll dependency of _hashlib.pyd
I: Skipping USER32.dll dependency of _hashlib.pyd
I: Skipping ADVAPI32.dll dependency of _hashlib.pyd
I: Analyzing c:\Python27\lib\site-packages\win32\win32api.pyd
I: Skipping USER32.dll dependency of win32api.pyd
I: Skipping ADVAPI32.dll dependency of win32api.pyd
I: Skipping SHELL32.dll dependency of win32api.pyd
I: Skipping VERSION.dll dependency of win32api.pyd
I: Skipping POWRPROF.dll dependency of win32api.pyd
I: Adding pywintypes27.dll dependency of win32api.pyd
I: Skipping KERNEL32.dll dependency of win32api.pyd
I: Analyzing c:\Python27\DLLs\select.pyd
I: Skipping WS2_32.dll dependency of select.pyd
I: Skipping KERNEL32.dll dependency of select.pyd
I: Analyzing c:\Python27\DLLs\unicodedata.pyd
I: Skipping KERNEL32.dll dependency of unicodedata.pyd
I: Analyzing c:\Python27\DLLs\bz2.pyd
I: Skipping KERNEL32.dll dependency of bz2.pyd
I: Analyzing C:\Windows\system32\pywintypes27.dll
I: Skipping ADVAPI32.dll dependency of pywintypes27.dll
I: Skipping USER32.dll dependency of pywintypes27.dll
I: Skipping ole32.dll dependency of pywintypes27.dll
I: Skipping OLEAUT32.dll dependency of pywintypes27.dll
I: Skipping KERNEL32.dll dependency of pywintypes27.dll
Warnings written to example\warnexample.txt
checking PYZ
rebuilding outPYZ1.toc because outPYZ1.pyz is missing
building PYZ outPYZ1.toc
checking PKG
rebuilding outPKG3.toc because outPKG3.pkg is missing
building PKG outPKG3.pkg
checking EXE
rebuilding outEXE2.toc because example.exe missing
building EXE from outEXE2.toc
Appending archive to EXE example\dist\example.exe

Binary has been created and placed in dist directory. We can run it:

..\pyinstaller-1.5.1\example\dist>example
Hello World

Basically, dist directory contains everything that should be distributed. By default, --onedir option is applied and PyInstaller creates a distribution that contains executable and other files it depends on. For example, distribution can contain following files:

..\example\dist\example\
msvcm90.dll
msvcp90.dll
msvcr90.dll
python27.dll
pywintypes27.dll
example.exe
example.exe.manifest
Microsoft.VC90.CRT.manifest
_hashlib.pyd
_socket.pyd
_ssl.pyd
bz2.pyd
pyexpat.pyd
select.pyd
unicodedata.pyd
win32api.pyd

If we want to have everything within a single binary, we need to apply --onefile option for Makespec.py. Build output is only a single file:

..\example\dist\example\example.exe

This single file is actually a self-extracting executable (bootloader) which contains all shared libraries and other helper files. When run, this executable extracts the following files into temporary _MEIXXXXX directory in user's Temp directory:

C:\Users\Bojan\AppData\Local\Temp\_MEI16522\
msvcm90.dll
msvcp90.dll
msvcr90.dll
python27.dll
pywintypes27.dll
copyupdate.exe.manifest
Microsoft.VC90.CRT.manifest
_hashlib.pyd
_socket.pyd
_ssl.pyd
bz2.pyd
pyexpat.pyd
select.pyd
unicodedata.pyd
win32api.pyd

This directory will be automatically deleted upon program termination.

Binary distributable should have its information embedded so File description, File versionProduct descriptionProduct version and Copyrights can be displayed when user does right-click >> Properties >>Details. This can be achieved by applying --version=PATH option where PATH is a path to the version resource file which can look something like this:

version.txt:

VSVersionInfo(
  ffi=FixedFileInfo(
    filevers=(1, 0, 0, 0),
    prodvers=(1, 0, 0, 0),
    mask=0x3f,
    flags=0x0,
    OS=0x40004,
    fileType=0x1,
    subtype=0x0,
    date=(0, 0)
    ),
  kids=[
    StringFileInfo(
      [
      StringTable(
        '040904B0',
        [StringStruct('FileDescription', 'This is a PyInstaller example application'),
        StringStruct('FileVersion', '1.0.0.0'),
        StringStruct('OriginalFilename', 'example.exe'),
        StringStruct('ProductName', 'example'),
        StringStruct('ProductVersion', '1.0.0.0')])
      ]),
    VarFileInfo([VarStruct('Translation', [1033, 1200])])
  ]
)

We can specify name of the executable by applying --name=NAME option where NAME is the name of the application, without extension (.exe) suffix.

This is one typical build configuration which includes some of options stated above:

..\pyinstaller-1.5.1>Makespec.py --onedir --version="d:\dev\workspace\example\src\version.txt" --name="example" "d:\dev\workspace\example\src\main.py"
wrote ..\pyinstaller-1.5.1\example\example.spec
now run Build.py to build the executable

..\pyinstaller-1.5.1>Build.py example\example.spec
I: Dependent assemblies of c:\Python27\python.exe:
I: x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none
checking Analysis
building Analysis because outAnalysis0.toc non existent
running Analysis outAnalysis0.toc
...
Appending archive to EXE example\build\pyi.win32\example\example.exe
checking COLLECT
building because outCOLLECT4.toc missing or bad
building COLLECT outCOLLECT4.toc

If you're using PyDev for Eclipse and you needed to add modules' paths to PYTHONPATH (External libraries) then you need to add all those paths to Makespec.py within --paths argument.

Links and references:

PyInstaller Manual

Thursday 18 August 2011

How to handle event when user clicks on a main menu item

When you create SDI or MDI application project in Visual Studio, Wizard automatically adds a menu bar to the main window. This default, statically created menu has three items "File", "Edit" and "Help". "File"'s submenu has only one item - "Exit", but we can add our custom items. Submenu is basically a popup window and framework notifies main window when this popup is about to become active by sending WM_INITMENUPOPUP message. We can capture this event, identify popup and change default processing of this message by overloading CWnd::OnInitMenuPopup in our class.

Let's say we have added (in the resource editor) "Foo" item in a "File" submenu. All items in a main menu are identified by their indexes. Index of the first item, "File", is 0, "Edit" - 1, etc. These numbers identify these items' submenus (popups) as well and they are passed as the second argument of OnInitMenuPopup method. Now we want to enable/disable menu item named "Foo", depending on the value of some flag. Following code shows how to achieve this:


Links and references:
MFC Dialog: How to enhance a dialog-based application with Menu, Toolbar...?

How to restrict resizing a window

We want to allow user to resize dialog up to certain minimum/maximum dimensions. This can be done by overloading CWnd::OnGetMinMaxInfo in your class. This method is WM_GETMINMAXINFO message handler in which you can set window's maximum and minimum size. This applies to any class derived from CWnd (so including classes derived from CDialog, CFrameWnd, CFormView...).

Links and references:

How to set the Minimum and Maximum window size while Resizing?
Control client area minimum size with MFC
MFC General: How to prevent a resizable window to be smaller than...?

Simple application with two panes and a splitter

Applications of this type are very common: we have some objects, or items, listed in a left pane and when we click on some item its attributes are displayed in the right pane. Items can be listed within a tree control, and right click context menu possibly provides actions we can perform on them. But let us focus now only on basic: a splitter window with two panes.

The quickest way to make application with such layout in MFC is by creating SDI (Single Document Interface) project. Document/View support is not required. Wizard creates CMainFrame class which is the main window of SDI application. Wizard also adds to it a View class - a child window that occupies the client area of the main frame:

Application looks like this:

SplitterTest1_5

We want to embed Splitter window into the main frame instead of this default View so we need to replace CChildView with instance of CSplitterWnd:

The right place to put code that creates controls in the main client area of the frame window is overloaded CFrameWnd::OnCreateClient.

Now how to overload parent's implementation of this method? The easiest way to insert various message handlers and overloads into MFC class is via class properties window:

  • go to class declaration in its header file
  • click on the class name
  • in the main menu in Visual Studio go to View->Other Windows->Properties Window
  • click Properties, Events, Messages or Overrides button in order to change properties or add event/messages handlers or to override base class methods

SplitterTest1_8

In this case, we click on Overrides, find and select OnCreateClient in the left side of the properties window, click in the value field and select Add OnCreateClient. This will automatically insert all necessary code in our class. This is default implementation of this overload:

We want to have static layout - left and right pane and a splitter bar all the time so will use CSplitterWnd::CreateStatic. Creating static splitter requires creating its panes within OnCreateClient.

A pane within a splitter window is usually a window derived from CView class. As we want to embed various controls (e.g. CTreeCtrl) in it, we will use CFormView, a class very similar to CDilaog. We could embed controls in a CView-derived class but using CFormView makes life simpler as allows using Visual Studio resource editor to design a form (properties, children controls) which is associated to CFormView object.

So how to add CFormView windows to our project? Just add a new Dialog in the resource editor and make sure its Style in Properties is set to Child. Optionally, remove Border and System menu. By default, this dialog has OK and Cancel button controls and we can leave them.

SplitterTest1_7

Right click on the dialog and pick "Add Class..." in order to associate a class with this resource. Select CFormView for the base class. I picked CLeftPaneFormView and CRightPaneFormView as names for my panes' classes.

SplitterTest1_6

Now when having panes ready, we can instruct our splitter to create them as its views:

If we run this application, we will see our dialogs with their children controls (OK and cancel buttons) embedded as the left and right pane in the splitter window - just what we wanted!

SplitterTest1_9

Throughout the code in main frame, panes can be accessed with this code:



Links and references:

TN029: Splitter Windows
MFC Tutorial
Visual C++ tutorial

Wednesday 17 August 2011

Message Box with custom commit buttons

Windows API and MFC framework offer ready to use Message Box dialog which can display, depending on a provided flag, one or group of more commit buttons with captions from a predefined set: OK, Yes, No, Cancel, Abort, Retry and Ignore. If dialog contains a question, it is a good practice to make buttons displaying specific responses, e.g. "Run" - "Don't run", "Apply" - "Don't apply" etc. The easiest and quickest way to achieve this is to create a new, custom dialog and add onto it buttons with desired text. Resource editor in Visual Studio creates by default dialog with two buttons, OK and Cancel and you can rename them, providing that new captions will match OK and Cancel functionality. For other buttons, add BN_CLICKED handlers (ON_BN_CLICKED entry in message map). The purpose of this dialog is just to pick the user's answer on a question and not to do any hard work. Caller should read return value of (custom) Message Box and take further actions depending on it.

Return value of the standard message box (one from a predefined set IDOK, IDYES, ...) identifies user's choice and we can return these predefined values or our custom return codes, e.g. ID_RUN, ID_APPLY...We can also reuse resource ID values of our custom buttons for this purpose. Standard Message Box is a modal dialog so our dialog needs to be created as modal (CDialog::DoModal). DoModal returns value that was passed to CDialog::EndDialog. This function should be called in our custom button BN_CLICKED handlers.

In the following example, our custom Message Box has button "Apply". Its BN_CLICKED handler is OnBnClickedButtonApply. Caller identifies user's choice with DoModal return value:

Our Message Box which has "Apply" button:

Caller:

Tuesday 16 August 2011

Axis2C logging and its documentation

I deployed Axis2C (1.6.0) on IIS (7.0) in order to host one web service. Axis2C is a Web Service engine with a logging feature - debug output can be redirected either to console or a text file. Log file is created in a directory with path constructed automatically by Axis engine at the web service startup. Axis installation directory path is kept in AXIS2C_HOME environment variable which must be added manually. During Axis2C deployment, additional information should be stored in a registry, at the path HKEY_LOCAL_MACHINE\SOFTWARE\Apache Axis2c\IIS ISAPI Redirector:
  • axis2c_home (string) - path to Axis2C installation directory (usually c:\axis2c)
  • log_file (string) - full path to log file log file name (read further this article for the explanation)
  • log_level (string) - trace, error, info, critical, user, debug, or warning
These registry entries can be added either manually or by running axis2_iis_regedit.js script which is included in Axis2C package.

I followed online documentation when deploying Axis and my service was running smoothly but I could not find log file. It was supposed to be in c:\axis2c\logs\axis2.log (log_file value I set) but directory was empty. Two things could be the root of the problem: engine either could not find path where to create a file or parent process didn't have enough rights to do so. Process Monitor is a good friend in these situations. I restarted World Wide Web Publishing Service and applied filter in Process Monitor to see only system events for w3wp.exe (IIS working process - one that loads Axis engine) only:

Process Monitor- Axis2C on IIS

Wow! Axis tried to create a log file at the very strange path: c:\axis2c\logs\c:\axis2c\logs\axis2.log. Axis engine was following this logic when creating path string:

log_file_path = $(AXIS2C_HOME) + "\logs\" + log_file

Documentation says:
"(add...) A String value with the name "log_file". The value is the absolute path of the log file.
Example: c:\axis2c\logs\axis2.log"


Even on the another page:
"Add a string value with the name log_file and a value of c:\axis2c\logs\axis2.log"

This seems to be wrong and correct version would be: log_file should contain log file name (e.g. axis2.log). When I applied this logic, log file appeared at the correct place!

I was curious about something else as well: what is axis2c_home registry value used for? As a test, I set it to some rubbish value: c:\axis2c123. I restarted IIS and this time set log_file to axis2.log.

To test whether Axis engine has been deployed successfully on IIS, you can use your browser: just type http://localhost/axis2/services and a page with a list of deployed services should appear. This didn't happen in my case, web page contained this message: An IIS server error occurred. An error occurred while initilizing Axis2/C. I checked log file (which was in expected directory: c:\axis2c\logs):

[error] ..\..\src\core\deployment\dep_engine.c(284) Repository path C:\axis2c123 does not exist
[error] ..\..\src\core\deployment\conf_init.c(56) Creating deployment engine failed for repository C:\axis2c123
[error] ..\..\src\core\deployment\dep_engine.c(284) Repository path C:\axis2c123 does not exist
[error] ..\..\src\core\deployment\conf_init.c(56) Creating deployment engine failed for repository C:\axis2c123

Seems that Axis engine used registry value axis2c_home (instead of environment variable AXIS2C_HOME as one would expect) to read installation directory required for proper initialization. I didn't dig further into this matter but only noticed that this confusion was a consequence of redundancy: installation directory was stored both in environment variable and registry - the same information was stored at two different places which is the practice that should be avoided.

I hope that next Axis2C release will be a bit more consistent in terms of configuration and with more accurate documentation.

Friday 12 August 2011

How to make your application UAC compliant

From Vista onwards Microsoft has been using security technology called User Account Control (UAC) which aims to prevent malware execution.

User account groups in Windows OS (from Vista on) range from those that are fully limited (e.g. "Users" group) to those with fully unrestricted access to computer ("Administrators"). Upon log on, standard user gets session that is assigned with an access token with restricted privileges. Member of Administrators group gets session assigned with two tokens - one with restricted (filtered) and one with unrestricted (elevated) privileges. In both cases all applications are run by default in the security context of the user with restricted token. This is because user-initiated applications inherit access token from explorer.exe (desktop) which always runs with restricted token. So, even if user logs on as Administrator, application they run will not have elevated rights but only rights of the standard user. For most of the applications running with restricted rights is not a problem but what if application needs to change certain parts of the file system (e.g. write into "Program Files" directory) or registry (HKLM keys), areas for which elevated rights are required? It depends on the application manifest (described in the next paragraph) and account type of the logged on user.

User that runs some application does not know what privileges it needs in order to run properly. Program will fail if it requires unrestricted privileges but is run without elevation, no matter by which user. To prevent this, a simple solution was invented: application itself carries information which tells OS which privileges (execution level) it needs. They are written in a manifest, a part of resources embedded into executable. Applications with manifest are signed or UAC compliant (don't confuse this with the term 'digitally signed').

If standard user runs signed application that requires elevated privileges, they are prompted (UAC Credential Prompt) to enter Administrator's credentials. If Administrator runs such application, they still need to confirm they allow this program to run (UAC Consent Prompt). So, in both cases user is aware that application they run requires unrestricted privileges and it is not run automatically. This prevents programs (potentially malware) to run unnoticed which is the basic idea of UAC.

What about applications that are not UAC compliant? No matter which user runs it UAC Consent Prompt appears first and program is run with restricted privileges unless user use "Run as Administrator" context menu option in which case standard user is prompted for Administrator's credentials.

Microsoft strongly suggest making all application UAC compliant as in future Windows releases applications without manifest will not be able to run with elevated rights at all.

So, how we can make our application UAC compliant if developing in Visual Studio?

Add linker option /MANIFESTUACGo to Properties->Configuration Properties->Linker->Manifest File and set Enable User Account Control to Yes.

How to set execution level our application requests from OS?

Add level attribute and its value to the option /MANIFESTUAC: /MANIFESTUAC:level='value'. Set UAC Execution Level to one of the following values:
  • asInvoker - no elevation; application will run with same rights as its parent process (process that started it). E.g. if you run Windows Commander as Administrator, and then you run some executable from it, that executable will be run as Administrator as well. Be aware that Windows Explorer (desktop) run as standard users, even for Administrators so applications run from it will run as standard users as well!
  • highestAvailable - conditional elevation; application will get highest permissions it can. Standard user can run it only with its, restricted permissions. Administrator in admin-approval mode needs to retype its credentials in order to run this application with administrator rights (elevation takes place here). Otherwise application is run with restricted token. 
  • requireAdministrator - elevation takes place always; application will run only with elevated rights. Administrator will run it with elevated rights. Standard user will be prompted for Administrator's credentials.
There is one more attribute of UAC manifest, called UAC Bypass UI Protection (/uiAccess='[true|false]') which determines whether application run with restricted token can send Windows messages to applications running with full privileges. If set to false, it enables UI Privilege Isolation.


References and useful links:

User Account Control (Wikipedia)
User Account Control (MSDN)
Teach Your Apps To Play Nicely With Windows Vista User Account Control

Thursday 11 August 2011

GUI design tips

Brief NSIS reference

NSIS compiler compiles setup.nsi script into two executables: Installer and Uninstaller. Script contains global statements, and statements within sections and functions. Compiler uses section/function names in order to determine which one belong to Installer and which to Uninstaller.

This is a short (and uncomplete) list of some commands and script features. For detailed reference please look at links at the bottom of the article.

OutFile inst_name - defines the name of the output (installer executable).
Example: OutFile "MyApp Setup.exe"

Name app_name - defines the name of the application (product) this installer installs. This name appears in a title bar of the installer's dialog.
Example: Name "My App"

ComponentText text_above text_inst_types text_inst_sect - defines text that appears in the installer dialog. text_above appears above other controls in the dialog, text_inst_types next to installation types select control and text_inst_sect appears next to components sections selection control. This command can be omitted as default text is provided: "Check the components you want to install and uncheck the components you don't want to install. Click Install to start the installation." for  text_above and "Select components to install:" for text_inst_sect.

!define symb_name value - defines symbol (its name and value). Like in C/C++ it can be used in conditional compilation and symbol replacement. ${symb_name} refers to the value of the symbol. Symbol is  replaced with value both in installer and uninstaller sections. Symbol can contain previously defined symbol - in a form of ${symbol}.
Examples:
!define COMPANY "MyCompany"

!define REGKEY "SOFTWARE\${COMPANY}\$(^Name)"
!define PRODUCT_VERSION 1.0.0.0
!define STR_COMPONENT1_DIR "Component1"
!define STR_MAIN_APP_DIR "MyApp"

$INSTDIR - predefined variable, contains path to installation directory; can be set by InstallDir,  ReadRegStr etc...
$PROGRAMFILES - predefined variable, contains path to "Program Files" directory
$STARTMENU - contains path to Start menu directory
$SMPROGRAMS - $STARTMENU\Programs
$SYSDIR - path to Windows system directory (C:\Windows\System32)

InstallDir dir_path - defines directory where the application will be installed. Predeclared variable (we don't need to declare it) $INSTDIR gets initialized with the value of the InstallDir argument.
Example:
InstallDir "$PROGRAMFILES\${STR_MAIN_APP_DIR }"


Var name - declares variable (all variables are global!)
Example:
Var UserName

Variable used in Installer and Uninstaller must be set in both sections! Value set in Installer is not available in Uninstaller - variable is uninitialized. It is therefore better to use symbols for const values (e.g. paths or path roots that do not change) that are used both in Installer and Uninstaller.

StrCpy dest_var src_str - copies src_str string to the destination variable dest_var. Can be used ONLY inside Section or Function!
Example:
StrCpy $UserName "admin"


SetOutPath out_path - sets destination path for copy/move operations. Sets $OUTDIR (output directory) value.
Example:
SetOutPath $INSTDIR
SetOutPath $TempDir
SetOutPath ${SOME_CUSTOM_DIR}

Section [options] section_name [index] - declares beginning of the section section_name, with index
   ...                                                            - here are placed section commands
SectionEnd                                               - declares section end

Sections appear in section selection control in the order they are listed in the script.
If section_name is omitted, empty string or begins with "-", that section is hidden - it will always be executed!

section_name of Uninstaller section must be "Uninstall" or to begin with "un.".

Option /o makes section unselected by default.

Function func_name - declares beginning of the function
...                                  - here are placed function commands
FunctionEnd                - declares the end of the function

func_name must start with "un." for function used in Uninstaller. This can be a bit annoying if you want to use the same function both in Installer and Uninstaller sections. With a help of a macro it is possible to map a single piece of code into two functions: Sharing functions between Installer and Uninstaller.

Call func_name - function call

Special type of (predefined) functions are callback functions.

Installer callback functions:
  • .onInit
  • .onInstSuccess
  • .onInstFailed
  • .onVerifyInstDir
  • .onNextPage
  • .onPrevPage
  • .onSelChange
Uninstaller callback functions:
  • un.onInit
  • un.onUserAbort
  • un.onUninstSuccess
  • un.onUninstFailed
  • un.onNextPage
MessageBox - displays message box
Example:
MessageBox  MB_OK "Hello!"
MessageBox  MB_OK Name is $name


File/directory functions:

SetOverwrite [on|off|try|ifnewer|ifdiff|lastused] - defines whether copy operation will overwrite existing file. on is default value.

File file_path - copies file from file_path path into current output directory. file_path can be either file's absolute path or path relative to the directory which contains this script.
Example:
File "3rdParty\abc-bin-1.0.0.2.zip"

Delete [/REBOOTOK] file_path - deletes file.
Example:
Delete /REBOOTOK $file_customers

CreateDirectory dir_path - creates directory
Example:
CreateDirectory ${WEB_DIR}


RMDir [options] dir_path - removes directory
/r - recursively delete its contents, directories and files. Without this, directory will be removed only if empty!
/REBOOTOK - if directory could not be removed on first attempt, it will be removed after rebootExample:
RMDir ${WEB_DIR}


Registry functions:

ReadRegStr var root_key subkey value_name - reads string from a registry into a variable
Example:
ReadRegStr $INSTDIR HKLM "${REGKEY}" Path - Reads value Path into $INSTDIR

DeleteRegKey [options] root_key subkey - deletes registry key
Example:
DeleteRegKey /IfEmpty HKLM "${REGKEY}\Components"

DeleteRegValue root_key subkey value_name - deletes value in registry
Example:
DeleteRegValue HKLM "${REGKEY}\Components" Main



DetailPrint text - prints text in edit control in main dialog.
Example:
DetailPrint "log_dir = $log_dir" - if log_dir was declared as Var
DetailPrint "web_dir = ${web_dir}" - if web_dir was defined as symbol

CreateShortCut shcut_name.lnk file.ext [options] - creates shortcut
References and useful links:

Nullsoft Scriptable Install System

Wednesday 10 August 2011

Visual Studio, Directories and Environment Variables

Visual Studio 2010 does not allow setting user-specific paths to include files and libraries at the global level but instead these paths must be set in the user property sheet VC++ Directories which is automatically added to every project. Go to Properties->Configuration Properties->VC++ Directories and there you'll find various paths:

  • Executable Directories - path that corresponds to environmental directory PATH
  • Include Directories - path that corresponds to environmental directory INCLUDE
  • Reference Directories - path that corresponds to environmental directory LIBPATH
  • Library Directories - path that corresponds to environmental directory  LIB
  • Source Directories
  • Exclude Directories

The fact that Visual Studio treats these paths as if they were set in environment variables allows us to use the same semantics used when setting environment variables. E.g. we can reference value of some arbitrary environment variable in the path. For example, if WINDDK_PATH is environment variable which value is path to Windows DDK, we can reference its value as $(WINDDK_PATH). Path to DDK API headers in Include Directories is then $(WINDDK_PATH)\inc\api.

NOTE 1: Value of arbitrary environment variable named ENV_VAR can be echoed in Command Prompt window by executing "echo %ENV_VAR%" command.

NOTE 2: Visual Studio/Command Prompt  must be restarted in order to fetch latest changes in environment variables.

MFC and strings

MFC framework uses CString and LPCTSTR as string data types. Code that uses MFC should therefore follow this convention and use these data types. Parts of code (possibly data/model or engine/controller parts) might be shared and portable and they should be using standard C++ string type (STL string). Conversion between MFC and standard strings is easy.

There is a MFC convention of how to pass strings as function arguments and how to return them:
- string as function input parameter: use LPCTSTR. e.g. SetName(LPCTSTR pszName);
- string as function out parameter: use CString&. e.g. GetName(CString sName);
- string as function return type: use const CString&. e.g. const CString& GetName();

Useful links and references:
Strings: CString Argument Passing
CString (MFC)

Tuesday 9 August 2011

How to get Windows taskbar position


Handle of the taskbar window (named Shell_TrayWnd) must be provided to SHAppBarMessage function which returns its coordinates. FindWindow returns window handle.



Usage:

How to add tray icon to MFC application


GUI application usually has its main window which can be minimized to tray and/or taskbar. When minimized to tray, application is not visible on the desktop but has its icon in the notification area of the taskbar (usually called tray). Through that icon user can access application, see its status and notifications.

Following code shows some basic usage of the tray icon in some MFC dialog:



References and useful links:

NOTIFYICONDATA Structure
Shell_NotifyIcon Function

Friday 5 August 2011

Some WinDbg commands for memory dump analysis

!analyze - displays information about the current exception (e.g. type, error code, place where it occurred, call stack)
   -v = show verbose output

.ecxr - switches debugger context to the one of the current exception (must be executed before other call stack commands!)

.frame - shows current frame (function) - specifies which local context (scope) will be used to interpret local variables, or displays the current local context.

.frame N - changes current frame to frame N (N is in hexadecimal format!). Frame with number 0 is the one where exception occurred and which is on the top of the stack.

Example:
.frame 0 - switches scope to function which is on the top of the stack
.frame 1- switches scope to function which called function from frame 0

k - displays stack trace for last set context.

kN - displays call stack for last N frames

kP - displays all frames (entire function call chain) from the call stack, with values of function parameters

!for_each_frame - instructs debugger to execute for each frame in the stack of the current thread

dv - Display Value. Displays the values of function parameters and values of local variables
   /t - show type information
   /v - show address

Example:
To show information about parameters and local variables of the last frame (function) in the stack use:
dv /t /v

To show entire function call chain with parameters and local variables we can use: 
!for_each_frame dv /t /v

dt - Display Type. Displays information (value, members, their values...) about variable or type
   /b - display embedded structures recursively

Example:
If myVar is some local variable from the last frame we can examine its members and their values with:
dt -b myVar

To (recursively) display the contents (members, their types and offsets) of some data type (e.g. CMyClass) use:
dt /b CMyClass

To display the state of some variable of type CMyClass which is at the address 0x00a7ab64 (address could have been obtained with dv) we can use:
dt -b CMyClass 0x00a7ab64


If CMyClass has a member of type T and its offset is for example +0x1f90, we can inspect T object with:
dt -b T 0x00a7ab64+0x1f90

db  
- display raw memory (128 bytes) starting from

If some local variable is pointer, we can examine memory it points to by using operator poi() which   returns value of pointer variable:

db poi(pData)
(db pData would output memory starting with address pData, not the one it points to!)

~
- displays brief list of all threads

~*
- displays brief list of threads, including Priority and Priority Class information

.cls - clear screen


References and useful links:

Debugger Reference(MSDN)
Common WinDbg Commands
WinDbg the easy way
Adventures In A 32-bit Minidump

Wednesday 3 August 2011

The order of elements in STL map


Have you ever been surprised by the order of elements when traversing the map? We insert elements with random keys but elements appear sorted by their keys when we traverse that map:



Output:

Map elements:

m["Austria"] = "Vienna"
m["Belgium"] = "Brussels"
m["Czech Republic"] = "Prague"

Maybe you expected that countries would be listed in the order of insertion, starting with Belgium but no, that wasn't the case. Let's unveil that magic which made our map sorted.

STL map is associative container so it provides an ability for fast retrieval of data based on keys. Fast lookup is provided by the container's implementation. Maps are usually implemented as binary search trees (ordered, sorted binary trees) where position of the new element depends on its key. Elements remain sorted after insertion; they are NOT stored in the order of insertion like those in vector! Insertion order is lost.

When a new element is being added to a map, its key is compared with keys of existing elements in the map. For each key type there must be some function (let's call it is_less) that accepts two objects of that type, compares them and returns true if left one has lower value than the right one. Hey, we used map but we never mentioned is_less in the code! Why? That's because std::map uses its default comparison function, or more precisely, functor.

Map constructor has four parameters of which last two are optional:



Traits argument is (according to this MSDN article) "The type that provides a function object that can compare two element values as sort keys to determine their relative order in the map. This argument is optional and the binary predicate less is the default value".

std::less is default functor used for keys comparison so, in our case, it was std::less⟨std::string⟩ that helped that wizzard to order elements during insertion.

There is NO way of preserving the order of insertion. The only thing we can do is to provide some custom comparison function/functor which will redefine the meaning of the 'is less' relation. It's a bit of cheating but this way we can insert our countries in reverse order. All we need is a functor 'is_less' which returns true when string1 is actually greater than string2:



With the map which uses this custom comparison function the output is:

Map elements:

m["Czech Republic"] = "Prague"
m["Belgium"] = "Brussels"
m["Austria"] = "Vienna"