Showing posts with label Computer architecture. Show all posts
Showing posts with label Computer architecture. Show all posts

Saturday, 28 January 2017

How to determine architecture on Linux

How to determine whether a given Linux is 32 bit or 64 bit?We can say system is 32-bit or 64-bit but there are actually three things that have to be observed separately:

  • CPU (processor) - can have 32-bit or 64-bit instruction set (it can support 32-bit or 64-bit kernel)
  • kernel (OS) - can be 32-bit or 64-bit
  • applications - can be 32-bit and 64-bit

Here is a list of some commands with output containing information on the system architecture.

arch



$ man arch

...
arch - print machine hardware name (same as uname -m)
...



Output example:


$ arch
x86_64



uname



$ man uname

...
NAME
uname - print system information

SYNOPSIS
uname [OPTION]...

DESCRIPTION
Print certain system information. With no OPTION, same as -s.
-a, --all

print all information, in the following order, except omit -p and -i if unknown:

-s, --kernel-name
print the kernel name

-n, --nodename
print the network node hostname

-r, --kernel-release
print the kernel release

-v, --kernel-version
print the kernel version

-m, --machine
print the machine hardware name

-p, --processor
print the processor type (non-portable)

-i, --hardware-platform
print the hardware platform (non-portable)

-o, --operating-system
print the operating system

--help display this help and exit

--version
output version information and exit
...



Output example:


$ uname -a
Linux bojan-IdeaCentre-K430 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ uname -m
x86_64



Instead of x86_64, 32-bit systems would return i686 or i386.

uname -p tells architecture supported by the processor and uname -m which kernel is loaded (kernel name).

/proc/cpuinfo


CPU info file also contains useful info:


$ cat /proc/cpuinfo

...
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
stepping : 9
microcode : 0x15
cpu MHz : 1749.273
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs :
bogomips : 6784.27
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:



We can observe flags section separately:


$ grep flags /proc/cpuinfo

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts



The flags of interest are:

  • lm - means Long mode cpu - 64 bit CPU
  • tm - means Protected mode - 32-bit CPU
  • rm - means Real Mode - 16 bit CPU

lshw


lshw tool extracts detailed information on the hardware configuration of the machine. CPU info can be obtained from its output:


$ sudo lshw -class cpu
[sudo] password for xxx:
*-cpu
description: CPU
product: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
vendor: Intel Corp.
physical id: 10
bus info: cpu@0
version: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
slot: SOCKET 0
size: 1708MHz
capacity: 3900MHz
width: 64 bits
clock: 100MHz
capabilities: x86-64 fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts cpufreq
configuration: cores=4 enabledcores=4 threads=8

/lib/systemd/systemd


We can also query the type of the systemd file (user space bootstrapper):


$ file /sbin/init
/sbin/init: symbolic link to /lib/systemd/systemd

$ file /lib/systemd/systemd
/lib/systemd/systemd: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=5d0c9b0db7931b487a8edd5faf4d5258d56e0abc, stripped

HOSTTYPE


HOSTTYPE environmental variable can also be used:


$ echo $HOSTTYPE
x86_64



References:

What do the flags in /proc/cpuinfo mean?
How to determine whether a given Linux is 32 bit or 64 bit?
How to determine Linux kernel architecture?


Friday, 7 October 2011

NSIS installer for 64-bit Windows

Here are some tips for creating (32-bit) NSIS installer which installs 64-bit application on 64-bit Windows.

There is no 64-bit NSIS Installer (yet) so only 32-bit version will be running on 64-bit host which means that 32-bit redirection will take place: by default its Installation Directory is "C:\Program Files (x86)" and it reads/writes only from HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node portion of the registry. This is not what we want so we need to enable access to native Win64 directories/registry for our Installer.

Our installer, as 32-bit application, will probably be used both for 32-bit and 64-bit installations. It therefore needs to be able to determine architecture of the CPU it runs on. x64 plug-in offers macro RunningX64 which returns true if installer is running on 64-bit Windows (under WoW64 emulation):

setup.nsi:
!include x64.nsh
...
${If} ${RunningX64}
   DetailPrint "Installer running on 64-bit host"
${EndIf}

Once we've detected 64-bit host, we need to:
  • set "C:\Program Files" as Installation Directory - use $PROGRAMFILES64 instead of $PROGRAMFILES
  • enable access to 64-bit registry - use SetRegView NSIS command

setup.nsi:
!include x64.nsh

;set initial value for $INSTDIR
InstallDir "$PROGRAMFILES\${MY_COMPANY}\${MY_APP}"
...
${If} ${RunningX64}
   DetailPrint "Installer running on 64-bit host"
   ; disable registry redirection (enable access to 64-bit portion of registry)
   SetRegView 64
   ; change install dir
   StrCpy $INSTDIR "$PROGRAMFILES64\${MY_COMPANY}\${MY_APP}"
${EndIf}

If installer needs to detect whether some 64-bit process is running, use FindProcDLL plug-in (there are couple of versions available but I found only this one - FindProcDLL_mod_by_hnedka.7z - working for me; please have a look at this forum thread). Download this archived file, unpack it and copy FindProcDLL.dll to your ..\NSIS\Plugins directory.

setup.nsi:
${If} ${RunningX64}
   FindProcDLL::FindProc "Some64BitProcess.exe"
   ${If} $R0 == 1
      DetailPrint "FindProcDLL::FindProc() returned 1 (process is running)"
   ${ElseIf} $R0 == 0
      DetailPrint "FindProcDLL::FindProc() returned 0 (process is not running)"
   ${Else}
      DetailPrint "FindProcDLL::FindProc() returned unexpected value"
   ${Endif}
${Else}
...

How to build Axis2C for 64-bit Windows

This article shows how to build Axis2/C package for 64-bit Windows (AMD64 processor architecture) from command line, by using NMAKE that comes with Visual Studio 2010. Building this package for 32-bit Windows is described here.

Step 1:    Download source distribution for Windows from Apache Axis2/C Releases page. It comes as axis2c-src-1.6.0.zip file. Unzip it to some folder, e.g. "..\axis2c-src-1.6.0"

Step 2:    Set NMAKE environment
..\axis2c-src-1.6.0\build\win32\makefile is Microsoft's NMAKE makefile and ..\axis2c-src-1.6.0\build\win32\configure.in is its configuration file. As I explained in the article "NMAKE and its environment", in order to compile code targeting 64-bit OS on 32-bit host NMAKE needs to use cross-compiler and linker, it needs to be able to access headers for 64-bit code, and 64-bit libraries. To achieve this we need to set environment variables in the current command prompt session which can be done in couple of ways:
  • run my batch file setenv.bat from the ..\axis2c-src-1.6.0\build\win32 directory providing "x64" as its argument
  • go to "C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64" and run vcvarsx86_amd64.bat
  • go to "C:\Program Files\Microsoft Visual Studio 10.0\VC" and run vcvarsall.bat providing "x86_amd64" as argument
After this, check whether nmake will use proper compiler and linker (x86_amd64 versions of cl.exe link.exe):

C:\DEVELOPMENT\Toolkits\Axis2C\axis2c-src-1.6.0\build\win32>cd "C:\Program
Files\Microsoft Visual Studio 10.0\VC"

C:\Program Files\Microsoft Visual Studio 10.0\VC>vcvarsall.bat x86_amd64
Setting environment for using Microsoft Visual Studio 2010 x64 cross tools.

C:\Program Files\Microsoft Visual Studio 10.0\VC>goto :eof

C:\Program Files\Microsoft Visual Studio 10.0\VC>cd "C:\DEVELOPMENT\Toolkits\Axi
s2C\axis2c-src-1.6.0\build\win32"

C:\DEVELOPMENT\Toolkits\Axis2C\axis2c-src-1.6.0\build\win32>where nmake
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\nmake.exe

C:\DEVELOPMENT\Toolkits\Axis2C\axis2c-src-1.6.0\build\win32>where cl
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64\cl.exe
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\cl.exe

C:\DEVELOPMENT\Toolkits\Axis2C\axis2c-src-1.6.0\build\win32>where link
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64\link.exe
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\link.exe

C:\DEVELOPMENT\Toolkits\Axis2C\axis2c-src-1.6.0\build\win32>

Step 3:    Modify configure.in

This file contains variables and flags for conditional compiling defined in makefile. If you don't need OpenSSL support, TCP server, using libxml2 (and so zlib1 and iconv), building Apache Axis2C engine module, libcurl, then modify configure.in to be like this:

#############################################################################
### Build Details ###
#############################################################################
#
# enables https support
ENABLE_SSL = 0
#
# build libcurl transport
ENABLE_LIBCURL = 0
#
# build axis2 with Libxml2 Parser. Axis2/C will be built with embeded guththila # parser by Default.
ENABLE_LIBXML2=0
#
# build tcp server in addition to http server
WITH_TCP = 0
#
# build with archive based deployment
WITH_ARCHIVE = 0
#
#
#############################################################################
### Dependant Binary Locations (Required) ###
#############################################################################
#
# libxml2 binary location ( axis2c is built with libxml2 )
LIBXML2_BIN_DIR =
#
# iconv binary location
ICONV_BIN_DIR =
#
# zlib binary location
ZLIB_BIN_DIR=
#
#
#############################################################################
### Dependant Binary Locations (Optional) ###
#############################################################################
#
# openssl binary location
# required if ENABLE_SSL = 1
OPENSSL_BIN_DIR =
#
# libcurl binary location, only required if libcurl transport is enabled
LIBCURL_BIN_DIR =
#
#
#############################################################################
### Apache Server module (required when building Axis2/C Apache Module) ###
#############################################################################
#
# apache binary location
APACHE_BIN_DIR =
#
# apache 2 server family
# To use apache 2.2 family, use APACHE_VERSION_IS_2_0_X = 0
APACHE_VERSION_2_0_X = 0
#
#
#############################################################################
### Compiler Options ###
#############################################################################
#
# C runtime LIBRARY OPTION ( Use /MD or /MT )
CRUNTIME = /MT
#
# Embed Manifest Files
EMBED_MANIFEST = 1
#
# debug symbols
# To build with debug symbols use DEBUG = 1
DEBUG = 0

Step 4:    Run NMAKE.

..\axis2c-src-1.6.0\build\win32>nmake install

Build output is in directory ..\axis2c-src-1.6.0\build\deploy:

..\axis2c-src-1.6.0\build\deploy\
                                 bin
                                 docs
                                 include
                                 lib
                                 logs
                                 modules
                                 samples
                                 services 
                                 axis2.xml
                                 ...

...and that's it! There is NO need to change any compiler/linker options in makefile. Just make sure you've set NMAKE environment properly - so x86_amd64 versions of cl.exe and link.exe are used!

You can check that all these binaries are 64-bit by using Dependency Walker.

NOTE 1: If you need to have different compiler/linker options in makefile, just add custom BUILD_X86 flag to configure.in:

# BUILD_X86 = 1 if nmake should create x86 binaries.
# BUILD_X86 = 0 if nmake should create x64 (amd) binaries.
BUILD_X86 = 0

makefile can use this file to separate compiler/linker settings for 32 and 64-bit builds (I have defined some symbols here and set /MACHINE to X64 but THAT IS NOT NECESSARY - build succeeds even without these options!):

...
...
!if "$(BUILD_X86)" == "1"
CFLAGS = /D "WIN32" /D "_WINDOWS" /D "_MBCS" /D "AXIS2_DECLARE_EXPORT" \
/D "_CRT_SECURE_NO_DEPRECATE" /D "_CRT_SECURE_NO_WARNINGS" \
/D "AXIS2_SVR_MULTI_THREADED" /W3 /wd4100 /MP10 /nologo $(AXIS2_INCLUDE_PATH) \
$(APACHE_INCLUDE_PATH)
!else
CFLAGS = /D "WIN32" /D "WIN64" /D "AMD64" /D "_WINDOWS" /D "_MBCS" /D "AXIS2_DECLARE_EXPORT" \
/D "_CRT_SECURE_NO_DEPRECATE" /D "_CRT_SECURE_NO_WARNINGS" \
/D "AXIS2_SVR_MULTI_THREADED" /W3 /wd4100 /MP10 /nologo $(AXIS2_INCLUDE_PATH) \
$(APACHE_INCLUDE_PATH)
!endif
...
...
!if "$(BUILD_X86)" == "1"
LDFLAGS = /NOLOGO /LIBPATH:$(AXIS2_LIBS) /LIBPATH:$(LIBXML2_BIN_DIR)\lib \
/LIBPATH:$(APACHE_BIN_DIR)\lib /LIBPATH:$(ZLIB_BIN_DIR)\lib
!else
LDFLAGS = /NOLOGO /LIBPATH:$(AXIS2_LIBS) /LIBPATH:$(LIBXML2_BIN_DIR)\lib \
/LIBPATH:$(APACHE_BIN_DIR)\lib /LIBPATH:$(ZLIB_BIN_DIR)\lib /MACHINE:X64
!endif

NOTE 2: If you need to build 64-bit Axis2C for deploying it as IIS module add this target to makefile:

iis_deploy: deploy axis2_core_without_server axis2_IIS_module copy_axis2_xml

and call nmake with it:

..\axis2c-src-1.6.0\build\win32>nmake iis_deploy

Your web service dll must be built for 64-bit Windows as well and must be built against 64-bit Axis2/C libraries.

Links and references:
Apache Axis2/C Installation Guide
How to: Enable a 64-Bit Visual C++ Toolset at the Command Line

Thursday, 6 October 2011

32-bit and 64-bit versions of Windows C++ application

How to set compiler and linker options in order to build some C++ application for 64-bit Windows? Sample application I will test this on is one that detects two things that should not be mixed:
  • detecting which processor architecture application has been compiled for (performed in compile time)
  • detecting which processor architecture application is running on (performed in run-time)
There is no portable way of performing these checks.

Compiler's target architecture can be detected by checking which pre-defined symbols (macros) have automatically been defined by compiler (this is compiler specific! - different compilers use different macros). Some Microsoft-specific are:

_WIN32 - defined for applications for Win32 and Win64. Always defined.
_WIN64 - defined for applications for Win64.
_M_AMD64 - defined for x64 processors
_M_X64 - defined for x64 processors
_M_IX86 - defined for x86 processors
_M_IA64 - defined for Itanium Processor Family 64-bit processors

These macros determine target processor - one that we are building application for (not processor we are building on). We don't need to add them manually as compiler will define them: _WIN32 and _M_IX86 for 32-bit applications and _WIN64, _M_AMD64, _M_X64 and _M_IA64 for 64-bit ones.

Another test can be checking the value of the sizeof(void*). It is 4 bytes in applications compiled for 32-bit architectures and 8 bytes in 64-bit applications, regardless on the processor architecture application is running on.

Host's processor architecture can be obtained in run-time in various ways, of which some of them (on Windows) are:
  •  __cpuid
  • check PROCESSOR_ARCHITECTURE and PROCESSOR_ARCHITEW6432 environment variables (or read value of PROCESSOR_ARCHITECTURE in registry on path HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment)
  • Use GetSystemInfo() or GetNativeSystemInfo()
main.cpp uses methods described above:

#include <windows.h>
#include <iostream>
#include <string>
#include <intrin.h>
using namespace std;

void CheckIs64BitAvailable()
{
bool b64Available = false;
int CPUInfo[4] = {0};
        __cpuid(CPUInfo, 0);
        b64Available = (CPUInfo[3] & 0x20000000) || false;

if(b64Available)
{
cout << "64-bit processor" << endl;
}
else
{
cout << "32-bit processor" << endl;
}
}

void CheckNativeSystemProcessorArchitecture()
{
SYSTEM_INFO si;
memset(&si, 0, sizeof(si));
GetNativeSystemInfo(&si);

switch(si.wProcessorArchitecture)
{
case PROCESSOR_ARCHITECTURE_AMD64:
cout << "PROCESSOR_ARCHITECTURE_AMD64 - x64 (AMD or Intel)" << endl;
break;
case PROCESSOR_ARCHITECTURE_IA64:
cout << "PROCESSOR_ARCHITECTURE_IA64 - Intel Itanium-based" << endl;
break;
case PROCESSOR_ARCHITECTURE_INTEL:
cout << "PROCESSOR_ARCHITECTURE_INTEL - x86" << endl;
break;
case PROCESSOR_ARCHITECTURE_UNKNOWN:
cout << "PROCESSOR_ARCHITECTURE_UNKNOWN - Unknown architecture" << endl;
break;
}
}

void CheckSystemProcessorArchitecture()
{
cout << "Calling GetSystemInfo()..." << endl;

SYSTEM_INFO si;
memset(&si, 0, sizeof(si));
GetSystemInfo(&si);

switch(si.wProcessorArchitecture)
{
case PROCESSOR_ARCHITECTURE_AMD64:
cout << "PROCESSOR_ARCHITECTURE_AMD64 - x64 (AMD or Intel)" << endl;
break;
case PROCESSOR_ARCHITECTURE_IA64:
cout << "PROCESSOR_ARCHITECTURE_IA64 - Intel Itanium-based" << endl;
break;
case PROCESSOR_ARCHITECTURE_INTEL:
cout << "PROCESSOR_ARCHITECTURE_INTEL - x86" << endl;
break;
case PROCESSOR_ARCHITECTURE_UNKNOWN:
cout << "PROCESSOR_ARCHITECTURE_UNKNOWN - Unknown architecture" << endl;
break;
}
}

void CheckEnvVariable_PROCESSOR_ARCHITECTURE()
{
char* pszProcArch = getenv("PROCESSOR_ARCHITECTURE");
if(pszProcArch)
{
cout << "PROCESSOR_ARCHITECTURE (environment variable): " << string(pszProcArch) << endl;
}
else
{
cout << "PROCESSOR_ARCHITECTURE (environment variable) not found" << endl;
}
}

void CheckEnvVariable_PROCESSOR_ARCHITEW6432()
{
// PROCESSOR_ARCHITEW6432 is defined only in WoW64 where reports the original native processor architecture
char* pszProcArch = getenv("PROCESSOR_ARCHITEW6432");
if(pszProcArch)
{
cout << "PROCESSOR_ARCHITEW6432 (environment variable): " << string(pszProcArch) << endl;
}
else
{
cout << "PROCESSOR_ARCHITEW6432 (environment variable) not found" << endl;
}
}

void CheckRegVal_PROCESSOR_ARCHITECTURE()
{
HKEY hKey;
DWORD dwBuffSize = 1023;
DWORD dwType;
char szValue[1024] = {0};

LONG lErrorCode = RegOpenKeyEx(
HKEY_LOCAL_MACHINE,
"SYSTEM\\CurrentControlSet\\Control\\Session Manager\\Environment",
0,
KEY_QUERY_VALUE,
&hKey);

if(lErrorCode!= ERROR_SUCCESS)
{
cout << "RegOpenKeyEx() failed. Error: " << lErrorCode << endl;
return;
}
 
RegQueryValueEx(
hKey,
"PROCESSOR_ARCHITECTURE",
NULL,
&dwType,
(LPBYTE)szValue,
&dwBuffSize);

cout << "PROCESSOR_ARCHITECTURE (registry value): " << string(szValue) << endl;

RegCloseKey(hKey);        
}

int main()
{
cout << "sizeof(void*) = " << sizeof(void*) << " [bytes]"<< endl;

CheckIs64BitAvailable();
CheckRegVal_PROCESSOR_ARCHITECTURE();

// Defined for applications for Win32 and Win64. Always defined.
#ifdef _WIN32
cout << "_WIN32 defined" << endl;
#endif

// _WIN64 is defined for applications for Win64 (they can run only on Win64)
#ifdef _WIN64
cout << "_WIN64 defined => target architecture is Win64; Process is running on native Win64" << endl;
CheckSystemProcessorArchitecture();
CheckEnvVariable_PROCESSOR_ARCHITECTURE();
#else
cout << "_WIN64 is not defined => target architecture is Win32" << endl;

BOOL bIs64 = FALSE;
IsWow64Process(GetCurrentProcess(), &bIs64);

if(bIs64)
{
cout << "Process is running under WoW64" << endl;
CheckNativeSystemProcessorArchitecture();
CheckEnvVariable_PROCESSOR_ARCHITEW6432();
}
else
{
cout << "Process is running on Win32" << endl;
CheckSystemProcessorArchitecture();
CheckEnvVariable_PROCESSOR_ARCHITECTURE();
}
#endif

// WIN32 is defined outside compiler - by the SDK or the build environment
#ifdef WIN32
cout << "WIN32 defined" << endl;
#endif

// WIN64 is defined outside compiler - by the SDK or the build environment
#ifdef WIN64
cout << "WIN64 defined" << endl;
#endif

// Pre-defined Architecture Macro: Intel x86; Defined for x86 processors
#ifdef _M_IX86
cout << "_M_IX86 defined: " << _M_IX86 << endl;
#endif

// Pre-defined Architecture Macro: AMD64; Defined for x64 processors.
#ifdef _M_X64
cout << "_M_X64 defined: " << _M_X64 << endl;
#endif

// Defined for x64 processors.
#ifdef _M_AMD64
cout << "_M_AMD64 defined: " << _M_AMD64 << endl;
#endif

// Pre-defined Architecture Macro: Intel Architecture-64 (Defined for Itanium Processor Family 64-bit processors.)
#ifdef _M_IA64
cout << "_M_IA64 defined: " << _M_IA64 << endl;
#endif

return 0;
}

This is application's output when run on 32-bit Windows (Win 7):
..\Release>ProcArchTest_x86.exe
sizeof(void*) = 4 [bytes]
32-bit processor
PROCESSOR_ARCHITECTURE (registry value): x86
_WIN32 defined
_WIN64 is not defined => target architecture is Win32
Process is running on Win32
Calling GetSystemInfo()...
PROCESSOR_ARCHITECTURE_INTEL - x86
PROCESSOR_ARCHITECTURE (environment variable): x86
WIN32 defined
_M_IX86 defined: 600

C:\DEVELOPMENT\RESEARCH\C++\ProcessorBitTest\Release>

This is application's output when run on 64-bit Windows (Win 7 emulated on VirtualBox):
C:\Users\Bojan\Desktop>ProcArchTest_x86.exe
sizeof(void*) = 4 [bytes]
32-bit processor
PROCESSOR_ARCHITECTURE (registry value): AMD64
_WIN32 defined
_WIN64 is not defined => target architecture is Win32
Process is running under WoW64
PROCESSOR_ARCHITECTURE_AMD64 - x64 (AMD or Intel)
PROCESSOR_ARCHITEW6432 (environment variable): AMD64
WIN32 defined
_M_IX86 defined: 600

C:\Users\Bojan\Desktop>

Now, let us build this application for 64-bit Windows. In Visual Studio, we need to create a new build platform: Build -> Configuration Manager -> Active Solution Platform, New; select "x64" and copy settings from Win32.

Let us compare compiler options for 32-bit and 64-bit Release configurations in order to see what are the differences:

Compiler options for Win32 Release:
/Zi /nologo /W3 /WX- /O2 /Oi /Oy- /GL /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /Gm- /EHsc /MT /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Fp"Release\ProcArchTest_x86.pch" /Fa"Release\" /Fo"Release\" /Fd"Release\vc100.pdb" /Gd /analyze- /errorReport:queue

Compiler options for x64 Release:
/Zi /nologo /W3 /WX- /O2 /Oi /GL /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /Gm- /EHsc /MT /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Fp"x64\Release\ProcArchTest_x64.pch" /Fa"x64\Release\" /Fo"x64\Release\" /Fd"x64\Release\vc100.pdb" /Gd /errorReport:queue

We can see that all options are the same except in case of /Oy and /analyze which are available only for x86 compilers.

What about linker options?

Linker options for Win32 Release:
/OUT:"..\Release\ProcArchTest_x86.exe" /INCREMENTAL:NO /NOLOGO "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /MANIFEST /ManifestFile:"Release\ProcArchTest_x86.exe.intermediate.manifest" /ALLOWISOLATION /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"..\ProcessorBitTest\Release\ProcArchTest_x86.pdb" /SUBSYSTEM:CONSOLE /OPT:REF /OPT:ICF /PGD:"..\ProcessorBitTest\Release\ProcArchTest_x86.pgd" /LTCG /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X86 /ERRORREPORT:QUEUE

Linker options for x64 Release:
/OUT:"..\x64\Release\ProcArchTest_x64.exe" /INCREMENTAL:NO /NOLOGO "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /MANIFEST /ManifestFile:"x64\Release\ProcArchTest_x64.exe.intermediate.manifest" /ALLOWISOLATION /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"..\ProcessorBitTest\x64\Release\ProcArchTest_x64.pdb" /SUBSYSTEM:CONSOLE /OPT:REF /OPT:ICF /PGD:"..\ProcessorBitTest\x64\Release\ProcArchTest_x64.pgd" /LTCG /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X64 /ERRORREPORT:QUEUE

We can see that linker options differ only in the value of /MACHINE option.

NOTE: /MACHINE option does not need to be set at all! Compiler inserts information about target architecture in OBJ files' headers. OBJ files created with Microsoft compiler have PE/COFF format which is described in Microsoft PE and COFF Specification. At the beginning of an object file, or immediately after the signature of an image file, is a standard COFF file header. Its first field is "Machine" whose value is the number that identifies the type of target machine (CPU type). Some values are: IMAGE_FILE_MACHINE_AMD64 (x64), IMAGE_FILE_MACHINE_I386 (Intel 386 or later processors and compatible processors - x86), IMAGE_FILE_MACHINE_IA64 (Intel Itanium processor family). Linker reads that information. If you compile your app with $(VCInstallDir)bin\x86_amd64\cl.exe linker will create amd64 binary even with /MACHINE Not Set! Cross-compiling with Microsoft (Visual Studio) tools is all about using correct compiler and linker! Compiler and linker options can actually be same for both 32-bit and 64-bit builds! When building 64-bit application in Visual Studio, just make sure that Executable Directories has (cl.exe and link.exe) paths in this order $(VCInstallDir)bin\x86_amd64;$(VCInstallDir)bin; and that AMD paths are listed first in Include and Library Directories (but all this should be done automatically upon creation of x64 project!).

Let us continue with analysis.

Although names of some libraries end with "32" for both x86 and x64 configurations, their different (32-bit or 64-bit) versions are actually used in different builds. Same for the compiler and linker - please read my article "NMAKE and its environment".

Let us try to run now our 64-bit application on 32-bit Windows OS:
..\x64\Release>ProcArchTest_x64.exe
This version of ..\x64\Release\ProcArchTest_x64.exe is not compatible with the
version of Windows you're running. Check your computer's system information
to see whether you need a x86 (32-bit) or x64 (64-bit) version of the program,
and then contact the software publisher.

C:\DEVELOPMENT\RESEARCH\C++\ProcessorBitTest\x64\Release>

This was expected as 64-bit application cannot run on 32-bit operating systems.
What about running it in its natural, 64-bit, enviroment - on 64-bit OS?

C:\Users\Bojan\Desktop>ProcArchTest_x64.exe
sizeof(void*) = 8 [bytes]
32-bit processor
PROCESSOR_ARCHITECTURE (registry value): AMD64
_WIN32 defined
_WIN64 defined => target architecture is Win64; Process is running on native Win
64
Calling GetSystemInfo()...
PROCESSOR_ARCHITECTURE_AMD64 - x64 (AMD or Intel)
PROCESSOR_ARCHITECTURE (environment variable): AMD64
WIN32 defined
_M_X64 defined: 100
_M_AMD64 defined: 100

C:\Users\Bojan\Desktop>

Conclusion:
  • application targets 64-bit architecture if _WIN64, _M_X64 and/or _M_AMD64 are defined and/or if sizeof(void*) is 8
  • reliable way of getting the architecture of the host processor in runtime is reading PROCESSOR_ARCHITECTURE value from registry or using GetSystemInfo()/GetNativeSystemInfo()
Advice:
Don't rely on WoW64 as it might not be enabled/installed on the target machine (e.g. WoW64 is optional on Windows Server 2003 R2). WoW64 is intended to support legacy 32-bit applications anyway. Build new applications for both x86 and x64 architectures. This way you can avoid some potential errors in applications that can occur due to directory and registry redirections under WoW64.

Links and references:

IsWow64Process function
64-bit (Wikipedia)
WoW64 (Wikipedia)
Use Visual Studio to build 64-bit application (MSDN blog)
Everything You Need To Know To Start Programming 64-Bit Windows Systems
Visual Studio Development Environment 64-Bit Support
HOWTO: Detect Process BitnessHow to: Configure Projects to Target Platforms (MSDN)
How to: Configure Visual C++ Projects to Target 64-Bit Platforms (MSDN)
How to determine whether a computer is running a 32-bit version or 64-bit version of the Windows operating system (MSDN)
Predefined Macros (MSDN)
You already know what your target architecture is (or at least you should)
How to detect programmatically whether you are running on 64-bit Windows
Detecting CPU architecture compile-time
Detect system architecture (x86/x64) while running
#ifdef for 32-bit platform
Determining 64-bit vs. 32-bit Windows
Use Visual Studio to build 64-bit application
Seven Steps of Migrating a Program to a 64-bit System

Monday, 3 October 2011

NMAKE and its environment

nmake is a Microsoft's command-line tool for building C/C++ applications. nmake comes with Visual Studio and Windows Driver Development Kit (DDK) and it comes in two versions: one for building 32-bit and another for building 64-bit binaries. Here is the list of its locations, depending on which Microsoft product is installed on your machine:

Visual Studio 2010:
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\nmake.exe
(32-bit binary; depends on 32-bit advapi32.dll, kernel32.dll and msvcr100.dll)

C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\amd64\nmake.exe
(64-bit binary; depends on 64-bit advapi32.dll, kernel32.dll and msvcr100.dll)

Visual Studio 2008:
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin\nmake.exe
(32-bit binary; depends on 32-bit advapi32.dll, kernel32.dll and msvcr90.dll)

C:\Program Files\Microsoft Visual Studio 9.0\VC\bin\amd64\nmake.exe
(64-bit binary; depends on 64-bit advapi32.dll, kernel32.dll and msvcr90.dll)

Windows DDK (I have some pretty old installed on my machine):
C:\WinDDK\6001.18001\bin\x86\nmake.exe
(32-bit binary; depends on 32-bit advapi32.dll, kernel32.dll and msvcr80.dll)

C:\WinDDK\6001.18001\bin\ia64\nmake.exe
(64-bit binary; depends on 64-bit advapi32.dll, kernel32.dll and msvcr80.dll)

Binaries from amd64/ia64 directories are 64-bit and cannot be run on 32-bit machines.

msvcrXX.dll is Microsoft Visual C Runtime library. XX is its version, specific for each version of Visual Studio: 80 for VS2005, 90 for VS2008 and 100 for VS2010. Binary linked against some msvcr library requires existence of that library on the machine it runs on. If it does not exist, it can be installed with Microsoft Visual C++ VS20XX Redistributable Package (x86 or x64).

If you want to run some particular nmake on your machine, make sure its path is listed in Paths environment variable. If there are more versions of nmake present on your machine, put path to desired version before paths to the other versions. E.g. put path "C:\Program Files\Microsoft Visual Studio 10.0\VC\bin" before "C:\Program Files\Microsoft Visual Studio 9.0\VC\bin" in Paths if you want to run VS2010 nmake. You can always check which nmake will be called from some arbitrary location if executing (from a command line):

where nmake

If nmake resides in multiple directories, all those directories will be listed but only instance from the first listed will be called,
nmake reads instructions from a file called makefile (full name of this file usually does not contain a dot and extension). Instructions of how to build a specific binary (or binaries) are grouped into named targets. Those instructions contain compiler and linker calls, provided with source files, paths to headers, dependencies, object file names and compiler and linker options. Tags for groups of targets or sets of instructions that don't make binaries (like creating or deleting files or directories) are pseudo-targets.

You need to provide target or pseudo-target name when calling nmake. You can provide path to makefile; without it nmake looks for it in the current directory.

makefile contains instruction to call Microsoft compiler, cl.exe and linker, link.exe. Just like nmake, they come with Visual Studio and Windows DDK.

Visual Studio 2010:
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\amd64\cl.exe
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\amd64\link.exe
(64-bit compiler and linker for AMD64 architecture)

c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\cl.exe
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\link.exe
(32-bit compiler and linker for building 32-bit applications)

c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64\cl.exe
(depends on advapi32.dll, kernel32.dll, msvcr100.dll, version.dll, psapi.dll and shell32.dll from C:\Windows\System32 and mspdb100.dll from c:\program files\microsoft visual studio 10.0\common7\ide)

c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64\link.exe
(depends on advapi32.dll, kernel32.dll, msvcr100.dll, psapi.dll and user32.dll from C:\Windows\System32 and mspdb100.dll from c:\program files\microsoft visual studio 10.0\common7\ide) (32-bit cross-compiler and linker for building 64-bit applications for AMD64 architectures)

c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_ia64\cl.exe
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_ia64\link.exe
(32-bit cross-compiler and linker for building 64-bit applications for Itanium architectures)

Visual Studio 2008:
c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\amd64\cl.exe
c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\amd64\link.exe

c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\cl.exe
c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\link.exe

c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\x86_amd64\cl.exe
c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\x86_amd64\link.exe

c:\Program Files\Microsoft Visual Studio 9.0\VC\ce\bin\x86_arm\cl.exe
c:\Program Files\Microsoft Visual Studio 9.0\VC\ce\bin\x86_arm\link.exe

c:\Program Files\Microsoft Visual Studio 9.0\VC\ce\bin\x86_mips\cl.exe
c:\Program Files\Microsoft Visual Studio 9.0\VC\ce\bin\x86_mips\link.exe

c:\Program Files\Microsoft Visual Studio 9.0\VC\ce\bin\x86_sh\cl.exe
c:\Program Files\Microsoft Visual Studio 9.0\VC\ce\bin\x86_sh\link.exe

Windows DDK:
c:\WinDDK\6001.18001\bin\ia64\ia64\cl.exe
c:\WinDDK\6001.18001\bin\ia64\ia64\link.exe
c:\WinDDK\6001.18001\bin\x86\amd64\cl.exe
c:\WinDDK\6001.18001\bin\x86\amd64\link.exe
c:\WinDDK\6001.18001\bin\x86\ia64\cl.exe
c:\WinDDK\6001.18001\bin\x86\ia64\link.exe
c:\WinDDK\6001.18001\bin\x86\x86\cl.exe
c:\WinDDK\6001.18001\bin\x86\x86\link.exe

Again, which compiler and linker will be called depends on paths listed in Path environment variable. Paths used can be checked by calling (from a command line):

where cl.exe
where link.exe

For example, if we want to build some 64-bit application for AMD architecture (which is usually named x64 and is more common than Itanium one) and we we want to build it on 32-bit machine, using VS2010, we need to use cross-compiler and so will add following paths to Path environment variable:
C:\Windows\System32; (path to dlls used by cl.exe and link.exe)
c:\program files\microsoft visual studio 10.0\common7\ide; (path to dlls used by cl.exe and link.exe)
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64; (path to cl.exe and link.exe)
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin; (path to nmake.exe)

If you try to run 64-bit compiler on 32-bit machine, you'll get message:
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\amd64>cl.exe /?
This version of c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\amd64\cl.ex
e is not compatible with the version of Windows you're running. Check your compu
ter's system information to see whether you need a x86 (32-bit) or x64 (64-bit)
version of the program, and then contact the software publisher.

You can run 32-bit compiler or 32-bit cross-compiler on 32-bit machine only:
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64>cl.exe /?
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

This is not enough for nmake build to be successful. Compiler reads paths to Visual C++ and Windows SDK (Windows Framework) headers from INCLUDE environment variable. These headers are coming with each version of Visual Studio and we need to make sure cl.exe will be using correct ones. For example, if we want to use VS2010, we will set INCLUDE as:
c:\Program Files\Microsoft Visual Studio 10.0\VC\include; (Visual C++ headers - part of Microsoft's implementation of C++ standard)
c:\Program Files\Microsoft Visual Studio 10.0\VC\atlmfc\include; (ATL/MFC headers - add this path only for this type of applications)
c:\Program Files\Microsoft SDKs\Windows\v7.0A\include; (Windows SDK headers)

Furthermore, linker reads paths to Visual C++ and Windows Framework libraries from LIB environment variable. These libraries are specific for each version of Visual Studio and target architecture. If using VS2010 and building binary for AMD64 architecture, we would set LIB as:
c:\Program Files\Microsoft Visual Studio 10.0\VC\lib\amd64; (Visual C++ libraries)
c:\Program Files\Microsoft Visual Studio 10.0\VC\atlmfc\lib\amd64; (ATL/MFC libraries- add this path only for this type of applications)
c:\Program Files\Microsoft SDKs\Windows\v7.0A\lib\x64; (Windows SDK libraries)

Of course, if your application depends on some other framework or package, INCLUDE needs to contain path to its headers and LIB path to its libraries.

I wrote one short batch file for setting Visual Studio environment (environmental variables) for x86 or x64 builds:
setenv.bat (caret character is used to break long lines):
@rem B.Komazec Setting environment for using Microsoft Visual Studio 2010 x86/x64 tools.
@echo off

@if "%1"=="x86" goto set_x86
@if "%1"=="x64" goto set_x64
@if "%1"=="" goto error

:set_x86
@echo Setting environment for using Microsoft Visual Studio 2010 x86 tools.

set INCLUDE=^
c:\Program Files\Microsoft Visual Studio 10.0\VC\include;^
c:\Program Files\Microsoft SDKs\Windows\v7.0A\include;

set LIB=^
c:\Program Files\Microsoft Visual Studio 10.0\VC\lib;^
c:\Program Files\Microsoft SDKs\Windows\v7.0A\lib;

set PATH=^
%SystemRoot%\system32;^
c:\Program Files\Microsoft Visual Studio 10.0\VC\bin;^
c:\program files\microsoft visual studio 10.0\common7\ide;

goto test_bin_locations

:set_x64
@echo Setting environment for using Microsoft Visual Studio 2010 x64 tools.

set INCLUDE=^
c:\Program Files\Microsoft Visual Studio 10.0\VC\include;^
c:\Program Files\Microsoft SDKs\Windows\v7.0A\include;

set LIB=^
c:\Program Files\Microsoft Visual Studio 10.0\VC\lib\amd64;^
c:\Program Files\Microsoft SDKs\Windows\v7.0A\lib\x64;

set PATH=^
%SystemRoot%\system32;^
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64;^
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin;^
C:\Program Files\Microsoft Visual Studio 10.0\Common7\ide;

goto test_bin_locations

:test_bin_locations
@echo on
where nmake
where cl.exe
where link.exe
@echo off
goto:eof

:error
@echo Usage: setenv.bat [x86^|x64]

goto:eof

For example, to set environment for building x64 (AMD64) application with VS2010, call this script providing "x64" as an argument:

setenv x64

Note that changes made with SET will remain only for the duration of the current CMD session. To check the value of some environment variable in command prompt window use ECHO command followed by environment variable surrounded with percent signs:

echo %Path%

You can also use batch files that come with Visual Studio:
Visual Studio 2010:
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\vcvars32.bat
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_amd64\vcvarsx86_amd64.bat
C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\x86_ia64\vcvarsx86_ia64.bat
C:\Program Files\Microsoft Visual Studio 10.0\VC\vcvarsall.bat

Visual Studio 2008:
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin\amd64\vcvarsamd64.bat
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin\x86_amd64\vcvarsx86_amd64.bat
C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat

Links and references:
NMAKE Reference (MSDN)
An introduction to Makefiles
Nmake Makefile Tutorial and Example
CL Environment Variables
LINK Environment Variables
Setting the Path and Environment Variables for Command-Line Builds

Wednesday, 21 September 2011

How to get processor architecture information on Windows

Is your machine 32-bit or 64-bit?

One way of getting this information is reading PROCESSOR_ARCHITECTURE value which can be found in Windows Registry at this location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment.

This value on my 32-bit Intel machine is x86 and on another 64-bit machine with Intel Xeon processor is AMD64.

64-bit vs 32-bit

Processors, installed operating systems and applications can be 32-bit or 64-bit. OS needs processor support (64-bit OS requires 64-bit processor) and Application needs operating system support (64-bit application requires 64-bit OS). Backwards compatibility is preserved (32-bit OS runs on 64-bit processor; 32-bit application runs on 64-bit OS).

Here is a short review of all possible combinations:

Operating System
32 64
P
r
o
c
32 OK x 32 A
p
p
x x 64
64 x OK
OK OK 32

Backward compatibility is not preserved in kernel mode: 32-bit drivers won't work on 64-bit OS!

  • 32-bit applications can run both on 32-bit and 64-bit operating systems. 64-bit Windows OS uses WoW64 emulation layer to support executing 32-bit applications which are, by default, installed in C:\Program Files (x86) directory, load 32-bit libraries from C:\Windows\SysWOW64 and use HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node portion of the registry.
  • 64-bit applications can run only on 64-bit operating systems. It is possible to run 64-bit operating system on (physically) 32-bit machine (by using some of Virtual Machines, e.g. Oracle's VirtualBox).

It is possible to build applications that target 64-bit architectures on 32-bit hosts. This process is called cross-compiling.