Bruce Eitman

Windows CE: Data Synchronization using Interlocked Functions

Bruce Eitman — Thu, 04 Sep 2008 00:42:05 GMT

Windows CE provides several ways to synchronize threads and access to data by multiple threads. These include the synchronization objects that are typical for a multithreaded operating system: mutex, semaphores, events and critical sections. These synchronization objects may be overkill for protecting access to a single variable. The Interlocked functions can be used to control access to a single 32 bit variable.

The interlocked functions are:

Function Name	Action
InterlockedCompareExchange	Compares two values and writes a third value if they are equal
InterlockedCompareExchangePointer	Compares two values and writes a third value if they are equal
InterlockedDecrement	Decrements a value
InterlockedExchange	Writes a new value to a location
InterlockedExchangeAdd	Adds a value to a location
InterlockedExhangePointer	Writes a new value to a location
InterlockedIncrement	Increments a value
IntelockedTestExchange	Compares two values and writes a third value if they are equal

These functions all perform atomic actions on the data, meaning that they perform the action without a context switch during the function handling. The functions work by calling into the kernel which then causes an exception. The action is then handled within the exception handler. The functions change a value that is pointed to by a pointer that is passed in. The pointer must point to a 32 bit value that is 32 bit aligned except for x86 processors (for a discussion on what happens if the pointer is not 32 bit aligned see: Platform Builder: Data Misalignment).

A few examples:

// Global data

DWORD Data = 0;

void Foo()

{

DWORD Result;

Result = InterlockedIncrement( &Data ); // Result will contain the original value of Data

Result = InterlockedCompareExchange( &Data, 3, 1 ); // if Data == 1 then Result will be 1 and Data will be set to 3

Result = InterlockedExhangeAdd( &Data, 6 ); // Result contains the original value of Data and Data will contain Data += 6
}

The Interlocked functions provide a safe and convenient way to write to a variable from multiple threads. I say convenient because these functions don’t require the overhead of other synchronization methods when managing small amounts of data. If on the other hand, you need to manage changing multiple values, the other synchronization methods may be a better choice.

Platform Builder: Data Misalignment

Bruce Eitman — Wed, 03 Sep 2008 20:04:20 GMT

I ran into a problem today with data misalignment. The problem would have taken a lot of time to find if I wasn’t already familiar with the problem (kicking myself now.)

I was porting an existing driver into an old platform to update the platform. So I know that the driver works, but when it started it would data abort. This particular driver is quite large and complex, so a simple read of the code would not be possible in a reasonable amount of time. I have some experience with tracking the cause of a data abort (Windows CE: Finding the cause of a Data Abort.) But this data abort was occurring in Coredll.dll, so I knew that it was because of something that the driver was passing into a function.

Thinking back on what I had done to port the driver into this platform here is what I did:

1. Copy the driver into the platform

2. Added some members to a data structure in a header file already in the platform

I know that the driver is well tested, but let’s review the data structure:

#pragma pack(1)

typedef struct _MYSTRUCT {

BYTE var1;

BYTE var2;

DWORD var4;

BYTE var5;

DWORD NewVar1;

DWORD NewVar2;

DWORD NewVar3;

} MYSTRUCT;

#pragma pack()

If you look at it closely, you may already have seen my problem. Of course depending on my driver code, this structure could be perfectly acceptable, but knowing my driver it is not. My driver increments NewVar1 using InterlockedIncrement() and decrements it using InterlockedDecrement(). If you aren’t familiar with these functions, these functions take as their parameter a pointer to a 32 bit location. InterlockedIncrment() then effectively does *ptr++ which when I pass in the address of NewVar1 is dereferencing a location that is not 32 bit aligned, which for the ARM processors that I work with causes an exception.

The alignment is not a problem when accessing the structure directly. If I had the following code:

MYSTRUCT *Ptr;

//Allocate some memory for Ptr…

Ptr->NewVar1++;

This would compile and run okay, although some extra assembly code would be generated to handle the alignment for me. The problem also would not exist if the structure wasn’t wrapped with the pragma pack, which tells the compiler not to add any padding to the structure to align the members.

The problem comes from passing the address of NewVar1 into another function.

The solution for my problem is to cause NewVar1 to be aligned by adding an extra byte before it in the definition.

Go to Summary of When Things Go Wrong

Tags: Data Abort

Books on My Shelf: Windows Embedded CE 6.0 Fundamentals

Bruce Eitman — Wed, 27 Aug 2008 03:10:52 GMT

Windows Embedded CE 6.0 Fundamentals by Stanislav Pavlov and Pavel Belevsky is one of the first books about developing a Windows CE device using Platform Builder availabe in several years.

As the name states, this book contains the fundamentals of developing using Platform Builder. It does not venture into too many details though, so this is a book for beginners, not beginning programmers, but beginners with Platform Builder.

This book is fairly short at 230 pages, as technical books go, so reading it doesn't take long and can be worth the time.

This book's success is a a discussion of the architecture of Windows CE. This is especially helpful becuase of the major changes to the Windows CE architecture in version 6.0. This includes a discussion of how processes are managed, kernel and user mode drivers and virtual memory.

The book falls short though in details, and maybe that is appropriate for a fundamentals book. It would have been good to have provided some source code examples to explain some of the material better. There are also some topics which are covered better in Platform Builder Help.

Overall, if you are new to Platform Builder or need information about the changes to the Windows CE architecture, this is a good book to read.

Platform Builder: Sources.cmn

Bruce Eitman — Sun, 24 Aug 2008 02:56:33 GMT

Sources.cmn is a build configuration system file that allows you to set common variables. This can be useful if more than one directory in the build tree need the a variable set to the same value because it can reduce your maintenance efforts.

Each build tree can use one sources.cmn file. When build.exe runs, it will determine the root of the build tree by looking for the top most folder with a Dirs file. Build.exe then sets BUILDROOT to the top most folder with a Dirs files. Makefile.def in Public\Common\OAK\Misc will include $(BUILDROOT)\sources.cmn if it exists.

Some of the variables that are commonly set in sources.cmn include CDEFINES, ADEFINES and INCLUDES.

CDEFINES and ADEFINES are used to set macros that are common to multiple build folders. These might include RAM and ROM sizes, but might also include OEM and CPU specific macros.

The include path, INCLUDES, can be set in sources.cmn which is very helpful, especially if and when you change the directory structure.

The Platform directory tree should have WINCEOEM set to 1, so sources.cmn is a good place to do this rather than setting it in each sources file.

Note: Starting with Windows CE 5.0, sources.cmn is no longer really an option. It is required because sources.cmn needs to at least set _COMMONPUBROOT, _ISVINCPATH, and _OEMINCPATH. These were set by build.exe in prior versions.

For more on sources files take a look at: Platform Builder: Sources Files and Platform Builder: Sources Files 2

Go to Summary of Building Windows CE

Tags: Build.exe, Sources

Platform Builder: Sources Files 2

Bruce Eitman — Fri, 22 Aug 2008 22:22:36 GMT

I was looking over my original post about sources files (Platform Builder: Sources Files) today and it occurred to me that there is room to improve. That post provided basic information to create a sources file that will build a driver. The following are other things that you can do within a sources file:

Set CFLAGS and AFLAGS

CFLAGS and AFLAGS cannot be set in a sources file. Instead, you will need to set CDEFINES and ADEFINES to set compiler and assembler command line flags.

Set CDEFINES, LDEFINES and ADEFINES

These variables are used to change the command line parameters to the compiler (CDEFINES), assembler (ADEFINES) and linker (LDEFINES).

As with all variables in a sources file the way to set the variable is <VAR NAME>=<DATA>. So, using CDEFINES for an example:

CDEFINES=-DMY_MACRO –DOTHER_MACRO=5

Which causes the compiler command line to define MY_MACRO and to define OTHER_MACRO=5. The –D is the compiler command line flag to define a macro. But wait, setting CDEFINES this way will cause any previous settings to be lost. This is exactly why we cannot use CFLAGS and AFLAGS, it is set this way in makefile.def. So a better way to set a variable is:

CDEFINES=$(CDEFINES) -DMY_MACRO –DOTHER_MACRO=5

By doing so, we are setting CDEFINES equal to the current value in CDEFINES plus defining MY_MACRO and OTHER_MACRO=5.

Skip Building

You can skip building a directory in the build tree by checking the component environment variable and setting SKIPBUILD. SKIPBUILD tells build to skip building the directory.

!if "$(BSP_NOTHISDRIVER)" == "1"

SKIPBUILD=1

!endif

This checks to see if the environment variable BSP_NOTHISDRIVER is set to “1”. If it is then sets SKIPBUILD=1. For those of us that don’t like variable with “NO” in them, you can use:

!if "$( BSP_THISDRIVER)" == ""

SKIPBUILD=1

!endif

This checks to see if BSP_THISDRIVER is not set. If it is not then sets SKIPBUILD=1.

Build Something Before or After Building the Directory

WINCETARGETFILE0 can be used to tell the build system to build a target before building the current directory. WINCETARGFILES tells the build system to build a target after building the current directory. I have given an example of using WINCETARGETFILES in Windows CE Platform Builder: Automatically putting files in the SDK during build so let’s look at WINCETARGETFILE0.

In this example, I have a DLL that is built in a different directory tree and want to extend the DLL by adding some new functions. The original DLL is built in two directories, one that creates a LIB file and the other that links the LIB to create a DLL. In this directory there is source code that adds some functions and a def file that exports the functions. For maintainability, I want to use the def file from the original DLL folder, but extend it to add new definitions. To do this, use WINCETARGETFILE0 set equal to DEFFILE:

WINCETARGETFILE0=$(DEFFILE)

Create or edit makefile.inc in the same folder as the sources file. Add a target to makefile.inc and build instructions. In this case the target is $(DEFFILE) and the instructions will copy the original def file to this directory and append the contents of Custome.def to it:

$(DEFFILE): Custom.def

xcopy ..\Original\BuildDll\Original.def .

type Custom.def >> Original.def

So the new def file will be created before building this directory and the def file will be available when the DLL is linked.

Tell the Solution Explorer about files to Display

The Solutions Explorer window uses sources files to determine which files to display and how to display them.

SOURCES not only tells build.exe which files should be built in the directory, but tells the Solutions Explorer which files to list in the “Source files” folder. The following will cause serial.cpp to display in the Source Files folder:

SOURCES=serial.cpp

FILE_VIEW_ROOT_FOLDER tells the Solutions Explorer window to which files to show in the root folder of the directory. The following will cause sources and makefile to display in the root folder:

FILE_VIEW_ROOT_FOLDER=Sources Makefile

FILE_VIEW_RESOURCE_FOLDER tells the Solutions Explorer window which files to display in the “Resource files” folder.

FILE_VIEW_RESOURCE_FOLDER=serial.rc

FILE_VIEW_INCLUDES_FOLDER tells the Solutions Explorer windows which files to display in the “Include files” folder.

FILE_VIEW_INCLUDES_FOLDER=serial.h

FILE_VIEW_PARAMETER_FOLDER tells the Solutions Explorer window which files to display in the “Paramter files” folder.

FILE_VIEW_PARAMETER_FOLDER=serial.bib serial.reg

Go to Summary of Building Windows CE

Tags: Build.exe, Sources

Windows CE: Running Batch Files

Bruce Eitman — Thu, 21 Aug 2008 23:53:28 GMT

Every once in a while I am asked about running batch files on Windows CE. My first response of course to ask why? But let’s assume that you are set on using a batch file. I know you probably think that batch files are easy to modify, but really I don’t think that they are and you may find that batch files for Windows CE aren’t that easy.

The first thing that you are going to need is a Command window, or DOS shell, to run your batch file in. It is likely that your system doesn’t have one, but if you have control over the content of your Windows CE OS ROM image, you can add one using Platform Builder or as your OS team to add one. If you don’t have one and don’t have control over the OS, then you will need to find one that you can use. I do have control over my OS, so I have never needed to look for one.

To demonstrate that batch files can run in Windows CE, and to check for some of the shortcomings, I created the following batch file:

@ECHO OFF

echo Running Batch.bat

set LOGOFILE="\Release\Logo.jpg"

set TESTFILE="\Release\Test.txt"

if EXIST %LOGOFILE% xcopy %LOGOFILE% \Windows

if EXIST %LOGOFILE% copy %LOGOFILE% \Windows

for /F %%i in (%TESTFILE%) Do (

echo %%i

)

CALL :ListDirs

Pushd \Windows

Dir /A:D

Popd

GOTO :EOF

:ListDirs

cd %1

for /F %%i in ('dir /A:D /B /S \Windows' ) Do (

Echo Dir %%i

)

GOTO :EOF

Careful, don’t think that everything in this file works. Some things do not work:

1. Xcopy isn’t available, but copy is

2. For loops aren’t suppprted, but if statements are

3. Call is supported, but not to functions in the batch file

4. Pushd and Popd aren’t supported, but cd is

I am sure that if I kept adding things to this batch file, I would have found more unsupported features that are supported in big Windows or DOS.

I think that it might be easier to write a simple application to do most of what you might want to do in a batch file. You can write an application that uses main() for Windows CE, and have for loops. But I am a C programmer so what is easy to me might not be so easy for you.

Platform Builder: Cloning an MDD Lib

Bruce Eitman — Thu, 21 Aug 2008 04:10:04 GMT

I have written about cloning code from the Public tree to a platform in the past, but someone recently asked about cloning an MDD lib build directory. More specifically the question was about cloning ufnmdd.lib in Windows CE 5.0, which is actually created by linking three separate libs together. In this article, I will walk through the process by cloning one of those libs.

Ufnmdd.lib is created by linking ufnmddbase.lib, ceosutil.lib and defbuslib.lib. I suspect that most people don’t really need to clone ceosutils.lib and defbuslib.lib, but instead really need to just clone ufnmddbase.lib for debugging purposes. So I will clone ufnmddbase.lib and show two different ways to then link the new lib with a USB function driver.

Cloning UFNMDDDBase.LIB

Because this is a statically linked library, the easiest way to clone it is to simply copy the folder to the platform and then make some changes to the sources file. For this example, I will clone ufnmddbase.lib into the MainstoneII platform. I point this out because you can refer to the MainstoneII platform in the WINCE500 build tree. The USB function driver in MainstoneII is a single directory that builds some source code and links with ufnmdd.lib among other libs.

The steps to add ufnmddbase.lib to the MainstoneII:

1. Move the current code down a folder from Drivers\USBFN to Drivers\USBFN\DLL (or any name you choose.) This makes way for us to add a new folder under USBFN for ufnmddbase.lib to build in.

2. Create a Dirs file in Drivers\USBFN and add the following to it:
DIRS =DLL

3. Test building the driver just to be sure that this didn’t break it already

4. Copy PUBLIC\COMMON\OAK\DRIVERS\USBFN\CONTROLLER\MDD to Drivers\USBFN\MDD

5. Add MDD to the Drivers\USBFN\Dirs before DLL:
DIRS=MDD DLL

6. Add RELEASETYPE=PLATFORM to the Drivers\USBFN\MDD\sources file

7. Test build again and check that ufnmddbase.lib is in MainstoneII\Lib sub folders

Now that ufnmddbase.lib has been cloned to the platform, there are two choices for using it within the USB function driver; link directly with the three libs in ufnmdd.lib or create a new folder to link the libs and create a new ufnmdd.lib.

Link Directly with UFNMDDBase.lib

For this option, edit Drivers\USBFN\DLL\sources to change the from linking with ufnmdd.lib to linking with the three separate libraries. Change the SOURCELIBS from:

SOURCELIBS= \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\pxa27x_usbfn.lib \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ufnmdd.lib

To:

SOURCELIBS= \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\pxa27x_usbfn.lib \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ceosutil.lib \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\defbuslib.lib \

$(_TARGETPLATROOT)\lib\$(_CPUINDPATH)\ufnmddbase.lib

Then rebuild the driver.

Create a New UFNMDD.LIB

For this option, create a new folder to link the three libraries to create a new ufnmdd.lib. The steps are:

1. Create a new folder: Drivers\USBFN\UFNMDD

2. Create a sources file in the new folder and include:

TARGETNAME=UFNMDD

TARGETTYPE=LIBRARY

RELEASETYPE=PLATFORM

SOURCELIBS= \

$(_TARGETPLATROOT)\lib\$(_CPUINDPATH)\ufnmddbase.lib \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ceosutil.lib \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\defbuslib.lib

SOURCES=

3. Change the SOURCELIBS from:

SOURCELIBS= \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\pxa27x_usbfn.lib \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\ufnmdd.lib

To:

SOURCELIBS= \

$(_COMMONOAKROOT)\lib\$(_CPUINDPATH)\pxa27x_usbfn.lib \

$(_TARGETPLATROOT)\lib\$(_CPUINDPATH)\ufnmdd.lib

4. Add UFNMDD to Drivers\USBFN\Dirs after MDD and before DLL:
DIRS=MDD UFNMDD DLL

This should give the bases for cloning an MDD lib file for a driver. If you wanted to also clone the other libraries, the technique is the same.

Tags: Build.exe

Windows CE: Automatically setting the Object Store Size

Bruce Eitman — Thu, 21 Aug 2008 01:40:32 GMT

A customer asked me an interesting question today. He wanted to know how to know how much space his system needed for the Object Store. That had me thinking that there could be a way to monitor the size and automatically adjust it, Windows Mobile does that. Of course Windows Mobile probably has a better way to manage the Object Store size than this, I am guessing that it monitors the size within FileSys and adjusts it as requests come in.

I wrote the following code just to see what could be done. I don’t necessary think that this is a good way to do it, but it is something that could be used to monitor the Object Store.

CheckObjectStoreSize() will check the current number of pages being used by the Object Store and will adjust the size to the maximum of:

- The documented minimum size of 128K

o #define DOCUMENTED_MIN_OS_PAGES( PageSize ) (( 128 * 1024 )/PageSize )

- Current number of pages being used plus MIN_FREE_PAGES, which is a random number that I picked

o #define MIN_FREE_PAGES( PageSize ) ((100 * 1024)/PageSize)

Because the number of pages being used is obtained using GetSStoreInformation() this function will both increase and decrease the size of the Object Store.

void CheckObjectStoreSize()

{

DWORD StorePages,

DWORD RamPages

DWORD PageSize;

DWORD CurrentNeededSize;

DWORD Status;

DWORD ObjectStorePages;

STORE_INFORMATION StoreInfo;

GetSystemMemoryDivision(&StorePages, &RamPages, &PageSize);

GetStoreInformation(&StoreInfo);

CurrentNeededSize = ((StoreInfo.dwStoreSize - StoreInfo.dwFreeSize)/PageSize) + 1 ;

ObjectStorePages = DOCUMENTED_MIN_OS_PAGES(PageSize);

// Check to see if the current Object Store needs to be bigger

if( ObjectStorePages < (CurrentNeededSize + MIN_FREE_PAGES( PageSize) ))

{

ObjectStorePages = (CurrentNeededSize + MIN_FREE_PAGES( PageSize));

}

// Do we have enough free space to increase the Object Store?

if (ObjectStorePages >= (StorePages + RamPages))

{

RETAILMSG(1,(TEXT("Out of memory to increase Object Store\n")));

return;

}

Status = SetSystemMemoryDivision (ObjectStorePages);

if (Status)

{

RETAILMSG(1,(TEXT("Failed to set Object Store: ")));

switch(Status)

{

case SYSMEM_MUSTREBOOT:

RETAILMSG(1,(TEXT(" SYSMEM_MUSTREBOOT\n")));

break;

case SYSMEM_REBOOTPENDING:

RETAILMSG(1,(TEXT(" SYSMEM_REBOOTPENDING\n")));

break;

case SYSMEM_FAILED:

RETAILMSG(1,(TEXT(" SYSMEM_FAILED (%d)\n", GetLastError())));

break;

default:

break;

}

I then tested the CheckObjectStoreSize() by calling it in a loop from WinMain(). The loop copies a file from the storage card to the Object Store 50 times followed by deleting the files on the next 40 times through the loop.

int WINAPI WinMain(

HINSTANCE hInstance,

HINSTANCE hPrevInstance,

LPTSTR lpCmdLine,

int nCmdShow

)

{

DWORD Counter = 0;

TCHAR FileName[ MAX_PATH ];

BOOL Toggle = FALSE;

while( 1 )

{

Sleep( 5000 );

wsprintf( FileName, TEXT("\\File%d.txt"), Counter++ );

RETAILMSG( 1, (TEXT("Copy to %s\n"), FileName ));

if( Counter > 50 )

{

Toggle = !Toggle;

Counter = 0;

}

if( Toggle )

{

if( !DeleteFile( FileName ))

RETAILMSG( 1, (TEXT("File Delete failed %d\n"), GetLastError() ));

}

else

{

if( !CopyFile( TEXT("\\Storage Card\\a.txt"), FileName, FALSE ))

RETAILMSG( 1, (TEXT("File Copy failed %d\n"), GetLastError() ));

}

CheckObjectStoreSize();

}

Tags: ObjectStore