December 30, 2013

Why your comment does not appear on my blog

It seems I have to clarify why some comments does not appear on my blog.
My blog is a place where I publish my personal views. It is not in anyway a public forum.

To be published on my blog, a comment must be:
  1. Directly related to the subject
  2. Add something complementing the point of view I expressed
  3. Be constructive, informative and helpful 
  4. Use the same language as the post you comment

The best way to not be published is:
  1. Use my blog to spread your own ideas or promote your own blog
  2. Being disrespectful
  3. Start a controversy
  4. Start a flame war

If your comment doesn't appears on my blog, you can:
  1. Express your ideas on your own blog
  2. Express your ideas in a public group such as "Delphi developers" community on G+
  3. Email me privately (My email can be found everywhere) 

December 28, 2013

Compiling to native code begets better performance

Todays with Mobile platforms such as IOS and Android, developers are faced with writing application targeted to several platforms. Of course developers want to do that with minimal effort. This is why the market of cross platform tools is growing.

Embarcadero do it well with RAD Studio XE5 (Includes Delphi and C++ Builder). There is an interesting article on NetworkWorld talking about the subject and comparing different solutions.



Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 25, 2013

Hotfix 3 for Delphi XE5 update 2

If you haven't noticed yet, there is a 3rd "hotfix" available for Delphi XE5, C++ Builder XE5 and RAD Studio XE5 update 2.

This hotfix contains compiled obj files for JPEG support.

Download from Embarcadero CodeCentral available for registered users of the mentioned products.



Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 18, 2013

Mandelbrot Explorer for Android and Windows

Using Delphi XE5, I rewrote my Mandelbrot fractal explorer with the FireMonkey component framework. The result is a working application for Android and Windows. To generate Windows or Android version, it is enough to just change the target operating system. Nice cross-platform application!
 
On the picture below, you see the Windows version running on my desktop and the Android version running on my Nexus 7 device.
 

The source code is making use of what I described in two previous articles:
http://francois-piette.blogspot.be/2013/12/firemonkey-android-windows-and.html
http://francois-piette.blogspot.be/2013/12/mandelbrot-fractal-explorer.html


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 17, 2013

Coding in Delphi ebook

Coding in Delphi ebook by Nick Hodges.

Available to registered users of Delphi XE5, RAD Studio XE5, and Embarcadero All-Access XE. This promotional item is available as a special promotion through December 31, 2013.

There is also a Google+ community: https://plus.google.com/communities/110978417023349293804

You can download it from http://cc.embarcadero.com/Item/29670

December 15, 2013

FireMonkey, Android, Windows and PostMessage

FireMonkey framework (FMX for short) is definitely able to use custom [Windows] messages much like we have always done with the VCL. And this is also true when using FireMonkey to build Android applications.

Both Windows and Android support a messaging system. It is well known by Windows developers who use it with PostMessage, GetMessage, PeekMessage and similar Windows API call. It is much less known by Android developers. Android has a “looper” API which has the same purpose as Windows own messaging system although it is implemented differently and has somewhat more features.

Often, we use FireMonkey framework to build multi-platform applications. Thanks to Delphi XE5, we can build an application for different targets such as Win32, Win64, Android, iOS and MAC OSx. If correctly written, the same application source code can be recompiled for different target and run unchanged. Embarcadero made a lot of efforts to hide differences between the supported platforms.

Speaking about the messaging system, it must admit that Embarcadero forgot to write the abstraction layer required for the platforms. They made some work but it is incomplete and undocumented. This is why I wrote it. At least for Win32, Win64 and Android which are the 3 platforms I currently use.

The layer I wrote is made of a single class I named “TMessagingSystem”. I made two different implementations: one for Android and one for Win 32/64. TMessagingSystem class allows you to register any number of custom messages to a form and associate a custom message handler. Of course it also allows you to call PostMessage to put a message into the message queue.

At the application level, you use the exact same code for Windows or Android. You just have to make use of one of the implementations. You’ll do that using a conditional compilation.

Before showing the implementation details, I will present a demo application. That you can target for Windows or Android without changing a single line.

Demo application for Windows and Android


I built a simple application to emphasize how to use TMessagingSystem. Actually it does not do anything very interesting. It is made of a single form having a button and a memo. When you click on the button, it starts a new thread which will periodically PostMessage a custom message to the main form. You can click many times on the button to start many threads. Each thread will do the same.



The image above shows on the left a screen dump of the application running under Win7 and on the right, the same application running on my Nexus7.

All you see is a memo with messages. Nevertheless, this is really one of the main usages of a messaging system: organize asynchronous operation between threads.

Each line looks like this:

8380] Thread=2 Count=8 ThreadID=7528

“8380” is the thread ID of the thread doing the display. This is always the same and is the main thread ID. “Thread=2” is the sequential thread number having generated the message, “Count=8” is the number of messages generated by this thread and finally, “ThreadID=7528” is the thread ID of the thread generating the message. The later change according to each started thread.


Demo application source code


unit FmxMultiplatformPostMessageDemoMain;

interface

uses
    System.SysUtils, System.Types, System.UITypes, System.Classes,
    Generics.Collections,
    FMX.Types, FMX.Controls, FMX.Forms, FMX.Graphics, FMX.Dialogs,
    FMX.StdCtrls, FMX.Layouts, FMX.Memo,
    FMX.Overbyte.MessageHandling;

const
    WM_SHOW_MESSAGE = WM_USER + 1;

type
    TWorkerThread = class(TThread)
    public
        MsgSys : TMessagingSystem;
        Id     : Integer;
        procedure Execute; override;
    end;

    TForm1 = class(TForm)
        RunThreadButton: TButton;
        DisplayMemo : TMemo;
        ToolPanel: TPanel;
        procedure RunThreadButtonClick(Sender: TObject);
    private
        FMsgSys      : TMessagingSystem;
        FThreadCount : Integer;
        procedure Display(const Msg: String);
        procedure WorkerThreadTerminate(Sender: TObject);
        procedure WMShowMessage(var Msg: TMessage);
    protected
        procedure CreateHandle; override;
        procedure DestroyHandle; override;
    end;

var
  Form1: TForm1;

implementation

{$R *.fmx}

{ TForm1 }

procedure TForm1.CreateHandle;
begin
    inherited CreateHandle;
    FMsgSys := TMessagingSystem.Create(Self);
    FMsgSys.RegisterMessageHandler(WM_SHOW_MESSAGE, WMShowMessage);
end;

procedure TForm1.DestroyHandle;
begin
    FreeAndNil(FMsgSys);
    inherited DestroyHandle;
end;
 
procedure TForm1.RunThreadButtonClick(Sender: TObject);
var
    WorkerThread : TWorkerThread;
begin
    Inc(FThreadCount);
    Display('Start thread ' + IntToStr(FThreadCount));
    WorkerThread                 := TWorkerThread.Create(TRUE);
    WorkerThread.MsgSys          := FMsgSys;
    WorkerThread.Id              := FThreadCount;
    WorkerThread.FreeOnTerminate := TRUE;
    WorkerThread.OnTerminate     := WorkerThreadTerminate;
    WorkerThread.Start;
end;

procedure TForm1.WorkerThreadTerminate(Sender: TObject);
begin
    Display('Thread ' +
            IntToStr((Sender as TWorkerThread).Id) +
            ' terminated');
end;

procedure TForm1.WMShowMessage(var Msg: TMessage);
var
    Buffer : PChar;
begin
    Buffer := PChar(Msg.LParam);
    Display(Buffer);
    FreeMem(Buffer);
end;

procedure TForm1.Display(const Msg: String);
begin
    Displaymemo.Lines.Add(IntToStr(GetCurrentThreadID) + '] ' + Msg);
end;

{ TWorkerThread }

procedure TWorkerThread.Execute;
var
    I      : Integer;
    Buffer : PChar;
const
    MaxLen = 100;
begin
    // For demo, let's do it 10 times
    for I := 1 to 10 do begin
        // Simulate some processing time by sleeping
        Sleep(1000);

        // Allocate memory to hold a message, take care of the ending nul char
        GetMem(Buffer, SizeOf(Char) * (MaxLen + 1));
        // Copy message to allocated memory, protecting overflow
        StrLCopy(Buffer,
                 PChar('Thread=' + IntToStr(Id) +
                       ' Count=' + IntToStr(I) +
                       ' ThreadID=' + IntToStr(GetCurrentThreadID)),
                 MaxLen);
        // Force a nul char at the end of buffer
        Buffer[MaxLen] := #0;
        // Post a message to the main thread which will display
        // the message and then free memory
        MsgSys.PostMessage(WM_SHOW_MESSAGE, I, LParam(Buffer));
    end;
end;

end.

This source code is really simple, isn’t it? The beauty is that it can be compiled for Win32, Win64 and Android targets without changing anything.

All the code depending on the platform has been moved to “FMX.Overbyte.MessageHandling” unit. That one takes care of calling the correct API function according to the compiler used. This is the power of OOP.

There is nothing special in the demo application except one thing: The worker thread generates messages to be displayed by the main thread. We have to take care of what happens with the storage used for the message. We cannot simply pass a string because messages are limited to two parameters of type WParam and LParam, both mapped to NativeInt. We can neither pass a reference to a string variable because it is possible a new message is generated before the previous is consumed (This happens if the main thread is heavily busy while the worker thread runs at full speed). We have to dynamically allocate storage for the message and pass the reference thru one of the message parameters. I’ve chosen to use a simple memory block allocated by GetMem and freed by FreeMem. The pointer is then passed thru the LParam parameter. The thread allocates the memory and the main thread frees it. The same allocation size is always used regardless of the message length. It is better for the memory allocator, limiting memory fragmentation.

How to use it?


TMessagingSystem class must be instantiated when the form is allocated a handle. It must be freed when the form’s handle is destroyed. After instantiation, or at any point in time, RegisterMessageHandler must be called for each custom message. That’s all!

Single unit, multiple platforms


We have seen in the demo code that the same unit to “FMX.Overbyte.MessageHandling” is used whatever the target platform is. The magic is in that unit. Here is very short source code:

unit FMX.Overbyte.MessageHandling;
{$DEFINE OVERBYTE_INCLUDE_MODE}
{$IFDEF ANDROID}
    {$I FMX.Overbyte.Android.MessageHandling.pas}
{$ENDIF}
{$IFDEF MSWINDOWS}
    {$I FMX.Overbyte.Windows.MessageHandling.pas}
{$ENDIF}

The magic is into the conditional compilation. Symbols ANDROID and MSWINDOWS are automatically defined by the compiler according to the target platform you compile for. So that small unit actually includes the Android or the Windows specific unit depending on the compiler target platform.

The two included units are just normal unit, well almost. You cannot include a unit into another one without having a problem with the “unit” line. You cannot have two such lines. This is why the symbol “OVERBYTE_INCLUDE_MODE” is defined. In the two included units, this symbol is used to conditionally compile the “unit” line.

Implementation for Android


Messaging system on Android platform is hidden in the “Looper” API. Basically, the idea is simple: Android monitors a list of handle for data availability. The list of handles is maintained by the API. You can add a new handle using ALooper_addFd API function. Each handle is associated with a callback function that Android calls when data is available.

As a handle, I use the read side of a pipe. A pipe, under Android as well as other operating systems, is like a first-in first-out queue. It has two ends identified by two handles. One is the writing end; the other is the reading end. What you write at one end is available for reading at the other end. Between both ends is a buffer. Reads and writes are asynchronous. If writing is faster than reading, the buffer is filled and nothing is lost.

This pipe is used here is the message queue. When PostMessage is called, a record with the parameters is written to the pipe. When data is available for reading, the looper API will call the LooperCallBack function we registered. From this callback, we read the pipe to remove one record at a time. When a record is read, the message number written in it is used to fetch the message handler to be executed.


{$IFNDEF OVERBYTE_INCLUDE_MODE}
unit FMX.Overbyte.Android.MessageHandling;
{$ENDIF}

interface

uses
    System.SysUtils, System.Types, System.Classes, System.SyncObjs,
    Generics.Collections,
    FMX.Platform.Android,
    Androidapi.AppGlue, Androidapi.Looper,
    Posix.UniStd, Posix.Errno, Posix.StrOpts, Posix.PThread;

const
    WM_USER         = 1024;

type
    LPARAM  = NativeInt;
    WPARAM  = NativeInt;
    LRESULT = NativeInt;

    TMessage = record
        Msg    : NativeInt;
        WParam : WPARAM;
        LParam : LPARAM;
        Result : LRESULT;
    end;
    TMessageHandler = procedure (var Msg: TMessage) of object;

    TMessagingSystem = class(TComponent)
    protected
        FPipeFD    : TPipeDescriptors;
        FData      : Byte;
        FHandlers  : TDictionary;
        FLastError : String;
        FCritSect  : TCriticalSection;
        procedure HandleMessage(var Msg : TMessage);
        function  CreatePipe: Integer;
        procedure ClosePipe;
        procedure InstallEventHandler;
        procedure UninstallEventHandler;
    public
        constructor Create(AOwner : TComponent); override;
        destructor  Destroy; override;
        function RegisterMessageHandler(uMsg    : NativeInt;
                                        Handler : TMessageHandler) : Boolean;
        function PostMessage(uMsg   : NativeInt;
                             WParam : WPARAM;
                             LParam : LPARAM) : Boolean;
        property LastError : String read FLastError;
    end;

    HWND   = TMessagingSystem;

function GetCurrentThreadID : TThreadID;

implementation

function LooperCallback(
    FileDescriptor : Integer;
    Events         : Integer;
    Data           : Pointer): Integer; cdecl;
var
    Len : Integer;
    Msg : TMessage;
    Obj : TMessagingSystem;
begin
    Result := 1;
    // Data contains a reference to our class
    if Data = nil then
        Exit;
    // Ready to cast to our class
    Obj := TMessagingSystem(Data);
    // Check if it's our ReadDes
    Obj.FCritSect.Enter;
    try
        if FileDescriptor <> Obj.FPipeFD.ReadDes then
            Exit;
    finally
        Obj.FCritSect.Leave;
    end;

    while TRUE do begin
        Len := __read(FileDescriptor, @Msg, SizeOf(Msg));
        if Len <= 0 then
            break;
        Obj.HandleMessage(Msg);
    end;
end;

{ TMessagingSystem }

constructor TMessagingSystem.Create(AOwner: TComponent);
begin
    inherited Create(AOwner);
    FCritSect  := TCriticalSection.Create;
    FHandlers  := TDictionary.Create;
    CreatePipe;
    InstallEventHandler;
end;

destructor TMessagingSystem.Destroy;
begin
    UninstallEventHandler;
    ClosePipe;
    FreeAndNil(FCritSect);
    inherited Destroy;
end;

function TMessagingSystem.CreatePipe: Integer;
var
    Status  : Integer;
    Val     : Integer;
const
    FIONBIO = $5421;
begin
    FCritSect.Enter;
    try
        if (FPipeFD.ReadDes <> 0) or (FPipeFD.WriteDes <> 0) then begin
            FLastError := 'Pipe already created';
            Result := -1;
            Exit;
        end;
        Status := Pipe(FPipeFD);
        if Status = -1 then begin
            Result := errno;
            FLastError := 'Pipe() failed. Error #' + IntToStr(Result);
        end
        else begin
            Result := 0;
            Val := 1;
            if ioctl(FPipeFD.ReadDes, FIONBIO, @Val) = -1 then begin
                Result := errno;
                FLastError := 'ioctl(FIONBIO) failed. Error #' + IntToStr(Result);
                Exit;
            end;
        end;
    finally
        FCritSect.Leave;
    end;
end;

procedure TMessagingSystem.ClosePipe;
begin
    FCritSect.Enter;
    try
        if FPipeFD.ReadDes <> 0 then begin
            __close(FPipeFD.ReadDes);
            FPipeFD.ReadDes  := 0;
        end;
        if FPipeFD.WriteDes <> 0 then begin
            __close(FPipeFD.WriteDes);
            FPipeFD.WriteDes := 0;
        end;
    finally
        FCritSect.Leave;
    end;
end;

procedure TMessagingSystem.InstallEventHandler;
var
    AndroidApp : PAndroid_app;
    Data       : Pointer;
const
    LOOPER_ID_MESSAGE_OVERBYTE = LOOPER_ID_USER;
begin
    AndroidApp := GetAndroidApp;

    Data := Self;
    ALooper_addFd(AndroidApp.looper,
                  FPipeFD.ReadDes,
                  LOOPER_ID_MESSAGE_OVERBYTE,
                  ALOOPER_EVENT_INPUT,
                  LooperCallback,
                  Data);
end;

procedure TMessagingSystem.UninstallEventHandler;
var
    AndroidApp : PAndroid_app;
begin
    FCritSect.Enter;
    try
        if FPipeFD.ReadDes <> 0 then begin
            AndroidApp := GetAndroidApp;
            ALooper_removeFd(AndroidApp.looper, FPipeFD.ReadDes);
        end;
    finally
        FCritSect.Leave;
    end;
end;

function TMessagingSystem.RegisterMessageHandler(
    uMsg    : NativeInt;
    Handler : TMessageHandler): Boolean;
begin
    FCritSect.Enter;
    try
        FHandlers.AddOrSetValue(uMsg, Handler);
    finally
        FCritSect.Leave;
    end;
    Result := TRUE;
end;

function TMessagingSystem.PostMessage(
    uMsg   : NativeInt;
    WParam : WParam;
    LParam : LParam): Boolean;
var
    Msg : TMessage;
begin
    Result := FALSE;
    FCritSect.Enter;
    try
        if FPipeFD.WriteDes = 0 then begin
            FLastError := 'Pipe is not open';
            Exit;
        end;
        Msg.Msg    := uMsg;
        Msg.WParam := WParam;
        Msg.LParam := LParam;
        Msg.Result := 0;

        if __write(FPipeFD.WriteDes, @Msg, SizeOf(Msg)) = -1 then begin
            FLastError := 'write() failed. ErrCode=' + IntToStr(errno);
            Exit;
        end;
    finally
        FCritSect.Leave;
    end;
    Result := TRUE;
end;

procedure TMessagingSystem.HandleMessage(var Msg: TMessage);
var
    Handler : TMessageHandler;
    Status  : Boolean;
begin
    FCritSect.Enter;
    try
        Status := FHandlers.TryGetValue(Msg.Msg, Handler);
    finally
        FCritSect.Leave;
    end;
    if Status then
        Handler(Msg);
end;

function GetCurrentThreadID : TThreadID;
begin
    Result := Posix.PThread.GetCurrentThreadID;
end;

end.

In that code, you’ll find a few data types frequently used in Windows applications. I used the same data types for compatibility with existing code.

TMessagingSystem class is very simple. Basically, it registers a pipe read handle with the looper API with an associated callback function. It also maintains a dictionary of message handlers. The key is the message number. The looper API also carries one pointer for you. It will give it back as an argument of the callback function. Here the pointer is used as a reference to the class instance, making is available when the callback function is called.

A critical section is used to avoid problems accessing the class data from several threads at the same time. Using this critical section makes the class fully thread safe.


Implementation for Windows


The Windows implementation makes obviously use of Windows own messaging API. There is no queue in the class because Windows queue is used.

FireMonkey forms does not provide any support for custom messages. This is not really a problem because a FireMonkey forms are just a Windows window. As any window, a FireMonkey form running on Windows has a HWND (Handle of WiNDow) and a window procedure handling all messages for the window.

To hook into this system, we must use standard Windows programming. By standard I mean it has always existed as far as I remember. What we need is to “subclass” the window. And surprisingly, this is very easy!

Windows internally maintain a structure for each window. In that structure you have all informations required for Windows to handle the window. This includes the pointer to the window procedure.

And Windows provides a function to access his internal structure. Our problem is just to get the current pointer to the window procedure and replace it with a pointer to our own procedure. From our own procedure, we will call the original procedure, or not. Our own window procedure has access to all messages sent/posted to the window, including those we add.

We have just one small problem: Windows does not know anything about a Delphi class instance. A window procedure is a simple procedure, not an object method. The problem is to get hand on our TMessagingSystem class instance from our own window procedure.

Fortunately Windows is incredibly well designed. We, as developer, can associate with any window a small piece of data called an “Atom” in Windows terminology. Once an “Atom” is created (It just has a name), you can associate the atom with any window along with a piece of data. That piece of data will be the reference to our TMessagingSystem class instance.

When called by Windows, our window procedure receives the handle of the window. We use it to fetch the piece of data we associated using the atom. From there we have access to TMessagingSystem class instance and check for the message to handle. if it is one of our registered messages, we just call the handler. If not one of our messages, the the original window procedure is called.

Here is the source code:

{$IFNDEF OVERBYTE_INCLUDE_MODE}
unit FMX.Overbyte.Windows.MessageHandling;
{$ENDIF}

interface

uses
    WinApi.Windows, WinApi.Messages,
    System.Classes, System.SysUtils, System.SyncObjs,
    Generics.Collections,
    FMX.Forms, FMX.Platform.Win;

const
    WM_USER = WinApi.Messages.WM_USER;

type
    TMessage        = WinApi.Messages.TMessage;
    WPARAM          = WinApi.Windows.WPARAM;
    LPARAM          = WinApi.Windows.LPARAM;
    TMessageHandler = procedure (var Msg: TMessage) of object;
    TWndProc        = function (hwnd   : HWND;
                                uMsg   : UINT;
                                wParam : WPARAM;
                                lParam : LPARAM): LRESULT; stdcall;

    TMessagingSystem = class(TComponent)
    protected
        FHWnd             : HWND;
        FHandlers         : TDictionary;
        FOriginalWndProc  : TWndProc;
        FLastError        : String;
        FCritSect         : TCriticalSection;
    public
        constructor Create(AOwner : TComponent); override;
        destructor  Destroy; override;
        function RegisterMessageHandler(uMsg    : NativeInt;
                                        Handler : TMessageHandler) : Boolean;
        function PostMessage(uMsg   : NativeInt;
                             WParam : WPARAM;
                             LParam : LPARAM) : Boolean;
        property LastError : String read FLastError;
    end;

function GetCurrentThreadId: DWORD; stdcall;

implementation

var
  MsgSysAtom       : TAtom;
  MsgSysAtomString : String;


function WndProc(hwnd: HWND; uMsg: UINT; wParam: WPARAM; lParam: LPARAM): LRESULT; stdcall;
var
    Msg     : TMessage;
    MsgSys  : TMessagingSystem;
    Handler : TMessageHandler;
    Status  : Boolean;
begin
    // Search if the window handle is associated with TMessageingInstance
    // We know this because we registered an atom for that purpose
    if GlobalFindAtomW(PChar(MsgSysAtomString)) <> MsgSysAtom then begin
        // Not found, just do default processing
        Result := DefWindowProc(hwnd, uMsg, wParam, lParam);
        Exit;
    end;
    // Fetch the atom property and cast it to a TMessagingSystem class
    MsgSys := TMessagingSystem(GetProp(hwnd, MakeIntAtom(MsgSysAtom)));

    // Now use the dictionary to see if the message is one we'll handle
    MsgSys.FCritSect.Enter;
    try
        Status := MsgSys.FHandlers.TryGetValue(uMsg, Handler);
    finally
        MsgSys.FCritSect.Leave;
    end;
    if Status then begin
        // Found the message and his message handler. Call it using
        // the TMessage record to hold the values
        Msg.Msg    := uMsg;
        Msg.WParam := wParam;
        Msg.LParam := lParam;
        Msg.Result := 0;
        Handler(Msg);
        Result := Msg.Result;
    end
    else begin
        // Not one of our messages, just execute original window procedure
        Result := MsgSys.FOriginalWndProc(hwnd, uMsg, wParam, lParam);
    end;
end;

{ TMessagingSystem }

constructor TMessagingSystem.Create(AOwner: TComponent);
begin
    if not (AOwner is TCommonCustomForm) then
        raise Exception.Create('TMessagingSystem.Create failed. Invalid owner');
    inherited Create(AOwner);
    FCritSect  := TCriticalSection.Create;
    FHandlers  := TDictionary.Create;

    // Find window handle corresponding to the owner form
    FHWnd := WindowHandleToPlatform(TCommonCustomForm(AOwner).Handle).Wnd;

    // If not already done, register the atom we'll use to associate
    // our messaging system with the window handle
    if MsgSysAtom = 0 then begin
        MsgSysAtomString := 'OverbyteMessagingSystem' +
                                     IntToHex(GetCurrentProcessID, 8);
        MsgSysAtom       := GlobalAddAtomW(PChar(MsgSysAtomString));
    end;

    // Associate our messaging system with the window handle
    SetProp(FHWnd, MakeIntAtom(MsgSysAtom), THandle(Self));

    // Subclass the form. That is change his handling procedure
    FOriginalWndProc := TWndProc(GetWindowLongPtr(FHWnd, GWLP_WNDPROC));
    SetWindowLongPtr(FHWnd, GWLP_WNDPROC, NativeInt(@WndProc));
end;

destructor TMessagingSystem.Destroy;
begin
    if Assigned(FOriginalWndProc) then begin
        SetWindowLongPtr(FHWnd, GWLP_WNDPROC, NativeInt(@FOriginalWndProc));
        FOriginalWndProc := nil;
    end;
    FreeAndNil(FHandlers);
    FreeAndNil(FCritSect);
    inherited Destroy;
end;

function TMessagingSystem.RegisterMessageHandler(
    uMsg    : NativeInt;
    Handler : TMessageHandler): Boolean;
begin
    FCritSect.Enter;
    try
        FHandlers.AddOrSetValue(uMsg, Handler);
    finally
        FCritSect.Leave;
    end;
    Result := TRUE;
end;

function TMessagingSystem.PostMessage(
    uMsg   : NativeInt;
    WParam : WPARAM;
    LParam : LPARAM): Boolean;
begin
    Result := WinApi.Windows.PostMessage(FHWnd, uMsg, WParam, LParam);
end;

function GetCurrentThreadId: DWORD; stdcall;
begin
    Result := WinApi.Windows.GetCurrentThreadId;
end;

end.

All the code is shown above. If you are interested by the complete project as source code, just drop me a private email.


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 11, 2013

Update 2 for RAD Studio XE5, Delphi XE5 and C++Builder XE5

If you have not noticed yet, Embarcadero published the second update for their latest tools: Delphi XE5 as well as RAD Studio XE5 and C++ Builder XE5.

More than one hundred bugs have been fixed. Thanks for the users having taken the time to write a bug report to Quality Central, the web tool used to keep track of bugs and feature requests.

Most fixed bugs are related to the mobile platforms and FireMonkey component set. Obviously the Windows,  RTL and VCL parts of the product are very stable.

The update is available freely for registered users of the product.
Download from http://cc.embarcadero.com/item/29662

There is also an article with a partial list of fixes: http://dn.embarcadero.com/article/43522

Update 16/12/2013: Hot fixes are available: HF1 and HF2


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 9, 2013

Practical Parallel Computing

Parallel computing is a form of computation in which many calculations are carried out simultaneously. This is almost the only way of taking full power of today’s computers.

CPU have reached the point where it is not possible to increase the frequency of operation. The only way to increase computation power is to multiply the CPU. Either fully using multi-CPU computer, or partially using multi-core and hyper threading.

Most of today’s PC are equipped with 4 cores CPU. High end workstations have multiple CPU each with multiple cores and even hyper threading. For example, my own computer is a HP Z600 workstation having two Intel Xeon processors each one having 4 cores and hyper threading. Using Windows task manager, you actually see 16 processors.

Not an easy task


To benefit from multi cores computers, you must write programs so that the problem is divided into smaller ones which are then executed in parallel.

Not all problems can be divided in smaller ones for parallel execution. Numerical computation is one domain where parallelism is often possible. In this article, I will show the programming for a heavy computing problem: exploring the Mandelbrot set. Mandelbrot set is well known. It is mostly shown as a beautiful colorful picture which may take a very long time to compute. Using parallel computing will result in a high performance gain provided you run on a modern computer.

Parallel computer programs are more difficult to write than sequential ones, because concurrency introduces several new classes of potential software bugs.

The most common concurrency problem is when one task writes some shared data which is read by another task. If nothing special is done, then the reader may read partial data! The writer starts writing following data while the reader has not finished reading previous data. This kind of bug is difficult to find because it does not always occur. It depends on how one task runs compared to other task. This is referred as a race condition. Most of the time, it works because the reading and writing are occurring at different times. Sometimes it does not work because reading and writing overlaps.

Multithreading


Multithreading is the way you write parallel programs using traditional languages such as C/C++, Delphi or C# running on traditional (PC) computers. Other languages or other hardware architecture exists, but this is beyond of this article which is related to PC programming.

A thread is an execution path managed by an operating system. Multiple threads are executed simultaneously by a processor when the processor has multiple cores. With a single core processor, you still have multithreading, but they are executed one after the other. Execution switches quickly between each thread, making the human eye see all executing in parallel. They are really executed in parallel with multi core CPU or when multiple CPU are available.

It is the operating system and the language run time that take care of the required housekeeping. As seen from the programming language, a thread looks mostly like a simple function call (or an object’s method call when using an object oriented language). The difference with a normal function call is that when using multithreading, the function returns immediately while the function’s code is executed in parallel with the caller’s code.

Synchronization


To avoid messing everything, the work accomplished by different threads has to be synchronized.

For example, data cannot be read while it is being written by another thread. Several threads can simultaneously read the same data, but when one is writing data, it must be the only one accessing the data.

Another example is computing. Adding a value to a variable must be executed by a thread having exclusive access to that data. This is, at another level, the same issue as we find in multiple user database application. In such application, we use a “record lock” so that a record can be read, some computation done and then written back in an atomic operation. That is, no other user can have access to the data while the first reads-computes-writes.

The very same problem exists with multithreading. Simply the data is not in a database but in memory. Every single variable has to be protected from simultaneous reading and writing. And some operations, such as adding a variable to another one must be done atomically. And by the way, if multiple threads are doing database access, we have to use the exact same “record lock” as we do for a multiple user application.

A last example: frequently, when a problem is divided in multiple parts for parallel execution, something must be done at one point in time to be sure that all parts are done before continuing. Imagine you use parallel processing to create a dataset in memory. You must wait until all threads are done before writing the result to the file.

Operating system and language runtime provide many synchronization mechanisms. For example: critical section, semaphore, mutex, queues and others.

Introduction to Mandelbrot set


Benoît Mandelbrot, who died on 10/14/2010, was a famous mathematician. He studied fractal geometry. He found that chaos can emerge from a very simple equation:

This equation is iteration where c is a constant. We start from a first value X0 and compute X1 using the formula. Using X1 we then compute X2 and so on. This is no fun but easy. Where it becomes interesting is when we do this calculation using complex numbers and when we look what happens after a lot of iterations. Depending on the constant c, we can see that the Xn value increases quickly or not. Sometimes, the value change but never become very large. Even if computing a lot of iterations, we cannot decide whether the value will become very large.

Complex numbers can be represented by two numbers forming the coordinates X and Y on a plane. This is exactly what we use to draw a colorful representation of the Mandelbrot set. Each pixel on the image is one value of the constant c in the iteration above. The color of each pixel is given by the number of iterations computed so that the Xn value becomes large. The number of iterations is limited to a maximum. If the maximum is reached, the pixel is colored in a specific color, usually black or white. The result looks like this:



Using Delphi, the code to compute the value of a given point is as follow:

function ComputeMandelPoint(P, Q : Extended; MaxIter : Integer) : Integer;
var
    X0, Y0, X, Y : Extended;
    R, M         : Extended;
    K            : Integer;
begin
    M  := 100;
    K  := 0;
    X0 := 0;
    Y0 := 0;

    while TRUE do begin
        X  := X0 * X0 - Y0 * Y0 + p;
        Y  := 2.0 * X0 * Y0 + q;
        K  := K + 1;

        R := X * X + Y * Y;
        if R > M then
            break
        else if K >= MaxIter then begin
            K := MaxIter + 1;
            break;
        end;
        X0 := X;
        Y0 := Y;
    end;
    Result := K;
end;

In this code, P and Q are the coordinates of one point in the complex plane. The image is computed for P in the range [-2.4 … 1.0] and Q in the range [-1.3 … 1.3] and a maximum number of iterations of 1000. Coloring is done using a continuous palette composed of 48 shades of blue like this:


Since the value of each pixel is 0 to the maximum number of iteration, the palette is used cyclically. An offset of 25 is given so that the white limit is positioned nicely.

The code here above implements the formula we saw earlier. The code layout may looks strange. It has been done like that for speed optimization.

Building an image from the Mandelbrot set


The image showed above is quite easy to obtain. Here is the set of operations
1. Create a bitmap of a given resolution (1024 x 768 pixel for example)
2. Associate each bitmap pixel to the couple P, Q according to the area you want to display. In the image above, X on the bitmap (0..1023) is mapped to P = [-2.4 … 1.0] and Y on the bitmap (0..767) is mapped to Q = [-1.3 … 1.3].
3. Iterate all X, Y on the bitmap to compute the value of the pixel as a number of iterations.
4. Map the pixel value in iteration to the color value according to the color palette you like.

Mapping bitmap pixel to Mandelbrot set point


The process of step 2 above is named “mapping”. In math language, it is a simple coordinate transformation from one coordinate system (bitmap) to another one (Mandelbrot set).

Graphically, we can represent a graphic with one system on the abscissa (horizontal axis) and on the ordinate (vertical axis) we have the other system:



We can express the slope of the line as seen from X, X1, Y, Y1 and from X2, X1, Y2, Y1 which are obviously the same:

We can rewrite this equation to have the Y value based on all others:


Let’s say we want to transform the X coordinate on a bitmap to the equivalent Y coordinate in the Mandelbrot set. For example, on the bitmap shown above we go from pixel 0 to pixel 1023 and on the Mandelbrot set we go from -2.4 on the left to 1.0 on the right. This gives:

X1 = 0, X2 = 1023, Y1 = -2.4, Y2 = 1.0

Now applying our equation, given an arbitrary pixel coordinate X, we can compute the equivalent value in the Mandelbrot set:

Mandel := -2.4 + (Bitmap – 0) * (1.0 - -2.4) / (1023 – 0)

Note that (Y2 – Y1) / (X2 – X1) which is the slope of the line in the above diagram is frequently named zoom factor.

We can do similar computation for the pixel coordinate Y, using the same zoom factor so that the image is not distorted.

If we put all this in a simple Delphi program, we will get the following code:

procedure TMandelSimpleForm.FormShow(Sender: TObject);
var
    Bitmap : TBitmap;
    X, Y   : Integer;
    P, Q   : Extended;
    V      : Integer;
    Slope  : Extended;
    Colors : array of TColor;
    T0     : Cardinal;
const
    MandelLeft   = -2.4;
    MandelRight  = 1.0;
    MandelBottom = -1.3;
    MaxIter      = 1000;
begin
    SetLength(Colors, 11);
    Colors[ 0] := RGB( 47,  47, 255);
    Colors[ 1] := RGB( 21,  21, 255);
    Colors[ 2] := RGB(  0,   0, 255);
    Colors[ 3] := RGB(255, 255, 255);
    Colors[ 4] := RGB(229, 229, 255);
    Colors[ 5] := RGB(203, 203, 255);
    Colors[ 6] := RGB(177, 177, 255);
    Colors[ 7] := RGB(151, 151, 255);
    Colors[ 8] := RGB(125, 125, 255);
    Colors[ 9] := RGB( 99,  99, 255);
    Colors[10] := RGB( 73,  73, 255);

    Bitmap             := TBitmap.Create;
    Bitmap.PixelFormat := TPixelFormat.pf24bit;
    Bitmap.Width       := Image1.Width;
    Bitmap.Height      := Image1.Height;

    Slope := (MandelRight - MandelLeft) / (Bitmap.Width - 1);

    T0 := GetTickCount;
    for Y := 0 to Bitmap.Height - 1 do begin
        for X := 0 to Bitmap.Width - 1 do begin
            P := MandelLeft   + X * Slope;
            Q := MandelBottom + Y * Slope;
            V := ComputeMandelPoint(P, Q, MaxIter);
            Bitmap.Canvas.Pixels[X, Y] := Colors[V mod Length(Colors)];
        end;
    end;
    Image1.Picture.Bitmap := Bitmap;
    T0 := GetTickCount - T0;
    Bitmap.Free;
    ShowMessage('Computed in ' + IntToStr(T0) + ' mS');
end;

This code is really complete. We don’t need more to have a nice Mandelbrot set shown on screen. To create the application, use the following steps:
  • Create a new VCL forms application
  • Enlarge the form so that a 1024x768 bitmap will fit
  • Drop a TImage on the form, make it 1024x768
  • Add the code above to the form’s OnShow even.
  • Save, compile and run.
On my computer, it takes 4400 mS to build the bitmap. The parallel computing version will run in 690 mS for the same bitmap. This is a 6 fold improvement. It deserves the effort. Those numbers are for a Win32 application. When rebuilt as a Win64 application, times are divided by 2.

Parallel computing of a Mandelbrot set


Overview of required steps:
  • Create a large bitmap to hold the full final image of the Mandelbrot set
  • Divide the total area into a lot of smaller areas
  • Start a number threads giving each one area to compute
  • When a thread finishes his area, submit the area for “painting” and give a new area, if any remains
  • When there is no more area to give to a thread, terminate the thread
  • When all threads are terminated, all areas are computed.
  • Since painting is incredibly faster than computing, short after the last thread finishes, the last area is painted and the full bit map is ready

Dividing Mandelbrot set computation in smaller chunks is easy: divide the bitmap into areas and delegate the computing for those areas to a number of threads.

In my Mandelbrot Explorer, I divided the bitmap as 128x128 pixel areas. If the image size is not a multiple of 128x128, then some areas are smaller and may even be a single pixel.

The areas are described by a class instance with all required parameters. All areas are kept in an object list.

Then a number of threads are created, for example 10 threads, and kept in an object list. Each thread receives one area to process. When it has finished processing the assigned area, the next available area is given to the thread. If no more area is available, the thread is terminated, removed from the list and freed.

Each thread computes the number of iterations for each pixel. Coloring is done by the main thread after computation is done. This is interesting because you can change color mapping without computing again. Coloring is quite fast: on my machine, it is almost instantaneous.

Submitting an area for coloring is done using a custom message posted to the message queue. We need to use that technique because submission is executed in the thread’s context and we want the main thread to do the coloring. Using custom messages is an easy way of inter-thread communication.

The code


The class describing each area is as follow:

TMandelArea = class
public
    Rect     : TGPRect;           // Area rectangle in full bitmap
    PixValue : array of Integer;  // Computed pixel value
    Zoom     : Extended;          // To compute (P, Q) from bitmap coord
    XOffset  : Extended;          // To compute (P, Q) from bitmap coord
    YOffset  : Extended;          // To compute (P, Q) from bitmap coord
    Colors   : TList;     // The list of colors for coloring
    MaxIter  : Integer;           // Max iterations when computing
    Index    : Integer;           // Index in areas list
    procedure Compute;            // Compte all pixels in area
    // Colorize the pixel in the area and draw the area into full bitmap
    procedure Colorize(FullBitmap  : IGPBitmap;
                       ColorInside : TColor;
                       ColorDiv    : Integer;
                       ColorOff    : Integer);
end;


TMandelArea is a very simple class. It has all the fields with parameters we talked about and two methods. One for computing the pixel values and one for colorize the pixels and transfer the result in the full bitmap.

What I named “FullBitmap” is an interface coming from GDI+. Windows GDI+ is a class-based API that provides two-dimensional vector graphics, imaging, and typography. It is faster than Windows GDI API which is used by Delphi runtime.

TMandelArea.Compute is quite trivial. It looks much like the method we have seen above in the simple Mandelbrot application. Here it is:

procedure TMandelArea.Compute;
var
    X, Y       : Integer;
    P, Q       : Extended;
    Index      : Integer;
begin
    SetLength(PixValue, Rect.Width * Rect.Height);
    Index := 0;
    for Y := 0 to Rect.Height - 1 do begin
        Q := (Y + Rect.Y) / Zoom + YOffset;
        for X := 0 to Rect.Width - 1 do begin
            P := (X + Rect.X) / Zoom + XOffset;
            PixValue[Index] := ComputeMandelPoint(P, Q, MaxIter);
            Inc(Index);
        end;
    end;
end;

Basically, the method just iterates thru all pixels in the area, transforms the coordinates from bitmap system to Mandelbrot set system and calls the function we have seen above to compute the pixel value. That value is stored in an array of integer. PixValue array is dimensioned before computing.

Computation has nothing to do with creating the result bitmap and coloring its pixels. The resulting bitmap, which will be shown on screen and saved to disk as JPEG, is created before anything else. I use a GDI+ bitmap which is handled thru in interface.

The programmer can access the data of a GDI+ bitmap. For that, it has to call “LockBits” which returns a record having among other things the pointer to actual bitmap data. For better performance, the lines of a bitmap data is always aligned to memory locations. See how Row is calculated below to see how to handle that.

Bitmap data is organized as RGB value, that is 3 bytes for red, green and blue components of each pixel color. In Delphi, a TColor is simply an integer value with the 3 bytes. This is why you see code like: Row^ := Byte(ColorValue shr 16) to extract one of the bytes and copy it to the bitmap memory.

procedure TMandelArea.Colorize(
    FullBitmap  : IGPBitmap;
    ColorInside : TColor;
    ColorDiv    : Integer;
    ColorOff    : Integer);
var
    BitmapData : TGPBitmapData;
    X, Y       : Integer;
    Row        : PByte;
    ColorValue : TColor;
    PixIndex   : Integer;
begin
    BitmapData := FullBitmap.LockBits(Rect, [ImageLockModeWrite],
                                      FullBitmap.PixelFormat);
    PixIndex := 0;
    for Y := 0 to Rect.Height - 1 do begin
        Row := PByte(BitmapData.Scan0) + (Y * BitmapData.Stride);
        for X := 0 to Rect.Width - 1 do begin
            ColorValue := ColorMandelPoint(PixValue[PixIndex],
                                           MaxIter,
                                           ColorInside,
                                           FALSE, 0,
                                           TRUE,
                                           ColorDiv,
                                           ColorOff,
                                           Colors);
            Row^ := Byte(ColorValue shr 16);
            Inc(Row);
            Row^ := Byte(ColorValue shr 8);
            Inc(Row);
            Row^ := Byte(ColorValue);
            Inc(Row);
            Inc(PixIndex);
        end;
    end;
    FullBitmap.UnlockBits(BitmapData);
end;


The thread class:

TMandelThread = class(TThread)
strict private
    FArea       : TMandelArea;
    FNewArea    : TMandelArea;
    FOnComputed : TComputedEvent;
public
    procedure Execute; override;
    property Area       : TMandelArea    read  FArea
                                         write FArea;
    property OnComputed : TComputedEvent read  FOnComputed
                                         write FOnComputed;
end;

implementation

procedure TMandelThread.Execute;
begin
    while Assigned(FArea) do begin
        FArea.Compute;
        FNewArea := nil;
        if Assigned(FOnComputed) then
            FOnComputed(Self, FArea, FNewArea);
        FArea := FNewArea;
    end;
end;

This thread class is incredibly straightforward. It has a single property “Area” which is the area to be computed, a single event “OnComputed” to be called when computation is done and one method “Execute” which is the thread’s entry point.

Execute method is a loop calling Compute method on behalf of the area to be computed and then trigger the OnComputed event to request the next area to compute. The event passes the current computed area (which will be submitted for coloring) and receives, as a var parameter, the new area to compute. If no new area is available, it receives a nil value and the thread terminates the loop.

The main code


The main code must create all the areas, all the threads, allocate areas to threads, colorize areas when computed by each thread and render the result on screen optionally saving it to a jpeg file.

Of course the main code also handles the user interface. I built a nice user interface to help the user explore the Mandelbrot set. I will not describe it here because it is just simple and easy user interface programming.

I will just show and describe the user interface:



The tool bar has edit fields to select the bitmap size, the maximum number of iteration, the number of threads to use for computation and color divisor and offset for coloring.

The buttons are to start computing, navigate to previous position, navigate to the next position, show the full Mandelbrot set, change the color palette, save the bitmap to disk file, change the inside color, zoom in and zoom out.

Using the mouse, the user may draw a rectangular area which is then computed. A double click anywhere on the bitmap recomputes a new bitmap centered on the double click point.

On the status bar, you see the time required computing the displayed image, the zoom factor, the coordinates of the image center and the cursor position coordinates with number of iterations at that point.

When the window is resized, the bitmap is as well. The screen always shows the complete bitmap, even if you select a very large bitmap.

If you load a new color palette, the bitmap is colored without computing again. This is almost instantaneous, even on large bitmaps.

The most interesting parts of the code are the method “MandelCreateImage” and the two event handlers for the thread.

procedure TMandelbrotExplorerForm.MandelCreateImage;
var
    Graphics  : IGPGraphics;
    Area      : TMandelArea;
    X, Y      : Cardinal;
    I         : Integer;
    NewThread : TMandelThread;
begin
    // Just do nothing if already terminating
    if FTerminate then
        Exit;

    // We cannot create a new image if threads are still computing
    // We set a flag so that no new area are given to thread and
    // defer processing until all threads are done
    if FThread.Count > 0 then begin
        FTerminate := TRUE;
        PostMessage(Handle, WM_TERMINATING, 0, 0);
        Exit;
    end;

    // Get parameters from user interface, use existing if invalid values
    FMaxIter         := StrToIntDef(MaxIterEdit.Text,  FMaxIter);
    FMandelWidth     := StrToIntDef(WidthEdit.Text,    FMandelWidth);
    FMandelHeight    := StrToIntDef(HeightEdit.Text,   FMandelHeight);
    FThreadCount     := StrToIntDef(ThreadsEdit.Text,  FThreadCount);

    // Validate all parameters
    if FMaxIter < 1 then
        FMaxIter := 1;
    if FMandelWidth < 1 then
        FMandelWidth := 1;
    if FMandelHeight < 1 then
        FMandelHeight := 1;
    if FThreadCount < 1 then
        FThreadCount := 1;

    // Send values back to user interface to replace invalid values
    MaxIterEdit.Text  := IntToStr(FMaxIter);
    WidthEdit.Text    := IntToStr(FMandelWidth);
    HeightEdit.Text   := IntToStr(FMandelHeight);
    ThreadsEdit.Text  := IntToStr(FThreadCount);

    FXOffset          := FCenter.X - (FMandelWidth div 2) / FZoom;
    FYOffset          := FCenter.Y - (FMandelHeight div 2) / FZoom;

    FFullBitmap       := TGPBitmap.Create(FMandelWidth, FMandelHeight,
                                          PixelFormat24bppRGB);
    FBackBrush        := TGPSolidBrush.Create(
                                 TGPColor.Create(GetRValue(clSilver),
                                                 GetGValue(clSilver),
                                                 GetBValue(clSilver)));
    Graphics          := TGPGraphics.FromImage(FFullBitmap);
    Graphics.FillRectangle(FBackBrush,
                           0, 0,
                           FFullBitmap.Width, FFullBitmap.Height);
    // Divide the image into AreaSize x AreaSize pixels
    FAreas.Clear;
    Y := 0;
    while Y < FFullBitmap.Height do begin
        X := 0;
        while X < FFullBitmap.Width do begin
            Area             := TMandelArea.Create;
            Area.Index       := FAreas.Add(Area);
            Area.Colors      := FColors;
            Area.MaxIter     := FMaxIter;
            Area.Zoom        := FZoom;
            Area.XOffset     := FXOffset;
            Area.YOffset     := FYOffset;
            Area.Rect.X      := X;
            Area.Rect.Y      := Y;
            Area.Rect.Width  := Min(AREA_SIZE, FFullBitmap.Width - X);
            Area.Rect.Height := Min(AREA_SIZE, FFullBitmap.Height - Y);
            // Make last rectangle on row or column has correct size
            if (Area.Rect.X + Area.Rect.Width) >= Integer(FFullBitmap.Width) then
                 Area.Rect.Width := Integer(FFullBitmap.Width) - Area.Rect.X;
            if (Area.Rect.Y + Area.Rect.Height) >= Integer(FFullBitmap.Height) then
                 Area.Rect.Height := Integer(FFullBitmap.Height) - Area.Rect.Y;
            Inc(X, AREA_SIZE);
        end;
        Inc(Y, AREA_SIZE);
    end;

    // Create all threads, but don't start them
    // Thread cannot be started because it would interfere with area allocation
    FAreaIndex := 0;
    for I := 1 to FThreadCount do begin
        NewThread                 := TMandelThread.Create(TRUE);
        NewThread.Area            := FAreas[FAreaIndex];
        NewThread.OnComputed      := MandelThreadComputed;
        NewThread.OnTerminate     := MandelThreadTerminate;
        NewThread.FreeOnTerminate := TRUE;
        FThread.Add(NewThread);
        Inc(FAreaIndex);
        if FAreaIndex >= FAreas.Count then
            break;
    end;

    // Set the counter pf to be colorized areas
    FColorizeCount := FAreas.Count;

    // Now start all thread, in reverse order. Reverse order is required
    // because threads may terminate and be removed from list, messing the
    // indexing.
    FStartTime := GetTickCount;
    for I := FThread.Count - 1 downto 0 do
        FThread[I].Start;
end;


// Warning: this is called in the context of the threads
procedure TMandelbrotExplorerForm.MandelThreadComputed(
    Sender        : TObject;
    const CurArea : TMandelArea;
    var   NewArea : TMandelArea);
begin
    if FTerminate then
        Exit;
    // We must pass the index instead of the pointer because the message
    // handling can be done much later when the area has been reallocated
    PostMessage(Handle, WM_COLORIZE, 0, CurArea.Index);
    FCritSect.Enter;
    if FAreaIndex < FAreas.Count then begin
        NewArea := FAreas[FAreaIndex];
        Inc(FAreaIndex);
    end;
    FCritSect.Leave;
end;


procedure TMandelbrotExplorerForm.MandelThreadTerminate(Sender : TObject);
begin
    // Simply remove the terminated thread from the thread list.
    FThread.Remove(Sender as TMandelThread);
end;

The FCritSect variable is very important. This is one of the synchronization mechanisms I talked above. A critical section works like this: only one thread at a time can execute the “Enter” method. Once the “Enter” method has been executed by a thread, any other thread calling “Enter” is made sleeping. When the “Leave” method is called, one and only one of the sleeping threads is awaken and continue execution. When this thread executes the “Leave” method, another thread is awaken and so forth.

A critical section MUST be used here because we increment FAreaIndex and use it to get the next area for the thread calling MandeThreadComputed.

Another part of the code requires some explanation. A flag “FTerminate” is used to signal the user wants to terminate the current computing. It is set my MandelCreateImage itself when called before all threads are done.

Since we cannot stop the thread immediately (It takes some time to stop a thread), we use another custom message WM_TERMINATING. The message handler will check if all thread are done. If not, it sleeps 0.25 sec and then post the message again. If all threads are done, it calls MandelCreateImage which will start a new image creation.

Here the message handler code:

procedure TMandelbrotExplorerForm.WMTerminating(var Msg: TMessage);
var
    TmpMsg : TMsg;
begin
    if FThread.Count <= 0 then begin
        // No more threads are computing.
        // Remove any WM_COLORIZE from message queue
        while PeekMessage(TmpMsg, Handle, WM_COLORIZE, WM_COLORIZE, 1) do;
        FTerminate := FALSE;
        MandelCreateImage;
    end
    else begin
        Sleep(250);
        PostMessage(Handle, WM_TERMINATING, 0, 0);
    end;
end;

The final word

I hope you enjoyed reading this article. All in all, the code is fairly simple. Actually the code for the user interface is longer!

If you are interested by the source code (Delphi XE5), just drop me a private email.

References:

Parallel computing: http://en.wikipedia.org/wiki/Parallel_computing
Multithreading: http://en.wikipedia.org/wiki/Multithreading_(software)#Multithreading
Synchronization objects: http://msdn.microsoft.com/en-us/library/windows/desktop/ms686364(v=vs.85).aspx
Mandelbrot set: http://en.wikipedia.org/wiki/Mandelbrot_set
GDI+: http://msdn.microsoft.com/en-us/library/windows/desktop/ms533798(v=vs.85).aspx
GDI+ for Delphi : http://www.bilsen.com/gdiplus


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 4, 2013

Upgrade your old Delphi version to XE5, any version

Sometimes Embarcadero is promoting his latest Delphi and C++ builder by allowing customer to upgrade from ANY older version. This is just happening now!

I'm really convinced that any Delphi user MUST update to the latest XE5, specially those still using the old D7 and even D5. Migration is not that difficult. Most of the time, well written code just requires a recompile.

Have a look at http://www.embarcadero.com/radoffer 

  1. Upgrade to XE5 from any earlier version (Upgrade price from any previous version)
  2. Bonus pack with up to $1,150 in free software
  3. Step Up to Ultimate Edition Free from Enterprise
  4. Get a free C++Builder iOS upgrade when you buy qualifying C++Builder XE5 or RAD Studio XE5 tools



Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

December 2, 2013

Mandelbrot Fractal Explorer

I wrote a nice Mandelbrot fractal explorer application. It is written using Delphi XE5 and use extended precision floating point calculation to be able to produce ultra high zoom images.

You can create bitmaps much larger than the screen size. This way you can have a high resolution printout.

The program use multithreading to take advantage for multicore CPU. Computation is quite fast.


You can download the utility from http://www.overbyte.be/arch/dump/MandelbrotExplorer.zip The zip file contains the executable as well as a number of color palette files (You can edit it with notepad if you like).

If there is some interest, I could write another blog article about the design of the application. Just add a comment to this blog post if you are interested.

The program make use of multithreading to compute several parts of the image simultaneously. This makes huge speed improvement for multicore CPU.

Also, GDI plus is used for graphic rendering which make the display very fast, even for ultra high bitmap resolution.

Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

November 28, 2013

What is ProcessMessages?

Sometimes, when you browse some application source code, you find loops calling Processmessages. What’s this? Well, it is a “Message pump” which retrieves and process messages from Windows message queues. More exactly from the current thread message queues.

Quick note: At the end of this article, you'll find why you should NOT use ProcessMessages. Before that, let's understand the concept.

The code you see is frequently like this:

FSomeFlag := TRUE;
while FSomeFlag do begin
    Application.ProcessMessages;
    Sleep(100);
end;

When looking at that code, you may think it will loop forever. Actually it will not! The loop variable “FSomeFlag” is for sure set to false in an event handler.

Application.ProcessMessages is implemented in Delphi runtime (Forms unit) as a loop checking Windows message queue for the current thread. The loop removes any message from the queue and dispatch processing, until the queue is emptied.

In the message queue, you have messages from the user interface (for example mouse move, button click, menu actions and much more) and from many activities occurring in the system such as inserting a removable disk, user wanting shutting down the computer, data received from a serial port, data packet received on the network, battery running out of charge and much more.

ProcessMessages will fetch all those events and process them. Translated to more common Delphi speaking, your event handlers will be called from ProcessMessages.

Try this:
Create a new VCL forms application, drop two buttons on the form and a memo. Declare a private Boolean member variable named “FSomeFlag”. Then write and run this code:

procedure TForm1.Button1Click(Sender: TObject);
begin
    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    while FSomeFlag do begin
        Application.ProcessMessages;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

procedure TForm1.Button2Click(Sender: TObject);
begin
    Memo1.Lines.Add('You clicked on Button2');
    FSomeFlag := FALSE;
end;

When running, click on Button1. This will execute Button1Click event handler which contains the loop. It will display the first message and then loop until FSomeFlag becomes FALSE.
After clicking on button1, then click on button2. This will execute Button2Click event handler which sets FSomeFlag to FALSE which in turn will break the loop still running in Button1Click event handler. You’ll see the message display by Button2Click event handler, followed by the second message in Button1Click event handler and you know the loop has ended.

This code has some surprising results. For example, click twice on Button1 before clicking on Button2. You will see the first message shown for each click on Button1. Then the message from Button2 when you click on it. And finally twice the second message from Button1Click event handler. What you see is the Button1Click event handler reentered. This is a king of recursive call. ProcessMessages will trigger all events, including new event of the same kind already executing.

Having you event handler reentered may have adverse effect on your code. You should probably avoid it by rewriting Button1Click like this:

procedure TForm1.Button1Click(Sender: TObject);
begin
    if FSomeFlag then begin
        Memo1.Lines.Add('You already clicked Button1');
        Exit;
    end;

    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    while FSomeFlag do begin
        Application.ProcessMessages;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

Checking FSomeFlag before the loop let us know that the loop is already looping and just exit instead of beginning a new loop.

There are more surprises: once you clicked on Button1, before clicking Button2, try to stop your application by clicking the close window button. It won’t stop!

The application doesn’t stop because VCL is written in such a way that an application is terminated when the main window closes. And that window cannot close while an event handler is still executing. Which event handler is execution? It is Button1Click which is still looping waiting for FSomeFlag to become FALSE. Click on Button2 and the loop will break, letting the window close and the application terminate.

It is likely that this behavior is unwanted. Easy to change. Rewrite Button1Click event handler like this:

procedure TForm1.Button1Click(Sender: TObject);
begin
    if FSomeFlag then begin
        Memo1.Lines.Add('You already clicked Button1');
        Exit;
    end;

    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    while FSomeFlag do begin
        Application.ProcessMessages;
        if Application.Terminated then
            break;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

The check for Application.Terminated does exactly what it name implies. Actually “Terminated” is a flag which is set when you click on the form’s close button. This is exactly the same mechanism as the one we implemented with FSomeFlag.

Finally, there is one more glitch: what happens if the user never clicks on Button2? Answer: the application loops forever. This is probably unwanted behavior (Remember that is a real application the flag FSomeFlag is set to FALSE no when a user clicks on a button but when some condition is met, for example when a removable media is inserted).

To fix this behavior, we may add a timeout. There are several possibilities to implement it. You may get the time within the loop and break if too much time elapsed. Or you may use a TTimer for that purpose. The code looks simpler with a timer but is not really visible so I prefer to get the time within the loop like this:

procedure TForm1.Button1Click(Sender: TObject);
var
    Timeout : TDateTime;
begin
    if FSomeFlag then begin
        Memo1.Lines.Add('You already clicked Button1');
        Exit;
    end;

    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    Timeout := Now + OneSecond * 10;
    while FSomeFlag do begin
        Application.ProcessMessages;
        if Application.Terminated then
            break;
        if Now > Timeout then begin
            Memo1.Lines.Add('Timeout');
            break;
        end;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

I have not explained yet why there is a Sleep(100) within the loop. If you remove it, everything seems to work the same way. Really? Not! use the task manager to see how much CPU is used by your application. Since it does almost nothing, task manager shows 0% of CPU used. If you click on Button1, you still see 0%.

Now remove Sleep(100) and test again. Task manager will show your application consuming a lot of CPU just for doing nothing sensible. Why is this?

The loop in Button1Click event will run at the maximum speed when there is no Sleep call. Depending on your CPU (Number of cores), this means the loop will consume almost all CPU time available using a single core. The loop will loop thousand or million times per second depending on your hardware.

When there is a call to Sleep, it will instruct the operating system to make the thread sleep for the remaining time slice and for approximately the time in millisecond. Using Sleep(100) will make your loop not run more than 10 times per second which will no use to much CPU.


Better solution?

I explained how ProcessMessages works and how to use it. Now it’s time to ask the good question: Should I really use ProcessMessages?

Most of ProcessMessages calls should and can be avoided. They should be avoided because there is a waste of CPU which slows down your application and the computer in general. They can be avoided by redesigning your application using event driven asynchronous pattern.

Most use of ProcessMessages are used when the developer has not correctly understood or is not even aware of the event driven asynchronous pattern. Such developer has a problem with asynchronous operation where your calls merely start an operation and give control back immediately while the operation itself is executed in the background and triggers an event when done.

In the simple case I used above, it is very easy to change the code so that no wait loop calling ProcessMessages is required. Let’s assume that some processing has to be done when the user has clicked Button3 and Button4 in any order.

The idea is to use a flag per event (Button click here) that must have occurred before the processing can take place. For each event, check all flag to see if processing can take place.

Here is the code:

procedure TForm1.Button3Click(Sender: TObject);
begin
   if FFlag3 then
       Memo1.Lines.Add('You already clicked Button3')
   else begin
       Memo1.Lines.Add('You clicked Button3');
       FFlag3 := TRUE;
   end;
   TryToExecute;
end;

procedure TForm1.Button4Click(Sender: TObject);
begin
   if FFlag4 then
       Memo1.Lines.Add('You already clicked Button4')
   else begin
       Memo1.Lines.Add('You clicked Button4');
       FFlag4 := TRUE;
   end;
   TryToExecute;
end;

procedure TForm1.TryToExecute;
begin
    // Check if both buttons have been clicked
    if (not FFlag3) or (not FFlag4) then
        Exit;
    // Both have been clicked
    Memo1.Lines.Add('Processing done');
    // Reset the flags so that the user can start again
    FFlag3 := FALSE;
    FFlag4 := FALSE;
end;

There can be cases where order of event is important or there are a large number of conditions. In those cases, the whole application should make use of a finite state machine. This will probably be the subject of a future article. For now, have a look at what Wikipedia says about it: http://en.wikipedia.org/wiki/Finite-state_machine

Update 16/12/2013: I wrote another article about how to make PostMessage multi-platform for Android and Windows: http://francois-piette.blogspot.be/2013/12/firemonkey-android-windows-and.html


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

November 24, 2013

OOP design and event driven programming

In a previous article, one of the 3 main design advices was “Use events to free your classes from things that can change”. In this article, I would like to talk a little bit more about it.

Following good OOP practice, you develop by encapsulating almost everything into specialized objects. This makes all the code and all data related to a given subject grouped into a single entity named object.

This encapsulation is quite easy. You first this about what the object is or made of, and what operation can be done on the object or with the object. What the object is or is made of becomes properties. Operations become methods.

Let’s see this in action with a very simple and basic example derived from one of my real world applications. You can skip the following 3 paragraphs if you are not interested by the actual application and only want to read software design part of this article.

First, a little background: As you may know I’m working for a company building – among other things – automated digital radiography (DR) system. A DR system is made of an X ray source, an X ray detector, a diaphragm, a manipulator, a control computer, an image processing system and a database system. It is automated because it does X ray inspection without human interaction. It takes hundreds of radiography fully unattended.

You can see a picture here. OK, when you don’t know what it is, it could be challenging to understand what you see. Actually in the background in orange you see a robotic arm. It holds a fork which support the X ray detector (White rectangular box on the left), the diaphragm (Yellow on the right) and the X ray source (just behind the diaphragm). On the foreground, you see the part being inspected (Component of the low pressure compressor of an aircraft engine). This component is secured on a rotating table you don’t see on the picture.

What my Delphi software does is drive the robot, the rotating table, the X ray source, the diaphragm, and the detector in coordinated movement to take X ray picture of all welds. Pictures are sent to an image processing system for examination and then stored in a database for later search and retrieve.

For such a complex system, it is very important to have a good software design. If you don’t, the application will be horrible to maintain and would probably quickly become unreliable.

Objects are defined at all levels in the application. High level objects making use of low level objects. Each object is specialized for his own purpose. If you don’t carefully think about the purpose, you’ll end up with a single huge object doing everything; or you get a myriad of object doing almost nothing each one. There is no rule to fix the boundaries. Your only guide I can give you is to always think about what the main purpose of the object is and concentrate on it. Everything that doesn’t fit the real purpose must be moved to another object with his own purpose.

Back to our real world example: Both the robot and the rotating table make use of serial communication (RS232 or RS485) between the computer and the electronic and embedded controller driving the motors. They use the same physical communication layer but different protocols. We can immediately see the object candidates, from low level to high level: Basic serial communication, robot communication protocol, rotating table communication protocol, robot itself and rotating table itself.

The basic serial communication will simply drive the computer serial port to send and receive characters. It has no knowledge of what represent the data sent or receive. It only knows how to send data and receive data.

Communication protocol object handle messages required to instruct a robot controller or a rotating table motor controller to do what it needs to. This object doesn’t know how to send characters to a serial port. It even doesn’t know it is making use of a serial port. But it knows how to format a message instructing the robot or the table to reach a given position. The communication protocol object has no idea what is the movement purpose, but it know how to request such movement.

Robot or rotating table objects are high level objects. They know what a robot or table is able to do, they know which sequences of instructions are required for everything a robot or table can do, at least from a low level point of view. The error to be avoided here is to put in that object something related to yet a higher level. That higher level is related to the coordination between the robot and the table, or related to the whole systems.

I have just scratched the surface of the OOP design for such an application. I don’t want to teach you how to build software for automated digital radiography system. I want to teach you how to write good Delphi code (Well, this applies to almost all object oriented programming language).

An object in the middle of a hierarchy receive order from the higher level and has to delegate work to the lower level objects. Those are simple method calls. For example, the robot having to move the arm to a given X, Y, Z position is space (You also need the 3 angles to fully define a position in space) will call a bunch of methods of the communication protocols, probably a message to send each parameter (coordinates and angles) to the robot controller. The communication protocol object will build messages, adding addresses, message number, checksum or CRC and similar items required to make a valid message. It will delegate the sending of the message to the lowest level object handling serial port communication. This object doesn’t know what the messages are but knows how to send it one byte or character at a time, with proper baud rate, parity, start and stop bit, and how to handle handshaking.

So far so good: we only used simple method calls up to now. We are now at the point where we need even driven programming!

The software is driving hardware. Moving a robot arm or rotating a table takes a huge time compared to the computer processing speed. What happens when the requested movement is done or when something went wrong? There is a data flow in the reverse direction! The hardware (Motor controller) sends a message thru the serial port to say – for example – the position has been reached.

The lowest level object receives bytes from the motor controller. After checking errors such as parity, it transmits those bytes to the communication protocol object which assembles complete messages. Messages are checked for validity (format, length, CRC, and so on). Complete messages are then used to notify the robot object or table object that the requested movement is done. It is likely that a single movement is part of a group of coordinated movement. The robot object knows about this coordination and collects all messages until the group is finished and only then forwards the information to the upper layer.

In an event driven software, the backward information flow is handled by events. This means that the software send a request for something and don’t care waiting until the requested operation is done. Rather, it handles notification when something happens, for example the end of requested operation or an error message. This event driven operation is frequently called asynchronous operation because requesting something is decoupled from waiting for it to be done.

Traditional programming, also known as synchronous programming or blocking programming, works by sending a request and waiting for the answer. This is simple and easy. Well easy until you have to do several things simultaneously. With synchronous programming you must then use multithreading to do several thing simultaneously. This works well but it is difficult to develop, debug and maintain. There are a lot of issues arising from thread synchronization. This is a very difficult matter.

Asynchronous programming or event driven programming solves those issues easily. There is no problem at all doing several things simultaneously since “things” are merely requests to do something. The request is almost instantaneous. There is no wait, no blocking. The program never wait that something is done. It just does the processing when it is done.

Think about Windows user interface. Your code never wait that the user clicks on a button. You just assign code to the event which is triggered when the user clicks on the button. The code is executed when the user clicks and you have nothing to do for that to happen.

This event driven behavior can be built into your own objects very easily. It fits very well along to the code you write for the user interface.

Here are the required steps:
  1. Think about which data you need to pass when the event is triggered
  2. Create a data type corresponding to the data found in step 1. Add a “Sender” argument. This data type is a pointer to a procedure of object.
  3. Create a protected member variable in your class to hold the data type from step 2. Usually this member variable name begins with “FOn”.
  4. Create a published property corresponding to the member variable.
  5. Create a protected “Trigger” virtual procedure.
  6.  Implement the trigger procedure
You want an example? The communication protocol object which receive data from the serial communication object will trigger an event when it has assembled a full message and this message in a message stating the position of the movement (Usually motor controller periodically send such message while moving). It is likely that the data is the actual position. This position is usually expressed as an integer count of a position encoder tick.

Step 1:
We need an integer.

Step 2:
type
  TPositionEvent = procedure (Sender : TObject; Position : Integer) of object;

Step3:
protected
  FOnXPosition : TPositionEvent;
  

Step 4:
  published
    property OnXPosition : TPositionEvent read  FOnXPosition
                                          write FOnXPosition;

Step 5:
protected
    procedure TriggerXPosition(Position : Integer); virtual;

Step 6:
procedure TMyObject.TriggerXPosition(Position: Integer);
begin
    if Assigned(FOnXPosition) then
        FOnXPosition(Self, Position);
end;

Usually I use a single source file for each individual object.
The complete code should looks like this:

unit RobotCommProtocol;

interface

uses
  Classes;

type
  TPositionEvent = procedure (Sender : TObject; Position : Integer) of object;

  TRobotProtocol = class(TComponent)
  protected
    FOnXPosition : TPositionEvent;
    procedure TriggerXPosition(Position : Integer); virtual;
  published
    property OnXPosition : TPositionEvent read  FOnXPosition
                                          write FOnXPosition;
  end;


implementation

procedure TRobotProtocol.TriggerXPosition(Position: Integer);
begin
    if Assigned(FOnXPosition) then
        FOnXPosition(Self, Position);
end;

end.

Carefully study how I named the various parts. Naming convention is very important to have readable and maintainable code. Keep naming same thing with same name, using prefixes or suffixes to make a distinction where required.

My event is supposed to return a position. Assuming we have several possible positions, I named everything related to the event “XPosition”. The data type is named “TPositionEvent” because the same event will apply to X, Y, Z and all others so the “X” has been dropped. It is to be used for an event so the suffix is “Event”. And it begins with letter “T” because it is a data type.

The property itself is named “OnXPosition” for obvious reasons. Think about the “OnClick” event of a TButton. This is similar.

The member variable has the same name as the property with an “F” prefix. This convention is almost always used.

The trigger procedure begins with prefix “Trigger” and become TriggerXPosition. When the object needs to trigger the OnXPosition event, it will call TriggerXPosition, passing the new position. The procedure is made virtual so that derived classes have a chance to override his behavior. For example, the derived class could enhance it be triggering another event when the value exceed some limit.

The unit containing the code has been named “RobotCommProtocol” because our object handles a communication protocol for a given robot. The object itself is named “TRobotProtocol” for obvious reasons. It derives from TComponent which makes possible to install the object as a component available in Delphi IDE component palette. There are other requirements which are out of this article scope.

There is much more to say about the topic. Please post a comment to the article to ask for the topics I should develop in the next article.



Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

November 17, 2013

Upgrade option for any older version of Delphi

Currently Embarcadero offers an exceptional upgrade option for any older version of Delphi, C++Builder and RAD Studio. The discount is up to 45%, valid up to the end of year.

This is a real opportunity if you still use an old Delphi. You will really benefit upgrading your old Delphi 5, 7 or whatever to XE5. I know, I've done it. Almost all my application have been ported to the latest Delphi version.

There are a lot of articles about porting old Delphi code to recent Delphi (Mostly pre-Unicode Delphi to post Unicode Delphi). And if you think you can't do this port yourself, I'm sure you'll find a Delphi expert who will help you. I suggest you look at Embarcadero MVP directory.


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

November 10, 2013

Delphi uses Excel to create a chart in a PDF document

Microsoft Excel exposes all his features thru a COM interface which can easily be used from a Delphi application. In this article, I use that feature to create a 3D pie chart from data available within a Delphi program and produce a PDF document.

I already talked about using Microsoft Office applications in this article. I gave examples using Word. In this article I use Excel for what it does very well: take an array of data and produce a nice chart.

Excel exposes a number of objects and this makes programming it a little bit confusing at first. The three most important objects are:

  • ExcelApplication: this is the whole Excel application.
  • WorkBook: This is a spreadsheet file
  • WorkSheet: This is a page within a workbook.

There are a lot of other objects or object collections. In this article we will use “Cells” and “Charts”. They are exactly what their names imply.

Each object or collection has a lot of properties and methods. This is where it becomes quite complex. Although most names are explicit, their use isn’t. Microsoft publishes a lot of documentation (http://msdn.microsoft.com/en-us/library/office/bb726434(v=office.12).aspx). Of course none of this documentation is written using Delphi syntax. Nevertheless it is of great help even if most samples are VBA or C#.

There are a large number of Office versions. The programming interface change slightly between each version but all in all, upward compatibility is excellent. The gold rule is to always use the oldest API version suitable for what you need to do. Because of upward compatibility, your application will generally work for the version you selected and all more recent versions.

For my sample application, I used Excel 2010. Microsoft reference is here.

In Delphi, you must use the correct components. See discussion in this article. What I said then for XE4 is valid for XE5 as well as previous versions.

My demo application is simple: A single VCL form with a single button. The button’s OnClick handler connect to excel, create a workbook having a worksheet, fill cells with simple data, create a new chart with the data, export the chart as a PDF file, close the workbook and Excel.

I hardcoded the data to keep the code simple. It is quite trivial to fetch data from anywhere, including some database. How the data is fetched is not today’s article object.

There are a number of traps when writing this kind of application. Most Office API functions have a lot of arguments. Most of them can be left empty. When you specify some argument the code may triggers an access violation or an OLE error. For example, when adding a chart, on argument specifies the chart type. I’ve found that using it will trigger an OLE error. I had to left it empty and then change the property ChartType to actually change the type of chart. This is really annoying because error messages are not explicit at all! It is a try and error play. It is time consuming.

The resulting code is very short and simple indeed:

procedure TForm1.Button1Click(Sender: TObject);
var
    WBook  : ExcelWorkbook;
    WSheet : ExcelWorksheet;
    Row    : Integer;
    WChart : ExcelChart;
    LCID   : Integer;
begin
    // Get the locale identifier for the user default locale
    LCID := GetUserDefaultLCID;
    //Connect to Excel application, this will launch excel
    ExcelApplication1.Connect;
    // Make excel visible (This is not required)
    ExcelApplication1.Visible[LCID] := TRUE;
    // Create a new workbook with a new sheet
    WBook  := ExcelApplication1.Workbooks.Add(xlWBATWorksheet, LCID);
    WSheet := WBook.ActiveSheet as ExcelWorksheet;
    // Add some data to the sheet
    WSheet.Cells.Item[1, 1] := 'Item';
    WSheet.Cells.Item[1, 2] := 'Quantity';
    for Row := 0 to High(Data) do begin
        WSheet.Cells.Item[2 + Row, 1] := Data[Row].Item;
        WSheet.Cells.Item[2 + Row, 2] := Data[Row].Quantity;
    end;
    // Create a new chart
    WChart := WBook.Charts.Add(EmptyParam, EmptyParam,
                               EmptyParam, EmptyParam, LCID) as ExcelChart;
    // Set the chart type
    WChart.ChartType := xl3DPie;
    // Set the tab name
    WChart.Location(xlLocationAsNewSheet, 'MyChart');
    // Export the chart as a PDF file
    WChart.ExportAsFixedFormat(xlTypePDF, 'MyChart.pdf', xlQualityStandard,
                               TRUE, FALSE, EmptyParam, EmptyParam,
                               TRUE,         // Open after published
                               EmptyParam);
    // Close the workbook, quit excel and disconnect
    WBook.Close(FALSE, EmptyParam, EmptyParam, LCID);
    ExcelApplication1.Quit;
    ExcelApplication1.Disconnect;
end;
--
Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be