November 28, 2013

What is ProcessMessages?

Sometimes, when you browse some application source code, you find loops calling Processmessages. What’s this? Well, it is a “Message pump” which retrieves and process messages from Windows message queues. More exactly from the current thread message queues.

Quick note: At the end of this article, you'll find why you should NOT use ProcessMessages. Before that, let's understand the concept.

The code you see is frequently like this:

FSomeFlag := TRUE;
while FSomeFlag do begin
    Application.ProcessMessages;
    Sleep(100);
end;

When looking at that code, you may think it will loop forever. Actually it will not! The loop variable “FSomeFlag” is for sure set to false in an event handler.

Application.ProcessMessages is implemented in Delphi runtime (Forms unit) as a loop checking Windows message queue for the current thread. The loop removes any message from the queue and dispatch processing, until the queue is emptied.

In the message queue, you have messages from the user interface (for example mouse move, button click, menu actions and much more) and from many activities occurring in the system such as inserting a removable disk, user wanting shutting down the computer, data received from a serial port, data packet received on the network, battery running out of charge and much more.

ProcessMessages will fetch all those events and process them. Translated to more common Delphi speaking, your event handlers will be called from ProcessMessages.

Try this:
Create a new VCL forms application, drop two buttons on the form and a memo. Declare a private Boolean member variable named “FSomeFlag”. Then write and run this code:

procedure TForm1.Button1Click(Sender: TObject);
begin
    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    while FSomeFlag do begin
        Application.ProcessMessages;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

procedure TForm1.Button2Click(Sender: TObject);
begin
    Memo1.Lines.Add('You clicked on Button2');
    FSomeFlag := FALSE;
end;

When running, click on Button1. This will execute Button1Click event handler which contains the loop. It will display the first message and then loop until FSomeFlag becomes FALSE.
After clicking on button1, then click on button2. This will execute Button2Click event handler which sets FSomeFlag to FALSE which in turn will break the loop still running in Button1Click event handler. You’ll see the message display by Button2Click event handler, followed by the second message in Button1Click event handler and you know the loop has ended.

This code has some surprising results. For example, click twice on Button1 before clicking on Button2. You will see the first message shown for each click on Button1. Then the message from Button2 when you click on it. And finally twice the second message from Button1Click event handler. What you see is the Button1Click event handler reentered. This is a king of recursive call. ProcessMessages will trigger all events, including new event of the same kind already executing.

Having you event handler reentered may have adverse effect on your code. You should probably avoid it by rewriting Button1Click like this:

procedure TForm1.Button1Click(Sender: TObject);
begin
    if FSomeFlag then begin
        Memo1.Lines.Add('You already clicked Button1');
        Exit;
    end;

    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    while FSomeFlag do begin
        Application.ProcessMessages;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

Checking FSomeFlag before the loop let us know that the loop is already looping and just exit instead of beginning a new loop.

There are more surprises: once you clicked on Button1, before clicking Button2, try to stop your application by clicking the close window button. It won’t stop!

The application doesn’t stop because VCL is written in such a way that an application is terminated when the main window closes. And that window cannot close while an event handler is still executing. Which event handler is execution? It is Button1Click which is still looping waiting for FSomeFlag to become FALSE. Click on Button2 and the loop will break, letting the window close and the application terminate.

It is likely that this behavior is unwanted. Easy to change. Rewrite Button1Click event handler like this:

procedure TForm1.Button1Click(Sender: TObject);
begin
    if FSomeFlag then begin
        Memo1.Lines.Add('You already clicked Button1');
        Exit;
    end;

    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    while FSomeFlag do begin
        Application.ProcessMessages;
        if Application.Terminated then
            break;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

The check for Application.Terminated does exactly what it name implies. Actually “Terminated” is a flag which is set when you click on the form’s close button. This is exactly the same mechanism as the one we implemented with FSomeFlag.

Finally, there is one more glitch: what happens if the user never clicks on Button2? Answer: the application loops forever. This is probably unwanted behavior (Remember that is a real application the flag FSomeFlag is set to FALSE no when a user clicks on a button but when some condition is met, for example when a removable media is inserted).

To fix this behavior, we may add a timeout. There are several possibilities to implement it. You may get the time within the loop and break if too much time elapsed. Or you may use a TTimer for that purpose. The code looks simpler with a timer but is not really visible so I prefer to get the time within the loop like this:

procedure TForm1.Button1Click(Sender: TObject);
var
    Timeout : TDateTime;
begin
    if FSomeFlag then begin
        Memo1.Lines.Add('You already clicked Button1');
        Exit;
    end;

    FSomeFlag := TRUE;
    Memo1.Lines.Add('First message: Start looping, click on Button2');
    Timeout := Now + OneSecond * 10;
    while FSomeFlag do begin
        Application.ProcessMessages;
        if Application.Terminated then
            break;
        if Now > Timeout then begin
            Memo1.Lines.Add('Timeout');
            break;
        end;
        Sleep(100);
    end;
    Memo1.Lines.Add('Second message: Loop done');
end;

I have not explained yet why there is a Sleep(100) within the loop. If you remove it, everything seems to work the same way. Really? Not! use the task manager to see how much CPU is used by your application. Since it does almost nothing, task manager shows 0% of CPU used. If you click on Button1, you still see 0%.

Now remove Sleep(100) and test again. Task manager will show your application consuming a lot of CPU just for doing nothing sensible. Why is this?

The loop in Button1Click event will run at the maximum speed when there is no Sleep call. Depending on your CPU (Number of cores), this means the loop will consume almost all CPU time available using a single core. The loop will loop thousand or million times per second depending on your hardware.

When there is a call to Sleep, it will instruct the operating system to make the thread sleep for the remaining time slice and for approximately the time in millisecond. Using Sleep(100) will make your loop not run more than 10 times per second which will no use to much CPU.


Better solution?

I explained how ProcessMessages works and how to use it. Now it’s time to ask the good question: Should I really use ProcessMessages?

Most of ProcessMessages calls should and can be avoided. They should be avoided because there is a waste of CPU which slows down your application and the computer in general. They can be avoided by redesigning your application using event driven asynchronous pattern.

Most use of ProcessMessages are used when the developer has not correctly understood or is not even aware of the event driven asynchronous pattern. Such developer has a problem with asynchronous operation where your calls merely start an operation and give control back immediately while the operation itself is executed in the background and triggers an event when done.

In the simple case I used above, it is very easy to change the code so that no wait loop calling ProcessMessages is required. Let’s assume that some processing has to be done when the user has clicked Button3 and Button4 in any order.

The idea is to use a flag per event (Button click here) that must have occurred before the processing can take place. For each event, check all flag to see if processing can take place.

Here is the code:

procedure TForm1.Button3Click(Sender: TObject);
begin
   if FFlag3 then
       Memo1.Lines.Add('You already clicked Button3')
   else begin
       Memo1.Lines.Add('You clicked Button3');
       FFlag3 := TRUE;
   end;
   TryToExecute;
end;

procedure TForm1.Button4Click(Sender: TObject);
begin
   if FFlag4 then
       Memo1.Lines.Add('You already clicked Button4')
   else begin
       Memo1.Lines.Add('You clicked Button4');
       FFlag4 := TRUE;
   end;
   TryToExecute;
end;

procedure TForm1.TryToExecute;
begin
    // Check if both buttons have been clicked
    if (not FFlag3) or (not FFlag4) then
        Exit;
    // Both have been clicked
    Memo1.Lines.Add('Processing done');
    // Reset the flags so that the user can start again
    FFlag3 := FALSE;
    FFlag4 := FALSE;
end;

There can be cases where order of event is important or there are a large number of conditions. In those cases, the whole application should make use of a finite state machine. This will probably be the subject of a future article. For now, have a look at what Wikipedia says about it: http://en.wikipedia.org/wiki/Finite-state_machine

Update 16/12/2013: I wrote another article about how to make PostMessage multi-platform for Android and Windows: http://francois-piette.blogspot.be/2013/12/firemonkey-android-windows-and.html


Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be

9 comments:

yeyo said...

but... in a really long loop, processing a lot of database records.... withouth processmessages the user can have a feeling of the app is frozen... ¿how to deal with this?

while not eof do begin
doTHing;
doOtherthing;
next;
end;
Thx for the blog :)

François Piette said...

Yeyo, even with ProcessMessages the user may think the app is frozen. You have several choices to solve this issue: use an asynchronous database component (ADO has an asynchronous mode for example) or put your blocking code into a thread. The thread will be blocked until the database stuff is done and will use PostMessage to signal work is done. (TThread.Synchronize make use of PostMessage as well).

Elisabeth Melander said...

One should never use ProcessMessages. Period!

I have never come across a use of ProcessMessage that couldn't relatively easy be solved better in another way.

To avoid your application appearing to hang, all you have to do is periodically poll the message queue:
PeekMessage(Msg, 0, 0, 0, PM_NOREMOVE);
You can also just call Update to have your form respond to and process WM_PAINT messages.

Even better: Display and update a progress dialog during long operations.

Jeroen Wiert Pluimers said...

ProcessMessages causes all sorts of re-entrancy problems. Do not promote using it.

If you insist on running tasks taking a long time in the main thread, then perform an occassional PeekMessage without removing it to keep Windows somewhat happy.

But ultimately long running tasks need to be in their own thread.

François Piette said...

@Jeroen It looks like you have not read the article to the end... I don't promote ProcessMessages. I said: "Most of ProcessMessages calls should and can be avoided."

Devia said...

I would even dare to go as far as saying Application.ProcessMessages can be a nightmare for someone trying to debug the source of a bug!

Personally I wouldn't say Avoid it … but instead Don't use it (unless really necessary)

Jeroen Wiert Pluimers said...

@Francois that note should have been the first sentence of your article. I read it and barely saw it, hence my previous comment: most people will regard your post as "oh, nice, ProcessMessages!".

François Piette said...

@Jeroen You are right. I edited the article to mention it in the beginning.

jaenicke said...

It is no problem to use ProcessMessages if one writes clean code. It should not be neccessary at many occasions, but it should be no problem if it is called somewhere.

Just an example:
The abort procedure of TPrinter calls Application.ProcessMessages. This procedure us calles for example within a call to TextOut.

If one has a problem with re-entrancy in ones code it is bad code. You can never know or ensure that ProcessMessages is not called in a call to a VCL method or inside a third party component. And it should not be neccessary.

So I do not deny that one should use ProcessMessages rarely, but if it is a problem for your code, this code is the problem, not ProcessMessages!