Internet Explorer Automation Part 2

Internet Explorer is a very nice program to automate. There are a large number of actions you can do programmatically from your own application. But when IE is already opened with a bunch of tabs, it is not a trivial task to programmatically select and activate the tab you want.

Here after, I will present all the code required to do that. It has been developed using Delphi XE3 but of course as automating IE is independent of the language, you should be able to translate my code to C#, C++ or any language supporting COM programming.

The code I present is basically in a single function with a number of small supporting functions. The main function is:

function WebBrowserSelectTabByUrl(
    const Wb           : IWebBrowser2;
    const Url          : String;
    out   HwndTopLevel : HWND) : Boolean;

You pass an existing IWebBrowser interface (see for example my previous article at http://francois-piette.blogspot.com/2013/01/internet-explorer-automation-part-1.html) and an URL. The function will select the tab having the given URL loaded, if any. It will also return a window handle that can be used to bring the actual window in the foreground or to restore it if it was minimized.

To achieve his goal, WebBrowserSelectTabByUrl is using a seldom know interface. I mean IAccessible (http://msdn.microsoft.com/en-us/library/windows/desktop/dd318466(v=vs.85).aspx). This interface is normally used by software written for the visual impaired person. This kind of software is able to discover almost every interface gadget on screen, return a description and perform a default action such as clicking on it if it is a button.

Internet Explorer is exposing a complete IAccessible interface for its entire user interface. And this is what I use to search for the tab rows displaying IE tabs, and get the URL assigned to each of the tab.

IAccessible interface and related definitions is defined in OleAcc unit which is an import from OLEACC.DLL type library. This unit also contains a lot of constants that were not included in the type library.

Beside the interface, there are a few API functions which give an IAccessible interface from a window handle or the reverse. We need two functions which are not defined in OleAcc and you’ll find the required import in the code at the end of this article. It is WindowFromAccessibleObject and AccessibleChildren.

IAccessible is just the programmatic way to interact with the underlying user interface gadgets. It is organized in an hierarchical tree. One you get an IAccessible interface for something, you can “travel” thru the tree to find what you need. Each gadget has a name. We are looking for “Tab Row” item. In Internet Explorer user interface, this represents the row usually below the address bar, where IE shows all tabs for all opened URL.

Once we get hand on the “Tab Row” gadget, we can iterate all of its descendants to find the one with the URL we are looking for. The URL is associated with each tab as a description. Actually the tab description is composed of the text IE show on the tab and the associated URL that IE shows in the address bar when the tab is selected.

Finally, when we have the IAccessible for the exact tab were looking for, we can invoke his default action programmatically. The net effect is the same as the effect a user produce when clicking on the tab.

There is still an issue: As IAccessible is made to help visually impaired users, the name of each gadget is localized. So “Tab Row” in English becomes “Onglet Ligne” in French! I have not found any way to discover the translation so I have to code a small routine querying the language from Windows configuration and use it to select the correct translation. If you use my code, you must add the language you need because I only programmed the English and French translation. See WebBrowserGetLocalizedTabRowName function at the end of this article. [The translation is probably somewhere in one resource in IE executable or DLL. Let me know if you know where it is]

The fastest way to find the first IAccessible interface we need is to travel Internet Explorer window tree. I used Microsoft Spy++ tool to see how those windows are organized. The outermost window handle is given my IWebBrowser interface in his HWND property. Then the hierarchy of window classes is “WorkerW” (or “CommandBarClass” depending on IE version), “ReBarWindow32”, “TabBandClass” and finally “DirectUIHWND”. In used the API function FindWindowEx to navigate thru the hierarchy. Yhe result is the functions WebBrowserGetDirectUIHWND.

From the DirectUIHwnd, we can get the IAccessible interface calling AccessibleObjectFromWindow. Let’s name it AccDirectUI.

The, as I said above, we have to traverse the IAccessible tree to find one with name “Tab Row” (Or the translated is you don’t use an English IE). This is FindAccessibleDescendantByName function. This is a classical tree traversal algorithm. The only complex thing is that there is a variant in the process. A cast and a call to QueryInterface are required to get hand on the IAccessible interface of the child.

Almost the same tree traversal is used from the “Tab Row” to find the right tab. Instead of checking the name, I check the description which contain the URL.

Enough story, here is the code:

function WebBrowserSelectTabByUrl(
  const Wb           : IWebBrowser2;
  const Url          : String;
  out   HwndTopLevel : HWND) : Boolean;
var
  HwndDirectUI     : HWND;
  AccDirectUI      : IAccessible;
  TabRow           : IAccessible;
  CandidateTab     : IAccessible;
  I                : Integer;
  LocalUrl         : String;
  HwndCandidateTab : HWND;
  ChildArray       : array of OleVariant;
  ChildDispatch    : IDispatch;
  ChildCount       : Integer;
  CountObtained    : Integer;
begin
  Result       := FALSE;
  HwndDirectUI := WebBrowserGetDirectUIHWND(Wb);
  AccessibleObjectFromWindow(HwndDirectUI, OBJID_WINDOW,
                             IID_IAccessible, AccDirectUI);
  if not Assigned(AccDirectUI) then
    Exit;

  TabRow := FindAccessibleDescendantByName(AccDirectUI, 
                           WebBrowserGetLocalizedTabRowName);
  TabRow.Get_accChildCount(ChildCount);
  if ChildCount <= 0 then
    Exit;
  SetLength(ChildArray, ChildCount);
  if AccessibleChildren(Pointer(TabRow), 0, ChildCount,
                        ChildArray[0], CountObtained) <> S_OK then
    Exit;
  for I := 0 to CountObtained - 1 do begin
    if VarType(ChildArray[i]) = varDispatch then begin
      ChildDispatch := TVarData(ChildArray[i]).VDispatch;
      if (ChildDispatch <> nil) and
         (ChildDispatch.QueryInterface(Ole2.TGUID(IID_IAccessible),
                   CandidateTab) = S_OK) then begin
        if not Assigned(CandidateTab) then
          continue;
        LocalUrl := WebBrowserUrlForTab(CandidateTab);
        if SameText(LocalUrl, Url) then begin
          CandidateTab.accDoDefaultAction(0);
          WindowFromAccessibleObject(CandidateTab, HwndCandidateTab);
          HwndTopLevel := FindIEFrameWnd(HwndCandidateTab);
          Result := TRUE;
          Exit;
        end;
      end;
    end;
  end;
end;


function WebBrowserUrlForTab(AccTab : IAccessible) : String;
var
  Desc : WideString;
  I    : Integer;
begin
  try
    SetLength(Desc, 1024);
    AccTab.Get_accDescription(CHILDID_SELF , Desc);
    if Desc <> '' then begin
      I := Pos(String(#13#10), String(Desc));
      if I > 1 then
        Result := Copy(Desc, I + 2, MAXINT)
      else
        Result := Desc;
      Exit;
    end;
  except
    Result := '??';
  end;
end;

// The IAccessible name for the tab row in Internet explorer is localized
// This function fetch the language code and return the appropriate value
// according to the current system default language
function WebBrowserGetLocalizedTabRowName : String;
var
  Lang : String;
begin
  Lang := GetLocaleStr(LOCALE_SYSTEM_DEFAULT, LOCALE_SISO639LANGNAME, '');
  if Lang = 'fr' then
    Result := 'Onglet Ligne'
    // YOU MUST ADD a "else if" clause for each language you want to support
  else
    Result := 'Tab Row'; // English
end;

function WebBrowserGetDirectUIHWND(Wb : IWebBrowser2): HWND;
begin
  // try IE 9 first:
  Result := FindWindowEx(Wb.HWND, 0, 'WorkerW', nil);
  if Result = 0 then begin
    // IE8 and IE7
    Result := FindWindowEx(Wb.HWND, 0, 'CommandBarClass', nil);
  end;
  Result := FindWindowEx(Result, 0, 'ReBarWindow32', nil);
  Result := FindWindowEx(Result, 0, 'TabBandClass', nil);
  Result := FindWindowEx(Result, 0, 'DirectUIHWND', nil);
end;


// Recursively trave the tree of descendant IAccessible interface object
// to search for the one having a given name.
function FindAccessibleDescendantByName(
  const AParent : IAccessible;
  const AName   : String) : IAccessible;
var
  ChildArray    : array of OleVariant;
  Child         : IAccessible;
  ChildName     : WideString;
  ChildDispatch : IDispatch;
  ChildCount    : Integer;
  CountObtained : Integer;
  I             : Integer;
begin
  Result := nil;
  Aparent.Get_accChildCount(ChildCount);
  if ChildCount <= 0 then
    Exit;
  SetLength(ChildArray, ChildCount);
  if AccessibleChildren(Pointer(AParent), 0, ChildCount,
                        ChildArray[0], CountObtained) <> S_OK then
    Exit;
  for I := 0 to CountObtained - 1 do begin
    if VarType(ChildArray[i]) = varDispatch then begin
      ChildDispatch := TVarData(ChildArray[i]).VDispatch;
      if (ChildDispatch <> nil) and
         (ChildDispatch.QueryInterface(Ole2.TGUID(IID_IAccessible),
                                       Child) = S_OK) then begin
        if not Assigned(Child) then
          continue;
        Child.Get_accName(0, ChildName);
        if SameText(AName , ChildName) then begin
          Result := Child;
          Exit;
        end;
        Result := FindAccessibleDescendantByName(Child, AName);
        if Assigned(Result) then
          Exit;
      end;
    end;
  end;
end;

// Given a HWND for a window deep in the hierarchy of windows, go back to
// the top level window which has the class name 'IEFrame'.
function FindIEFrameWnd(Hndl : HWND) : HWND;
var
    H     : HWND;
begin
    H := Hndl;
    while TRUE do begin
        if SameText(GetClassName(H), 'IEFrame') then begin
            Result := H;
            Exit;
        end;
        H := GetParent(H);
    end;
end;

function WindowFromAccessibleObject(
             pAcc      : IACCESSIBLE;
             var phwnd : HWND) : HRESULT; stdcall;
             external 'oleacc.dll';

function AccessibleChildren(
             paccContainer     : Pointer;
             iChildStart       : LongInt;
             cChildren         : LongInt;
             out rgvarChildren : OleVariant;
             out pcObtained    : LongInt) : HRESULT; stdcall;
             external 'oleacc.dll';

The first part of this article is at:
   http://francois-piette.blogspot.be/2013/01/internet-explorer-automation-part-1.html

This article is at:
   http://francois-piette.blogspot.be/2013/01/internet-explorer-automation-part-2.html

Follow me on Twitter