March 26, 2019

ICS V8.60 has been released

ICS V8.60 has been released at:

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

ICS is a free internet component library for Delphi 7, 2006 to 2010, XE  to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo and 10.3 Rio, and C++Builder 2006 to XE3, 10.2 Tokyo and 10.3 Rio. ICS supports VCL and FMX, Win32, Win64 and MacOS targets.

The distribution zip includes the latest OpenSSL 1.1.1 win32, with other versions of OpenSSL being available from the download page.

Major changes in ICS V8.60 include:

1 - Added several new components and sample applications created by Magenta Systems Ltd over the past 17 years and previously distributed and installed separately to the ICS distribution.  Bundling them with ICS makes installation and updating easier, and allows existing ICS samples to make use of many the new components, such as UTF-8 file logging. All the components have new names so existing applications using the originally distributed versions will still work, but it's recommended updating existing applications for the new ICS versions. The added components include IP stream logging, SMTP Mail Queue, Time Server and Client,  Whois client, blacklisting of malicious IP addresses, file logging functions, file indexing, copying and deleting, FTP indexing, multiple file uploading and downloading, HTTP page parsing and URL downloading.

2 - In THttpCli, only follow relocation for 3xx response codes, not 201 Created, but keep Location property for 201 which is often response to a POST and may be needed by the application.

3 - When starting TWSocket Connect, the IP address chosen for DNS lookup is now saved in ASCII as AddrResolvedStr which is useful in connect OK or failed events to see whether an IPV4 or IPv6 address was chosen, and which was used if DNS offered multiple IPs. AddrResolvedStr is exposed as a property in TWSocket, THttpCli, TSslHttpRest and TFFtpClient and reports in failed connection events.  Other clients will be added soon.

4 - Various samples have been updated to ease testing of IPv6 and to save the diagnostic window activity to a disk log file, OverbyteIcsHttpsTst, OverbyteIcsHttpRestTst, OverbyteIcsTimeTst, OverbyteIcsX509CertsTst.

5 - Added round robin DNS lookup if DNSLookup returns multiple IP addresses, so they are used in turn after a failure when a component is called repeatedly without being freed.  This is implemented in THttpCli,
TSslHttpRest TFFtpClient and TIcsTimeClient.  Other clients will be added soon.  There is a new OnSelectDns event to override round robin lookup and make your own choice.

By default, the DNS lookup in ICS components ignores IPv6 addresses and  always use the first IPv4 address offered, when there is more than one. This is usually implemented in the OnDnsLookupDone event in the application
or high level component.  So if that first address does not respond, the application never tries any other addresses.

This has become more of a problem when enabling applications for IPv6, by changing SocketFamily from the default sfIPv4, to sfAny, sfAnyIPv4 (prefer IPv4), sfAnyIPv6 (prefer IPV6) or sfIPv6 (only IPv6), when IPv6 addresses
may also be returned as well as IPv4.  Due to routing or firewall issues, IPv4 and IPv6 might not both be available and so connection will fail if that address is chosen first.  Previously it was necessary to restrict SocketFamily so only the working family is attempted.

The DNS round robin implementation relies on keeping the last successful connected IP address, so it can be re-used for subsequent connections, but looping through any alternative addresses if the last connection failed, for subsequent connection attempts. Most existing applications use the OnDnsDone event to select the connection IP address so the round robin code is added there.

Newer applications making use of TWSocket ComponentOptions wsoAsyncDnsLookup should added code in OnDNSLookupDone and update the DnsResult property which is then used by Connect.

6 - IcsHosts supports two new TSslSrvSecurity server security levels, sslSrvSecTls12Less and sslSrvSecTls13Only, the former disables TLS1.3 in ICS servers if TLSv1.3 fails (perhaps a bad client implementation) while the second only supports TLSv1.3.

7 - Up to date C++ packages are included for 10.2 Tokyo and 10.3 Rio. Information on installing ICS for C++ 2007 may be found at: https://en.delphipraxis.net/topic/844-use-ics-with-c-builder-2007/
Sorry, currently don't believe it's easy to update the ICS source code to avoid the changes needed for C++ 2007.
 
Full release notes are at: http://wiki.overbyte.eu/wiki/index.php/ICS_V8.60
 

March 24, 2019

New home for ICS support

ICS (Internet Component Suite) is used by hundreds of thousands developers around the world. After more than 20 years, ICS is still actively developped. Lastest version can be downloaded from ICS download page and from the subversion repository if you need the very latest update. And if you need easy download and install, get it from Embarcadero GetIT! directly from RAD Studio/Delphi IDE.

You may need help using ICS and for that purpose, a lot of demos are included in the distribution. Of course you may ask the community using the web forum at Delphi-PRAXiS. This is an English forum.

Enjoy!
Fran├žois PIETTE
Embarcadero MVP
ICS Author

January 2, 2016

DirWatch: Delphi 10 in action

In this article, I will show you how I used Delphi 10 Seattle to create a nice application aimed at watching a directory tree. Source code provided, see at the end.


Background:


I frequently use shared directories as repository for files. Some user on the network just drops the file in the directory where I can grab it. The user is supposed to notify me when he has dropped a file.  Unfortunately some user forgets to notify e and the file stay there for a long time before I notice it myself. In other situation, I like to know which files are affected by a given application or by a setup.


DirWatch will be of great help in those situation and many others. DirWatch will monitor a whole directory tree, even a full disk, for changes and send me an email when a change is detected. A change means a file or subdirectory is added / renamed / deleted / modified.


How it works:


Part 1: Getting notifications


Using Windows, it is actually very easy to be notified of changes in the file system. There is a specific and simple API for the purpose. The main API functions are FindFirstChangeNotification, FindNextChangeNotification and ReadDirectoryChanges. The actual work is done by Windows!


This notification API works much like listing a directory: you do a FindFirst, then if successful, you read the changes that occurred and then call FindNext to be notified for the next change. As simple as that.


OK, things can be a little bit more complex if you want your application to stay responsive you should resort to asynchronous Input/Output and multithreading. That code already exists on the web in numerous instances. I selected to use DWScript implementation because it is simple and efficient and also because I use DWScript for other project and find this open source product excellent. DWScript is available under Mozilla Public License Version 1.1.


dwsDirectoryNotifier is available from DWScript project (https://bitbucket.org/egrange/dwscript). Direct link to the source file at the time of writing is


Part 2: Sending an email


To programmatically send an email, you have many solutions available. In my opinion, the best solution is to directly address the email server, either company email server or Internet Service Provider email server. This may sound difficult but it is not much. An email server makes use of a protocol named SMTP, just like your browser make use of a protocol name HTTP. SMTP stands for Simple Mail Transfer Protocol and as its name implies, it is simple! SMTP is a text based protocol and involves the exchange of a few messages to send an email.


SMTP becomes a little bit more complex when you start talking authentication, compression, and security but it basically remains simple.


I have implemented the SMTP protocol in an open source freeware I published nearly 20 year ago. I mean the Internet Component Suite (ICS for short). See http://wiki.overbyte.be. ICS is a huge component library. We will only use the SMTP component.


ICS (Internet Component Suite) is available from http://wiki.overbyte.be/wiki/index.php/ICS_Download and also can be installed directly in Delphi 10 Seattle using Embarcadero GetIT (Delphi menu Tools / GetIi Package Manager, type ICS into the search box and select "ICS for VCL").


Part 3: be discreet


When I use a utility which I don't have to interact with, I like to have it very discreet. I don't even want to see it on my screen. There are two good solutions for this: write a service application or write a "tray-icon" application. I selected the second which is slightly easier to write and debug.


A tray-icon application is a normal Windows application which once minimized is only noticeable by its icon displayed in the systray (You know, this little area usually on the bottom right of the screen where the time is displayed).


Using Delphi 10, you create an application as a "VCL form application" and drop a TTrayIcon component on it. You have just two event handler to write: FormResize and TrayIconDoubleClick. FormResize is used to hide the form once it is minimized and TrayIconDoubleClick is used to make the form visible again when the user double clicks on the small icon.


There is a side problem when building a tray-icon application: you need a small 16x16 icon. If you don't create your own 16x16 icon, Windows will shrink the large icon to create the small one. The result is frequently horrible.


To create the multiple resolution icon file, I used IconMagick (http://www.imagemagick.org) command line utility "convert" to combine icons created using my favorite paint application.


convert DirWatchIcon32x32.ico DirWatchIcon16x16.ico DirWatchIcon.ico


Once DirWatchIcon.ico was created, I associated it with the application using Delphi project options dialog.


Please note that TrayIcon component dropped on the form must be loaded with the DirWatchIcon16x16.ico file otherwise window will use the 32x32 icon file resized to 16x16 which gives an horrible result.


Part 4: Work offline


I wanted to be notified even for changes occurring while DirWatch was not running. This is especially useful if the watched directory is not local but a shared resource on a file server.


When DirWatch starts, I create notifications for the changes occurred since last run by comparing the list of files actually on disk with the list of files that where on disk when DirWatch was stopped.


This means that at program startup, DirWatch enumerates all the files and directories in the watched directory. This could take some time if the directory contains millions of files!  On my hard drive with about one million files, it takes approximately 15 seconds to enumerate the files, compare that list with the previous list and send the email for the changes.


The implementation I made makes use of Delphi 10 Seattle generics. Internally the list of files is not a list but a dictionary. This gives fast access to each file data. Each file (Or directory entry) is stored with his name, attributes, size and timestamp. This is used to find out what has changed with a very low chance to miss something. Of course it could be possible to write a program which saves the timestamp of a file, update some bytes in the file without changing the size and then restore the timestamp. Actually this never occurs in real life!


It is interesting to look at the code I wrote for the dictionary. There are a few involved classes: TDirDictionary, TDirEntry, TDirDictionaryPair, TDirDictionaryObject and TDirDictionaryEnumerator.


I implemented TDirDictionary by delegation instead of inheritance. I frequently do that to have complete freedom about the interface I expose. In most cases, delegation is a better encapsulation that inheritance. Look at the source code for details and you'll quickly see the code is really very simple.


Part 5: Data persistence


DirWatch must be configured with some parameters. Of course, the directory which must be watched but also the SMTP server information (Those information are the same that Outlook uses for a classical POP3/SMTP account).


I also like that my applications remember where their window was and which size they had when I run the application again.


For all those purposes, I like to use the good old INI file. It is easy to use within the program and easy you me to edit with my favorite text editor. I store the INI file in the "Loacl AppData" folder in the user profile.  The Windows API function SHGetFolderPath is used to query the exact location of the "Local AppData" folder.


Part 6: Hide sensitive data


Sending an email may sometimes involve a usercode and password that the DirWatch must know. As explained above, those values are stored in the INI file and this is a problem if they are saved in clear text.


I wrote two functions Decrypt and EncryptRestore for the purpose. One trick is used to allow encryption without using a special program. When the data is enclosed in angle bracket in reverse order, then it's assumed not encrypted but needing to be encrypted. Then once encrypted the resulting value is saved between angle brackets.  So "]MyName[" will result in "MyName" and a flag is set so that EncryptRestore will produce "[Qyaze2==]" which is an encrypted value. If no angle bracket is used at all, then the value is assumed not encrypted. Here in this implementation I used very weak encryption: base64 encoding. In a fully secured application you should use a real encryption mechanism providing strong security.


Part 7: Command line options


Some information cannot be stored in the INI file and yet be changed at run time. For example the name and location of the INI file itself can be changed.
-i IniFileName        Select INI file
-h            Show this help
-m            Start minimized
-a AppName        Select application name
-c CompanyName    Select company name


To parse command line argument, I used code largely inspired by by code from Stehan Huberdoc  (http://stefan.huberdoc.at/comp/software/delphi/sandkasten.html) which is an implementation of the well-known GNU GetOpt command line parsing function (https://en.wikipedia.org/wiki/Getopt).


Part 8: Source code


Source code is split into two units: DirWatchMain.pas which is the main and only form of the application and DirWatchProcess.pas which contains the code which does the actual work without any user interface. Of course other units are used as explained above: some from ICS, some from DWScript and GetOpt.


Download links:




Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be