Distributed Application Case Study: Part I - The Problem
For the past year-and-a-half, I have been involved in the design and implementation of a system that pulls data out of corporate application databases, integrates it, and presents it as a series of charts. This has been implemented with SQL Server 2005 for the database & integration side of things, and a C#-based Windows Forms application with calls to Dundas Chart and Map for the charts. The cliché-de-jour for this type of application is a dashboard, and if you want to call it that, so be it.
To date, this application has been built using a monolithic approach – the database and application all reside on a single development machine. I’ve always assumed it would wind up on a production server in a very similar configuration. As the day approaches when the application goes into production, however, it has become increasingly evident that decoupling the user interface from the query engine is a mandatory requirement. The user interface is blocked for up to 30 seconds — or more, if the database call didn’t time out — during which the user simply has to sit and stare at a locked up application.
Externally, I visualize the user being able to set the parameters of their query, click the Search button, and have control of the application return to them more-or-less instantly. A progress bar, updates to a status line, or other amusement will keep the users informed of running query’s progress.
While the query is running, it seems obvious you should still be able to perform other, unrelated activities. Setting up a chart for printing would be a good example. Resizing the window or panels within the window would be another. Heck, even minimizing the application while waiting for the query to complete would be desirable option for many users. But alas, none of that is possible given the current state of affairs.
Internally, all database access is implemented through a series of stored procedures. The client application provides a series of parameters, passes them to the stored procedure, and then waits for the resulting dataset to be filled. The application is blocked at this point — the user cannot continue with any other use of the application until such time that particular line of code is finished executing. This is the beast that sometimes runs to 30 seconds, or worse yet, times out, in which case, the best you can hope for is a well-handled exception which is then handed back to the user.
There are basically three, distributed computing mechanisms that can be used to achieve the desired objective of decoupling the user and their interface from the slow-running query; web services, .NET Remoting, and Microsoft Message Queuing. In addition, multithreading within a single application domain could be used to achieve a similar result. But most important, regardless of the which approach is chosen, it will be necessary to implement it asynchronously – which is the ability for the client application to carry on doing it thing while the query is running on another machine.
Of the four choices, my initial inclination is to go with web services. The application is intended for users that reside inside the corporate firewall, but there is a pretty good chance a subsequent version will be made available to users on the outside. It’s my assumption that using web services is the better choice, given the relative ease with which XML-based web service traffic can be made to traverse the firewall, and other potential obstacles. On the down side, web services will not likely deliver the performance of the more closely coupled .NET Remoting and Message Queuing. But given the volume of data that is being manipulated by this application, this lower performance is not considered a factor.
Multithreading is not really being seriously considered given the relative complexity of the implementation, coupled with the fact that it does nothing to break the application into small, more manageable chunks — another important objective of this exercise.
This series of posts is intended to document my approach to solving this problem. This is not a theoretical exercise; the problem I’m trying to solve is real, and the solution, assuming the outcome of this exercise is successful, will go into a production application. I can’t make a specific commitment as to how often I will post, or when I’ll be finished, but I’ll do my best to keep this blog current with progress.
Thank you very much for reading, and please let me know what you think.