|
Autumn 2001
|
In Workspace this issue, a software house faces up to the fundamental differences between traditional green-screen solutions and the new world of web enablement. John Calder of Pixieware explains how they tackled field-by-field processing in web applications.
When users “surf the Web” most of them are not aware that they are always making and breaking contact with Web sites. Web sites stay connected for only as long as it takes to feed us the requested documents, pictures and media and then we look at the detached copies of this material now parked temporarily in our machine’s local memory. Moving from one page to another page of a Web-site is a complete new re-connection, and navigation within a Web-site gives you very little speed advantage over someone else linking to the page as new, from a quite different Web-site.
Such a protocol has its advantages for simple actions such as passively reading documents and it conserves bandwidth well. Figures like 1000 plus visitors “hitting” a Web-site at once or “a million hits in one day!” are not as dramatic as they look. Most visitors will only be working the system for a few seconds to get pages which may keep them occupied for minutes of reading time. That million hits in one day could translate to between 10 and 500 users actively connected at any one time, and any Web event that pulls a big crowd also causes devices like “caching proxy servers” to kick in all through the communication channels and help out with feeding local copies of repeating transmissions like pictures.
Our Pick programmers were not pleased with our early Web experiments, with the way the Web “makes and breaks” along with its deeply embedded standard that the “browser document window” completely clears on any exchange of data with the Server. Therefore the common Web app programming model is based on expecting the user to complete a screen full of data entry, then send the lot in for processing in one form-submit batch. This was common practice with IBM mainframes and terminals in the 70s and 80s and our Pick colleagues did not like it then and they don’t like it now! The objection is the lack of validation-as-you-go. It is too easy to make an error in data entry that ruins a screen load or more of typing and not know about it until after a lot of time is wasted.
Other objections include:
Web batch input/output practice is highly incompatible with the valuable resource of existing terminal apps based on field-by-field validation.
There is a high programming workload of tracking and managing the state of the user within an app.
There are security issues around the Web’s freedom of navigation from page to page.
The challenge was on to make browsers act more like terminals so they can support an “evolution not revolution” approach to Web-enabling existing apps.
Our work up until early 2001 was based on Microsoft’s “Remote Scripting” technology. Quoting from http://msdn.microsoft.com/scripting:
“With remote scripting, developers can now create seamless, interactive Web applications in which the browser can call scripts on the server without reloading the Web page. Prior to remote scripting, developers would have to require the user to reload the calling page, often several times, to interact with the server. This created a slower, disjointed, user experience and inefficient use of the server.”
“Remote Scripting” worked well for us, but required additional installations on the Webserver and the use of some complicating additions to the user session (e.g. the loading of an “RSPROXY” Java applet). “RSPROXY” supports IE4 and above. Netscape 4 can be handled with considerable difficulty. But Netscape 6 is a no-go.
Internet Explorer 5 and above have a suitable transmission method built-in, the “XML Download Behaviour”. This is fast, and makes for much simpler app installation on the webserver. The app only depends on the basic features of IE without any issues around Java or any other installation optional extras.
The “XML Download Behaviour” has become our preferred method, but we do have another slightly slower but more flexible one. Our Web app templates are easily changed to the “Array of IFRAMEs” method. The use of separate invisible windows (“frames”) to take the hit of being erased at each conversation step is an old idea. We have been experimenting with it since late 1998. Its main problem is that it fails to manage requests in queues, as in a user working fast from field-to-field can cause new transmissions to destroy earlier ones in progress. We have recently solved that problem by using three IFRAMEs which rotate in turn at being the transmission device. In theory, the array of IFRAMEs demands more processing power from the client. In practice, most recent machines, can run it almost as fast as the XML Download Behaviour. Apart from supporting IE4, the IFRAME object is now available in Netscape 6 so at the time of writing we have successfully translated our client-side BASIC VBscript template page into JavaScript to support Netscape 6 as well.
Protocol
A transmission method needs something meaningful to transmit. Our protocol in brief:
Servers can control and rewrite the supplied template Web page with delimited string commands.
Input Fields have ids of “fx1”, “fx2”, “fx3”
etc. Other active objects use other prefixes,
eg “gx8” is a grid, it has rows “gx8x1”, “gx8x2”
etc and input fields within rows have ids like “gx8x2x4”
Example Server transmissions:
GROWINS|gx8x3 insert a new row into grid number 8 above row number 3 VALID|fx3=John colour field fx3 green to show validation and display “John” in it VALUE|fx3=John^FOCUS|fx6 display “John” in fx3 and focus on fx6 |
Example Client transmission:
fx3|Andrew user has entered “Andrew” in field fx3 |
The similarity of purpose to Microsoft SOAP and the use of something called “XML Download Behaviour” as a transport mechanism naturally inspires the question “why isn’t this protocol XML-based ?”
The purpose of XML is bulk data transfer with definition of the meaning of each item and sub-item. This situation of thin-client screen-interface is a different need. At each step only small quantities of data are transferred in a simple format, therefore an approach based on Richard Pick’s principle of single-character delimiters is appropriate. Our parsing routine is openly published as source code. It is small clear and fast, our experiments show it to be at least 10 times faster than XML equivalents going through an XML parser. Now that we have translated the routine into JavaScript, it has a more universal reach than the variety of XML-parsing objects, and it has no dependence on browser proprietary optional extras like Java Applets or ActiveX objects.
Compared to Microsoft SOAP, our FBF protocol is:
Microsoft’s “XML Download Behaviour” is probably named to support Microsoft’s current “get XML mentioned with everything” policy. It serves as an object for the browser to receive any text, which could include XML tags.
Browser-as-super-Terminal, other considerations
The protocol is a big step towards turning the browser into a super-terminal which we can quickly bring into the Pick fold. To complete the picture and make this “evolution not revolution” easier and more economical, some other recent browser advances work to our advantage.
CSS Positioning. HTML version 4 supports absolute x-y positioning of labels(<DIV>s). IE5+ also supports direct positioning of input fields. Netscape 6 and IE4 require them to be parked inside a positioned <DIV>. This means that existing terminal screen designs and layouts can be used in Web apps by replacing @(x,y) with a Web equivalent. We call ours SUBROUTINE WEBDU for labels and SUBROUTINE WEBIU for input text fields, textarea boxes, buttons and checkboxes.
We use a browser popup window workspace of 720 pixels x 500 pixels, so multiplying existing x values by 9 and existing y-values by 22 nicely translates character-based co-ordinates to Web.
Excerpt of BASIC Code from WEBDU:
* Translate positioning from character co-ords x = (xraw * 9):’px’ y = (yraw * 22):’px’ |
Example of call to WEBDU for a label:
In Terminal-serving BASIC this was:
SC = SC:@(4,2):”Supplier Code”)
In the Web app version, this becomes:
CALL WEBDU(4,2,0,0, “Supplier Code “, “”, “”, SC) |
Then to define an active GUI input field, do this:
CALL WEBI5(22,2,0,0, “”, “fx0”, “”, SC) |
DHTML, dynamic creation on the fly of page elements. With our Web App page able to have an ongoing conversation with the server, it is possible for that page to be a standard blank slate, like a terminal, with the server able to rewrite part or all of it (FBF keywords “IREPLACE”, “INSERT”). The issues around control of user freedom of navigation are solved at a stroke. Here the entire app consists of only one template page and the user is locked down to that page. Leaving it or closing it cues the app to logoff and disconnect.
Pick-to-Webserver methods, persistence, non-persistence
A Web-page talking to a Pick app requires a Webserver in between acting as a relay station.
This article has been about the Webserver-to-Page step of a two-step transmission. The question arises, what about that other Pick-to-Webserver step? Of course we use our own product PixieWeb software for this. It is a “socket-with-attitude” small and light enough to be persisted in the Session Object of the IIS Webserver.
When run this way it can maintain a terminal-like “persistent” = ongoing conversation with conventional BASIC programs. These programs converse with the Web page with the familiar PRINT and INPUT statements. This makes the Webserver role quite minimal with only three or four template scripts required and the entire Web app being run with BASIC.
Such a configuration does mean that each connected Web app user requires a licensed port for the entire user session. The classicWeb app batch-submission non-persistent approach for all its complexity does offer the sweetener of saving on licensed ports because you only need them for the moment the user finishes and submits an entire form.
Field-by-Field is probably most needed for expert data-entry by in-house employees, or an elite of trusted outside-world associates, for who enough licensed ports would already exist anyway for their current terminal work.
So could the advantages of Field-by-Field and Licensed-port-savings be combined? It is possible but the coding for Field-by-Field validation and field-navigation needs to move to the Webserver, eg as VBscript which is a little like BASIC and could be adapted from existing BASIC Input/Output code. The BASIC programs would play a back-server role of data READ/WRITE at the start/end of a form in a similar way to the batch approach.
Could Field-by-Field be used with connectivity providers other than PixieWeb software? Naturally we have not been motivated to try! To fully support BASIC Field-by-Field, providers need to offer a persistent connection option and be friendly to existing BASIC code. For example a possible but untested PixieWeb software emulator may be Raining Data’s FlashConnect using its “W3Input” method. Providers who are non-persistent only could support a hybrid of Webserver and BASIC coding.
FBF offered as an Open Standard
We regard the Field-by-Field protocol, scripting source code and Webserver templates as supporting examples and documentation for PixieWeb. That is we sell PixieWeb software, we make “FBF” freely available on an open source model, requiring only that our copyright notice remain in the resources so we get the promotional benefit.
Working example Web apps, and more details at: http://203.109.148.138:7301/pixieinfo/rsdemointro.htm
The FBF protocol is published online at: http://www.pixieware.com/PixieWeb/PWDocu7.htm
If any non-Pick enthusiasts read this article, FBF and PixieWeb software has been tried and tested on another database environment, filePro, where a B2B Web-site is being successfully developed with this technology.
John Calder
Pixieware Software (NZ)
Last Updated: 29 Nov 2001