Field-by-Field Web Application
Development: By John Calder, Pixieware Software
W H E N U S E R S
"surf the Web" most of them are not
aware that they are always making and breaking
contact with Web sites. Web sites stay connected
for only as long as it takes to feed us the
requested documents, pictures and media and then
we look at the detached copies of this material
now parked temporarily in our machine's local
memory. Moving from one page to another page of a
Web site is a completely new re-connection, and
navigation within a Web site which gives you very
little speed advantage over someone else linking
to the page as new, from a quite different Web
site.
Such a protocol has
its advantages for simple actions such as passively
reading documents and it conserves bandwidth well.
Figures like 1,000 plus visitors "hitting"
a Web site at once or "a million hits in one
day!" are not as dramatic as they look. Most
visitors will only be working the system for a few
seconds to get pages which may keep them occupied for
minutes of reading time. That million hits in one day
could translate to between 10 and 500 users actively
connected at any one time, and any Web event that
pulls a big crowd also causes devices like
"caching proxy servers" to kick in all
through the communication channels and help out with
feeding local copies of repeating transmissions like
pictures.
Our early Web
experiments had difficulties gaining acceptance with
our "legacy coding" colleagues, due to the
way the Web "makes and breaks" connections,
along with its deeply embedded standard that the
"browser document window" completely clears
on any exchange of data with the Server. Therefore
the common Web application programming model is based
on expecting the user to complete a screen full of
data entry, then send the lot in for processing in
one form-submit batch, which is the way the web
normally works, not FBF like Pixie. This was common
practice with IBM mainframes and terminals in the
'70s and '80s, and our colleagues did not like it
then and they don't like it now! The objection is the
lack of validation-as-you-go. It is too easy to make
an error in data entry that ruins a screen load or
more of typing and not know about it until after a
lot of time is wasted.
Other
objections include:
A The
web batch input/output practice is highly
incompatible with the valuable resource of existing
terminal applications based on field-by-field (FBF)
validation. We call our technique FBF, which allows a
field to be edited or validated individually, rather
that processing a complete form. At the back-end the
programming is a little different, but
terminal/legacy programs are "better"
because they can do this field-by-field validation
and many/most web applications cannot?
B
There is a high programming work-load of tracking and
managing the state of the user within an application.
As indicated earlier, the programmer has to do things
to keep track of where the program was up to when a
session is broken, and then when it starts again it
needs to begin where the last session stopped.
C There
are security issues If you are not careful, one can
use the back/forward buttons to get around password
protections, so as an example, the Web's freedom of
navigation from page to page needs to be considered.
The challenge was on,
to make browsers act more like terminals so they can
support an "evolution not revolution"
approach to Web-enabling existing text-based
applications (e.g. COBOL, PICKBasic, filePro). What
FBF allows is the ability to re-use programs
(re-engineer) and evolve into GUI/Web, rather than
throw everything out and start again (revolution).
Our work up until early 2001 was based on Microsoft's
"remote scripting" technology. Quoting from
Microsoft:
"With remote
scripting, developers can now create seamless,
interactive Web applications in which the browser
can call scripts on the server without reloading
the Web page. Prior to remote scripting,
developers would have to require the user to
reload the calling page, often several times, to
interact with the server. This created a slower,
disjointed, user experience and inefficient use
of the server."
"Remote
scripting" worked well for us, but required
additional installations on the Web server and the
use of some complicating additions to the user
session (e.g., the loading of an "RSPROXY"
Java applet). "RSPROXY" supports IE4 and
above. Netscape 4 can be handled with considerable
difficulty. But Netscape 6 is a no-go.
Internet Explorer 5
and above have a suitable transmission method
built-in, the "XML Download Behavior." This
is fast, and makes for much simpler application
installation on the Web server. The application only
depends on the basic features of IE without any
issues around Java or any other installation optional
extras.
The "XML Download
Behavior" (a Microsoft term for a web-server
behaviour which we hook into) has become our
preferred method, but we do have another (refer
following on IFRAMES) which is slightly slower but a
more flexible one. Our Web application are easily
changed to the "Array of IFRAMEs" method.
The use of separate invisible windows
("frames") to take the hit of being erased
at each conversation step is an old idea. We have
been experimenting with it since late 1998. Its main
problem is that it fails to manage requests (from the
browser) in queues, as in a user working fast from
field-to-field can cause new transmissions to destroy
earlier ones in progress. We have recently solved
that problem by using 3 IFRAMEs which rotate in turn
at being the transmission device. In theory, the
array of IFRAMEs demands more processing power from
the client. In practice, most recent machines can run
it almost as fast as the XML Download Behavior. Apart
from supporting IE4, the IFRAME object is now
available in Netscape 6, so at the time of this
writing, we have successfully translated our
client-side BASIC VB-script template page into
JAVASCRIPT to support Netscape 6.2/above as well.
THE PROTOCOL
A transmission method
needs something meaningful to transmit. Our protocol
in brief:
S
E R V E R S - can control and rewrite the supplied
template Web page with delimited string commands.
I
N P U T F I E L D S - have ids of "fx1",
"fx2", "fx3", etc. Other active
objects use other prefixes, e.g., "gx8" is
a grid, it has rows "gx8x1",
"gx8x2", etc., and input fields within rows
have ids like "gx8x2x4".
The similarity of
purpose to Microsoft SOAP and the use of something
called "XML Download Behavior" as a
transport mechanism naturally inspires the question
"Why isn't this protocol XML-based?"
The purpose of XML is
bulk data transfer with definition of the meaning of
each item and sub-item. This situation of thin-client
screen-interface is a different need. At each step
only small quantities of data are transferred in a
simple format, therefore an approach based on the
principle of single-character delimiters is
appropriate. Our parsing routine is openly published
as source code. It is small, clear and fast; our
experiments show it to be at least 10 times faster
than XML equivalents going through an XML parser. Now
that we have translated the routine into JavaScript,
it has a more universal reach than the variety of
XML-parsing objects, and it has no dependence on
browser proprietary optional extras like Java Applets
or ActiveX objects. Basically XML is used for moving
bulk data, FBF moves only what is needed and is very
efficient. We paint a screen initially, and only
refresh that which is needed to update the screen.
Compared to Microsoft
SOAP, our FBF behavior is our interactive technique
for interfacing screens to applications.
2
- W A Y -
server-calling-browser-functions as well as
browser-calling-server-functions. To this writer's
best knowledge, SOAP only offers 1-way
browser-calling- server-functions and with less
flexibility.
U
N I V E R S A L - simplest, freely available
programming toolkits, e.g., Java Script, to implement
it.
I
D E N T I C A L - on IE and Netscape.
FA
S T E R -
because FBF is not carrying unnecessary baggage in
order to appear to be fashionable. As indicated
elsewhere we only send/transmit what is needed.
F
R E E -
with open source code provided, simple principle
familiar to many, and easily customizable.
F
R I E N D LY - to COBOL and similar routines inside
COBOL, because of commands and data moving to/from
the browser in a dynamic-variable format.
Microsoft's "XML
Download Behavior" is probably named to support
Microsoft's current "get XML mentioned with
every-thing" policy. It serves as an object for
the browser to receive any text, which could include
XML tags.
Example Server
transmissions:
GROWINS|gx8x3 -
insert a new row into grid number 8 above row
number 3
VALID|fx3=John -
colour field fx3 green to show validation and
display "John" in it
VALUE|fx3=John^FOCUS|fx6
- display "John" in fx3 and focus on
fx6
Example Client
transmission:
fx3|Andrew - user
has entered "Andrew" in field fx3
BROWSER AS
SUPER-TERMINAL, OTHER CONSIDERATIONS
The protocol is FBF a
big step towards turning the browser into a
super-terminal which we can quickly bring into the
COBOL fold. To complete the picture and make this
"evolution not revolution" easier and more
economical, some other recent browser advances work
to our advantage.
CSS
P O S I T I O N I N G. HTML version 4 supports absolute x-y
positioning of labels (DIV). IE5+, Netscape 6.2+ and
Mozilla 1.3+ also support direct positioning of
(INPUT) fields.
This means that
existing terminal screen designs and layouts can be
used in Web applications by replacing (x,y) terminal
positioning with a Web equivalent. We call our
SUBROUTINE WEBDX for labels and SUBROUTINE WEBIX for
input text fields, text area boxes, buttons and
checkboxes. Text-screens use column and row
positioning in a 80 by 22 character framework which
renders all characters to be equal size, font and
colour (with some mods. applicable). In the web world
the characters are based on pixels and can be any
size, font or colour from one to the next. The
text-screen character transmissions have to altered
to conform to pixels.
We use a browser
pop-up window workspace of 720 pixels x 500 pixels,
so multiplying existing x-values by 9 and existing
y-values by 22 nicely translates character-based
coordinates to Web.
Excerpt of
COBOL code from WEBDX
Define xpx and ypx
01 xpx.
02 xnumpx PIC
9(5) VALUE ZERO.
02 px PIC X(2)
VALUE "px".
01 ypx.
02 ynumpx PIC
9(5) VALUE ZERO.
02 px PIC X(2)
VALUE "px".
Translate positioning
from character co-ords
MULTIPLY xraw BY 9
GIVING xnumpx.
MULTIPLY yraw BY
22 GIVING ynumpx.
Example of
call to WEBDX for a label
MOVE 4 TO xraw. *
field x co-ord - row
MOVE 2 TO yraw. *
field y co-ord - col
MOVE 0 TO Width. *
field width in pixels
MOVE 0 TO Height.
* field height in pixels
MOVE
"Supplier Code " & X'00' TO sText.
CALL
"Webdx" USING BY CONTENT xraw yraw
Width Height
BY REFERENCE sText
NULLsID NULLsClass NULLsAttrs StringSegment
StringLength.
D
H T M L,
dynamic creation on the fly of page elements.
With our Web
application page able to have an ongoing conversation
with the server, it is possible for that page to be a
standard blank slate, like a terminal, with the
server able to rewrite part or all of it (FBF
keywords "IREPLACE", "INSERT").
The issues around control of user freedom of
navigation are solved at a stroke. Here the entire
application consists of only one template page and
the user is locked down to that page. Leaving it or
closing it cues the application to log off and
disconnect.
COBOL-TO-WEB
SERVER METHODS, PERSISTENCE, NON-PERSISTENCE
A Web page talking to
a COBOL application requires a Web server in between,
acting as a relay station. This article has been
about the Web-server-to-Page step of a two-step
transmission. The question arises, what about that
other COBOL-to-Web-server step? Of course we use our
own product "PEWeb" software for this.
PEWeb addresses the second issue, while our FBF
addresses the first issue. PEWeb a
"socket-with-attitude," small and light
enough to be persisted in the Session Object of the
IIS Web-server. When run this way it can maintain a
terminal-like, "persistent" ongoing
conversation with conventional COBOL programs. These
programs converse with the Web page with the familiar
DISPLAY and ACCEPT statements. This makes the Web
server role quite minimal with only three or four
template scripts required and the entire Web
application being run with COBOL. Such a
configuration does mean that each connected Web
application user requires a licensed port for the
entire user session. The common non-FBF-FORM-
Web-app-based approach for all its complexity does
offer the sweetener of saving on licensed ports, but
by coding your own make-and-break technique a user
may run more end-users than licenced ports. Otherwise
if you have 100 ports, only 100 users can
connect.because you only need them for the moment the
user finishes and submits an entire form.
Field-by-Field is
probably most needed for expert data-entry by
in-house employees, or an elite group of trusted
out-side-world associates, for whom enough licensed
ports would already exist anyway for their current
terminal work.
So could the
advantages of Field-by-Field and
licensed-port-savings be combined? Yes.
Non-persistence (and port sharing) can be achieved by
storing control data between "make and
break" connections. The WebServer
"Session" object can act as such a
datastore or provide a SessionID which COBOL
programmers can use as an ID key for their own
datastores.
FBF Summary
- 75 % of business
data is still processed in COBOL (Source:
Gartner).
- There are 180
billion to 200 billion lines of COBOL in use
worldwide (Gartner).
- 15 % of new
applications are still written in COBOL
(Gartner).
- Replacement costs
for COBOL systems, estimated at $US25 per
line, are in the hundreds of billions of
dollars" (Tactical Strategy Group).
So we view this FBF
model as an effective and inexpensive way to upgrade
COBOL applications with a new graphical user
interface. There is a small learning curve as COBOL
experience is fused with the simple to learn
scripting language DHTML.
The initial investment
is small, and legacy applications can be migrated
over a period of time rather than taking a "big
bang" approach. With PEWeb you can extend the
life of your COBOL applications by years.
We regard the
Field-by-Field protocol, scripting source code and
Web server templates as supporting examples and
documentation for PEWeb. PEWeb software is sold with
these components included. We hope that the
"FBF" protocol develops into a commonly
used model for web enabling (GUI-izing), within many
text-based legacy environments.
A companion product
PEDirect is also available. This is a proprietary GUI
terminal emulator which handles the FBF protocol in
exactly the same manner as a browser, but it connects
directly to COBOL rather than through the webserver.
More details available
from Tantus Software (USA/Canada agents). Working
Examples at: http://24.163.65.111/demo.htm
The FBF protocol is
published online at: http://www.pixieware.com/PixieWeb/PWDocu7.htm
If any enthusiast,
outside of PICK or COBOL reads this article, and
would like to try FBF and PEWeb software, we welcome
testers on other text-based language environments
where B2B Web sites may be developed with this
technology. Promising tests on VAX basic and C are
underway at the time of writing.
John Calder is
chief designer and developer of the PixieWeb software
(now PEWeb for COBOL) from Pixieware Software.
Pixieware Software is a developer of legacy systems
rejuvenation software. You can find out more by
visiting www.pixieware.com. (This article has
been modified from the PICK original, for COBOL, by
Dave Johnstone of Pixieware Software.)
