Tony Marston's Blog About software development, PHP and OOP

Why you should build your web application back-to-front

Posted on 6th January 2013 by Tony Marston
Introduction
Why "front end" and "back end"?
My background with Administrative/Management Applications
The "wrong" development sequence
The "right" development sequence
The Front End website is nothing but a small window into the Back End application
The requirements of the back end management application
Satisfying those requirements
The Proof of the Pudding
Conclusion
References
Comments

Introduction

This article addresses the notion that when a new business is created which requires a website, the software which is developed for that business is, more often than not, designed and written in the wrong sequence. I have been designing and building business applications for several decades (refer to My background with Administrative/Management Applications) and I can safely say that each application has three major parts:

  1. The data, and the database in which it is stored.

    The data is the organisation's most precious asset as it records (or is supposed to) all the important events such as income and expenditure, thus showing if the organisation is making a profit or a loss. Those organisations which continually make a loss do not tend to last for very long, so an accurate picture of it's financial health is vitally important.

    The data is held in a third-party piece of software known as a Database Management System (DBMS). This allows the data to be stored in structures known as tables which are subdivided into columns. The DBMS provides standard functions to get data in and out of the database, but the structure of the database has to be designed specifically by the software developers to cover the requirements of the organisation. It is important to note that the design of the database provides the foundation for the entire application - if there are flaws in the foundations then the entire structure will be fragile and liable to collapse. It is not possible, for example, to mask bad database design with clever coding. Refer to The Relational Data Model, Normalisation and effective Database Design for a description of this design process.

    While the structure of the database may evolve over time, it is possible, but quite rare, for that structure to be moved from one vendor's DBMS to another. Reasons for such a move might be cost, speed, reliability or functionality. It is a relatively quick and painless process to rebuild an existing data structure in another DBMS then migrate the data to that DBMS, although such things as stored procedures and database triggers can be more problematic due to differences in the procedural language used by each DBMS.

  2. The administrative/management application (admin app) which is used by members of staff to view and maintain that data.

    This is the piece of software which allows events such as purchases and sales to be recorded in the database, and which extracts data in the form of reports which allow management to see how the organisation is performing so that they may make any adjustments necessary to keep the profits up and the expenditure down. Every part of the database is accessible through this software, although there are usually security controls in place to ensure that each person can only access those parts of the system which are relevant to their position within the organisation's structure. Some components in this application may be quite simple, such as the maintenance of look-up tables, but others may involve the spreading and retrieval of data across multiple tables, or the processing of quite complex business rules.

    Unlike the DBMS which is rarely changed, the admin app may be changed more often. As programming languages and styles come in and out of fashion, or the requirements of the business expand or evolve, then the software may need to be changed to keep up. Sometimes existing components may be modified to incorporate changes in requirements. Sometimes additional components, such as web services for example, may be added in without the need to replace any existing components. Sometimes parts of the software may be rewritten entirely, either in the existing language or a new one, on a piece-by-piece basis, or the entire application may be replaced in one fell swoop.

  3. A website which allows purchases to be made online instead of in-store.

    Since the advent of the internet it is now possible for customers to make purchases without visiting a physical store. The events known as "enter sales order" and "record payment" are now input by the customer via the website instead of by a data entry clerk employed by the organisation. This takes care of order entry, but the follow-on process known as order fulfilment still has to be done, and this can only be done by members of staff using the administrative application (admin app).

    Although some functions such as order entry can now be performed primarily through the website instead of the admin app, it is important to note that they should still be available in the admin app. The functionality of the website is in addition to, not instead of, the administrative application used by members of staff.

Why "front end" and "back end"?

The Wikipedia article titled Front and back ends contains the following statements:

The front end is responsible for collecting input in various forms from the user and processing it to conform to a specification the back end can use. The front end is an interface between the user and the back end.

This implies that the front end is the user interface and the back end is the database, and the two go together to make a single application. Each component has its own responsibilities within the application and cannot exist without the other.

The same article also contains a slightly different definition:

In content management systems the terms front end and back end may refer to the end-user facing views of the CMS and the administrative views respectively.

This implies that the front and back ends are different views into the same application, with the front end for the end-users and the back end for the administrators. By "same application" I mean that each view accesses the same database(s) and processes the same data validation and business rules, but simply supplies a different "look and feel" in the user interface. In some cases the only shared component is the database as there are no sharable code modules, either because the code was badly modularised, or because the two ends were written in different languages. I have worked with several organisations which had legacy desktop applications for use by the administrators which could not be extended to include a web interface, so a completely new web application had to be built from scratch.

Using the second statement as a guide the customer-facing website is therefore the "front end" or "front office" while the administrator-facing application is the "back end" or "back office". Does this distinction exist in the real world? Yes, it does. If you visit a typical high street store there is an area at the front where customers can view the products and make purchases. Behind the sales counter is a door marked "staff only" which gives access to the back of the store where all the operations which support the front office take place. These support operations cover such things as deliveries, returns, inventory control, accounting, et cetera, and are much more complicated than the simple "display products, accept purchases" of the front office. If the store is part of a larger chain then some of these back office activities may actually be performed in a central head office.

More and more organisations are augmenting, or in some cases actually replacing, the bricks and mortar store with its electronic equivalent, the website. The website requires special software of its own, but this is still in addition to the software used by the support staff either in the back office or head office. This is such an important point that I shall repeat it just to make sure that it sinks in - an organisation's website will require special software, but this is in addition to and not instead of the software used by the back office/head office staff. It is also important to note that the front end website software is significantly smaller, sometimes by a factor of 100, than the back end administrative software. After all, "order entry" is a very small process when compared with "order fulfilment".

My background with Administrative/Management Applications

A lot of the current collection of front-end-only programmers seem to think that an administrative/management application is nothing more than a simple collection of CRUD screens. In this they are seriously wrong. If you are one of these people then either your application is incredibly simple or you are incredibly naive. Those of us "oldies" who have dealt with "proper" business applications which do serious work rather than simply look sexy know different.

Before I switched to web development using PHP just after the turn of the century I spent several decades designing and developing business applications, by which I mean such applications as order processing (sales and purchase), inventory/stock control, shipping, invoicing, accounting, manufacturing, et cetera. You can view the full list in my CV. These were all done using compiled languages (initially COBOL and then UNIFACE) using a variety of hierarchical, network and relational databases and were only accessible on "green screen" VDUs or desktop PCs by the organisation's own staff. Then the internet arrived, and gradually more and more organisations wanted their own websites, initially just as a showcase for their products and later to provide the ability for customers to purchase goods online without having to visit a physical store. As the pre-internet compiled languages did not have the ability to provide a web interface, the addition of a website could only be provided by developing a totally separate application in a totally different language, with the only common component being the organisation's database, as shown in figure 1:

Figure 1 - Front End web site, Back End administrative application

front-end-back-end-04 (3K)

Because these two applications were written in different languages, an old one for the desktop and a new one for the web, they usually required different teams of developers with different skills. Because they used different languages there was little or no ability to share components, so a lot of functionality was duplicated. Some languages, like UNIFACE for example, were originally developed for desktop applications but had web capabilities retro-fitted at a later date. This was achieved by moving from the 2-Tier to the 3-Tier Architecture so that a common set of Business layer components could be shared by both a desktop Presentation layer and a web Presentation layer. This meant that the same language could be used for both the web front end and the desktop back end, which provided the possibility to create components which could be shared by both applications (desktop and web). After having worked on desktop UNIFACE applications for seven years I joined a team who attempted to use the new web-enabled version to create an internet application for a major government department. This project failed dismally, so I decided to take a different approach - instead of using a language originally designed for desktop applications which had a web interface retro-fitted I would start with a language which was specifically designed for the web and see if it was possible to develop a back end administrative application with it. I chose PHP as the language, and after having familiarised myself with the basics I set about redeveloping the framework which I had previously developed in both COBOL (refer to the Functional Specification and separate User Manual) and UNIFACE (refer to the User Guide).

My experience with UNIFACE exposed me to new ideas which I wanted to use in my new PHP framework - the 3 Tier Architecture (UNIFACE), component templates, and XML processing. I disliked intensely the method that UNIFACE used to create web pages as it was very clunky and labour intensive. I would have much preferred the ability to create web pages from XML documents using XSL transformations, but sadly this was not possible.

My preliminary investigations into the capabilities of PHP proved to my satisfaction that it could handle the creating of XML documents and their transformation into HTML using an XSLT processor. It did not have inbuilt facilities to support the 3 Tier Architecture, so I designed and wrote my own (refer to What is the 3-Tier Architecture and the 3 Tier Architecture within RADICORE). It did not have inbuilt facilities for component templates, so I designed and wrote my own (refer to Transaction Patterns). I redesigned the Role Based Access Control (RBAC) system which I had first used in my COBOL framework, and also redesigned the Audit Logging system which I had first used in my UNIFACE framework. And thus the RADICORE framework was born.

The "wrong" development sequence

Today's generation of script kiddies seem to all suffer from the same problem - they seem to think that the website is the be-all and end-all of the business, and the ability to design and build sexy looking pages using the latest gimmicks in HTML, CSS and JavaScript is all they need. They usually follow this development sequence:

  1. They knock up a few fancy looking web pages, with the emphasis on colours, fonts and imagery, then try to connect these pages to a database using a server-side language such as PHP.
  2. They leave the database till last as they treat data storage as a "mere implementation detail", and consider looks to be more important than functionality. They haven't a clue about how to design a properly normalised database, and make so many mistakes that my last encounter with such people prompted me to write a new article entitled Database Design - are you Novice, Ninja or Nincompoop?
  3. They sometimes leave out the building of an admin app altogether (which is why this number is faded in figure 2) as they think that the database is going to be so simple that it can be maintained with an off-the-shelf utility program such as phpMyAdmin, MySQL Workbench, HeidiSQL or Webyog. If you are one of these people then either your application is incredibly simple or you are incredibly naive.

Figure 2 - Front-to-Back development sequence (Wrong!)

front-end-back-end-05 (5K)

The use of a primitive utility program to maintain the database instead of a proper admin app begins to cause problems when you realise that it does not have any of the following features:

A typical mistake is designing the database for events which only happen in the front end only to find out later on that this cannot handle events which only happen in the back end. Take credit card payments for example - it is wrong to assume that an order can only have a single payment, therefore in a one-to-many/parent-child relationship the payment is the "one" to the order's "many". What happens if you need to void the payment, or issue a refund? An experienced person will immediately know that a PAYMENT, VOID and REFUND are all types of credit card transaction, and as an order can have any number of transactions the relationship should be order (one) to payment transaction (many). An ugly workaround is to is to make the VOID or REFUND transaction the parent of a dummy order, and then find some way of relating the real order to the dummy order. This then makes the simple process of satisfying the query "show me all the credit card transactions for order X" a lot less simple than it would be with a properly designed database.

When these script kiddies realise that they actually need a proper back end admin app instead of a general purpose utility program, usually because of a sudden requirement for management reports, or other functionality which is never accessible in the front end (such as payment refunds), they try to build one in exactly the same style as the front end website and using exactly the same framework or tool set. Because they are inexperienced children they fail to realise that a front end website and back end administrative application are totally different animals, and a framework for building front ends is seriously deficient when it comes to building back ends. It may be possible, just as it is possible to build a house with match sticks, but everything has to be designed and assembled by hand one piece at a time. This is is a very laborious process, costly in terms of both time and money. Because each of these pieces is custom made instead of off-the-shelf there is also a strong possibility that the design and/or implementation could be faulty or inadequate, which leads to even more problems, delays and expense.

I have seen what happens when a bunch of these inexperienced children tries to design and build from scratch what is already provided as standard in the RADICORE framework, and it is not a pretty sight. Hilarious, yes. Pitiful, yes. Productive, no. When asked why they don't use a proper framework that was built just for this purpose all they can do is answer with "We are already using framework X, and we don't see any reason to use another".

The "right" development sequence (in my humble opinion)

In the introduction I identified the three major parts of a web application and their relative importance to each other. It is my belief that these parts should therefore be developed in the sequence which relates to their importance, as shown in figure 3, which is:

  1. The database design.
  2. The back end administrative application.
  3. The front end website.

Figure 3 - Back-to-Front development sequence (Right!)

front-end-back-end-06 (3K)

Some common arguments against this sequence that I hear from some programmers are:

The Front End website is nothing but a small window into the Back End application

To some people this statement is obvious, while to others it is unfathomable. How is it possible to say such a thing? What is my reasoning? What is my logic? As we are dealing with business applications which have a website, the easiest thing to do is to draw analogies between electronic shopping and bricks-and-mortar shopping. I think that even the more feeble minded among you should not have too much difficulty with that.

If a business wishes to open a store in the high street, where does it start? Does it start with that area in front of the sales counter, or that area at the back of it? How can you start selling products until you have products to sell? As well as having products on display in the sales area you will need a supply of spares (known as inventory) in the back office so that you can replenish what is sold. When you make sales you need to record what has been sold in the form of sales receipts and ensure that the funds appear in your bank account. As well as sales receipts you may also need to create invoices and credit notes. If customers do not take their purchases with them then you will need shipment facilities. If you have inventory then you also need replenishment mechanisms so that you can reorder from your suppliers when stocks are getting low. Ordering from suppliers requires purchase order processing as well as sales order processing. You will also need mechanisms for dealing with customer returns, possibly with a Returned Material Authorisation (RMA) as well as returns to suppliers, possibly with a Non Conformance Report (NCR). If you have inventory you will need mechanisms for performing stock checks so that you can reconcile what has been purchased against what has been sold and what remains in stock. And so the list goes on.

If the high street store is part of a chain then its particular "back office" might be quite small as a lot of activities may be centralised in a head office. This could mean that a lot of business rules, procedures and processes are controlled from head office and that each individual store has only a small number of activities which it handles itself.

Any experienced businessman will tell you that opening a bricks-and-mortar store in the high street is always the last thing to be done in a long chain of activities and never the first. You have to build the infrastructure before you can build the showcase. You have to finish the foundations before you start the edifice. The high street store may appear to be the most important as it is the only thing that the public sees, but the high street store is nothing without the infrastructure and support mechanisms behind it. The only time that you should be aware of the infrastructure is when it is not there, or when it fails.

As it is with bricks-and-mortar stores then so should it be with their electronic equivalents, the website. Just as a high street store cannot exist without the infrastructure to support it, neither can an electronic store (the website) exist without its administrative application. Just as in the world of bricks and mortar the infrastructure is built up first and the stores in the high street are opened up last, so should it be in the world of electronic stores. Just as in the physical world the high street stores use the procedures and processes laid down by the management team in head office, in the electronic world the front-end websites should use the procedures and processes built into the back-end management application. Just as in the physical world each high street store is merely a small representation of the business as a whole, in the electronic world each front-end website is nothing more than a small window into the much larger back-end management application - it needs nothing more than a small presentation layer which connects into the back-end business layer. Just as in the real world any high street store can receive a make-over without having any effect on the infrastructure or the head office, in the electronic world a website's presentation layer can receive a make-over without having to rewrite any of the business logic in the back-end administrative application.

When I say that the back-end management application is much larger than the front-end website, how much is "much". Is it a percentage, or is it orders of magnitude? The needs of the front-end website, the electronic store, are quite simple:

  1. Display products.
  2. Accept sales.

The needs of the back-end management application, the infrastructure, are more complex:

  1. Maintain product data - products, product categories (which may have hierarchies), product features (availability, applicability and compatibility), product prices (standard prices, contract prices), surcharges (such as as sales tax and shipping), discounts (either fixed value or a percentage).
  2. Maintain party details - customers and suppliers, organisations and people, classifications, roles, contact mechanisms (postal addresses, email addresses, telephone numbers).
  3. Contract prices and/or discounts, either for specific parties or for party categories.
  4. Customer Relationship Management (CRM) - emails received, emails sent, notes on conversations.
  5. Content Management System (CMS) - maintaining text for static pages.
  6. Maintain order details - sales orders, order adjustments, item adjustments, purchase orders, non-conformance reports, transfer orders (for inventory), refunds and voided payments.
  7. Invoicing - invoices, credit notes, PDF generation, financial reports.
  8. Shipments - customer shipments, customer returns, supplier shipments, supplier returns, transfers (in and out), packages, package contents, returned material authorisations (RMAs) and non-conformance reports (NCRs).
  9. Inventory - facilities, containers, items, receipts, pick lists, issuances, stock checks, variances, lot numbers, reorder guidelines.
  10. Management reports.

I have actually built a management application as a package which is used by several different organisations as a common back-end to each of their bespoke front-end websites (refer to The Proof of the Pudding), so I can easily compare the number of screens/functions in the front end with the number in the back end. My management application has a total of 2,000 screens while the front end websites have an average of only 20. This gives a ratio between the back end and front end of 2,000:20 or 100:1, which is the same as saying that the back end management application is 100 times larger than the front end website, or that the website is only 1% the size of the management application.

So when you are thinking of building a business with a website, be aware that the total amount of software that you will need to design and build does not begin and end with the website itself, it could be up to 100 times larger. If your budget and schedule only covers the website and you have not made any provision for the management application then you could be in for quite a shock.

The requirements of the back end management application

Once you have identified all the functions that are required in your back end administrative/management application, and designed the database to support all those functions, what else do you need? Those of us who have been around the block more than once and have dealt with a variety of business applications could come up with a list containing the following:

  1. A proper development framework which has facilities to help generate application components as well as having executables to help run your application. Most so-called "frameworks" are actually nothing more than a collection of libraries, as explained in What is a Framework?
  2. Sample applications that demonstrate the capabilities of the framework which you can run and debug to see how they work.
  3. An architecture which allows the generation of components which can easily be shared with any number of front-end websites (yes, there may be more than one).
  4. The ability to quickly create simple CRUD screens for each and every table in the database, thus doing away with the need for a primitive utility application such as phpMyAdmin.
  5. The ability to handle changes in the database structure with the minimum of effort.
  6. The ability to customise any screens quickly and easily so that you can add or remove columns, or re-arrange the sequence in which they are displayed.
  7. The ability to modify functions so that they may be able to display data from multiple tables in a single screen.
  8. The ability to modify functions so that they may be able to write to multiple tables from a single screen.
  9. The ability to add the processing of business rules or additional data validation to any screen.
  10. The ability to create functions from a collection of standard patterns so that screens with similar structure and behaviour, but different content, can be quickly created without having to re-invent the wheel each time.
  11. The ability to create a customisable structure of menus so that functions can be organised into whatever sequence and hierarchy that you decide are the most appropriate.
  12. The ability to quickly navigate from any screen to a related screen without having to step through a series of menus.
  13. The ability to automatically pass the identity of selected entries to other screens so that entries on related tables may be quickly located and displayed.
  14. The ability to add additional screens in order to perform different functions.
  15. The ability to create online help text for each screen.
  16. A logon mechanism so that only authorised users can access the application.
  17. An access control system so that permission to view or modify individual tables can be given to only those users who need to do so.
  18. A selection of additional security features which may be required for applications which deal with sensitive information.
  19. Automatic session management.
  20. An audit logging system so that you can track any changes made to the database.
  21. An optional workflow system so that you can create sequences of events which are followed automatically.
  22. Internationalisation (I18N) options which can support multiple languages with the minimum of effort.

Satisfying those requirements

You may be asking if it is actually possible to satisfy all of those requirements. I am here to tell you that not only is it possible, it has actually been done. All these features are available in the open source framework called RADICORE.

  1. A true Framework

    Unlike other pseudo-frameworks which are nothing more than simple libraries RADICORE is based on the 3 Tier Architecture which provides the following:

    RADICORE is also platform independent, browser independent and database independent.

  2. Sample applications

    Unlike other pseudo-frameworks which can only describe how to build your application and then leave you to grope in the dark, RADICORE comes with a selection of prototype applications which you can install and run. These provide working examples of most of the Transaction Patterns, so you can see how they work and debug the running code should you wish to. There is also a tutorial which gives you step-by-step guidance on how to build one of these prototypes from scratch, and an FAQ which answers common questions.

  3. Architecture.

    With my experience of various architectures I firmly believe that the best one for this situation is the 3 Tier Architecture with its separate components in the Presentation, Business and Data Access layers. This is similar to, but not the same as the Model-View-Controller design pattern. The back end application and the front end website can each have their own Presentation layers, but can share the Business and Data Access layers, as shown in Figure 12. It is even possible to have multiple front ends at the same time, such as a B2C site, a B2B site, and a web services portal.

  4. Quick generation of CRUD screens

    The process for creating basic CRUD screens with default behaviour is very simple:

    1. Import the database table into the Data Dictionary.
    2. Export each table to provide the basic class file and structure file which are used in the business layer.
    3. Generate user transactions from the library of Transaction Patterns.

    Selecting the LIST1 pattern will, for example, also create an entire family of forms containing the ADD1, ENQUIRE1, UPDATE1, DELETE1 and SEARCH1 patterns. It will also add the relevant entries to the MENU BAR and NAVIGATION BAR so that you can run the tasks immediately after they have been created without any additional effort. All this can be done in 5 minutes without having to write a single line of code - no PHP, no HTML and no SQL. Can your pseudo-framework do the same?

  5. Changes in Database structure

    Changes to the structure of database tables are not uncommon during the life of an application, and while other frameworks may require significant effort to synchronise the software with those changes, there is no such difficulty with RADICORE. The same mechanism used to create the original table class files can be used to update them:

    1. Import the updated table structure into the Data Dictionary.
    2. Export that updated structure to overwrite the structure file.

    Note that there is no need to update the original class file as it does not contain any properties or methods which need to be changed. The only time the class file needs to be amended is when it refers to any changed fields in any additional validation or business rules.

  6. Customise the default screens

    If you wish to change the default screen for any task there is no need to modify any HTML as the layout is defined in a simple screen structure file which can be found in the screens/en subdirectory with a name in the format <table>.<pattern>.screen.inc. This file either has the horizontal format for LIST screens or the vertical format for detail screens. You can change the field labels, move fields around in the data area, and add or remove fields. You are not restricted to only those fields which exist in the underlying table - any field name which can be found in the output array is valid. This includes fields obtained from other tables via a JOIN, or fields which are constructed at runtime.

  7. Read from multiple tables in a single screen

    By default the SQL query which is generated when reading from the database is SELECT * FROM <tablename>, but this can be customised. It is possible to automatically include JOINs to parent tables by specifying the field(s) you wish to be returned from the parent tables in the Data Dictionary, as documented in Using Parent Relations to construct sql JOINs. It is possible to customise the query manually as described in FAQ 8: How to extend the SQL SELECT statement.

  8. Write to multiple tables from a single screen

    By default an insert, update or delete operation will only affect a single database table, the one which is accessed by the table class on which the operation is requested. This can be overridden to include any number of updates to any number of tables using the procedure which is outlined in FAQ 147: How can I update several tables in a single operation?

  9. The ability to add business rules

    Business rules go beyond the primary data validation which simply checks that a field's data is consistent with its type and size. It may require secondary validation so that the value in one field can be compared against the value in another, or it may require adding or updating records in other tables. For example, the business rule which states "when a product's price is updated the new price must also be written to the price history table" can be satisfied by placing the relevant code in the _cm_post_updateRecord() method of the PRODUCT class. It is also possible to issue a redirect to another task so that the other task can be processed automatically.

  10. The ability to create transactions from a catalog of patterns

    Unlike front end websites where the page structures and layouts are completely free-form and can vary quite significantly between one website and another, the pages for back end management applications are limited to the display of database data in rows and columns and are therefore more predictable. The number of operations which can be performed on this data is also quite predictable (Create, Read, Update and Delete) therefore it is possible to design a series of templates or patterns which can be used to generate tasks (user transactions) with the minimum of effort. All that is required is to use the Generate PHP script function in the Data Dictionary to select a database table, select a pattern, then press the SUBMIT button to generate the script (or family of scripts). In most cases this will also add the task to the menu bar and any child tasks to the navigation bar so that you can immediately run the newly generated task(s) without further ado.

    Note that each generated task will use default behaviour and a default screen layout (although a small number of patterns do not actually have screens), but any of these defaults can be overridden or extended as required by adding code into the relevant customisable methods of the database table class(es). It is never necessary to amend any of the page controllers.

  11. Customisable Menus

    You do not have to write any code to display menu options or deal with any options when they are selected as this is handled automatically by the framework. The entries which are displayed in the menu bar are obtained from the MENU table in the MENU database which is maintained by the Maintain Menu Items task. It is possible to create a menu structure of any complexity or depth. Note that you do not have to create different menu structures for different roles - the same structure can be used by everybody because at runtime the access control mechanism will automatically filter out any options which the user is not allowed to access.

  12. Customisable Navigation Buttons

    You do not have to write any code to display navigation buttons or deal with any options when they are selected as this is handled automatically by the framework. The entries which are displayed in the navigation bar are obtained from the NAVIGATION_BUTTON table in the MENU database which is maintained by the Maintain Navigation Buttons task. These buttons can be displayed in any sequence. Note that you do not have to create different sets of buttons for different roles - the same structure can be used by everybody because at runtime the access control mechanism will automatically filter out any options which the user is not allowed to access.

  13. Automatic passing of context to child screens

    A common requirement when displaying data from database tables is displaying data from a child table which is related to a selected row in the parent table. For example, from within the LIST CUSTOMERS (LIST1 pattern) screen you may wish to see those sales orders which are related to a selected customer. This requires the following steps from the user:

    The framework will then automatically perform the following:

    The mechanics of this feature are documented in Appendix I: Passing context between objects.

    Note that details of the selected rows are *NOT* passed via the URL as this could present a security issue. This data is maintained within the session data on the server and is never sent to the client.

  14. Additional functions

    As well as creating tasks to perform the basic CRUD operations on a database table it is also possible to create tasks which display data in a different way or which perform some specialised operations. It is possible to create more than one task for a database table from the same pattern as each task has its own unique identifier. In order for a different task to execute different code while referencing a database table there are two basic choices:

    For example, as well as the basic "Update Sales Order" task you may have other updates which do the following:

    Basically there is no limitation to the custom code which can be executed. All you have to do is put the relevant code into the relevant method of the relevant class.

  15. Online Help text

    Software without adequate user documentation is not appreciated by users. Documentation which exists but is not easily accessible is also not appreciated by users. If a user is working in a screen and wishes to find out more about that particular screen, such as what it does, how it works or what values can be used then the best way to satisfy that wish is to have a button or hyperlink clearly identified as "HELP!" which the user can press. The RADICORE framework provides a hyperlink on every screen in the top right-hand corner of the menu bar.

    Creating the text which is used by this standard facility is also easy - simply use the Maintain Help Text task which is available as an option on the navigation bar in the List Task screen. This text should be relevant to the current task, but can also contain links to other documents, such as any documentation which covers the entire application.

    This is described in more detail in Appendix F: Online Help facility.

  16. Logon screen

    As this is a web application it is possible that anyone could access it if they knew the URL. The first level of security is to deny access to everyone unless they first pass through a logon screen. The creation of user_ids is restricted to the system administrator, so self registration is not an option. Although the first password is also assigned by the system administrator, users can change their own passwords at will. They may also be required to change their passwords if certain limits are passed. Passwords are stored in an encrypted form within the database.

    Addition logon options are also available:

    This logon process means that it is not possible to bookmark URLs, nor is is possible to send URLs to others.

  17. Access Control

    Instead of having a single screen which allows all the CRUD operations to be performed on a particular table, the fact that the RADICORE framework creates a separate task for each operation means that access can be allowed or denied at the individual task level, as described in Appendix B: Task Access.

    The information on the ROLE_TASK table is checked at runtime when constructing the contents of the menu bar and the navigation bar, and any tasks which are not accessible to the current user will be filtered out. This means that the user will only be able run those tasks which he/she is permitted to run, and all others will be invisible and therefore inaccessible.

  18. Additional security features

    If you are writing an application which handles sensitive data you may require a more extensive list of security features. The full list of features which are available within RADICORE is documented in The RADICORE Security Model.

  19. Session management

    Web applications use the stateless HTTP protocol which requires special handling. RADICORE session management covers all the known situations including Back Button Blues and the ability to have multiple sessions open on the same device at the same time.

  20. Audit logging

    It is sometimes required, especially when the application deals with sensitive data, that any changes made to that data are recorded in a separate audit database so that they can be viewed at a later date. RADICORE has an inbuilt Audit Logging system which is instantly available and which does not require any extra effort by any developers. This implementation differs from other traditional methods as follows:

  21. Activity based workflow engine

    Sometimes when a particular task is performed, such as 'Take Customer Order', this has to be followed by a series of other tasks in a particular sequence such as 'Charge Customer', 'Pack Order' and 'Ship Order'. Without a Workflow Engine these subsequent tasks must be selected and processed manually, which is where mistakes and inefficiencies can arise.

    RADICORE comes with an inbuilt Activity Based Workflow Engine which can be turned on if required. This engine is based on Petri Nets which deal with structures containing objects such as places, transitions, arcs and tokens. It is not necessary to modify any code to activate the workflow engine - all you have to do is define a workflow process containing a series of places, transitions and arcs, and when triggered by the activation of the starting task a new workflow case will automatically be generated. This will then cause the next workitem in the workflow process to appear as a prompt in the Menu/Home Page screen. As each transition/transaction is processed its token will be moved to the next place, thus enabling the next step in the sequence until there are no more steps.

    In this implementation neither the application nor the workflow engine are aware of the internal workings of the other. The only link between the two is that a workflow transition points to a user transaction on the TASK table.

  22. Internationalisation

    Sometimes it is a requirement that the application be made available in more than one language, either the back end, the front end, or both. It could be a very expensive exercise, both in terms of money and time, to introduce this feature into an existing application, so it would be advantageous if this capability were available from the start so that adding support for a new language would involve no more effort than simply providing the relevant translations for each piece of text.

    The internationalisation features within RADICORE are described in the following documents:

    The functions used in the back end can also be used in the front end, which means that only one source of translated text need be maintained.

The Proof of the Pudding

The proof of the pudding is in the eating, as they say, so to prove that what I have written is not just another wild theory but an established fact here are details using an application which I have developed using the ideas which are described in this document.

In October 2005 I was contacted by a web design agency called Agreeable Notion Ltd who had developed a small logistics application for a Scottish Distillery. This needed to be expanded to cover the company's growing needs, and one of the critical requirements was a Role Based Access Control system to control who had access to what. The design agency had no knowledge of such systems, but after having read my articles on the subject they asked if they could use my framework as it had not yet been released as open source. I gave them a pre-release copy, but in June 2006 they asked if I could take on the development myself as the customer's timescales were particularly tight. This I did, and the project was delivered on time and within budget.

This particular project was a bespoke system for a single client, but as the design agency had another client who also needed an upgrade to his system I was asked if I could design and build a general purpose package which could be used by a variety of different clients with different needs. The idea was to provide what looked and acted like a bespoke system but with an off-the-shelf price. This was a challenge that I could not resist, so I accepted and began work on it straight away.

My starting point was, of course, the database. I had previously come across the Data Model Resource Book by Len Silverston which contains a collection of proven database schemas for core business functions. These designs cover a wide variety of business types and a wide variety of options. There is a sample of the PRODUCT schema in one of my prototype applications. I tackled this task in the following stages:

I started this in January 2007, and by July 2007 I had completed the first iteration containing implementations for Len Silverston's PRODUCT, INVENTORY, PARTY, ORDER, SHIPMENT, INVOICE and REQUEST databases. I then set about working on the first client for this package, now called TRANSIX, a company called Wedding Rings Direct who sell custom made jewellery online. They had an existing website which used an older and less flexible database design, so the upgrade required the following steps:

Since the initial implementation for Wedding Rings Direct they have added additional front-end websites in the USA, Germany and Sweden which all share the same back office database in the UK, thus taking advantage of RADICORE's Internationalisation capabilities. Multi-currency capabilities were already built into the package, so this was handled without any problem. Other functionality has also been added, such as the sending and receiving of emails through the back office application instead of an external email client, and automatically sending out emails to customers when the status of their order is changed.

Since the initial implementation for the jewellery company the TRANSIX package has been implemented for clients in other types of business as diverse as children's clothing and photographic prints, thus proving the flexibility and power of Len Silverston's database designs.

As with any software package, virtually all the clients want some additional or different processing in certain places. The trick here is to identify when a change can be made to the core package and therefore become universally available, and where the change is so unique that it needs to be made available to only one client. The RADICORE framework can deal with these customisations as a series of plugins which are kept in client-specific directories, thus avoiding the possibility of installing plugins which should not be processed by the wrong client. These plugins can specify custom processing which can be instead of or in addition to the core processing.

Conclusion

The main points of this article, which have been proven with the success of the TRANSIX application, can be summarised as follows:

The TRANSIX application was developed according to the above principles, starting with an industrial-strength database design (thank you, Len Silverston) and a back-end management application which was developed using a specialised back-end framework (thank you, RADICORE). The same database design and management application can be used by many different organisations to support front-end websites which can be totally unique. Several websites have been built using TRANSIX as the back end, and they contain an average of only 20 screens, which is insignificant compared with the 2,000 components in TRANSIX. Because these front-end websites are relatively small and can share a lot of components which already exist in the back-end they are also very fast and economical to develop.


References


Script Kiddies

I refer to the latest generation of web programmers as "Script Kiddies" for the following reasons:


counter