I have often been told by my fellow programmers that I am not following "best practices", therefore everything I produce can be regarded as crap and should be ignored. This criticism is unjustified as I *DO* follow best practices, just not the same ones as them. There is no such thing as a single set of practices which every programmer must follow, just as there is no single true religion and no single master race. Those of you who follow a set of rules blindly and without question are in serious danger of becoming nothing more than a bunch of Cargo Cult Programmers. I am a pragmatist, not a dogmatist, therefore I choose to follow only those practices which produce the best results.
When Robert C. Martin, known as 'Uncle Bob', first published his Single Responsibility Principle he stated that he was consolidating the ideas of Separation of Concerns and the the notions of Coupling and Cohesion which were introduced by Larry Constantine, and amplified by Tom DeMarco, Meilir Page-Jones and many others. These can be described as follows:
It should therefore be obvious that software which achieves high cohesion and loose coupling is better than software which achieves low cohesion and tight coupling. That is precisely what I have achieved with my framework, so I regard all accusations of the contrary to be baseless and without foundation.
The purpose of this document is to describe how, in a database application, the use of a separate class property for each column in a database table is a bad practice simply because it promotes tight coupling.
Every programmer is taught that using a separate class property for each table column is standard practice, and they follow this rule automatically without thinking. It's the "without thinking" part which separates the boys from the men, the wheat from the chaff, the chumps from the champs, and if you don't examine all the possibilities then how can you be sure that you are dealing with the situation in the optimum way? When dealing with data which comes in from external sources, such as the user interface or the database, you should first ask yourself one simple question - does the data arrive in separate pieces, or does it arrive in chunks?
I have written database applications in three different languages - COBOL, UNIFACE and PHP - where all the data from external sources arrived in chunks known as records, structs or composite data items. The only type of application I can think of where the data arrives in separate pieces is something like an aircraft control system where the aircraft is covered in sensors which take readings at random intervals and sends them to the central processor in real time. It does not accumulate the data so that it can be sent in chunks. As each sensor detects a change it send the new reading to the central processor. It does not assemble a batch of readings and then send them in one go.
While software, such as UNIFACE, responds to mouse events in what is known as character mode, other software, such as that using VPLUS forms on the HP3000, runs in block mode. This means that input fields on the form are not sent to the program until the ENTER key is pressed. For HTML forms the ENTER key has been replaced by the SUBMIT button.
When I began teaching myself PHP by running through the sample code which I found in books and online tutorials I noticed that everyone seemed to have the habit of disassembling the $_POST array into into component parts before loading those parts into separate class properties, as shown in Example 1 or Example 2 below.
Example 1: POST array disassembled
<?php require 'classes/person.class.inc'; $dbobject = new Person(); $dbobject->setUserID ( $_POST['userID'] ); $dbobject->setEmail ( $_POST['email'] ); $dbobject->setFirstname ( $_POST['firstname']); $dbobject->setLastname ( $_POST['lastname'] ); $dbobject->setAddress1 ( $_POST['address1'] ); $dbobject->setAddress2 ( $_POST['address2'] ); $dbobject->setCity ( $_POST['city'] ); $dbobject->setProvince ( $_POST['province'] ); $dbobject->setCountry ( $_POST['country'] ); $dbobject->insertPerson($db); if (empty($dbobject->errors)) { $result = $dbobject->commit(); } else { $result = $dbobject->rollback(); } // if ?>
Example 2: Passing each column as a separate argument
<?php require 'classes/person.class.inc'; $dbobject = new Person(); $dbobject->insertPerson($_POST['userID'], $_POST['email'], $_POST['firstname'], $_POST['lastname'], $_POST['address1'], $_POST['address2'], $_POST['city'], $_POST['province'], $_POST['country'])); if (empty($dbobject->errors)) { $result = $dbobject->commit(); } else { $result = $dbobject->rollback(); } // if ?>
I could only assume that everybody did it this way because they were taught to do so, and nobody dared to question was was being taught by their supposed superiors. I was not taught to do it this way, so I examined the possibilities with the intention of making my own decisions. I had already begun to see that in PHP arrays were simple yet powerful and flexible, so I wondered if I could pass in the array in one piece instead of breaking it down into its component parts, and what the effects would be. A quick five minute test showed me that I did not need to address any of the component parts until I was inside the object which processed the array, so I removed all the unnecessary lines as shown in Example 3 below:
Example 3: POST array left intact
<?php require 'classes/person.class.inc'; $dbobject = new Person(); $dbobject->insertPerson($_POST); if (empty($dbobject->errors)) { $result = $dbobject->commit(); } else { $result = $dbobject->rollback(); } // if ?>
This resulted in less code outside of the object to load in its data, while inside the object it is just as easy to access each column using $fieldarray['column'] as it is $this->column.
Note that the $_POST array is references as $fieldarray on all method signatures within the abstract table class.
A novice programmer might think that saving a few lines of code up front is simply not worth the effort of breaking a long-standing tradition and upsetting the delicate sensibilities of a bunch of ill-trained dogmatists, but as I spent more and more time on my prototype I discovered that this single decision had a snowball effect and allowed me to implement more and more labour-saving techniques. There were several tasks I could achieve more easily that would otherwise have been impossible if not more difficult.
When creating a database application the first thing you should recognise is that you are writing software which communicates with entities in a database, and those entities are called "tables". Each has its own set of columns (attributes or properties) but is subject to exactly the same set of CRUD operations. One way to deal with the fact that multiple classes share the same operations is to put those operations into an abstract table class so that they can be shared by every concrete table class using the mechanism known as inheritance. Refer to The meaning of "abstraction" for more details.
Every database application then requires hundreds, sometimes thousands, of user transactions which either put data into or get data out of the database. These transactions all follow the same pattern in that they perform one or more CRUD operations on one or more tables.
Example 4: Standard code for an SQL INSERT
$fieldarray = $this->validateInsert($fieldarray); if (empty($this->errors) { $fieldarray = $this->dml_insertRecord($fieldarray); } // if return $fieldarray;
Notice here that unlike my junior-grade, poorly trained, ill-informed brethren I do not insist of using separate methods to load(), validate() and store() as I learned decades ago that when you always have to perform the same sequence of functions in the same sequence then it is more efficient to place that sequence of function calls in its own wrapper function. This replaces multiple function calls with a single function call, but the result is the same.
Example 5: Standard code augmented with "hook" methods
$fieldarray = $this->pre_insertRecord($fieldarray); if (empty($this->errors) { $fieldarray = $this->validateInsert($fieldarray); } if (empty($this->errors) { $fieldarray = $this->dml_insertRecord($fieldarray); $fieldarray = $this->post_insertRecord($fieldarray); }
This demonstrates how easy it is to expand the contents of a wrapper function to perform extra processing without having to amend those places where the wrapper is called. This is especially useful when employed in an abstract class as the Template Method Pattern, which is an essential part of a framework, requires the ability to run variable "hook" methods (shown above in yellow) at pre-set places in the processing sequence.
After creating a class for each entity it is necessary to have a separate module which instantiates that class into an object so that it can call any of the methods on that object. Using nomenclature from the Model-View-Controller design pattern the entity is known as the Model while the module which calls its methods is known as the Controller. Example 3 shows a Controller which communicates with the Person entity. This Controller is tightly coupled to the Person entity because it specifies "person" in both the class name and the method name. This means that it cannot be used with any other class. Out of curiosity I wondered if I could reference a class using a variable name instead of a hard-coded name, as shown in Example 6 below:
Example 6: Loading the class name into a variable (a component script)
<?php $table_id = "mnu_user"; // identifies the model $screen = 'mnu_user.detail.screen.inc'; // identifies the view require 'std.add1.inc'; // activates the controller ?>
Example 7: Referencing the class name from a variable (a controller script)
<?php require "classes/$table_id.class.inc"; $object = new $table_id; $fieldarray = $object->insertRecord($_POST); if (empty($object->errors)) { $result = $object->commit(); } else { $result = $object->rollback(); } // if ?>
You will notice here that I have changed the insert<entity> method into the more generalised insertRecord method as this can be used on any entity within the application. Every Model class inherits the same set of common table methods from the same abstract table class in order to support the standard CRUD operations which apply to every table in the database. As every Controller communicates with its Model(s) using methods which are defined in the abstract class this means that every Controller can be re-used with any Model. I have 45 reusable Controllers, one for each Transaction Pattern, which can be used with any number of Models, so that is an enormous amount of reuse.
Because of my 20 years experience before switching to PHP I was well aware that all user input should be validated before it is written to he database, but I was surprised to see so much hand-written code was needed to perform this validation in all the PHP code samples which I encountered. The same pieces of logic were being used over and over again with different property names, and I immediately saw the potential savings by defining the logic only once and then calling it separately for each property. This was made easy by having all the table columns (properties) in a single $fieldarray variable which was used as both an input and output argument on every method call with the abstract table class. This array contains a series of name=value pairs, so I spotted straight away that if I had a second array of name=specification pairs, where specification is itself an array which identifies the specifications for each table column (type, size, decimal places, minimum/maximum values, etc) then it would be a simple matter to write a standard routine to compare the values in $fieldarray with the specifications in the $fieldspec array.
I already knew that it was possible to extract the details of each table's columns directly from the database and store it in a format that was accessible programmatically, so I use this knowledge to construct my $fieldspec array. This then enabled me to write a standard validation class which is called automatically from within the abstract class.
When validating user input it is easy to check that each piece of data conforms its column specifications in the database as there is always a 1-to-1 link. But what happens if you wish to compare data from multiple columns? Even with RADICORE you have to do this with custom code, but how where do you put this custom code and how do you call it? The use of an abstract class provides the answer in the form of the Template Method Pattern. The abstract class contains a mixture of invariant methods with fixed implementations interspersed with a number of variable or "hook" methods with empty implementations. These "hook" methods are called automatically in the superclass, but do nothing unless they are overridden by customised methods in a subclass.
Example 8: Example of an empty hook method in the abstract class
function _cm_whatever ($fieldarray) // perform custom processing. { // custom code goes here return $fieldarray; }
Note that every hook or customisable method has the "_cm_" prefix. The developer never has to write any code to call a "hook" method as the necessary code already exists within the abstract class. This ensures that the code adheres to the Hollywood Principle (don't call us, we'll call you) which differentiates a framework from a library.
Example 9: Example of a hook method in a subclass to provide custom validation
function _cm_commonValidation ($fieldarray, $originaldata) // perform validation that is common to INSERT and UPDATE. { if ($fieldarray['start_date'] > $fieldarray['end_date']) { // 'Start Date cannot be later than End Date' $this->errors['start_date'] = getLanguageText('e0001'); // 'End Date cannot be earlier than Start Date' $this->errors['end_date'] = getLanguageText('e0002'); } // if return $fieldarray; }
By looking at Example 1 and Example 2 above you should see that the code can only deal with a specific set of columns that belong to a single specified table. This means that the script must have prior knowledge of what columns it expects to be passed to it. If different sets of columns are involved, such as with a partial update, then different versions of that script will be required for each different set of data. It should also be noted that it is not possible to load data into a property which does not exist in the target object.
This mirrors the way in which in which the forms worked in my earlier languages. Each form or screen was a mixture of labels and editable fields, had to be defined and compiled before it could be used and loaded into memory before it could be processed, after which you could write data to it or read data from it using a single pre-defined and pre-compiled data buffer. Each form therefore required its own buffer to match its data, and it was up to the programmer to ensure that the buffer used by the program was an exact match to the buffer used by the form.
The HTTP protocol does not use compiled forms, and a PHP program does not need to know the structure of the form, or even its identity, before it can receive data from the client browser. All data is received in either the $_GET or $_POST variable, which are both arrays. Unlike programs which require a static structure, the array in PHP is a dynamic structure whose contents do not have to be predefined. You do not tell PHP what data to expect, the request is sent from the client to a nominated script, and it is up to that script to process whatever data it has been given. When building or refreshing the screen the entire HTML documented, not just the data, is returned in full to the client.
It is usual for the contents of an HTML document to match the contents of a single database table, but it is possible for an update to only specify a subset of a table's contents. It is also possible for the client to send data which needs to be spread across several tables. By tying each table class to a specific set of columns, as shown in Example 1 and Example 2, where each column has its own property, it is not possible to deal with a varying number of columns. It would require different sets of scripts, both calling and receiving, to deal with each combination of columns.
By leaving the GET/POST array intact RADICORE does not need separate Controllers or separate methods to deal with these variations. It does not matter how much or how little data is
passed from the Controller to the Model as no columns are identified in the Controller. It is up to each Model to process whatever data it is presented with in its input array, which is normally called $fieldarray. The software knows what columns exist in each table simply by examining the contents of the $fieldspec array. If a column is missing from this array then it does not exist in that table and therefore is filtered out before being sent to the Data Access Object where the SQL query is constructed and executed. If a column is missing from $fieldarray there there is no data to be processed for that column.
If data in $fieldarray needs to be sent to an additional table then it must be processed one table at a time. This will require code in either the _cm_post_insertRecord or _cm_post_updateRecord methods of the initial table class similar to that shown in Example 10 below.
Example 10: Writing data to an additional table
function _cm_post_insertRecord ($fieldarray) // perform custom processing after database record has been inserted. { $dbobject = RDCsingleton::getInstance('other_table'); $other_data = $dbobject->insertRecord($fieldarray); if ($dbobject->errors) { $this->errors = array_merge($this->errors, $dbobject->getErrors()); } // if return $fieldarray; } // _cm_post_insertRecord
Note here that it is not necessary to filter the contents of $fieldarray so that it only contains the data that is relevant for 'other_table' as anything which does not belong in that table will be automatically ignored.
In my COBOL days I worked with hierarchical databases such as Data General INFOS and network databases such as HP IMAGE, neither of which used an SQL interface. Instead each operation used a specific API which had three arguments - the table name, a list of columns, and a data buffer to hold the values for those columns. All these arguments were compiled, static and unchangeable. JOINS to other tables were not possible, so it was necessary to retrieve data from different tables separately using different procedure calls with different arguments.
The databases available to PHP are all relational and therefore use the SQL protocol. This does not use static APIs as every CRUD operation is performed using a single API and a single query string. The result of each query is not a static structure but a dynamic array.
RADICORE uses a single Data Access Object to access its database, with separate versions for each supported DBMS (currently MySQL, PostgreSQL, Oracle and SQL Server). This object can construct and execute any query for every table in the database, and the result is always a multi-level array, an indexed array of rows and an associative array of columns for each row. By leaving this array intact I do not have to waste time writing code to transfer the contents of this array into individual class properties, which would make it impossible to deal with those columns which did not have properties because they were not part of the table..
When application data is extracted from each table object for inclusion in the View module it is done so with a single call to the getFieldArray method which returns all the application data. It does not matter how the data got into the array, just that it is there. Note that any named column which does not exist in the current screen structure file or report structure file will be ignored.
While my set of Transaction Patterns initially covered the ADD 1 and UPDATE 1 patterns to handle one record at a time, I found that as I began to encounter several scenarios which needed to handle multiple records at a time. This led to the creation of the ADD 4, ADD 5 and ADD 7 patterns which use the insertMultiple method, and the MULTI 2, MULTI 3, MULTI 4, MULTI 5 and MULTI 6 patterns which use the updateMultiple method.
Later another complication arose when I found that I needed to allow different processing on different rows, but the solution I devised was incredibly simple. This was based on the fact that having a single $fieldarray variable to contain all the application data, both from the user interface and the database, meant that I can add as many values as I liked without causing the current operation to fail. Thus I could add a flag or switch to a row's data in order to alter how that individual row was processed. These flags all have a prefix of 'rdc_' to indicate that they are there for the RADICORE framework to process and are not part of the application data. These are listed in FAQ89: What reserved words exist within the RADICORE framework?. An example of their use is shown in Example 11 below:
Example 11: Using flags to alter how each row is processed
// this code is within the updateMultiple method
foreach ($fieldarray as $rownum => $rowdata) {
if (array_key_exists('rdc_to_be_ignored', $rowdata)) {
$unchange_count++; // do nothing with this record
} elseif (array_key_exists('rdc_to_be_inserted', $rowdata)) {
$fieldarray[$rownum] = $this->insertRecord($rowdata);
if (!empty($this->errors)) {
// perform error handling
} elseif ($this->numrows > 0) {
$insert_count = $insert_count + $this->numrows;
} // if
} elseif (array_key_exists('rdc_to_be_deleted', $rowdata)) {
$fieldarray[$rownum] = $this->deleteRecord($rowdata);
if (!empty($this->errors)) {
// perform error handling
} elseif ($this->numrows > 0) {
$delete_count = $delete_count + $this->numrows;
} // if
} else {
$fieldarray[$rownum] = $this->updateRecord($rowdata);
if (!empty($this->errors)) {
// perform error handling
} elseif ($this->numrows > 0) {
$update_count = $update_count + $this->numrows;
} // if
} // if
} // foreach
As stated in the introduction, OOP best practices are supposed to have their roots in the ideas of Separation of Concerns, High Cohesion and Loose Coupling, but as far as I can see the practice of splitting the input arrays into their component parts so that each part can be loaded into its own class property, as shown in Example 1 and Example 2, causes nothing but tight coupling. This leads to the ripple effect when dealing with database changes. By specifying individual column names in method signatures you are forcing those signatures to change when the list of columns changes, so when you make changes to this list you have to make corresponding changes to all places where those method signatures are referenced.
The RADICORE approach of leaving the input array intact means that I can make an infinite number of changes to the contents of that array and I don't have to modify any method signatures. It is not possible to have a level of coupling which is looser than that.
The advantages of not disassembling the POST array can be summarised as follows:
$fieldarray and all the column specifications available in a single $fieldspec array.Do still think that having a separate class property for each table column is a good idea?
Here endeth the lesson. Don't applaud, just throw money.
Here are some other common practices which, in my humble opinion, do not deserve to be called "best practices":