How Important is the Role of Testing?

Simon Morris, Research & Development Director

Introduction

In such an uncertain economic climate organisations need to be far more competitive, one simple way to stay ahead of competitors is for businesses to make sure their products get to market first. This can mean a business has an advantage over its competitors by being the initial occupant of a market segment. This advantage can be based purely on the fact that the first entrant can gain control of resources that followers may not be able to match. Sometimes however, the first mover is not able to capitalise on its advantage, which means the opportunity is there for another organisation to gain ‘second-mover advantage’.

This theory plays heavily in the technology testing arena. Organisations need to develop a balance between being first to market and being the best in the market. In most businesses the sales and marketing team will push for new products to be ready in time for certain market conditions, to be ahead of competitors and to position the organisation as the leader in its market place. This however is somewhat of an issue when it comes to testing security products because the two tactics do not sit well together. Testing is not a process that can be rushed, it is essential for businesses to make sure their products meet their customer’s objectives and needs and yet the product needs to be ready in time to ensure a competitive edge. So how best can an organisation realise this balance between competitiveness and product accuracy?

Getting the Right Balance

Organisations need to take a semi formalised approach to testing in order to keep a ‘real world’ aspect. Many organisations in the past have used mathematical testing, which has proved a product to be accurate and safe to use however once the human factor has been introduced to the equation the product has crashed. Some organisations do not use validation testing and instead release a BETA so that they can make corrections as they go along. These tactics only work to a certain extent; organisations need to prioritise the risks involved with the product and how accurate the testing needs to be e.g. online banking products must be safe and have no bugs/vulnerabilities before they are introduced to customers and so must undergo a meticulous testing process.

Organisations also need to ensure that the testing process is as efficient, accurate and fast as possible. In many cases and particularly programming, there are millions of lines of code that have been created and written by a programmer who is no longer with the company and so the code is almost impossible to understand, even if a bug is found it is extremely difficult to fix and so the new programmer needs to start from scratch again.

Testing is hugely important when developing a new product and many issues can be easily overlooked in the rush to get a product to market. Unfortunately testing is often seen as an overhead as organisations are too eager to reach the end product and cannot see the tangible return-on-investment testing can bring.

Communication is also imperative in the product development process; many programmers still struggle to articulate their ideas and plans and very often find themselves under pressure from marketing and sales to deliver a product far quicker than it should be. If it was up to programmers to get products out to market however it would never happen. There needs to be a clear middle ground/compromise between these two business departments. Getting it right is the bouncing point.

Importance of Testing

Firstly an organisation needs to test the stability of a new product, but simple questions such as does it do what it says on the tin and does it do what the marketing and sales department has asked for; are very often be overlooked. Also the ease of use of a product is vital, it needs to be aimed at the right audience because even if it is a remarkable piece of code if it’s not something that can be used easily by the customer then the project objectives have not been met.

One of the main reasons Apple has been a success is because of its focus in the early days on the human computer interaction aspects of its products. Apple tested its user interfaces to assure they could be used by anyone and everything was where a user would expect to find it. Businesses must follow this example to keep customers satisfied.

To ascertain the quality of a product perimeter checking is vital situations where a perimeter that allows a user to enter a number between one and ten must be tested for situations where a user enters a number outside of that range as this can crash the product and it must be able to cope with irregular input.  On the flip side of this products must also be tested on its data validation. Here at Pentura we have seen examples in the past where some of the leading fire wall products allow users to enter any kind of data and only at the end of the process does the product crash because the data was wrong. Organisations must ensure the information that users enter, no matter how random, does not break the product.

There is tremendous pressure on software engineers to get new products out quickly and, at Pentura the aim is to catch as many problems as possible before a product goes to market. Organisations today are realising the value in a carefully planned product development process and the value in testing all possibilities. Many of the exploits on banking sites that are banded around the media are from bad coding. Banks have realised this over the last few months and are now investing in accurate code checks.

How should an organisation implement testing accurately?

In order to make the product development process as streamlined and efficient as possible there are a number of tactics an organisation can use. With time being a very important factor in product engineering it is not possible for businesses to stay competitive if products need to be constantly redeveloped and bugs removed. With so many products released way ahead of time without adequate testing the only way to stay ahead of competitors and maintain customer satisfaction is to take the ‘second-mover advantage’.

A well respected and recognised way to manage the software lifecycle to follow the seven stages of the ‘Waterfall Model’, a sequential software development process so called because the product development progress is seen as flowing steadily downwards, like a waterfall through the seven phases of

1. Requirements specification

2. Design

3. Construction (AKA implementation or coding)

4. Integration

5. Testing and debugging (AKA Validation)

6. Installation

7. Maintenance

Each of these phases must be completed accurately and precisely before moving on to the next. To ensure the testing process is not seen as the only section in the product development process to filter out any bugs and analyse the success of the product, each phase must be completed correctly before moving on to the next. The more time spent in the early stages of a software production cycle the better the results and cost efficiencies at the later stages. It has been shown that a bug found in the early stages such as requirements specification or design is cheaper in terms of money, effort and time, to fix than the same bug found later on in the testing phase.

It is very difficult however to ensure every phase of a software product's lifecycle is perfected, this is why testing is still a very important and necessary step in the product development process. One way of ensuring a product is meeting the original objectives is to break the process down into smaller projects or ‘quick wins’. This allows developers to clearly see if they are on target to deliver on customer objectives at the end of each of the smaller projects.

Some long term software engineering projects can last years and as new programmers come through the business, objectives can get misinterpreted and misunderstood, the code that was originally written is difficult for another programmer to translate and understand and very often projects have to started again from scratch. It makes sense to break large projects into smaller sections so there are clear benchmarks where objectives can be reviewed, re-set and monitored and new programmers can be introduced with minimal disruption.

Another way of minimising time spent developing and writing new code is to use smaller sections that have been implemented before and tested successfully and then interlink these with new code.

A business can develop lots of libraries of code to be more holistic. This creates a type of jigsaw of lots of smaller product parts that a business may need at a later date giving it the building bricks for future products of the projects you might need to do.

There are no regulatory measures for writing code, it is generally understood and an unwritten rule that programmers will annotate and document their coding so that should another programmer need to edit, and develop their code further the annotations will allow them to do this with ease.

Conclusion

The issues of time and understanding the ins and outs of product development will not be fixed over night, the best way to ensure that product is ready to go to market is by setting up targets and milestones to monitor the progress of a product, this can be done using the modulisation options in tools such as Pascal and Modular 2. ADA a defence language promoted this modularisation plan so that they didn’t have a million lines of code that were impossible to break up.

It is important to put the final product into perspective, if it’s not going to be deployed into a high risk environment then testing is not as essential and perhaps the BETA technique can work well in this environment. In high risk environments it is essential to spend more time testing because the impact of a bug can be catastrophic in the banking industry.

In a perfect world there would be more staff and time available to test and ensure consistency. Consistency is important to ensure everyone can understand the objectives. Marketing and product engineer departments need to set realistic time allocations to ensure they manage customer expectations well.

Software testing should provide accurate information about the quality of a product or service with respect to the context it is intended to operate in. Software testing should also provide an objective and independent view of the product to allow an organisation business to appreciate and understand the risks involved in implementing the software. It must validate and verifying that a product meets business and technical requirements that were agreed in the early stages and it must work as expected.

Testing can never completely highlight all the bugs and faults within a product. Instead, it identifies how well and product will work in a particular environment. Every software product has a target audience and when an organisation develops or invests in a software product, it must ensure the product will address the needs of its end users, its target audience and its purchasers.

Pentura is exhibiting at Infosecurity Europe 2010, the No. 1 industry event in Europe held on 27th – 29th April in its new venue Earl’s Court, London. The event provides an unrivalled free education programme, exhibitors showcasing new and emerging technologies and offering practical and professional expertise. For further information please visit www.infosec.co.uk

Courtesy: Eskenzi PR