Quality assurance and SAS application development Search And Solve

Transcription

1 Search And Solve page 1 Quality assurance and SAS application development Search And Solve Ing. B. Dudink MBA, Drs. A. Geerts, Ir. M. Klijn, Drs. I. König, Ir. E. Lekkerkerker-Smit. In general, successful applications meet higher quality requirements then do less successful applications. Creating high quality applications serves the purposes of both the developer and the client. The writers of this essay argue for the existence and importance of quality requirements in the course of application development. These quality requirements exist next to the well known functional requirements. Introduction Who can not think of examples of SAS applications that a re functional and technically satisfactory but do not meet the clients expectations?! In the course of our work we have encountered numerous examples. For instance, a time-record sheet application that enables users to gain insight into budget use. Or a registration system that maps the geographic locations of a large company s branchoffices. Both SAS applications comply with the client s functional requirements and technical specifications. Nonetheless after a short period of time the applications were no longer being used. We believe there to be requirements, other than the functional requirements, which need to be fulfilled in order to increase the chances of success of an application. These additional requirements are being called quality requirements. Quality Requirements Many applications will be used by many different people. These people all have their own specific expectations with regard to the application. But how to satisfy all these implicit expectations? Traditionally, the only requirements that are agreed upon and documented, are the functional requirements. These are regarded necessary requirements. But other requirements, the quality requirements, are important as well. Figure 1 illustrates both types of requirements. For years the SAS Institute has offered macro s which allowed you to create and train neural nets and decision trees. These macro s were very flexible, but not very accessible. That is, if you did not understand the principles guiding neural nets and decision trees or you were no SAS-guru, there was no way you could successfully apply the macro s. (Applying them needed some serious studying, even by SAS-gurus) But since the introduction of the Enterprise Miner it is even possible for SAS-illiterates to apply neural nets and decision trees successfully. (The Enterprise Miner is a mature environment with a nice click-drag-and-drop GUI, which is, compared to any standard, very user friendly). Figure 1: Functional and quality requirements In order to be used, code for neural nets in SAS have to satisfy certain requirements. We can discern two types of requirements: functional requirements and quality requirements. Having a neural net SAS macro is satisfying functional requirements. Having a GUI accompanying the macro is satisfying quality requirements.

2 Search And Solve page 2 In general, applications should not be restricted to satisfying mere functional requirements. Applications should also satisfy requirements that exist beyond this level; these are quality requirements. Quality requirements when compared to functional requirements, are more subjective in nature (not everyone loves GUI s), more difficult to measure (is there a way to count GUI-ness). The most important characteristic of quality requirements is that they tend to promote the usability of applications. Though we address two kinds of requirements, functional and quality requirements, the difference between them can be very blurry at times. One can argue endlessly about what is a functional and not a quality requirement (or the other way around). Point is there are requirements an application must fulfil which are not being addressed by the usual rules of good software engineering, but are very important to the success of an application. These requirements are called quality requirements. Quality Requirements and Applications Development When applications are being developed it is necessary to learn which requirements are to be satisfied to what extent. Any good introductory book on software engineering will tell you how to uncover and satisfy functional requirements. Unfortunately this is not the case for quality requirements. In this section we would like to give some guidelines on how to satisfy quality requirements. In order to satisfy quality requirements, one has to identify, quantify and test these. In practice there is only a very limited set of qualities, which is relevant to the application at hand. Which set of quality requirements is considered to be relevant should be the result of the cooperation of all people (or their representatives) having direct interest in the application. As for every relevant quality one has to determine to what extend it should be satisfied by the application, and one also has to determine how satisfaction should be measured. But how to determine which qualities are relevant? Is there a list of possible qualities from which I can choose the qualities relevant to my application at hand? Yes. In fact there happen to be a lot of quality lists, which are equal in nature and serve the same purpose. One of those lists is the ISO/IEC 9126-model (see figure 2). ISO: Quality of a software product is a set of attributes of a software product by which it is described and evaluated. A software quality characteristic may be refined into multiple levels of sub-characteristics The key point is that there should be a quality model to at least the sub-characteristic level for a software product. Functionality: suitability, accuracy, interoperability, compliance, security and traceability. Reliability: maturity, fault tolerance, recoverability, availability and degradability. Usability: understandability, learnability, operability, explicitness, customisability, attractivity, clarity, helpfulness and user-friendliness. Efficiency: time behaviour and resource behaviour. Maintainability: analysability, changeability, stability, testability, manageability and reusability. Portability: adaptability, installability, conformance and replaceability.

3 Search And Solve page 3 Figure 2: ISO 9126-model Once you have chosen the qualities that are relevant to your application, you should find ways to measure to what extent qualities are satisfied. (Knowing how to measure qualities, allows you to set your standards like cut-off points) Unfortunately there is no common standard telling you how to measure and satisfy this or that quality. Here you are on your own. But we will help you by illustrating testing the user-friendliness-quality in figure 3. In order to be used as a requirement that can be fulfilled, the notion of user-friendliness must be poured into some measurable form; the user-friendly requirement must be quantified. One is very well justified to set user-friendliness to some kind of rating. Assume every user may give the application a rating on a scale of 1 to 5; and there are 10 users who can give a rating. Before the application has been fully developed everyone agrees that the ratings should add up to 35. Once the application has been developed and all functional requirements haven been satisfied, the users can test the user-friendliness of the application. Fortunately for everyone the ratings add up to 42. The application is user-friendly! Figure 3: User-friendliness After the requirements have been quantified the application can be created and subsequently tested on actually having met the requirements. The developer can do this testing as far as he has some quantified metrics at his disposal. Having passed the developers testing is one thing, getting accepted by the customer is something else. The customer himself will test the application on, by all agreed upon, quality requirements. In case of the example of figure 3, the customer will have 10 users test the application. If the rating total is at least 35 the application is considered to be user-friendly, in any other case the application must be reviewed or rejected. Quality requirements and SAS Of course, meeting quality requirements is just one of many challenges an application developer faces. Meeting these requirements should therefor be incorporated into an overall application development methodology. SAS Institute has defined such an overall methodology, the SAS Rapid Warehousing Methodology (figure 4). Though this methodology has been promoted as a guide in building data warehouses, it can very well be used for the realisation of any other construct/application.

4 Search And Solve page 4 Assessment Requirements Review Design Construction Deployment Figure 4: SAS Rapid Warehousing Methodology When trying to incorporate the stress on quality requirements in the SAS Rapid Warehousing Methodology, it is clear one should gather the quality requirements in the Requirements Phase / Gather Requirements -module (figure 5). Initiate project Gather requirements Define subjects Produce requirements doc Agree to proceed Project Plan Subject Model Requirements Documen Project initiation Document Project Plan Figure 5: Requirements Phase

5 Search And Solve page 5 Acceptance by the customer of the application according to the quality requirements can be done in the Construction Phase / Conduct Test -module of the SAS Rapid Warehousing Methodology (figure 6). Populate data warehouse Conduct test Hand over Agree to proceed Develop exploitation application Design Specification Test Plan Maintenance Document Acceptance Figure 6: Construction Phase Stretching the focus of the SAS Rapid Warehousing Methodology to quality requirements, does not call for a Gathering Requirement extension, or whatever other extension of the methodology. The SAS Rapid Warehousing Methodology is general enough to allow the extra attention to quality requirements. A Case on Quality Requirements In this section we would like to demonstrate the use of the SAS Rapid Warehousing Methodology when uncovering and testing quality requirements during application development. The application was to solve the next problem for a large Dutch bank: Our cashiers should be able to see all relevant data of a client, when given a few key-values. All relevant data are stored in an easily accessible database. In the requirement phase of the SAS Rapid Warehousing Methodology we gathered the requirements as follows: First, our client and we tried to figure out what requirements, as proposed by the ISO-model, were of interest to their objective. The important requirements turned out to be: 1. Suitability: does the application solve the problem technically? 2. Accuracy: are the cashier s questions answered correctly by the application 3. Interoperability: the application must interact with other systems (as the database is not stored in a SAS-format) 4. Fault tolerance: the application should not go berserk when for example the cashier writes a character in a number field 5. Understandability: the intended meaning of the application should come natural to the user 6. Operability: the use of the application should come natural 7. Clarity: this requirement is easily satisfied as the systems doesn t perform various different functions

6 Search And Solve page 6 8. Time behaviour: the application will be used on-line while communicating with the customer, so it has to be very fast 9. Resource behaviour: the application should not use too much resources Since the relevant requirements have been identified, it was necessary to figure out how to quantify each of the requirements, and set standards for acceptable performance (quality, that is). Requirements 1, 2 and 3 correspond to the necessary requirements as known from the functional requirements. Not satisfying these requirements means not being anywhere near to a successful application. Fault tolerance can be quantified as the number of crashes or times of not-wanted behaviour of the application when confronted with false inputs. Setting the standard was not easy, but we agreed that the application should not crash on keyboard input, nor should it crash on wrong table-input. Testing these qualifications was done using standard tests available on the net. Understandability, operability, and clarity were related to the extent to which the users were able to work naturally with the application. We rated these qualities as the number of different questions users would ask during a trial run (which should be long enough to explore the application to the fullest). This was easy to test. Time behaviour and resource-behaviour are qualities that are easy to measure on a clock or I/O-basis. Testing by the user and acceptance of the application was done during the construction phase of the SAS Rapid Warehousing Methodology. As the testing was being done it was clear to us that we should not measure understandability as the number of questions regarding it, but rather as the number of different questions or the number of different sorts of questions. As thinking of ways of measuring quality we found out we could be very creative. So we knew that Sometimes we had to be very creative in order to measure quality. Keeping this in mind we knew we should not be too strict when comparing test-results with pre-set cut-offs. Sometimes the cutoffs were stretched, sometimes they were tightened. When stretching and tightening is allowed, why bother the measurements at all? Well at first quality measurement forces some serious quality focus, and quality focus is our main focus. Second, when the test results are seriously off-mark one has a very persuasive indication of a false problem conception. Knowing you re wrong, is of course the first step to be taken when wanting to do better. Evaluating our case we can say that when developing this small application it has been important to identify and quantify the relevant requirements. Compared to the usual approach, the customer turned out to be more involved and satisfied. And Finally No doubt everybody wants maximum quality. There is also no question hardly anybody dares to define quality, as the concept of quality is always prone to endless discussions, even though it is very easy to define some quality metrics, as long as one chooses to. Therefore our message to you is: take advantage of your knowledge of the usefulness of quality requirements.

7 Search And Solve page 7 Bibliography ISO 9126, Information Technology software product evaluation quality characteristics and guidelines for their use Praat and Suerink. Inleiding EDP-auditing QAI, Effective Methods For Information Systems Quality Assurance Weerahandi and Hausman. Software Quality Measurement Based on Fault Detection Data Zeist and Hendriks. Specifying Software Quality with the Extended ISO-model Zeist, Hendriks, Paulussen and Trienekens. Kwaliteit van softwareprodukten SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. Other brand or product names are trademarks of their respective companies.

8 Search And Solve page 8