How to Test A SOA System W6 Carsten Feilberg, Strand & Donslund A/S, Denmark
How to test a SOA system Carsten Feilberg Strand & Donslund A/S Hannedal 9 DK-2860 Søborg www.s-d.dk
What this talk will address Background setting the scene Understanding what SOA means What s in an environment, anyway Organise it all So why is it so slow, then? Rounding up and.. questions
Understanding what SOA means A service is something, which is independant of other (external) services and which offers some kind of functionality. Service Oriented Architecture is the principle of basing a systems functionality upon a number of services.
Understanding what SOA means The user = the tester sees the system as a common user interface we don t see the mess of services behind it. Alive-Test shows the services are open Smoketest shows the client is hooked up on the services and then you re ready to start testing
Understanding what SOA means What goes into such a system? Standard systems Std-frame systems Customized ( home made ) Very static probable not able to create or alter existing services. Take it or leave it -systems. More customizable, but within certain limits. Can be extended and tailored to most of our needs Completely open for change Will often be (mis-)used to make up for the failure of other subsystem Solutions
Understanding what SOA means Data ownership All systems are maintained in their own rights we have no influence on that. However by usage we need some data not to change unnoticed.. Synchronizing We cannot allow some subsystems to fail synchronization. And in general we have only few means as to figure out if it has happened. State of the system It has no meaning to talk about the system being up or not. Subsystems, yes, but not the system itself. What part of the system may be down, and still allow some value? Performance Distributed Virtual Data Model We cannot allow the product subsystem to delete product X, if it s involved in the sales subsystem but the two systems may not know of each others existance. Back up and restore (recovery) All sorts of problems pop up at a closer look
What s in an environment, anyway Simple (non-working) environment model SOA model Production Production Staging? User Test Test Test Test (may use staging instead) Stable Dev Stable Dev Development Development Development It all depends on what you d like to do at any given time. You can do with less, but not much less. What do we connect to when we address foreign services? And what s Staging connected to? Are all subsystems present in all environments?
What s in an environment, anyway Release Management It s a big job to find out what works together, to help coordinate between subprojects and rightout to install the systems. Remember many may have to release at the same time due to interface compliances.. Beware of external services Often you re about to hook up to something which is also in test or even development or worse: real bloody production. Release Manager is the first to receive new versions needs to know where what is has to communicate with all subprojects and help them coordinate Now does any of that sound familiar? Data in each environment We probably need to be able to synchronize data between environments especially the two low-level ones. Where do we get data from? In reality, we need to release data too.. Careful about licenses Some subsystems may require licenses with all these environments you can become very poor indeed. How do you get access to each environment? Embrace the world of virtuality.. It s a true gift for the SOA tester. Do you have to convert data as well?.. and if yes, do you have any place to do so? Mind you, conversion routines must be developed and tested too..
Organise it all Programme Programme Project Project Project Project Project Project Decentralised Efficient - inside the project Focusing on own tasks and problems Global problems are annoyances Highly specialised knowledge Would-be global problems may not be timely identified Centralised Focus on the system as a whole Able to attend global as well as local problems All problems are to be solved All have common knowledge of everything May be difficult to address problems and tasks in time Difficult while managing several contracts at different states No matter what you choose it s a matter of time only..
Organise it all Bugs A bug may be anyones problem you need a defect manager to deal with it. It s a big job. If possible, setup a common bug handling system make it open to all. Acknowledge that not all contributors may want, need or be able to use a common system. Programme Project Project Project Where are you? There s all the difference in the world sitting in one of the projects, trying to deliver it to a client, and sitting in the programme trying to make it all work together. The Free Sweeper You d like a bug to be hunted down across all the subsystems, but unless you provide someone who can do this, it will not be done.
So why is it so slow, then? It s the environment! It s the s! Someone s using the system.. Your pc is just too slow.. It s the others fault! A likely story but SOA does add some neat little features of its own merits..
So why is it so slow, then? appl db switch web broker db pc switch appl switch appl db db switch web appl db Introducing the system backbone by which packages travel all the way down and back.. Are they all configured correctly? Imagine they are all writing to extensive logs too.. How many round trips and detours?
So why is it so slow, then? There s no easy way You may have to start studying all sorts of logs, build models on white boards to ease understanding the problems, read books, search the internet, buy consultants Example (if from you can the find a.net-world good one). Talk to and involve all who may have any idea of what s going on. maxconnections = 48 / 2 maxiothreads = 100 / 20 maxworkerthreads = 100 / 20 minfreethreads = 1056 / 8 minlocalrequestfreethreads = 912 / 4 It s important to check the configuration and to observe more s together while they run the perfmon-tool is more valuable than you should think Tracking and debugging performance issues is cross-project work
Rounding up and.. questions SOA means work for testers and a lot of work. Things are mingled together and it's hard to separate them. You get problems with setting up test data, maintaining your test across different environments and figure out, where the bug actually is. SOA doesn't mean you should forget all you normally do. It all still applies. There's no need to put all your efforts into integration testing, it smells much more like functionality testing. Operations, installation and performance are crucial test areas as well. You need more roles either under the test manager, or closely connected to test: operations, installation and release management - but also someone needs to be dedicated to defect management, perhaps more than one! - you may also get a lot of value from a free sweeper, who can do the initial analysis of bugs to help figure out, what's actually wrong, and who should mend it. But - it's also great fun, and a huge challenge.