Cloud testing.jpg
What is different about testing using the cloud?

The arrival of the cloud computing has presented both challenges and opportunities for testing. Chief amongst the opportunities is access to flexible resources to support testing activities.

For example:

  • Affordable test environments that can be switched on and off as required
  • Speed of test environment availability and management, using features such as cloning, resetting environments back to a known state and running tests in parallel on multiple environments
  • Cheaper resource costs for computing commodities such as memory, disk space, and processing power. This allows for faster test execution and support for tasks such as performance testing or compatibility testing which had previously been cost prohibitive or too complex to use legacy resources
  • Global reach, allowing distributed testing to support test resources spread in multiple sites and simulate geographical spread of your customers

But to get these benefits, it is key to understand the model of the cloud provider. For example, where are their servers based, how they distribute the workload, how they manage data persistence, network traffic limits or modelling, security protection or restrictions and what minimums service level is provided backed by any service level agreements (SLA's)

You will also need to decide how you are going to monitor the tests, environments and any results you may need

Finally you will need to understand any architecture or technology differences between the cloud test environments and the production environments, for example network latencies, different virtualisation models and differing splits in the roles of the components of the system. This includes understanding how the cloud test environments will connect with any legacy systems.

A challenge which will catch the unwary is getting test data that satisfies both testing requirements and any security sensitivity requirements. This can place constraints on the type of cloud you could use and the location of that cloud.

What is different about testing cloud based solutions?

Many of the issues above apply equally well to testing solutions that reside in the cloud. But good practices for testing applications, solutions and enterprises also apply in the same way to cloud based solutions. Some companies fall into the trap of believing because it is quicker to change or scale a cloud solution then they could take more risks and react to problems if they arise. This can result in companies underestimating the types of problem that can impact of their reputation as well as the remedial costs of fixing something it would have cost a lot less to fix in development.

If you are testing cloud solutions, it is worth spending time on the following non-functional testing areas

  • Performance testing. In theory, cloud solutions could be limitlessly scalable. This places a greater need on performance testing the model and components of the solution compared to that required on traditional systems.
  • Security testing. There are many different types of cloud setup’s and security for some of these can present a much higher risk. So more testing in this area is required, which can be difficult in some cases, due to the lack of control of the full end to end environment.
  • Failover testing. Due to cloud models typically distributing the load, it is more likely that system components will failover to others, so this type of testing becomes a higher priority compared to the past.
  • Connectivity testing. This is particularly true of legacy systems, where connectivity and performance of these connections can be more risk prone in cloud solutions due to incompatibilities, use of code or methods that no one has tried yet, complex paths for these connections or less than clear boundaries between responsibilities or ownership of components.

Contact acutest