Using Cascade

So when should you use Cascade? Well developers will make up their own minds, but I will outline the context within which I intend that Cascade plays a part.

Microservices

The Microservice Architecure is the preferred architectural style that many businesses are adopting to support their business processes. The reasons for this, is that this architecture closely aligns with many tried and tested IT principles.

  • Single Responsibility Principle - A component's scope is limited to few responsibilities as possible.
  • Liskov's Substitution Principle - A component can be replaced as long as it meets the original contract (semantically as well as syntactically).
  • Interface Segregation Principle - More interfaces are better than one monolithic interface.

These are classic Object Orientated Design principles, and that's not really an accident. What is true for software components running together within a process, is naturally true for processes that interact with one another over the network.

Here is a typical example:

The Microservice Architecture

And you can imagine a likely journey that this business supports. It's an online shop.

  • The user visits the website and browses the shop.
  • She adds items to a shopping cart.
  • She then decides to checkout.
  • As a part of checking out, she is offered the option of registering.
  • She enters her delivery address.
  • She then makes a payment.
  • And a confirmation email is sent.

So we now have a growing number of discrete components developed by different teams or obtained off-the-shelf that can run on different machines and that talk to each other according to defined contracts. You have the ability to deploy, update, scale and replace these components independently of each other. They can run them on virtualized hardware and on PaaS systems. You can choose different persistence stores and messaging systems. We have much greater flexibility and scalability now.

How do developers go about testing microservices in this environment? The Agile methodology calls for tests, but what kind of tests? Well, common practice is to have tests structured along the lines of the Test Pyramid, where we have many units tests, some functional tests, and a few end to end tests that test everything. If you wish, you can add manual tests which are even fewer in number. There are a couple of things to bear in mind with this picture.

  • Units within microservices that are developed within house are composed by third party tools such as Spring, and wrapped in transport level tools such as the JAX-RS implementations. So unit tests don't cover all the code in use, even though, paradoxically, they are really the only thing that provides coverage reports.
  • Some microservices are obtained off-the-shelf, so no unit tests or functional tests are possible or even likely to be required.
  • Some functionality is not implemented by Java or similar technology stack. Your option at this point is to duplicate that code in its original context or stub it out during testing. Triggers on databases, database aggregations and views, filters on JMS queues, and nginx driven circuit breakers are all examples.

So what we are left with is the prospect of either writing many journey style tests that will drive multiple microservices through many different states across the entire cluster. Which will be incredibly expensive tests in terms of time. Or we test nothing, or something in between.

Clearly, we should apply as much automation to this as possible which brings me to the Continuous Integration System.

Continuous Integration

So what happens in practice? A typically Agile team sets up a Continuous Integration System using something like Jenkins or Teamcity. This system runs the pipeline on every commit and proves that the code works. This system has two requirements:

  • Developers need to share code if they work collaboratively on the same codebase. They need a third party system to validate every change allowing them to share working code rather than sharing broken code. Timely results here are every bit as important as accuracy since code that is 95% accurate is still shareable, while code that will only be available hours later costs developers many 'man-days' over time.
  • Continuous Integration provides artefacts for release into production. Here the goal is to test as exhaustively as possible. Automation is used to increase the volume of tests that execute. Timely results are much less significant than accuracy.

So development teams setup a build pipeline that achieves these aims.

The Continuous Integration Pipeline

There are many variations on this. But in general the earlier stage meets the first requirement that allows developers to collaborate, and the second phase which can repeat at different levels of granularity, validates the subject at progressively lower levels of stubbing. Which brings me to stubbing...

Stubbing

So we are setting up the second stage of the pipeline for our subject microservice and we now have an artefact and we are going to deploy it into a environment where we will test it running against a actual database and other microservices. But we immediately run into some issues. The first major issue is that it talks to other services. We face our first big challenge. We could deploy all the services to the test environment, or only a part.

  • If we deploy all the services, we need to deploy all of their databases. There is a transitive problem in that those services have dependencies as well. We need to support those. The more services and databases we introduce, the costlier it is to deploy. Also each component introduces a technical risk that it will not behave as a state machine and will break the automated tests. (More on this later.)
  • Alternatively, we deploy a subset. This is really the most common choice as the first option often just isn't practical. Now, its a question of granularity. How big is our subset of services? And we now have to support the downstream dependencies that we will exclude from the test context. We introduce Stubbing. The tests and stubs now need to support a scenario, because often a user journey is driven by existing state within the services and not solely by user input.

So lets test the two components that are primarily concerned with our workflow; the online shop website and the shopping cart service. These two services are very likely to be developed by the same team. The other services will form part of the service backdrop the organisation offers to all their sales channels.

Testing the Online Shop

We have deployed five stubs in this example. All of the downstream dependencies have been met. And the stubs don't have transitive dependencies. They express and validate the 'Contract' that those services meet. So by integrating with the stub, we argue that we are meeting the contract's that those substituted services offer. And we can deploy and test a subset of the grand micro service diaspora.

But now our journey depends on some starting state. The starting state is related to the particluar journey we want to run. Our stubs must behave in a particular way that is specifically related to the test. So before the test executes, Cascade (any end to end testing framework actually) talks directly to the stubs and sets up behaviour.

And our subject system might modify state in other services. So those stubs should support the verification of interactions made by our subject. Cascade would then query those stubs after the execution of the test so that all of the interactions of the subject are validated.

State Machines

At this point we face our second big challenge. So far we have depicted a black box being dropped into a sand box and tested. But automated testing in contrast to exploratory testing, is very dependent on the subject system being deterministic. Most applications are not state machines. They do things on their own that change their state in unpredictable ways. A human tester would observe these changes and accept that they are valid, but an automated test does not have that capacity. The problems are:

  • Initial starting state.
  • Date based logic.
  • Random numbers.
  • Asynchronous jobs.
  • Multi threaded behaviour.

Initial starting state

The subject system often requires some seed data. The test cases you wish to execute simply might not make any sense without this starting data. The subject system might not offer a valid production method for setting this starting state. So we either:

  • Change the production system to expose a test interface that the testing infrastructure uses to coerce the data into the correct starting state. These interfaces are often very powerful, and not guarded by the normal authentication system. So these interfaces as often very dangerous, but unfortunately unavoidable as I will explain later.
  • Or the testing infrastructure connects to the test database directly and modifies its state. Some argue that this breaks encapsulation because the test code needs to interpret the database schema which is encapsulated by the subject, but I don't agree. The testing infrastructure's responsibility is to assert that the subject works. If I'm faced with the choice of having a test that breaks encapsulation or having no test, I choose to have the test interpret the underlying schema.

Date based logic

Many business processes are very dependent on date. Often some business processes don't make sense unless they occur at the end of the month, or at the end of the year. For a simple test to validate any date it needs to know what date it should have. That means either setting the date up front, or assuming the date won't change in the meantime. The latter option introduces a race condition which might make a very expensive test fail.

The temporal problem is especially acute when you have multiple microservices that are all operating together and need to agree on the date that is used.

  • You can introduce a Time Machine Service that manages time across all your microservices. This solution can suffer from performance problems. Making a network call for the time can be very expensive for a supposedly trivial thing.
  • Another option is to have each microservice support the notion of static time and offer a means to set the time to an arbitrary point. This option introduces security concerns.
  • A compromise between the first two ideas, might be best. A single service that serves time that is cached by each microservice. Each microservice would offer a test interface that can invalidate its cache.

Random numbers

Of course random numbers are the very antithesis of determinstic processes. They need to be addressed in very similar ways to dates.

Asynchronous events

These are typically jobs that execute at some later point in time. This issue is closely related to the date problem mentioned above, but there is an additional component.

Some business process might require that some other 'job' is executed at some later point in time. It is not practical to run a test that tests this end-to-end, since the test would have to wait for the job to execute, which might be a while. So this kind of thing is typically handled by querying the job store and testing if the job is scheduled. If it is, then we either modify the date, or setup another job that will execute immediately. The results of that job are then validated.

To support this kind of thing, requires the exposing of end points or the interrogation of database state.

Multi-threaded behaviour

Testing across multiple different processes is an inherently multi-threaded environment, which means you will be dealing with race conditions. This often means that we never truly have a state machine. What we have a system that is eventually a state machine, which means we need to build in a polling mechanism for synchronising our testing process with the execution of the subject system.

Summary

So what problem is Cascade trying to solve?

Modern IT infrastructure is composed on many heterogeneous processes, some developed in-house and others obtained off-the-shelf. These processes often have persistent stores, or run in non-traditional contexts, such as PaaS, or the Cloud, or as discrete processes collaborating in a Docker container.

If we choose to test this wider system, we are interested in our tests being as true as possible which means running those processes in an environment as close as is possible to the production setting. And we wish to test as exhaustively as possible. This leads us to a compromise; we use automated testing test exhaustively, but we convert our processes to state machines or stub out processes that don't conform so that the subject system becomes deterministic. We depend on manual testing for what remains.

Our interest then is in having the most efficient configuration of tests possible given the complexity of the enviroment and the cost of the tests.

And that's it for this chapter. Ready to Get Started?