Today I want to discuss how we test online travel sites. The approach to testing is quite similar to how we test online trading sites. Let’s review the basics of that approach:
Create a Threat Profile for the Travel site
Create the Test Plan
Perform the Tests
Prepare the Report
The Threat Profile is the list of all threats to the application. “Threat” is the goal of an adversary – it’s what he wants to achieve. As the owner of a travel site, the threats are what you are worried about. Thus, the threat profile for an online travel site like Orbitz or Travelocity would include these threats:
... cancels the tickets booked by another user
... buys tickets at a lower price than quoted
... buys tickets on flights where no more tickets are available
... steals the credit card details of other users
... gets the travel itinerary of all users
... changes the itinerary of other users
... deletes hotels from the inventory
The Threat Profile helps us focus our testing. It helps us pursue the attacks an adversary is most interested in. [We discuss this in more detail in our post “Why we love Threat Profiles”.]
It takes us about a day or two to prepare the Threat Profile. The test engineers then share the Threat Profile with you to get your feedback: have we missed something? Have we exaggerated a threat? The Threat Profile drives the Test Plan. For an online travel site, the Threat Profile might identify about 50 threats.
Next comes the Test Plan. The test plan first maps each threat in the Threat profile to the relevant pages in the application. E.g. for the threat “an adversary cancels the tickets booked by another user”, we identify the “cancel tickets” page as the relevant page. Once the relevant pages are clear, we then decide which all attacks to try on that page to realize the threat. For instance, to test “an adversary cancels the tickets booked by another user”, we choose Variable manipulation and SQL Injection on the Cancel Tickets page. Similarly, for each Threat in the Threat Profile, we identify the relevant pages and the attacks. We put a lot of thought to create the test plan. We have a master reference checklist of all attacks to help us - we pick attacks for the Test Plan from that checklist.
After a senior reviews the Test Plan and approves it, we begin the testing. Usually we get a few new ideas once we begin the test, we then update the Test Plan and continue the Test. The tests themselves are a combination of manual and automated checks.
When we find a vulnerability, we capture step-by-step screenshots of the attack. In the final report, we walk through the attack with the aid of these screenshots. The report describes the solution for fixing each vulnerability and also provides references to good papers on the topic.