Main Menu

Keep Your Pen Tester Honest: What a Merchant or Service Provider Should Expect


As Seen in BizTalk in the St. Louis Business JournalIn 2014, the Verizon Data Breach Investigations Report analyzed 1,367 reported security incidents, nearly half of which involved POS and web application attacks. In both types of attacks, threats are overwhelmingly coming from external sources. Hackers are quickly realizing that smaller targets may not have the resources to keep their data secure.

For anyone that accepts card payments, PCI should be a familiar term and so should penetration testing. Recently, the IT Audit and Security team at Brown Smith Wallace performed an external pen test for a mid-sized company that uncovered a host of vulnerabilities — the client was shocked. Brown Smith Wallace was engaged to perform further security work, including an internal penetration test and internal vulnerability assessment.

According to the Verizon 2015 PCI Compliance Report, compliance for penetration tests (or pen tests) is only 67 percent, and overall compliance in this area is down since 2013. The PCI Security Standards Council (PCI SSC) has released more prescriptive guidance for penetration tests in an attempt to help organizations of all sizes, budgets and sectors evaluate and implement a pen testing method. The new guidance will help ensure clear expectations between customers, security testers and the PCI SSC. With the new guidance, another dip in compliance is expected as merchants, service providers, Qualified Security Assessors (QSAs) and pen testers get up to speed. The guidance is intended to mostly help the pen tester, but merchants and service providers need to be aware of certain components.

Vulnerability Scan vs. Penetration Test
The new guidelines are particularly useful as they offer detailed explanations of the differences between a vulnerability scan and a penetration test. A vulnerability scan merely uses automated tools to scan the surface of the environment, whereas a pen test adds a human layer to the attack to reveal true risks and impacts. A pen test will always be carried out by a person to exploit vulnerabilities to circumvent or defeat the security features of an organization’s system components.

What Does the Guidance Mean for Pen Testers?
The guidance is valuable for pen testers and QSAs because it sets expectations for pen tester qualifications, pre-engagement preparation (including review of past threats and vulnerabilities) and pen testing methodologies. It also sets the expectation that any findings that impact the security posture of the assessed entity should be reported.

What Does the Guidance Mean for Merchants and Service Providers?
The new guidance can help merchants and service providers keep their pen testers honest. The following is a checklist to consider when going through a pen test:

  1. Make sure pen testers are studying your environment and asking questions pre-engagement.
  2. Make sure you agree on a set of rules of engagement.
  3. If your pen testers find card holder data (CHD), they MUST notify you immediately.
  4. Pen test reports should include certain elements; the PCI SSC guidance provides a good checklist. Some of those elements are:

    1. Statement of Scope
    2. Statement of Methodology
    3. Testing Narrative
    4.  Findings
  5. The pen tester should never leave your card holder data on their machines.
  6. If personnel within your organization perform the external and internal penetration tests, then they should read and follow the new PCI guidance in full.

A QSA will focus on reducing the scope of a pen test as much as possible in order to lower costs. For example, third party systems — such as cloud-hosted websites and applications — should be considered as part of the merchant’s pen test scope. However, if the third-party service provider has an Attestation of Compliance (AOC), then their environment can be excluded from the pen test scope.

Gogel_Bill (3)For more information about penetration tests, contact Bill Gogel, IT Advisory Manager, at 314.983.1363 or



Back to Page