The Plynt Certification Criteria is one year old. We're examining the cuts and bruises, re-reading customer feedback and debating the next version of the criteria. The most heated discussions are around:
Is criterion #20 "New authentication token on log in" really required?
Should we consider low risk threats in criterion #2, "Defend against Threat Profile"?
How should we certify modules that integrate with other apps?
Criterion #20 has received most feedback as being too stringent, and tough to solve. Basically, #20 insists that the token used to track an authenticated session must take a new value after login. Seems obvious, but many platforms including JSP and classic ASP do not change the value of the session cookie on login. This is vulnerable to session fixation, and a variant we described 2 years ago. The debate is whether session fixation and the variants are serious enough to make this mandatory for certification. Criterion #2 is criticized as being too broad, that it covers even low risk threats if the threat profile is really comprehensive. The debate is how to deal with low risk threats - who decides if a threat is low risk, what framework should we use. And, what's the best way to certify modules? In some of our tests, the app did not have a login page - the app was a module that plugged into other apps. In such cases, should we test the full app before certifying? Or exclude the the criteria related to authentication. It would be great to hear your thinking. Please mail me your feedback at: email@example.com. I shall keep you updated about the criteria.