8 Measuring success

 

This chapter covers

  • Determining whether your application security program is effective
  • Learning which metrics should be gathered and visualized
  • Identifying who needs to know about the program’s success
  • Getting feedback from your clients
  • Using your metrics to drive improvement

You’ve developed a program that addresses security at the different stages of the development pipeline. You have a roadmap that plots the midterm and long-term goals of the application security team going forward. But how do you know whether the program is effective and that all that hard work that you and your team have put in has paid off? Gathering metrics is a priority for any project or program to ensure that the returns are there from the initial purpose of the project. But metrics are also used to determine whether the project is on track and will complete with the expected outcomes.

For security projects, specifically, the metrics are not much different. You still want to know whether the project is on track and will have the expected outcome. However, they are also used to determine whether the processes you have are working, whether the security posture of the organization is getting better, and whether the tools you use are effective. This can be helpful if you are looking at new tools to fill a gap or a competitor to a current tool so that you have baselines and an opportunity to compare current state with a potential new tool.

8.1 What to measure

8.1.1 Measuring the effectiveness of your tools

8.1.2 Tuning the tools based on feedback

8.1.3 Measuring the effectiveness of your processes

8.1.4 Measuring the mean time to remediate

8.1.5 Optimizing the mean time to remediate

8.2 Gathering effectiveness with KPIs

8.2.1 Building the KPIs

8.2.2 Setting KPI targets

8.2.3 Driving change based on KPIs

8.3 Getting feedback

8.3.1 Getting feedback from conversations

8.3.2 Getting feedback from surveys