The Safety Dance: Risk Assessment for DevOps Practitioners
A presentation at DevOpsDays São Paulo 2020 by Daniel "phrawzty" Maher
The Safety Dance Risk Assessment for DevOps Practitioners DevOpsDays São Paulo 2020
The Safety Dance Risk Assessment for DevOps Practitioners DevOpsDays São Paulo 2020
Your Safety Dancer Daniel Maher Twitter: @phrawzty Email: daniel.maher@datadoghq.com Web: www.dark.ca
Your Safety Dancers Daniel Maher ← that’s me! Andrew Krug ← shout out!
What we’re going to do today
Risk: A Crash Course
What is risk? What is safety science? Kaplan and Garrick (1989) What can go wrong? What is the likelihood? (probability) How bad could it be?
What is risk? What is safety science? Classic calculation R=f(s,p,c) Risk = f (scenario, probability, consequence)
Qualitative vs Quantitative Qualitative reasoning ranks likelihood using a scale score metric. 👍 👎 Speedy Light data gathering? Easy Low precision Light data gathering!
Qualitative vs Quantitative Quantitative reasoning uses data to reason about probabilities and consequences. 👍 👎 Accuracy Time Data Relative risks (adjacent)
Hybrid Models T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832019302649?via=ihub. [Accessed: 15-Oct-2020].
What is risk? What is safety science? Hybrid calculation R=f(s,p,c,k) Risk = f(scenario, probability, consequence, knowledge) knowledge = (mix of both quantitative and qualitative data)
The Knowledge Dimension (This is where you come in!)
Risk in the age of tech Triforce of fear! DDoS Failed Deploy Certificate Issues Load Issues Reputation Information Security Tampering SQL Injection Impersonation Authentication Bypass Productivity Data Exfiltration SQL Injection XSS Financial
Risk in the age of tech Triad of consequences! Headline news? Brand damage Customer trust Reputation Information Security How long to restore? How much forensics? Time to rollback? Employee Safety Doxxing Productivity Fines! Legal damages Identity insurance Financial
Risk in the age of tech What we know vs. what we feel Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely
Risk in the age of tech What we know vs. what we feel Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely
Risk in the age of tech What we feel High 3 + Maximum 4
7?
Risk in the age of tech What we know 0.5 confidence 0.9 confidence 0.7 confidence High 3 Maximum 4 Hi/Max 3.5 1.5 + 3.6
5.1
Real risks in the real world
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk in the age of tech, in all industries G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk in the age of tech, in tech itself G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk budgets (Hope for the best; plan for the worst)
You may have heard this song before…
What is your risk budget based on?
Base your budget on your data!
Risk budgets You already have the data… Tools Signal Visibility & Detection Signal Time-based risk
Risk budgets You just need to think about it differently
- Qualitative risk = Time-based risk Comprehensive Visibility
Using your data to level up!
Rapid risk assessment event_probability = ( recommendations * max_impact ) a.k.a the more things are wrong the more likely an incident will occur
Use the common language and framework SEV-1 SEV-2 SEV-3 SEV-4 SEV-5 INFO WARN ERROR CRITICAL Low 1 Medium 2 LOW MED HIGH High 3 Maximum 4
Dynamic Risk A method for calculating : Assign a weight to the number of findings Low 1 Medium 2 High 3 WARNING = 1 point CRITICAL = 20 points likelihood = ( points * 0.25 ) Since we’re using a 4 point scale “id”: “W41”, “type”: “WARN”, “message”: “S3 Bucket should have encryption option set”, Maximum 4
Aggregate the data
Use the data
So in summary…
In summary… tl;dr – The best risk assessments balance both speed and accuracy, with a healthy dose of testable fact – Your environment is already giving you risk signals – You are an important piece of the security puzzle! – Risk budgets are a tool that organisations can add to their toolbox in order to help their business go fast and stay safe
In summary… Keep the party going! Mozilla Rapid Risk Assessment: https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html Rapid Risk Assessment Training https://www.youtube.com/watch?v=jxpuafW-H8U Better Reliability Through SLOs (Fique em Casa Conf 2020) https://www.youtube.com/watch?v=JOFYhFbrsK8 Verizon Data Breach Investigations Report 2020 https://enterprise.verizon.com/resources/reports/dbir/ @phrawzty | daniel.maher@datadog.com | www.dark.ca
Daniel Maher // @phrawzty // dark.ca And remember: We can dance—everything’s under control.
Risk management is relatively new to the tech scene but in reality insurance teams, government, and finance have been using risk assessments to make decisions for years. In this talk, I’ll demonstrate how to apply classical risk management concepts to modern DevOps practices. You’ll learn how to build a culture of risk management through a common, shared measurement framework, and how to implement a powerful tool for any organisation: the risk budget.
Video
Resources
The following resources were mentioned during the presentation or are useful additional information.
Buzz and feedback
Here’s what was said about this presentation on social media.
