The Safety Dance Risk Assessment for DevOps Practitioners DevOpsDays São Paulo 2020
Slide 1
Slide 2
The Safety Dance Risk Assessment for DevOps Practitioners DevOpsDays São Paulo 2020
Slide 3
Your Safety Dancer Daniel Maher Twitter: @phrawzty Email: daniel.maher@datadoghq.com Web: www.dark.ca
Slide 4
Your Safety Dancers Daniel Maher ← that’s me! Andrew Krug ← shout out!
Slide 5
What we’re going to do today
Slide 6
Risk: A Crash Course
Slide 7
What is risk? What is safety science? Kaplan and Garrick (1989) What can go wrong? What is the likelihood? (probability) How bad could it be?
Slide 8
What is risk? What is safety science? Classic calculation R=f(s,p,c) Risk = f (scenario, probability, consequence)
Slide 9
Qualitative vs Quantitative Qualitative reasoning ranks likelihood using a scale score metric. 👍 👎 Speedy Light data gathering? Easy Low precision Light data gathering!
Slide 10
Qualitative vs Quantitative Quantitative reasoning uses data to reason about probabilities and consequences. 👍 👎 Accuracy Time Data Relative risks (adjacent)
Slide 11
Hybrid Models T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832019302649?via=ihub. [Accessed: 15-Oct-2020].
Slide 12
What is risk? What is safety science? Hybrid calculation R=f(s,p,c,k) Risk = f(scenario, probability, consequence, knowledge) knowledge = (mix of both quantitative and qualitative data)
Slide 13
The Knowledge Dimension (This is where you come in!)
Slide 14
Risk in the age of tech Triforce of fear! DDoS Failed Deploy Certificate Issues Load Issues Reputation Information Security Tampering SQL Injection Impersonation Authentication Bypass Productivity Data Exfiltration SQL Injection XSS Financial
Slide 15
Risk in the age of tech Triad of consequences! Headline news? Brand damage Customer trust Reputation Information Security How long to restore? How much forensics? Time to rollback? Employee Safety Doxxing Productivity Fines! Legal damages Identity insurance Financial
Slide 16
Risk in the age of tech What we know vs. what we feel Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely
Slide 17
Risk in the age of tech What we know vs. what we feel Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely
Slide 18
Risk in the age of tech What we feel High 3 + Maximum 4
7?
Slide 19
Risk in the age of tech What we know 0.5 confidence 0.9 confidence 0.7 confidence High 3 Maximum 4 Hi/Max 3.5 1.5 + 3.6
5.1
Slide 20
Real risks in the real world
Slide 21
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Slide 22
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Slide 23
Risk in the age of tech, in all industries G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Slide 24
Risk in the age of tech, in tech itself G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Slide 25
Risk budgets (Hope for the best; plan for the worst)
Slide 26
You may have heard this song before…
Slide 27
What is your risk budget based on?
Slide 28
Base your budget on your data!
Slide 29
Risk budgets You already have the data… Tools Signal Visibility & Detection Signal Time-based risk
Slide 30
Risk budgets You just need to think about it differently
- Qualitative risk = Time-based risk Comprehensive Visibility
Slide 31
Using your data to level up!
Slide 32
Rapid risk assessment event_probability = ( recommendations * max_impact ) a.k.a the more things are wrong the more likely an incident will occur
Slide 33
Use the common language and framework SEV-1 SEV-2 SEV-3 SEV-4 SEV-5 INFO WARN ERROR CRITICAL Low 1 Medium 2 LOW MED HIGH High 3 Maximum 4
Slide 34
Dynamic Risk A method for calculating : Assign a weight to the number of findings Low 1 Medium 2 High 3 WARNING = 1 point CRITICAL = 20 points likelihood = ( points * 0.25 ) Since we’re using a 4 point scale “id”: “W41”, “type”: “WARN”, “message”: “S3 Bucket should have encryption option set”, Maximum 4
Slide 35
Aggregate the data
Slide 36
Use the data
Slide 37
Slide 38
So in summary…
Slide 39
In summary… tl;dr – The best risk assessments balance both speed and accuracy, with a healthy dose of testable fact – Your environment is already giving you risk signals – You are an important piece of the security puzzle! – Risk budgets are a tool that organisations can add to their toolbox in order to help their business go fast and stay safe
Slide 40
In summary… Keep the party going! Mozilla Rapid Risk Assessment: https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html Rapid Risk Assessment Training https://www.youtube.com/watch?v=jxpuafW-H8U Better Reliability Through SLOs (Fique em Casa Conf 2020) https://www.youtube.com/watch?v=JOFYhFbrsK8 Verizon Data Breach Investigations Report 2020 https://enterprise.verizon.com/resources/reports/dbir/ @phrawzty | daniel.maher@datadog.com | www.dark.ca
Slide 41
Daniel Maher // @phrawzty // dark.ca And remember: We can dance—everything’s under control.
