DevSecOps

What Why and How?

Anant Shrivastava
@anantshri
NotSoSecure Global Services
 About: Anant Shrivastava
Director NotSoSecure Global Services
Sysadmin / Development / Security
Trainer / Speaker: BlackHat, Nullcon, RootConf, RuxCon, IPExpo, C0c0n
Project Owner: Android Tamer, Code Vigilant
Contributor: null, G4H, OWASP and more
https://anantshri.info (@anantshri on social platforms)

2
 Agenda
● What is DevSecOps?
● Why do we need DevSecOps?
● How do we do DevSecOps?
● Integrate Security in DevOps Pipeline
● Tools of Trade
● Sample Implementation (On Prem and Cloud Native)
● Case Studies

3
 Disclaimer
● I will be listing a lot of tools, It’s not an exhaustive list

● I don't endorse or recommend any specific tool / vendor

● Every environment is different: Test and validate before

implementing any ideas

4
 What is DevSecOps?
Effort to strive for “Secure by Default”
● Integrate Security via tools
● Create Security as Code culture
● Promote cross skilling

5
 Why do we need DevSecOps?
● DevOps moves at rapid pace, traditional security just can’t
keep up
● DevSecOps makes it easier to manage rapid pace of
development & large scale secure deployments
● DevSecOps allows for much smoother scaling of process
● Security as part of process is the only way to ensure safety

6
 Shifting Left saves cost & time

Source Suite of
CI/CD Staging/
Developer Code Build Production Monitoring Security
Server QA
Repository Test

7
 Shifting Left saves cost & time

Source Suite of
CI/CD
Developer Code Build Staging/QA Production Monitoring Security
Server
Repository Test

1 SQL Injection
Automated Source Fewer Man Day Effort
Code Review No New Deployments

8
 How do we do DevSecOps?
● DevSecOps is Automation + Cultural Changes
● Integrate security tools into your DevOps Pipeline
● Enable cultural changes to embrace DevSecOps

9
 Injecting Sec in DevOps
Artifact
Repository
Build Artifacts
versioning against
code commits

CI/CD Server

Code
Developer
Repository
Pre-Build Post-Build QA/Staging Production
Pre-Commit Static Application
Secrets Dynamic Manual Web
Hooks Security Security in Iaac
Management Application Application
IDE Plugins Testing(SAST) Compliance as
Security Pentesting
Source Code
Testing(DAST) Business Logic
Composition Alerting &
Analysis (SCA) Flaws Monitoring

Vulnerability Management

10
DevOps ---> DevSecOps
DevOps Pipeline

DevSecOps Pipeline
 Pre-Commit Hooks

● Sensitive information such as the access keys, access

tokens, SSH keys etc. are often erroneously leaked due to
accidental git commits
● Pre-commit hooks can be installed on developer’s
workstations to avoid the same
● Work on pure Regex-based approach for filtering sensitive
data
● If developers want they can circumvent this step hence use
it like a defense in depth but don't fully rely on it

12
 IDE Security Plugin
● IDE Plugin's provide quick actionable pointer to developers

● It is useful to stop silly security blunders

● Work on pure Regex-based approach

● If developers want they can circumvent this step hence use

it like a defense in depth but don't fully rely on it

13
 Secrets Management
● Often credentials are stored in config files
● Leakage can result in abuse scenario
● Secrets Management allows you to
tokenize the information

14
 Software Composition Analysis
● We don’t write software's, we build on frameworks
● Biggest portion of software is now third party libraries
● Major languages provide module managements
● PIP, NPM, Gems, go get, perl cpan, php packager and more
● Software Composition Analysis performs checks to identify
vulnerable/outdated 3rd party libraries

15
 Static Analysis Security Testing
● White-box security testing using automated tools
● Useful for weeding out low-hanging fruits like SQL Injection,
Cross-Site Scripting, insecure libraries etc
● Tool by default configured with generic setting, needs
manual oversight for managing false-positives

16
 Dynamic Analysis Security Testing
● Black/Grey-box security testing using automated tools
● SAST may not get full picture without application deployment
● DAST will help in picking out deployment specific issues
● Results from DAST and SAST can be compared to weed out
false-positives
● Tools may need prior set of configuration settings to give
good results

17
 Security in Infrastructure as Code
● Infrastructure as a code allows you to document and
version control the infra
● It also allows you to perform audit on the infrastructure
● Docker / K8s infra relies on base images
● Environment is as secure as the base image
● Base images need to be minimal in nature and need to be
assessed to identify inherited vulnerabilities

18
 Compliance as Code
● Compliance could be industry standard (PCI DSS, HIPAA,
SOX) or org specific
● Compliance is essentially a set of rules and hence can be
converted into written test cases
● Having written code format this can again be version
controlled

19
 Vulnerability Management
● All the tools discussed above result in report fatigue
● Every tool has a different style of presentation
● A central dashboard is required to normalize the data
● Vulnerability Management System can then be integrated to
bug tracking systems to allow devs to work on items

20
 Alerting and Monitoring
● Monitoring is needed for two end goals
● Understand if our security controls are effective
● What and where we need to improve

● To test Security control effectiveness:

● When did an attack occur
● Was it blocked or not
● What level of access was achieved
● what data was bought in and bought out
21
 Asset Monitoring
● With recent advancements assets now should include
anything and everything where organization data resides

● With rapid development & provisioning the asset inventory

can't be a static inventory

● We need to monitor the assets constantly both on premise

and Cloud
Reference: https://redhuntlabs.com/blog/redifining-assets-a-modern-perspective.html

22
 Sample Implementation - Java
A simplistic flow of DevSecOps Pipeline incorporating various stages

https://www.youtube.com/watch?v=7ILrzYLpr14
 Tools of The Trade
Microsoft Threat
Threat Modelling Tools ThreatSpec
Modeling Tool

Pre-Commit Hooks truffleHog Git Hound

Software Composition Analysis Retire.js

Static Analysis Security Testing

(SAST)

IDE Plugins CAT.net

Secret Management Keywhiz

Preference given to open-source tools; we don’t endorse any tool 24
 Tools of The Trade
Vulnerability Management

Dynamic Analysis Security

Testing (DAST)

Security in Infrastructure as
Code

Compliance as Code
Docker Bench for Security

WAF

Preference given to open-source tools; we don’t endorse any tool 25

 To be or Not to Be in Pipeline
● API / command line access
● Execution start to final output should be 15 minutes max
● Tools should be Containerizable / scriptable
● Minimal licensing limitations (parallel scans or threads)
● Output format parsable / machine readable (no to stdout,
yes to json / xml)
● Configurable to counter false negatives / false positives

26
 Pipeline Optimization
● Pipeline to be tweaked based on Milestone (Initiative/Epic/Story)
● Remember initial onboarding is tedious
● Ensure dataset dependent tool get frequent data refresh
● Sample optimization
● Only CSS Changes: no need for SCA
● Only pom.xml or gradle changes: no need of SAST
● If Infra as code has zero changes skip or fast track infra scan
● Ensure to run full (non optimized) pipeline periodically
 Does Programming Language Matter
● Different programming languages need different tools for
static analysis and software composition analysis
● Some tools support multiple languages like sonarqube
● Others are focused on one language

28
 Language Specific Tools
Languages Software Composition Analysis Source Code Static Analysis

JAVA graudit
ClearlyDefined

PHP graudit

Python graudit

DotNet
.NET DotNET Retire Security
SafeNuGet Guard

Ruby/Rails Brakeman graudit

npm-check
Node JS NodeJsScan
ClearlyDefined

Preference given to open-source tools; we don’t endorse any tool 29

DevSecOps Lab - Ruby
DevSecOps Lab - PHP
DevSecOps Lab - Python
DevSecOps Lab - NodeJS
 What about Cloud
● The Threat Landscape changes
- Identity and Access Management
- Asset Inventory
- Billing

● Infrastructure as Code allows quick audit / linting

● Focus more on:
- Security groups
- Permissions to resources
- Rogue / shadow admins
- Forgotten resources (compromises / billing)
34
 Cloud Native Approach to Security
● Different Service Providers Approach Security Differently
● All of them provide some of the ingredient in-house
● Irrespective of Cloud provider some tools will still need to
be sourced
● Static Code Analysis Tool
● Dynamic Code Analysis Tool
● Software Composition Analysis
● Vulnerability Management Tool

35
 AWS Cloud Native DevSecOps

https://youtu.be/i38-YQsnqfw 36
 Cloud Native Dev[Sec]Ops
Conventional Infra AWS Azure GCP
Source Code Management Bitbucket, Github, Gitlab etc.. AWS CloudCommit Azure Repos Cloud Source Repositories
Infrastructure As a Code Chef, Puppet, Ansible more.. Amazon CloudFormation Azure DevTest Labs Cloud Code

AWS CodeBuild
Jenkins, Bamboo, Gitlab,
CI/CD Server AWS CodeDeploy Azure Pipelines, Azure Test Plans Cloud Build, Tekton
Travis CI, Circleci more
AWS CodePipeline

jFrog Artifactory,
Artifactory Repository Amazon S3 Azure Artifacts Cloud Firestore
Sonatype Nexus, more..

EC2 Virtual Machines, Compute Engine,

VMWare,
Stg/Prod Servers ECS (Elastic Containers) Azure Lab Services, App Engine,
On-premises servers
EKS (Elastic Kubernetes) Azure Kubernetes Service (AKS) Shielded VMs

Monitoring & Alert Nagios, Graphite, Grafana AWS CloudWatch Azure Monitor, Network Watcher Access Transparency
AWS Firewall Manager, AWS
Firewall Modsecurity Azure Firewall Application Gateway
WAF
DLP MyDLP, OpenDLP Amazon Macie Azure Information Protection Cloud Data Loss Prevention
Azure Advanced Threat
Threat Detection Snort, Kismet Amazon GuardDuty Event Threat Detection (beta)
Protection
Vulnerability Scanning OpenVAS, Nessus Amazon Inspector Azure Security Center Cloud Security Scanner
Secrets Management Hashicorp Vault, Docker Secrets AWS Secrets Manager Azure Key Vault Secrets management

37
 Cultural Aspect
● Automation alone will not solve the problems
● Encourage security mindset especially if outside sec team
● Cultivate/Identify common goals for greater good
● Build allies (security champions) in company
● Focus on collaboration and inclusive culture
● Avoid Blame Game

Security team should try to eliminate the need of dedicated security team
38
 Security Champion
● Bridge between Dev, Sec and Ops teams
● Single Person per team
● Everyone provided with similar cross skilling opportunities
● Incentivize other teams to collaborate with Sec team
○ Internal Bug bounties
○ Sponsor Interactions (Parties / get-togethers)
○ Sponsor cross skilling trainings for other teams

39
 Security Enablers
People Process Technology
• Build relationships between • Involve security from get-go • Templatize scripts/tools per
teams, don’t isolate (design or ideation phase) language/platform
• Fix by priority, don’t • Adopt security to devops
• Identify, nurture security
attempt to fix it all flow don’t expect others to
conscious individuals
• Security Controls must be adopt security
• Empower Dev / ops to programmable and • Keep an eye out for simpler
deliver better and faster automated wherever and better options and be
and secure, instead of possible pragmatic to test and use
blocking. new tools
• DevSecOps Feedback
• Focus on solutions instead process must be smooth
of blaming and governed

40
Generic Case Study

41
 Case Studies – Fannie Mae

https://www.slideshare.net/DevSecOpsDays/fannie-mae-devsecops-journey-with-chitra-elango-and-john-willis 42
 Case Studies – Fannie Mae

https://www.slideshare.net/DevSecOpsDays/fannie-mae-devsecops-journey-with-chitra-elango-and-john-willis 43
 Case Studies – ABN Amro

https://www.slideshare.net/derweeksglobal/abn-amro-devsecops-journey 44
 Case Studies – ABN Amro

https://www.slideshare.net/derweeksglobal/abn-amro-devsecops-journey 45
 Negative Case Studies

t io n
g ur a
o n fi
i sc
ts M
d A sse
C l o u

Prevention: Continuous monitoring and review of cloud assets and config

46
 Is it Enough?
● Rite of passage by periodic pen test and continuous bug bounty
● It's not just important to get feedback but to also action on them
● Risk Acceptance Documentation should be the worst case scenario
not your first bet

47
 Who Watches the Watcher
● Did we secure the security controls
● DevSecOops: If attacker controls security
tools / build chain It has limitless power
● Ensure the same practice is followed back
again for these tools
● Security role doesn't means you get to
circumvent the rules
● Follow basic security hygiene we always
keep talking about
● Secure configuration
● Patching Policy
48
 References
• https://www.blackhat.com/docs/us-17/thursday/us-17-Lackey-Practical%20Tips-for-Defendin
g-Web-Applications-in-the-Age-of-DevOps.pdf

• https://www.sonatype.com/hubfs/2018%20State%20of%20the%20Software%20Supply%20
Chain%20Report.pdf

• https://snyk.io/opensourcesecurity-2019/
• https://scaling-threat-detection.awssecworkshops.com/
• https://www.veracode.com/state-of-software-security-report

49
 Key Takeaways
● Security is everyone responsibility
● Embrace security as an integral part of the process, use feedback to
refine the process
● DevSecOps is not a one size fit all: your mileage will vary

@anantshri
[email protected]
50