June 2023To CISA,
We are the authors of the Threat Modeling Manifesto. We were excited to see the release of Shifting the Balance of Cybersecurity Risk: Principles and Approaches for Security-by-Design and -Default and appreciate the chance to comment.
We fully endorse and embrace the drive to prioritize Security-by-Design and Default as a central value to the development process. As practitioners see threat modeling as an integral part of the Secure Development Lifecycle, we were happy that Shifting The Balance mentions “tailored threat models” as mechanisms to further its aims. If Security-by-design is the goal, then threat modeling is the means.
This letter provides some background on threat modeling, illustrates how threat modeling can buttress the 3 principles espoused in Shifting the Balance, identifies some specific opportunities to integrate it into the document, and closes with information about the authors.
Our key message is that Shifting the Balance should go further in integrating and advocating threat modeling in any future revision, and/or in related/supporting documents.
Software manufacturers should [work] to identify and enumerate prevalent cyber threats to critical systems, and then include protections in product blueprints that account for the evolving cyber threat landscape.
This quote could be taken as a definition of threat modeling. We urge you to do so. CISA can leverage a widespread understanding held by developers and security practitioners by using this common term.
As delineated in the Threat Modeling Manifesto (threatmodelingmanifesto.org), threat modeling centers on asking four fundamental questions:
Dr. Suzanne Schwartz of the Food and Drug Administration has stated that threat modeling is how we bring structured, systematic and comprehensive security to medical devices such that they are safe and effective. We believe that her definition should not be limited to those devices. Threat modeling both spans strategic activity and informs the tactical execution, and we’ll expand on exactly how. Before we do, it is worth noting that asking “What can go wrong” and “What are we going to do about it” delivers strategy through tactics and provides a common language for everyone.
Each of the Shifting the Balance Principles can be enhanced, enforced, enabled, and empowered through Threat Modeling. In this section, we discuss how that can work.
Threat modeling enables organizations to anticipate and address future security and privacy challenges. Without threat modeling by product creators, customers will inevitably carry a larger part of the security burden when products are deployed in the real world.
Threat modeling informs the builder of the product or service about security burdens associated with a design. The customer is the beneficiary of this security and privacy analysis, and ultimately the mitigations that are enacted in the product prior to its release.
As experienced practitioners, we have seen repeatedly how development organizations embrace Threat Modeling once they realize the advantages of identifying potential issues before they can surface in the field where they will lead to additional cost and burden, coupled with loss of confidence by the customer.
Threat modeling helps identify where the system may encounter dangers, and documented threat models are records of security concerns and blueprints for security controls. Sharing the threat model provides transparency to the security plan and orients all parts of the organization towards the risk that has been identified and what the appetite for it might be. Making the analysis transparent to those who might usually not have access to it will help keep security measures and awareness consistent and encompassing.
When radical transparency is discussed, there is a tendency to get bogged down in a concern that we are providing a “roadmap for attackers.” We encourage CISA to clearly state that attackers are doing just fine without roadmaps, and that concern takes a back seat to transparency.
CISA should think carefully about what this principle means in practice. Should companies be sharing lists of the concerns they’ve identified and what they do to address each? This would certainly be radical transparency (and enable accountability, as we’ll discuss in the next section.) Should companies be sharing a subset, perhaps residual risks or risks that have been transferred or accepted? This form of transparency also helps customers to make decisions about what mitigating controls they need to operate.
Similarly, each organization will need to think about what it does for its customers, including how much is available publicly, and how threat model transparency might replace the avalanche of unique product security questionnaires.
The FDA has published “Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions Draft Guidance for Industry and Food and Drug Administration Staff” which discusses how transparency can be conveyed through labeling, including adequate directions for use.
That guidance, with its clear technical grounding, might be usefully contrasted with disclosures to investors, which might be seen to combine a laundry list of possible future issues and specific facts, both massaged into unreadable pablum by attorneys. Common business practices including usability testing and A/B testing can inform the effectiveness of transparency measures.
The question of accountability for published threat models is a fascinating and complex one, especially in light of forthcoming attestation requirements. Is accountability centered on the process or the outcome? Does a reasonable investment suffice, even if mistakes creep in? What counts as reasonable for “typical” software, say a SaaS product marketed for general business use? It is vital to ensure that this call for transparency does not act as a deterrent to any organization to engage in threat modeling. Companies embracing threat modeling should be in no way disadvantaged for doing so.
Threat modeling requires organizational capability to analyze designs, and to document those decisions. In our Manifesto, we presented the value of doing threat modeling over talking about it and a culture of finding and fixing design issues over checkbox compliance. These points align closely to the Shifting the Balance Principle of building organizational structure and leadership.
We applaud the fact that you are calling out “a tailored threat model” as a component of the product deployment process. This tailoring must start with senior executives and be implemented through a threat modeling program, and must set the standard for adherence to the performance of threat modeling at a specific scope (such as a user story, functional requirements spec, or whatever is appropriate for the organization.)
Organizations need to develop the capacity to threat model via training, support and help, tooling, and other capability elements. We have discussed in the Manifesto a journey of understanding, and journeys take time. Starting simply with the Four Questions helps your organization learn what to do, and a willingness to ask “Did we do a good enough job” leads to agile iteration and improvement.
Organizational staff needs to be involved broadly in threat modeling. Decisions that impact security can range from tactical decisions made by a single programmer to complex architectural choices. Everyone having familiarity with security and the skills to threat model is a better pattern than what we call “hero threat modelers.” There are many elements to the anti-pattern of a small team of security analysts, including the cost of bringing them up to speed on each feature and the tradeoffs involved, the delays in scheduling meetings, and the review effect where nothing is ever actually quite the way the reviewer would have done it.
Organizational structure requires time in schedules for threat modeling and sometimes for redesign. (Like measuring twice, cutting once, this appears to take more time, until you account for the time and material wasted by measuring once, cutting once.)
Leadership also includes support for positive escalations, focused on helping the organization make decisions aligned with Shifting the Balance, rather than trying to bolt on security as an after-thought. It also includes making time for developers to exercise their security abilities, participate in threat modeling activities and follow up on their findings.
We, the authors of the Threat Modeling Manifesto and friends, applaud this effort. At the same time, in conclusion, the work will benefit from more specificity about how to implement security-by-design through threat modeling. We suggest that it’s strategically important to define what radical transparency means with respect to threat modeling. We had a spirited dialog on the question of ‘reveal the full threat model’ versus ‘reveal the unaddressed risks’ versus “reveal the addressed risks”. In addition, with whom to be “radically transparent: share with customers versus publicly available.
We would also like to see clarification of the relationship between security guidance and radical transparency. For example, if a SaaS provider chooses to not follow a cloud provider’s configuration guidance, does CISA advocate the SaaS provider tell its customers? More generally, where along the supply chain and how far along the supply chain is appropriate for transparency? How does this align with the attestation requirements of emerging requirements such as OMB M–22–18?
Additionally, there are more specific opportunities, including:
Performing threat modeling would address the tactics listed in the original publication and inform the technical teams on the choice of techniques, controls, and tactics that will prevent flaws identified during threat elicitation.
This collaboration between business leaders and technical teams extends from the early stages of design and development, through customer deployment and maintenance.
Manufacturers are encouraged to make hard tradeoffs and investments, including those that will be “invisible” to the customers.
These quotes apply to threat modeling at the strategic level by informing the amount of risk the tactical ones would be willing to accept.
We thank CISA for allowing us to comment.
Adam Shostack. Author, Threat Modeling: Designing for Security, Threats: What Every Engineer Should Learn from Star Wars. Affiliate Professor, UW Allen School.
Alyssa Miller. CISO Epiq Global, Author, Internationally Recognized Speaker/Educator on Application Security
Avi Douglen. Vice Chair OWASP Global Board of Directors, CEO Bounce Security
Brook S.E. Schoenfield. Author, Securing Systems: Applied Security Architecture and Threat Models, Building In Security At Agile Speed, Secrets Of A Cyber Security Architect, and others, faculty University of Montana
Chris Romeo. CEO Kerr Ventures and host of the “Threat Modeling Podcast”
Fraser Scott, Threat Modeling practitioner and creator of ThreatSpec
Irene Michlin AppSec Engineer and conference speaker
Izar Tarandach. Author, Threat Modeling: A Practical Guide for Development Teams
Jonathan Marcil. Application Security Engineer and workshop facilitator
Kim Wuyts. Researcher, Creator of the LINDDUN privacy threat modeling methodology
Matthew Coles. Author, Threat Modeling: A Practical Guide for Development Teams
Marc French, CISO & Managing Director, Product Security Group & conference speaker
Robert Hurlbut Principal AppSec Architect, Threat Modeling Lead, and conference speaker
Sarah-Jane Madden CISO, AppSec Educator and conference speaker
Sebastien Deleersnyder, CTO Toreon, OWASP Threat Modeling Playbook project lead, OWASP SAMM project lead.
Stephen de Vries. CEO IriusRisk, Contributor to OWASP ASVS Guide, OWASP Testing Guide
Zoe Braiterman. Information Security Consultant, OWASP member
All affiliations are listed for identification purposes only.