June 2023

We are the authors of the Threat Modeling Manifesto. We were excited to see the release of Shifting the Balance of Cybersecurity Risk: Principles and Approaches for Security-by-Design and -Default and appreciate the chance to comment.

We fully endorse and embrace the drive to prioritize Security-by-Design and Default as a central value to the development process. As practitioners see threat modeling as an integral part of the Secure Development Lifecycle, we were happy that Shifting The Balance mentions “tailored threat models” as mechanisms to further its aims. If Security-by-design is the goal, then threat modeling is the means.

This letter provides some background on threat modeling, illustrates how threat modeling can buttress the 3 principles espoused in Shifting the Balance, identifies some specific opportunities to integrate it into the document, and closes with information about the authors.

Our key message is that Shifting the Balance should go further in integrating and advocating threat modeling in any future revision, and/or in related/supporting documents.

Software manufacturers should [work] to identify and enumerate prevalent cyber threats to critical systems, and then include protections in product blueprints that account for the evolving cyber threat landscape.

This quote could be taken as a definition of threat modeling. We urge you to do so. CISA can leverage a widespread understanding held by developers and security practitioners by using this common term.

Background: About Threat Modeling

Delivering “Secure by design” requires threat modeling. By identifying and analyzing dangers, we can determine the most effective ways to mitigate them, generate the necessary security requirements for a secure design, development, and operations, and also gauge residual risk.

As delineated in the Threat Modeling Manifesto (threatmodelingmanifesto.org), threat modeling centers on asking four fundamental questions:

1. What are we working on?
2. What can go wrong?
3. What are we going to do about it?
4. Did we do a good enough job?

Dr. Suzanne Schwartz of the Food and Drug Administration has stated that threat modeling is how we bring structured, systematic and comprehensive security to medical devices such that they are safe and effective. We believe that her definition should not be limited to those devices. Threat modeling both spans strategic activity and informs the tactical execution, and we’ll expand on exactly how. Before we do, it is worth noting that asking “What can go wrong” and “What are we going to do about it” delivers strategy through tactics and provides a common language for everyone.

Improving  Software Product Security Principles

Each of the Shifting the Balance Principles can be enhanced, enforced, enabled, and empowered through Threat Modeling. In this section, we discuss how that can work.

Principle: The burden should not fall solely on the customer

Threat modeling enables organizations to anticipate and address future security and privacy challenges. Without threat modeling by product creators, customers will inevitably carry a larger part of the security burden when products are deployed in the real world. 

Threat modeling informs the builder of the product or service about security burdens associated with a design. The customer is the beneficiary of this security and privacy analysis, and ultimately the mitigations that are enacted in the product prior to its release.

As experienced practitioners, we have seen repeatedly how development organizations embrace Threat Modeling once they realize the advantages of identifying potential issues before they can surface in the field where they will lead to additional cost and burden, coupled with loss of confidence by the customer.

Principle: Embrace radical transparency and accountability

Threat modeling helps identify where the system may encounter dangers, and documented threat models are records of security concerns and blueprints for security controls. Sharing the threat model provides transparency to the security plan and orients all parts of the organization towards the risk that has been identified and what the appetite for it might be. Making the analysis transparent to those who might usually not have access to it will help keep security measures and awareness consistent and encompassing.

What is radical transparency for threat models?

When radical transparency is discussed, there is a tendency to get bogged down in a concern that we are providing a “roadmap for attackers.” We encourage CISA to clearly state that attackers are doing just fine without roadmaps, and that concern takes a back seat to transparency.

CISA should think carefully about what this principle means in practice. Should companies be sharing lists of the concerns they’ve identified and what they do to address each? This would certainly be radical transparency (and enable accountability, as we’ll discuss in the next section.) Should companies be sharing a subset, perhaps residual risks or risks that have been transferred or accepted? This form of transparency also helps customers to make decisions about what mitigating controls they need to operate.

Similarly, each organization will need to think about what it does for its customers, including how much is available publicly, and how threat model transparency might replace the avalanche of unique product security questionnaires.

The FDA has published “Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions Draft Guidance for Industry and Food and Drug Administration Staff” which discusses how transparency can be conveyed through labeling, including adequate directions for use.

That guidance, with its clear technical grounding, might be usefully contrasted with disclosures to investors, which might be seen to combine a laundry list of possible future issues and specific facts, both massaged into unreadable pablum by attorneys. Common business practices including usability testing and A/B testing can inform the effectiveness of transparency measures.

What is accountability for published threat models?

The question of accountability for published threat models is a fascinating and complex one, especially in light of forthcoming attestation requirements. Is accountability centered on the process or the outcome? Does a reasonable investment suffice, even if mistakes creep in? What counts as reasonable for “typical” software, say a SaaS product marketed for general business use? It is vital to ensure that this call for transparency does not act as a deterrent to any organization to engage in threat modeling. Companies embracing threat modeling should be in no way disadvantaged for doing so.

Principle: Build organizational structure and leadership

Threat modeling requires organizational capability to analyze designs, and to document those decisions. In our Manifesto, we presented the value of doing threat modeling over talking about it and a culture of finding and fixing design issues over checkbox compliance. These points align closely to the Shifting the Balance Principle of building organizational structure and leadership.

We applaud the fact that you are calling out “a tailored threat model” as a component of the product deployment process. This tailoring must start with senior executives and be implemented through a threat modeling program, and must set the standard for adherence to the performance of threat modeling at a specific scope (such as a user story, functional requirements spec, or whatever is appropriate for the organization.)

Organizations need to develop the capacity to threat model via training, support and help, tooling, and other capability elements. We have discussed in the Manifesto a journey of understanding, and journeys take time. Starting simply with the Four Questions helps your organization learn what to do, and a willingness to ask “Did we do a good enough job” leads to agile iteration and improvement.

Organizational staff needs to be involved broadly in threat modeling. Decisions that impact security can range from tactical decisions made by a single programmer to complex architectural choices. Everyone having familiarity with security and the skills to threat model is a better pattern than what we call “hero threat modelers.” There are many elements to the anti-pattern of a small team of security analysts, including the cost of bringing them up to speed on each feature and the tradeoffs involved, the delays in scheduling meetings, and the review effect where nothing is ever actually quite the way the reviewer would have done it.

Organizational structure requires time in schedules for threat modeling and sometimes for redesign. (Like measuring twice, cutting once, this appears to take more time, until you account for the time and material wasted by measuring once, cutting once.)

Leadership also includes support for positive escalations, focused on helping the organization make decisions aligned with Shifting the Balance, rather than trying to bolt on security as an after-thought. It also includes making time for developers to exercise their security abilities, participate in threat modeling activities and follow up on their findings.

Specific Opportunities for Improvement

We, the authors of the Threat Modeling Manifesto and friends, applaud this effort. At the same time, in conclusion, the work will benefit from more specificity about how to implement security-by-design through threat modeling. We suggest that it’s strategically important to define what radical transparency means with respect to threat modeling. We had a spirited dialog on the question of ‘reveal the full threat model’ versus ‘reveal the unaddressed risks’ versus “reveal the addressed risks”. In addition, with whom to be “radically transparent: share with customers versus publicly available.

We would also like to see clarification of the relationship between security guidance and radical transparency. For example, if a SaaS provider chooses to not follow a cloud provider’s configuration guidance, does CISA advocate the SaaS provider tell its customers? More generally, where along the supply chain and how far along the supply chain is appropriate for transparency? How does this align with the attestation requirements of emerging requirements such as OMB M–22–18?

Additionally, there are more specific opportunities, including:

1. On page 4, in secure by design:

1. Add a threat modeling definition along the lines of “analyzing representations of a system to highlight concerns about security and privacy characteristics.”
2. In the paragraph starting “Secure information technology”, change “use a tailored threat model during…” to “tailor your threat modeling process and methodology in ways appropriate for your situation and staff skills.”
3. In the paragraph starting “The authoring agencies” change “requires the investment of significant resources by software manufacturers at each layer of the product design and development” to “significant resources both in analyzing and refining designs (threat modeling) so that security is a first class participant, and also in documenting both the analysis done and its results. The analysis that’s done might be documented in the form of decision records (‘we chose to split components A and B to improve isolation…’) so that future revisions do not accidentally change those features.” The results would be security features and controls, tests to ensure that defensive coding works, and documentation of how residual risk is handled.
4. To confirm the threat model at runtime, does secure by default also result in a requirement for suppliers to include security configuration validation capabilities, to discover if those defaults have been changed?

2. On page 5, the recommendation to “...prioritize features, mechanisms, and implementation of tools that protect customers rather than product features that seem appealing but enlarge the attack surface.” could be viewed as stifling business and innovation in order to achieve security. Adding new product features is a critical business driver, and increasing the attack surface does not necessarily mean that the feature is more susceptible to attack. We would suggest that this be modified to embrace adding new product features as long as they are threat modeled and implemented securely.
3. On page 7, define what “tailored” means so that readers receive operational insight into what they are expected to do to build threat models and what they may expect to gain by threat modeling.
4. On page 7, please clarify the need for partnerships with an organization's customers. We think that many of the elements in 3a-c require threat modeling to be a part of product engineering more than they require a partnership with customers. For example, proposed security controls should be derived from threat modeling, and perhaps validated with customers. This is important because customer partnership is expensive both for the organization and for those customers, who might be using dozens or hundreds of software packages. So getting feedback may be hard, and getting feedback in timelines that align with agile product lifecycles will be even harder. Threat modeling provides an opportunity to model “what can go wrong” and “what are we going to do about it” on a timescale that is better aligned to, serves what we understand the purpose of paragraph 3 to be.
5. On page 8, “Secure-by-design Tactics” highlights some elements of the NIST SSDF, however it does not include the practice “Produce Well-Secured Software” with task PW.1.1 “Use forms of risk modeling such as threat modeling…”. We suggest including this practice in the highlighted items because it speaks directly to the topic of designing secure software.
6. On page 11, Secure by default tactics, in the context of threat modeling, consider the personas and role, and the permissions of that role. Specify the intent for products and applications to practice least privilege when it comes to what each role can do.
7. On page 11, Secure by default tactics, under consideration of the user experience consequences of security settings, we recommend adding a reference to threat modeling as an approach to help the developers deduce the security and privacy consequences of a security setting. Threat modeling makes this tactic come to life.
8. On page 11, ‘The authoring agencies believe that developing written roadmaps and executive support that prioritize these ideas into an organization’s most critical products is the first step to shifting towards secure software development practices.” We recommend adding “, such as threat modeling”.
9. On page 11, we recommend modifying this sentence, “It is important for the manufacturers to create meaningful incentives for customers to stay current and not allow them to remain vulnerable indefinitely.” to “and use threat modeling as a vehicle to convey the risk to Customers of remaining vulnerable indefinitely”.
10. On page 12, hardening vs loosening guides, our question is why would we want loosening guides to exist in our industry? As security professionals, we do not want Customers to loosen default security settings. We do not see the point in making this easy to do.
11. On page 12, recommendations for customers, we recommend asking vendors if they threat model, and the depth of their approach. Customers should understand the full lifecycle of security practices that their vendors perform.
12. On page 13, resources, we recommend adding the Threat Modeling Manifesto as a resource for consumers, along with books such as Threat Modeling: Designing for Security, Threat Modeling: A Practical Guide for Developers, and Secrets of a Cyber Security Architect. (All by authors of this letter.)
13. Add References to works that more fully explain and describe threat modeling including NIST SSDF, FDA’s Pre-market cybersecurity guidance (even though the latest version remains in draft), other bodies, as well as other publicly available material; there is much good information to draw from.

Performing threat modeling would address the tactics listed in the original publication and inform the technical teams on the choice of techniques, controls, and tactics that will prevent flaws identified during threat elicitation.

This collaboration between business leaders and technical teams extends from the early stages of design and development, through customer deployment and maintenance.

Manufacturers are encouraged to make hard tradeoffs and investments, including those that will be “invisible” to the customers.

These quotes apply to threat modeling at the strategic level by informing the amount of risk the tactical ones would be willing to accept.

We thank CISA for allowing us to comment.

About the Authors

We originally came together to capture and promote effective practices in threat modeling, which we use as an umbrella term for structured methods to inform decisions made during design.

Adam Shostack. Author, Threat Modeling: Designing for Security, Threats: What Every Engineer Should Learn from Star Wars. Affiliate Professor, UW Allen School.

Alyssa Miller. CISO Epiq Global, Author, Internationally Recognized Speaker/Educator on Application Security

Avi Douglen. Vice Chair OWASP Global Board of Directors, CEO Bounce Security

Brook S.E. Schoenfield. Author, Securing Systems: Applied Security Architecture and Threat Models, Building In Security At Agile Speed, Secrets Of A Cyber Security Architect, and others, faculty University of Montana

Chris Romeo. CEO Kerr Ventures and host of the “Threat Modeling Podcast”

Fraser Scott, Threat Modeling practitioner and creator of ThreatSpec

Irene Michlin AppSec Engineer and conference speaker

Izar Tarandach. Author, Threat Modeling: A Practical Guide for Development Teams

Jonathan Marcil. Application Security Engineer and workshop facilitator

Kim Wuyts. Researcher, Creator of the LINDDUN privacy threat modeling methodology

Matthew Coles. Author, Threat Modeling: A Practical Guide for Development Teams

Marc French, CISO & Managing Director, Product Security Group & conference speaker

Robert Hurlbut Principal AppSec Architect, Threat Modeling Lead, and conference speaker

Sarah-Jane Madden CISO, AppSec Educator and conference speaker

Sebastien Deleersnyder, CTO Toreon, OWASP Threat Modeling Playbook project lead, OWASP SAMM project lead.

Stephen de Vries. CEO IriusRisk, Contributor to OWASP ASVS Guide, OWASP Testing Guide

Zoe Braiterman. Information Security Consultant, OWASP member

All affiliations are listed for identification purposes only.