PSIRT — Explained

Soumen Mukherjee
14 min readSep 22, 2020

1. Introduction to PSIRT

Product Security Incident Response Team is a group of teams or individual people within an organization whose primary responsibility is to be first point of contact for any security misshaping for any of the product or services offered by the organization. The core focus of psirt is on the identification, assessment, and disposition of the risks associated with security vulnerabilities identified, disclosed or published in public domain with or without the due knowledge of the organization.

The product or services could be any software, hardware or managed service or any of its reusable component ( API , Library , SDKs ) which are offered either free of charge or against a standard commercial contract .

The PSIRT is promoted by the security engineering initiative of the organization and is an integrated part of the engineering or R & D group and is an equal stakeholder in the business unit or organizations product / service deliverables.

PSIRT v/s CSIRT

Enterprise CSIRT is primarily focused on the security of computer systems , corporate infrastructure , data centers and/or networks (vpn’s , leased lines , data links ) within an organization. While the PSIRT team’s focus is on the product offering both on-prem and hosted inclusive.

1.1 Why do you need PSIRT

As enterprises grow both in terms of business revenue and product / solution offering, there is a corresponding build up of an integrated ecosystem around the very products and solutions offered. In such a setup any security loophole if left untreated can lead to huge losses both in terms of reputation and revenue for the customers and customer’s customer. Hence it is a matter for foremost importance for any organization to have an active response plan in place whose existential purpose is dedicated to be the single point of contact for managing the receipt, investigation, and public reporting of information about security vulnerabilities and issues related to the org product and solution offerings.

Today’s competitive environment has lead to emergence of various covert forces which use the security vulnerabilities of the product as a means of negative publicity in competition. Entities like Ethical Hackers are subcontracted for taking up corporate funded hacking activity against the competition with an intention of either giving them a bad press or making them divert key resources from critical projects to the un-invited security concerns. The state is also not to be left behind , state sponsored entities have also been known to behind such coercive activities against commercial enterprises for various geopolitical reasons. Having a well-defined PSIRT team with an elaborated internal workflow helps organizations mitigate such situations in a well-organized and planned manner.

1.2 What Constitutes a PSIRT

The PSIRT team in it’s very basic form can be a group of people who manages the show or in a more mature form a subunit comprising various groups distributed yet organized in manner to form a cohesive unit designed to work in an automated workflow. The structure can also vary due to the domain of business ,product portfolio, customer segment, product development strategies, organizational structure , size of business and focus of services. Hence, there is no silver bullet or a single one-size-fits-all product security incident response strategy or team template for all organizations to follow.

If we however try to generalize the approaches or models used by organizations world wide , each structure or model will fit into one of the follow three model.

· Distributed

· Centralized

· Hybrid

1.2.1 The Distributed Model

The distributed PSIRT model consists of a small core psirt group which works closely with point of contacts or members of the product team to address the report security incidents or vulnerabilities in product or the services provided by the organization. The small PSIRT core operations drives the following key responsibilities.

The Distributed Model

· Defining policies, processes, procedures, internal/external security SLA’s and guidelines for the triage, analysis, remediation, and internal/external communication of fixes, mitigations or other advisory information to address security vulnerabilities.

· Working with the core product / service engineering group to establish a matrix of (tiered) product security engineering representatives throughout the organization.

· Offering leadership, guidance and stakeholder support regarding product security vulnerability response and duly informing the management on potential risk to the business.

· Acting as the central point of the entire organization for incoming security vulnerabilities where the economies of scale benefit from a central point of control.

· Notifying and working with the Product Owner/Manager and the security engineer of new security vulnerabilities, assisting in the development of remediation plans, and drafting/publishing communication of a fix or mitigation, including incident management.

This model is beneficiation to an organization with a large and diverse product portfolio because the cost of the PSIRT mission is budgeted across the organization. This model also allows the PSIRT mission to scale by leveraging the skilled people in the product engineering / services teams.

The challenge with the Distributed PSIRT model is that the people responsible for performing the triage and delivering the fixes for security vulnerabilities are not directly controlled by and do not report to the PSIRT Operations which often leads to priority conflicts between security fixes and functional deliverables. Also different product/service areas may put their own best interest ahead of the organizational PSIRT activities.

1.2.2 The Centralized Model

The Centralized model is little more elaborate and formal and has a larger PSIRT staff drawn from multiple departments that report to one or more senior executives responsible for the organization’s product security. This model might have a structure similar to the following:

The Centralized Model

The core constituent entities of this Model are the following with the respective responsibilities.

· PSIRT Program Management Department: The core responsibilities being Creates policies, processes procedures, and guidelines for the triage, analysis, remediation, and communication of fixes for security vulnerabilities. Manages the operations of the overall PSIRT initiative and the ticketing system and represents PSIRT leadership to the organization.

· PSIRT Security Intelligence and Triage: This group Monitors various external sources for security vulnerabilities. Assesses the initial impact of security vulnerabilities to the organization’s product portfolio.

· PSIRT Remediation and Communications: This is a highly technical group and Directly provides code fixes for security vulnerabilities to the product engineering teams.

This model is well suited for smaller organizations and/or an organization with a homogenous product portfolio. This model concentrates and cultivates a high level of security skill and expertise into one area of the organization.

The challenge with this model is in the cost of maintaining a centrally specialized team that does not scale as well if the product portfolio grows and/or becomes more diverse. Also at times building a team of experts capable of providing direct fixes to the product team leads to non-technical conflicts which further leads to completely unproductive resolution meetings and discussions. Major decisions will need to be made with the different functional manager’s approval and cooperation which bring in it’s own set of challenges and impediments.

1.2.3 Hybrid Model

The Hybrid model as the name suggest is a variation that includes characteristics of both the Distributed and Centralized models. An organization may choose to implement some characteristics and features of both models, creating a hybrid that considers the following factors:

· Organizational corporate structure and size

· Product portfolio size and diversity

· Product development strategy

The Hybrid Model

The advantage is of course the flexibility of both the centralized and distributed model along with an option to balance the characteristics from either of the two models.

1.2.4 Other Important Considerations

It is important for a PSIRT to have the autonomy to maintain an independent and objective position on the organization’s product security vulnerabilities. Irrespective of what Model is adopted, in either case It is important that a PSIRT report to an executive of the company who confirms the authority of the PSIRT or a person with considerable authority and clout is part of the core PSIRT group .

Considering the stakeholders’ needs and requirements is a critical part of defining the PSIRT strategy. With time as the PSIRT process continues to mature and scale, and the composition or reporting structure of the team should change. The driving force of change and maturity of a PSIRT is its key stakeholders. Stakeholders are often defined by the model adopted by the organization as well as the size of the organization. It is critical to maintain and nurture positive relationship between stakeholders and the PSRIT organization as a whole.

One final consideration in the formation of the product incident response team and strategy is influencers. The influencers may be invisible and can pop up anywhere on the timeline. The influencers are industry and government standards, legislation, regulations, and trends. These influencers may impose greater requirements on the formation, strategies, policies, and operational characteristics of a PSIRT and should be respected and adopted in due course.

2. PSIRTs benefit for Organizations

Irrespective of the model chosen, it will define the scope and operational activities of a PSIRT, but not necessarily change the actions an organization needs to take with respect to addressing security vulnerabilities in their products. While the overall PSIRT activities helps in refining the scope of the capabilities, actions and responsibilities directly attributed to the PSIRT rather than those distributed throughout the organization. There are certain soft benefits which are a direct result of the PSIRT activities within the organizations.

General PSIRT Activity Benefits

2.1 Ongoing Process and Policy Development

PSIRTs establish the organization’s policies with respect to product security. The needs of the business drive and dictate the requirements of PSIRT and not the other way around. Before the PSIRT policies can be implemented, they must be reviewed and imbued with authority by the organization’s leadership. Approved policies must be followed with clear procedures that, when followed, ensure the organization’s compliance with these policies.

2.2 Security Education

The greatest mistake that can be made when deploying the PSIRT mission, policies and procedures is to have it viewed as a separate responsibility or requirement. Therefore, it is critical to educate all members of the organization on product security basics and the role they play. The entire organization must be included, enabled, and empowered to meet the PSIRT policy requirements. Hence as an action, the management insures to conduct frequent training programs for the teams which are oriented towards cyber security , product security , security best practices and alike. The PSIRT policies also enable due justification to investment towards automation of various security processes which are aligned with the defined policies for the organization.

2.3 Timely redressal of security vulnerabilities

In today’s time it is a strict NO NO to have a product with un patched vulnerabilities in a corporate setup , every organization demands a time bound fix to identified or public disclosed vulnerabilities in any product or service. The defined policies and SLA as part of the PSIRT activity comes to the rescue of the organization in such case. The organization can provide a proper response to a security query from a customer. This also enables the product to qualify for enterprise deployment despite having unpatched security vulnerabilities.

2.4 Safeguard against corporate sponsored / ethical product hacks

Ethical hackers and corporate sponsored product hacks are so very common and easily accessible service today. The modus operandi is to do an ethical hack and then contacting the organization to outline the found security issues and vulnerabilities. Having a PSIRT team enables the organization to establish a formal protocol to handle such disclosures and negotiate a reasonable time for the Product / service development team to fix and address the vulnerabilities.

2.5 Minimize Bad Press

With over-enthusiastic press and paid press all around, it is a common scenario for press to sensationalize security findings by independent or paid security researchers. As a matter of protocol, the press generally approaches the organizations looking for some feedback and comments on public disclosures. The PSIRT group along with corporate communications and legal form a formidable alliance to make sure that the press queries are suitably addressed in line with the welfare of the organization. A structured and well-articulated response also makes the press go on the backfoot and force them to write a balanced report.

2.6 Strong rebuttal to Law suites

Security and Data Privacy are also governed by various government regulations and compliance rules these days. Hence it is common to find public interest groups filing lawsuits citing violation of Government laws in case some security incident with an organization’s product or service comes into limelight. The groups are also at times funded by competition and leave no stone unturned to make it extremely difficult for the organization to defend its stance. The PSIRT group in such situations has a key role to play in making sure that the issues are addressed at the earliest and no momentum is lost within the organization towards the fix. The PSIRT champions the cause of the organization as a prompt fix many times turns the tide in favour of the organization.

3. PSIRT, Responsible Disclosure and Bug Bounty Program

Many organization today are running one or more of security response programs, these are namely PSIRT, Responsible Disclosure and Bug Bounty. PSIRT adopts one of the models as explained earlier and is also driven forward by the organization’s security objective.

Responsible disclosure, provides and interface which can be a Web Form hosted with the corporate website or can be a simply an email id . This interface is provided with a view of letting ethical hackers, security researchers and customers share information and insights into security vulnerabilities that they might have come across during there regular course or investigation of the organization’s product and security. An input to the regular disclosure program usually triggers a PSIRT response.

Bug Bounty programs takes the responsible disclosure program to a next level where the organization commits itself to provide some monetary benefits to the reporter of the security issues besides handling of the issues towards a fix through the PSIRT program. Some Bug Bounty programs also introduce a competitive league where the reporters are also granted points (along with monetary reward ) which let them compete against fellow security researchers as well. This is more to do bring a level of challenge and popularity to the program and is usually done by using a Software Platform which allows the reporters to create their login and set their “Avatars” reflecting their personality and likes.

3.1 Compliance Standard

A common factor however for the three is a set of internationally accepted standard defined by ISO/IEC 29147:2018 Information technology — Security techniques — Vulnerability disclosure. The overall goal of vulnerability disclosure is to reduce the risk associated with exploiting vulnerabilities. Coordinated vulnerability disclosure is especially important when multiple vendors are affected

The ISO/IEC 29147:2018 standard provides requirements and recommendations to vendors on the disclosure of vulnerabilities in products and services. Vulnerability disclosure enables users to perform technical vulnerability management as specified in ISO/IEC 27002:2013 section12.6.1[1]. Vulnerability disclosure helps users protect their systems and data, prioritize defensive investments, and better assess risk. This standard provides:

· Guidelines on receiving reports about potential vulnerabilities.

· Guidelines on disclosing vulnerability remediation information.

· Terms and Definitions that are specific to vulnerability disclosure.

· An overview of vulnerability disclosure concepts.

· Techniques and policy considerations for vulnerability disclosure.

· Examples of techniques, policies and communications.

4. Common Pitfalls

While organizations continue to invest in and adopt PSIRT practices and groom them to maturity over a period of time , there are instance when the same very organizations falter on a security response causing monetary and reputation damage which may take years to rebuild. A detailed analysis by experts on such failures have outlined some common mistakes which these organizations did over the course of their PSIRT journey. A brief outline of such factors is outlined in the next section.

4.1 Poor Vulnerability Investigation

It has been seen that due to lack of appropriate review controls the reported vulnerability is either poorly investigated or even worst is poorly documented during the investigation of the responsible security engineer. This leads to a whole cycle of re-investigation just prior of the release of the fix just to remember what the issue was. This may often lead to delay and runs the risk of details being missed out. The ideal remediation is to define a formal format to capture the details of the vulnerability during the initial investigation itself and have a review mechanism in place before proceeding with the planning the fix.

4.2 Failure to Prioritize

It has been found that organizations take too long to prioritize the security issues for fix, either they end up wasting lot of time triaging or they end up triaging issues incorrectly. This leads to a very inappropriate and delayed security response which may lead to substantial business damage to the customer. The way to avoid this is for the organization to come up or adopt a prioritization business model that considers both technical and business impact . This is to be supported with the definition of Acceptable Business Risk as well.

4.3 Incorrect Role Definition

For the PSIRT organization it is very important to have the right person play the right role. While at times much emphasis is given in the structure of the group , not much importance is given to the persons who are places in the structure and the corresponding roles that they need to play. This leads to lack of clarity and picture to the stakeholders which further leads to inaccurate decisions. Lack of proper role play also gives a very inefficient PSIRT group making the management question the ROI. Hence a proper role fitment exercise is must when evaluating an individual for the PSIRT organization.

4.4 Improper Communication

It is generally said that in business, politics and life Communication is the Key to success. This is even true for PSIRT. For Security response team it is even more critical since they do both internal (with product / service development Team) and external (with security incident informer) communication. A gap or lag of communication in either of the channel is disastrous. The timing of the communication also plays a key role in making sure to attract adequate business focus internally and also to thwart any attempt from the external informer from extracting extra mileage from the reported incident.

Conclusion

Securing any organization’s product and service is Journey and not a destination. Reaching one milestone should immediately point you to the next one. The threat landscape is continuously evolving and so should we. New technologies are getting created and new ways of thinking are developed, this leads to new threats arise and old ones stagnate. The Security response has to evolve regularly keeping up with the ever-changing threat landscape. Some tips for the PSIRT group which can help us keep up to date are outlined below.

· Keep doing regular investment in people by means of training of technology and skills.

· Review your processes and practices regularly with all the internal groups challenging the status quo should be the new normal.

· Security awareness across the organization as per the roles is a must. Outline definitive goals in that regard to a formal learning and people development group.

· Be well connected in the industry by means of being member of formal groups and early adopters forum

· Embrace compliances and certifications as they help you both internally and externally.

· It always flows from the top , so do insure that respective business heads are all aligned with the overall security positioning of the organization.

References

· ISO/IEC 29147 : https://www.iso.org/standard/72311.html

· Responsible Disclosure : https://www.barco.com/en/about-barco/legal/responsible-disclosure

· PSIRT Framework : https://www.first.org/

--

--