Risky Business

By Dr. Scott Rudge

“Risk Analysis” is a big topic in pharmaceutical and biotech product development. The International Conference on Harmonization (ICH) has even issued a guidance document on Risk Analysis (Q9). Despite the documentation available and the regulatory emphasis, these tools remain poorly understood. They are used to justify limiting the extent of fundamental understanding that can be gained on a process, while simultaneously used as a cure-all for management challenges that we face in pharmaceutical process development.

Risk analysis focuses only on failure modes. Failure mode effect and analysis (FMEA) was developed by the military, first published as MIL-P-1629, “Procedures for Performing a Failure Mode, Effects and Criticality Analysis” in November 1949. The procedure was established to discern the effects of system failure on mission success and personnel safety. Since then, we have found a much broader range of applications for this type of analysis. The methodology is used in the manufacture of airplanes, automobiles, software and elsewhere. People have devised “design” FMEAs and “process” FMEAs (DFMEA and PFMEA). These are all great tools, and help us to design and build better, safer products with better, safer processes.

Where Risk Analysis Falls Down

The FMEA is such a great hammer, it can make everything look like a nail. And when regulatory authorities are encouraging companies to use Risk Analysis for product design and process validation, the temptation to apply it further can be overwhelming. In particular, risk analysis is used in three inappropriate ways, in my estimation:

 

1. Decision analysis
2. Project management
3. Work avoidance

Quite often, risk analysis tools are used to guide decisions. Here, the pros and cons of selecting a particular path are weighed, using the three criteria of risk analysis (occurrence, detectability and severity) as guides. However, not every decision path leads to a failure mode or an outcome that could be measured as a consequence. And detectability and occurrence may not be the only or most appropriate factors by which to weight a consequence. There are excellent decision tools that are designed specifically for weighting and evaluating preference criteria. A very simple tool is the Kepner Tregoe (KT) decision matrix. Decision analysis uses very detailed descriptions of the decision to model the potential outcomes. The KT decision matrix is sufficient for many decisions, large and small. But if you really want to study the anatomy of a decision, decision analysis is the most satisfactory method.

 

FMEAs are sometimes inappropriately applied in project management, to assign prioritization and order in which tasks should be completed. This may be handy in some instances, but is somewhat misleading or inappropriate in others. The riskiness of an objective should not be the sole determinant in its prioritization. Quite often, the fundamentals or building blocks need to be in place in order to best address the riskiest proposition in a project. Prematurely addressing the pieces of a project that present the greatest risk of failure may lead to that failure. On the other hand, being “fast to fail” and eliminate projects that might not bear fruit with the least amount of resources spent, is critical to overall company or project success. Project management requires consideration of failure modes, but also resource programming and timeline management. FMEAs can be an element of that formula, but should not be the focus.

 

Finally, and perhaps most painfully, FMEAs are used to justify avoiding work. Too often, risk analysis is applied to a problem, not to identify the elements that are most deserving of attention, but to justify neglecting areas that do not rank sufficiently high in the risk matrix. Sometimes, the smallest risks are the easiest to address, and in addressing them, variability can be removed from the process. Variability is the “elephant in the room” when it comes to pharmaceutical quality, as has been concisely pointed out by Johnston and Zhang.

 

The FMEA is a stellar tool, and it is applicable to problems in design, process and strategy across many industries. Its quantitative feel makes practitioners feel as though they are actually measuring something, and can make fine distinctions between risks that they were unable to articulate before. However, the FMEA can be applied rather too widely, or sometimes unscrupulously, yielding bad data and bad decisions.

Ten Steps to Choosing your Contract Manufacturer

For many young companies, choosing a contract manufacturer, or CMO, for their lead pharmaceutical candidate is critical. Choose the wrong contractor, and you could be faced with delays and cost overruns with which your investors and patients won’t be very sympathetic. While there is no guarantee that you will always make the right decision, here are some tips that can help you make your choice in an organized, thoughtful, meaningful and objective way:

1) Make a list of all the possible suppliers. Such lists may be purchased, but they are also easily assembled from internet searches. In fact, you can do a little pre-screening with your own internet search.

2) Screen potential suppliers with a phone call. You will probably speak with a business development or sales person representing the contractor, but usually these people are quite knowledgeable about their company’s capabilities, and common problems encountered in the industry. We recommend you not reveal too many details about your project, and be prepared mostly to listen. However, you should have three or four key capabilities or proficiencies that you are looking for in all of the potential vendors. If possible, try to rule out vendors who do not meet your “must-have” requirements at this stage. Stay tuned for a blog on how to establish your showstopper list, it’s a critical exercise, and may extend beyond key capabilities and proficiencies.

3) Keep a matrix, and record the date you first contacted the vendor, when they responded to you, the status of any confidentiality agreements, and all the contact information that you can gather (email addresses, cell phone numbers, main switchboards and extension numbers). Also note the responses that each contact had relative to your three or four show-stopper criteria.

4) Meet with your team, and select three to five potential vendors to request a proposal from. We don’t recommend more than five: getting good, comparable proposals is a lot of work, like 2n times the work, where n = the number of proposers. Not to mention the work that contract manufacturers go through to read your RFP and prepare a proposal. Your three or four showstopper criteria should help you limit the number of proposers; if necessary you can begin to narrow down based on “nice-to-have” criteria as well. You may also deliberately choose to look at a range of vendors that represent different strengths/weaknesses (for example, do you prefer a “one-stop shop” that is convenient, but maybe not the best at everything, or a “specialty” vendor that provides higher levels of expertise, but will require you to select and manage multiple vendors?).

5) Solicit proposals. Most contract manufacturing seekers have a Request for Proposals (RFP) process that includes a document. These RFP documents vary from one page requirements descriptions, to lengthy, legalistic documents that require a team, and a month, to respond to. You should do what is comfortable for your organization. There needs to be enough information so that the vendor is able to respond with a meaningful proposal. There is some legal danger, particularly with intellectual property, so it’s not a bad idea to get your RFP reviewed by your legal counsel. And you should only send an RFP to a vendor after they have signed a mutually agreed confidentiality agreement.

6) Score your proposals. Find some basis to make apples to apples comparisons. RMC uses a modified Kepner Tregoe analysis, but many forms of analysis will work. You should have determined how you will assess and weight qualitative data before you begin. And in doing so, you should not under-estimate intangible factors: the ability to communicate, time zone differences, good references (you’ve checked, right?) are some examples. At this stage you should be ranking and scoring on “nice-to-have” criteria as well as comparing cost/timeline, capabilities, capacity, and viability of the business. You might form your opinion of the viability of the business by reading annual reports and press releases, and by assessing how busy the manufacturing area and support labs look during your visit.

7) Visit the top two or three vendors in person. Vendors may not allow a formal quality audit prior to signing a contract, but be sure to bring your quality representatives even for an informal “technical visit”. If the project is large, you may take the resistance to a pre-use quality audit as a red flag. Again, spend as much time as possible listening, rather than talking. Get a tour, and copies of all presentations. Ensure that you have a meeting between the decision makers for both sides as well as aligning discussions between key technical and quality personnel. If there are disputes or further work to do, your decision makers must have a good working relationship.
Your visit is also your best opportunity to break past the business development group and take a true measure of the business. Chat with the people in the lab or production area if you can. Look over the state of the equipment, the cleanliness of the facility and the stock in the warehouse. Check their flexibility-- what can they make happen for you, vs. what will have to be run past someone in another building, or another city? Ultimately, you should think about hiring a contract manufacturer similarly to how you hire an employee, by hiring for expertise as well as fit with your team.

8) Consider entering contract negotiations with at least two vendors. Things can go wrong in negotiations, and your position is stronger if you can legitimately walk away. We typically don’t let any of the final three candidates off the hook until the ink is dry on the final contract. If your budget can justify it, having a second contractor performing development work and verifying the primary contractor’s results is an excellent idea. It may ultimately spread your risk in supply chain, and give you leverage in negotiating commercial supply agreements later on.

9) Revisit your analysis tool. You may learn new things in your contract negotiations that cause you to adjust your evaluation. Don’t be afraid to be frank if you feel like terms have been changed since the selection was made. This is a good reason to keep a back up vendor.

10) You must now manage the project according to the criteria by which you selected your supplier. Hold your supplier and yourself accountable to these criteria. For example, after selecting a vendor because they can meet a very aggressive timeline, do not put the project timeline at risk by failing to order back up critical path supplies, in case the primary order doesn’t arrive, or fails to meet specifications on arrival. We will have more to say about managing a contract manufacturer in a future blog.

Choosing the right manufacturing partner is critical for your success as a drug developer. Spend the time required to make a good decision. This time should be spent gleaning as much tangible and non-tangible information on all your options as possible, and then objectively comparing it. You should have an idea about how you’re going to evaluate and weight non-tangible factors into your decision. And once you have made the choice, manage according to your criteria. Although everyone has their own methods for vendor selection, these are some suggestions that have worked well for us and our clients. If you have questions or comments, please visit http://www.rmcpharma.com/ or email us at info@rmcpharma.com.