Tech Talk

Pharmaceutical Quality Metrics, Part 2 - What Gets Measured Gets Done

In the Spring 2014 Journal (http://bit.ly/112SmcM), we introduced the topic of quality metrics. Since then, interest has grown considerably so we thought an update on progress was timely.

What’s the Big Deal?

The push for quality metrics took a major step forward in July 2012, when the US Food and Drug Administration Safety and Innovation Act (FDASIA) was passed.

• Section 705 of FDASIA requires FDA to replace the periodic inspection frequency with a risk-based inspection schedule. Risk is assessed based on compliance history and the inherent risk of the drug being manufactured
• Section 706 gives FDA authority to obtain certain records from a manufacturer in lieu of or in advance of an inspection. Essentially, any document that is discoverable during an on-site visit is subject to this regulation
• FDASIA Section 711 drives revisions to cGMP regulations to improve oversight of the manufacturing process and improve the detection of emerging safety and quality signals
  ICH Q10 reaffirmed the combined position of industry and its regulators:

“Performance indicators that measure progress against quality objectives should be established, monitored, communicated regularly, and acted upon as appropriate as described in Section 4.1 (Management Review of the Pharmaceutical System).”

So, in summary, performance indicators should be chosen to monitor the effectiveness of the pharmaceutical quality system and can include processes such as corrective and preventive action (CAPA), deviations, complaints, audits and regulatory inspections.

• FDA is actively seeking input on how quality metrics could be used to support the riskbased inspection schedules. The incentive for manufacturers is to develop metrics that objectively and sufficiently indicate the safety of their products and the effectiveness of their quality systems
• These metrics would be submitted periodically to FDA (probably once per year), perhaps as part of the Annual Product Review (APR). In return, the top performers can expect regulatory relief in the form of fewer on-site inspections and fewer Prior Approval Supplements
• Work to develop consensus on quality metrics began in earnest at a joint FDA - Parenteral Drug Association (PDA) conference in December 2013. Russell Wesdyk of FDA’s Center for Drug Evaluation and Research (CDER) Office of Strategic Products (OSP) delivered the opening plenary session. He assured attendees that the Agency will not use these metrics as a “restaurant-style grade”
• FDA is in listening mode. What metrics will work? How do we define them? How should they be used? What algorithm will be used to stratify risk? Wesdyk stated that the Agency would like to see three broad categories of metrics:
  • Product quality – with focus on the patient
  • Site quality – focusing on manufacturing performance
  • Quality system effectiveness – focusing on the quality system inspection technique (QSIT) six-system model. The first two
    categories would be collected at the Agency and used to adjust the inspection schedule, and the third category would be evaluated during the periodic inspections

Break-out sessions and presentations at the FDA-PDA conference were quick to point out the challenges that must be addressed.

• Can we agree on definitions?
• How do we balance lagging indicators with predictive measures?
• How do we objectively evaluate risk across such a broad industry spectrum of manufacturing processes, drug types and patient indications?

The International Society for Pharmaceutical Engineering (ISPE) responded to this challenge by holding two well-attended public meetings and issuing a white paper in December 2013: “ISPE Proposals for FDA Quality Metrics Program”. You can obtain current information on the ISPE Quality Metrics Initiative on the ISPE website (www.ispe.org/quality-metricsinitiative). Manufacturers who participate in the ISPE Quality Metrics Pilot Program can influence the selection of metrics and definitions, obtain a blinded comparison to the industry average among technology platform peers and get a head start on metric implementation and data collection. As of October 28, 2014, 18 companies and 44 sites are participating in the pilot. If you would like to participate, contact [email protected].

What’s Next?

Based on FDA and industry input, ISPE has proposed 14 metrics and pilot company experience with these metrics will drive changes before the end of the year.

Definitions for many of these metrics are available in the ISPE quality metrics white paper referenced above. Since life is far from perfect, we know that some of the proposed metrics will be difficult to measure, such as the CAPA effectiveness rate and quality culture. That said, articles in previous issues of the Journal could help.

Note! If you do not like the proposed metrics, here is your opportunity to get involved and influence the outcome! Metrics that drive appropriate behaviors and actions by firms are needed. Deviation reoccurrence rates are clearly going to help companies focus on CAPA effectiveness. Furthermore, a metric that reduces reoccurring events and encourages straightthrough processing will result in improved product quality.

Steven Mendivil, co-chair of the December FDA-PDA, provided some useful insights:

“Both the absolute value and trends of any given metric or suite of metrics might be valuable relative to making both direct comparisons (segmenting products and sites) and promoting continuous improvement.”
“Various factors make comparing raw metric data between companies very difficult. It isn’t only about the number (the metric) – it is about the trend and variability that measures risk and drives continuous improvement.”

Your Call to Action 1. ISPE has provided the opportunity to comment and influence – make use of it! - 2. Move from data overload to becoming information savvy – start trending! Trend charts are the easiest way to detect emerging safety and quality signals, and can provide an early warning that quality systems or processes are unstable.Prompt action can prevent adverse trends from developing into higher reject rates and product shortages. Forward thinking companies are using trend analysis, evaluating process capability and using continuous process verification to their advantage as a tool to improve process reliability - 3. Before selecting any measure, consider the behavior it will drive - 4. Get the balance between leading (80%) and lagging (20%) indicators right - 5. Less is more. Avoid ‘death by measure’ – the more you measure the less you know