Imported Food Safety: FDA's Targeting Tool Has Enhanced Screening, but Further Improvements Are Possible
Highlights
What GAO Found
The Food and Drug Administration's (FDA) Predictive Risk-based Evaluation for Dynamic Import Compliance Targeting (PREDICT) tool uses a variety of data and analyzes data by applying rules—conditional statements that tell PREDICT how to react when encountering particular information—to generate risk scores for imported food. Many of the data used by PREDICT come from internal FDA sources, such as FDA databases. PREDICT also uses data from sources outside of FDA, such as other federal agencies, states, and foreign governments. Some of the data are open source data—information that is publicly available, such as information from newspapers and websites. FDA's Office of Regulatory Affairs (ORA) relies on other FDA offices and federal agencies to provide open source data for PREDICT, but ORA does not have a documented process for identifying, obtaining, and using the data, relying instead on an ad hoc process. Federal standards for internal control call for agencies to document internal controls. Without a documented process for identifying, obtaining, and using open source data, FDA does not have reasonable assurance that ORA will consistently identify, obtain, and use such data for PREDICT.
The implementation of the FDA Food Safety Modernization Act (FSMA), enacted in 2011, will provide PREDICT with additional data for estimating the risk of imported food. FDA identified FSMA provisions likely to generate data, including the Foreign Supplier Verification Program (FSVP), which requires importers to verify that their foreign suppliers use processes and procedures that provide the same public health protection as applicable U.S. requirements. Because FSVP and other FSMA-related programs are still new and not yet fully implemented, the details of how PREDICT will use the data have not been worked out. However, according to FDA officials, the data will be useful. For example, FSVP data will identify suppliers that are not in compliance with standards, and PREDICT will use those data in assigning risk scores to imports from those suppliers.
FDA has assessed the effectiveness of PREDICT by monitoring of key data and by conducting an internal evaluation of the system, and it has implemented many, but not all, of its recommendations from the evaluation. GAO's analysis of FDA data from fiscal year 2012 through 2014 shows that in general, PREDICT is working as intended: imported food with higher-risk scores is more likely to be physically examined and to be found in violation of food safety standards or labeling requirements. In May 2013, FDA completed an evaluation of PREDICT that examined five key processes. As a result of the evaluation, FDA developed 24 recommendations for improving PREDICT and prioritized these recommendations based on feasibility and impact. According to FDA, the agency has fully implemented 15, partially implemented 6, and not implemented 3 of the recommendations. FDA officials said that the agency has not fully implemented all recommendations because of a lack of resources. However, federal standards for internal control specify that agencies are to ensure that the findings of reviews are promptly resolved. To that end, agencies are to complete, within established time frames, all actions that correct or otherwise resolve the matters brought to management's attention. Establishing a timeline for implementing the remaining recommendations as resources become available would help ensure that PREDICT continues to remain an effective tool for screening imported food.
Why GAO Did This Study
Imported food makes up a substantial and growing portion of the U.S. food supply. FDA is responsible for oversight of more than 80 percent of the U.S. food supply. However, because the volume of imported food is so high, FDA physically examines only about 1 percent of imported food annually. In 2011, FDA implemented PREDICT, a computerized tool intended to improve FDA's targeting of imports for examination by estimating the risk of imported products.
GAO was asked to review how FDA is using PREDICT to protect the U.S. food supply. This report examines (1) the data used by PREDICT and how PREDICT analyzes these data to identify high-risk food shipments for examination, (2) how implementation of FSMA will affect PREDICT, and (3) the extent to which FDA has assessed the effectiveness of PREDICT and used the assessments to improve the tool. To address these issues, GAO analyzed FDA documents, interviewed FDA officials, and analyzed data from fiscal years 2012 through 2014.
Recommendations
GAO recommends that FDA take two actions to improve the effectiveness of PREDICT: (1) document the process by which FDA is to identify, obtain, and use open source data and (2) establish a timeline for implementing the remaining recommendations from FDA's 2013 evaluation of PREDICT. FDA generally agreed with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Health and Human Services | To further enhance FDA's PREDICT tool and its ability to ensure the safety of imported food, the Secretary of Health and Human Services should direct the Commissioner of FDA to document the process for identifying the type of open source data to collect, obtaining such data, and determining how PREDICT is to use the data. |
FDA told us in July 2019 that they had established a project charter with the goal of drafting a procedure in coordination with FDA field offices. The draft is under review, and FDA said that it anticipated finalizing it in the first quarter of FY 2020. We are following up with FDA and will report on any actions taken once we receive FDA's update, likely in the summer of 2020. In May 2020, FDA said that its Office of Regulatory Affairs (ORA) had modified the Global Watch work activity in the Compliance Management System (CMS) database to enable FDA foreign staff to submit information through CMS. This allows FDA headquarters and the various FDA Centers to easily access detailed information related to FDA-regulated products and determine the need to adjust FDA import surveillance activities. In addition, FDA's Office of Enforcement and Import Operations' Division of Import Operations finalized an internal Quality Management System procedure (SOP-00419) titled "Global Watch: Utilizing Open Source Information from FDA's Foreign Offices for Import Screening" on October 4, 2019. This document details the appropriate routing and triaging of information related to FDA-regulated products. However, the document explicitly states that it is not intended to "describe how open source information and intelligence is gathered." Therefore, these procedures do not document a process to identify what types of open source data to collect and how FDA obtains such data, which is part of our recommendation. While FDA's modifications to the Global Watch work activity has increased the ability for staff to submit open source information through CMS, SOP-00419 does not address how staff identify or obtain the information they submit. FDA's response does not provide evidence that the agency is identifying and collecting open source information in a systematic way.
|
| Department of Health and Human Services | To further enhance FDA's PREDICT tool and its ability to ensure the safety of imported food, the Secretary of Health and Human Services should direct the Commissioner of FDA to establish a timeline for implementing, as resources become available, the remaining recommendations from FDA's 2013 evaluation of PREDICT. |
In our 2016 report on FDA's import screening tool, called PREDICT, we found that FDA had assessed the effectiveness of PREDICT by ongoing monitoring of key program data and by conducting an evaluation of the tool in 2013. We also found that FDA has implemented many, but not all, of the 2013 evaluation's 24 recommendations to improve PREDICT. We therefore recommended that FDA establish a timeline for implementing, as resources become available, the remaining recommendations from FDA's 2013 evaluation of PREDICT. In August 2017, FDA provided information to GAO describing the actions taken as a result of our recommendation. FDA has addressed or started to address some of the 9 open recommendations. However, FDA believes recent events have made some of these 2013 recommendations obsolete. One recommendation that FDA did implement was the recommendation to optimize review times for low risk products. The agency has focused on the implementation of the Automated Commercial Environment (ACE), a centralized system operated by U.S. Customs and Border Protection for all transactions related to imports and exports. ACE has contributed to improved sample targeting and optimizing review times for low-risk products. Initial analysis indicates that "Automated May Proceeds" have increased in ACE, and the percentage of import lines requiring manual review has decreased. In 2014, prior to ACE, only 26 percent of entries were "Automated May Proceeds." In 2016, after full implementation of ACE, 62 percent of lines were "Automated May Proceeds." FDA has not fully addressed other recommendations. In some cases, the agency has started to address its recommendation. For example, it has started to address its recommendations to develop a rule testing system and process, and to develop performance metrics to assess PREDICT's performance on an ongoing basis. The agency is in the process of developing a test that has several functions pertaining to rule development. Once fully deployed, the test would allow FDA to push a rule into live data to see how it functions against other rules. It would also let FDA see if the rule produces the desired outcome. PREDICT will therefore allow for more accurate rule design. FDA expects full deployment of the test rule in the coming year, resources permitting. In other cases, FDA maintains that the development of ACE has preempted the need to fully implement the 2013 study recommendations. For example, the agency told us that implementation of ACE has taken priority in FDA's continued efforts to review and improve the entry admissibility process, in place of the 2013 recommendations. Therefore, although FDA did not address all of the recommendations it made to itself in 2013, it did implement those that were still relevant to helping improve the PREDICT system. We therefore believe FDA has met the intent of our recommendation.
|