Data Mining:

Agencies Have Taken Key Steps to Protect Privacy in Selected Efforts, but Significant Compliance Issues Remain

GAO-05-866: Published: Aug 15, 2005. Publicly Released: Aug 29, 2005.

Additional Materials:

Contact:

David A. Powner
(202) 512-6240
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Data mining--a technique for extracting knowledge from large volumes of data--is being used increasingly by the government and by the private sector. Many federal data mining efforts involve the use of personal information, which can originate from government sources as well as private sector organizations. The federal government's increased use of data mining since the terrorist attacks of September 11, 2001, has raised public and congressional concerns. As a result, GAO was asked to describe the characteristics of five federal data mining efforts and to determine whether agencies are providing adequate privacy and security protection for the information systems used in the efforts and for individuals potentially affected by these data mining efforts.

The five data mining efforts we reviewed are used by federal agencies to fulfill a variety of purposes and use various information sources, including both information collected on behalf of the agency and information originally collected by other agencies and commercial sources. Although the systems differed, the general process each used was basically the same. Each system incorporates data input, data analysis, and results output. While the agencies responsible for these five efforts took many of the key steps required by federal law and executive branch guidance for the protection of personal information, they did not comply with all related laws and guidance. Specifically, most agencies notified the general public that they were collecting and using personal information and provided opportunities for individuals to review personal information when required by the Privacy Act. However, agencies are also required to provide notice to individual respondents explaining why the information is being collected; two agencies provided this notice, one did not provide it, and two claimed an allowable exemption from this requirement because the systems were used for law enforcement. In addition, agency compliance with key security requirements was inconsistent. Finally, three of the five agencies completed privacy impact assessments--important for analyzing the privacy implications of a system or data collection--but none of the assessments fully complied with Office of Management and Budget guidance. Until agencies fully comply with these requirements, they lack assurance that individual privacy rights are being appropriately protected.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Administrator of the General Services Administration should ensure that the appropriate information security measures defined in OMB and NIST guidance are applied to the systems used in the Citibank Custom Reporting System data mining effort, including the development of a risk assessment, a system security plan, a tested contingency plan, the performance of regular testing and evaluation, and the completion of certification and accreditation by agency management.

    Agency Affected: General Services Administration

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of the General Service Administration (GSA), which contracts to use Citibank?s Custom Reporting System (CCRS), we recommended that the Administrator of GSA ensure that appropriate information security measures defined in Office of Management and Budget (OMB) and National Institute of Standards and Technology (NIST) guidance are applied to the systems used in the CCRS data mining effort, including the development of a risk assessment, a system security plan, a tested contingency plan, the performance of regular testing and evaluation, and the completion of certification and accreditation by agency management. As we recommended, GSA developed a risk assessment report and system security plan, and regularly performed testing and evaluations on the Citibank Commercial Cards System that it uses. The CCRS is a subcomponent of this system. In addition, the CCRS was certified and issued an Authority to Operate Accreditation in November 2007. The performance of regular system testing and evaluation against NIST guidance helps ensure that this data mining effort includes adequate privacy protections.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of the Treasury should direct the Commissioner of the Internal Revenue Service to revise the privacy impact assessment for the Internal Revenue Service's Reveal system to comply with OMB guidance, including analyses of the information to be collected, the purposes of the collection, the intended use of the information, how the information is to be secured, and opportunities for impacted individuals to comment.

    Agency Affected: Department of the Treasury

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of IRS Reveal, we recommended that the Secretary of the Treasury direct the Commissioner of the Internal Revenue Service to revise the privacy impact assessment for the Reveal system to comply with OMB guidance, including analyses of the information to be collected, the purposes of the collection, the intended use of the information, how the information is to be secured, and opportunities for impacted individuals to comment. Since we reported in August 2005, IRS Reveal has migrated to the Criminal Investigation System Domain (CI-1) General Support System (GSS), and now has a revised PIA. The CI-1 GSS PIA was completed by the IRS Office of Privacy in April 2006 and includes Reveal as a component. This CI-1 GSS PIA was approved by the Director of the Office of Privacy in May 2006. The PIA complies with OMB guidance; specifically, it includes analyses of the information to be collected, the purposes of the collection, the intended use of the information, and how the information is to be secured. The revised PIA should help ensure that this data mining effort includes adequate privacy protections.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of the Treasury should direct the Commissioner of the Internal Revenue Service to apply the appropriate information security measures defined in Office of Management and Budget (OMB) and National Institute of Standards and Technology (NIST) guidance to the systems used in the Reveal data mining effort, specifically, the performance of regular system testing and evaluation against NIST guidance.

    Agency Affected: Department of the Treasury

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of Internal Revenue Service's (IRS) Reveal system, we recommended that the Secretary of the Treasury direct the Commissioner of the IRS to apply the appropriate information security measures defined in Office of Management and Budget (OMB) and National Institute of Standards and Technology (NIST) guidance to the systems used in the Reveal data mining effort, specifically, the performance of regular system testing and evaluation against NIST guidance. The Reveal system was migrated to the Criminal Investigation System Domain (CI-1) General Support System (GSS). The Reveal system is therefore addressed in the CI-1 GSS Interim Authority to Operate (IATO) Certification on June 23, 2005, and Accreditation on August 11, 2005. The subsequent Authority to Operate (ATO) Accreditation was issued on March 23, 2006, for the CI-1 GSS. Based on the ATO Accreditation, for the CI-1 GSS, which Reveal was migrated to, the IRS has implemented this recommendation. The performance of regular system testing and evaluation against NIST guidance helps ensure that this data mining effort includes adequate privacy protections.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of Agriculture should direct the Administrator of RMA to make the completed privacy impact assessment available to the public, as appropriate.

    Agency Affected: Department of Agriculture

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. For the system used by the Department of Agriculture's (USDA) Risk Management Agency, we found that a privacy impact assessment had not been made available to the public, even though it did not contain any sensitive information that would prevent its public release. We therefore recommended that the department revise the assessment and release it to the public, as appropriate. In September 2006, USDA revised the system's privacy assessment. The assessment was subsequently released to the public on the department's Web site. By providing this information to the public, USDA is better able to balance the operational needs of the program with individuals' rights to privacy.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of Agriculture should direct the Administrator of RMA to have the completed privacy impact assessment approved by the chief information officer or equivalent official.

    Agency Affected: Department of Agriculture

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of the system used by the Department of Agriculture's (USDA) Risk Management Agency (RMA), we found that the system's privacy impact assessment was not approved by the Department's Chief Information Officer (CIO), as required by the E-Government Act of 2002. We therefore recommended that the Department have its privacy impact assessment approved by the CIO, or equivalent official. In response, USDA's CIO approved the privacy impact assessment for RMA's data mining system in September 2006. By ensuring a thorough review of the system's privacy impact assessment, USDA will be better able to balance the operational needs of the program with individuals' rights to privacy.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of Agriculture should direct the Administrator of RMA to revise the privacy impact assessment for the RMA data mining effort to comply with OMB guidance, including analyses of the intended use of the information it collects, with whom the information will be shared, how the information is to be secured, opportunities for impacted individuals to comment, and the choices made by the agency as a result of the assessment.

    Agency Affected: Department of Agriculture

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of the system used by the Department of Agriculture's (USDA) Risk Management Agency (RMA), we found that the system's privacy impact assessment did not address all of the information required under Office of Management and Budget (OMB) guidance. We recommended that USDA revise the assessment to address the required elements, including analyses of the intended use of the information it collects, with whom the information will be shared, how the information is to be secured, and opportunities for impacted individuals to comment. In September 2006, USDA completed a revised assessment for the RMA data mining system, which substantially addressed the OMB requirements outlined above. As a result of this more comprehensive assessment, the Department should be better able to balance the operational needs of the program with individuals' rights to privacy.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of Agriculture should direct the Administrator of RMA to develop and implement procedures that ensure the accuracy, relevance, timeliness, and completeness of personal information used in the RMA data mining effort to make determinations about individuals.

    Agency Affected: Department of Agriculture

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining efforts. In regards to the Risk Management Agency (RMA), we recommended that the Secretary of Agriculture direct the Administrator of RMA to develop and implement procedures that ensure the accuracy, relevance, timeliness, and completeness of personal information used to make determinations about individuals. As we recommended, RMA developed and published these procedures in a June 2008 handbook for its data validation system. This handbook is posted on RMA's website. By developing these procedures, RMA can better ensure the quality of records used to make determinations about individuals.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of Agriculture should direct the Administrator of RMA to apply the appropriate information security measures defined in OMB and NIST guidance to the systems used in the RMA data mining effort, specifically, the development of a complete system security plan, a tested contingency plan, and regular testing and evaluation of the systems used in the effort.

    Agency Affected: Department of Agriculture

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of the Risk Management Agency's (RMA) data mining system, we recommended that the Secretary of Agriculture direct the Administrator of RMA to apply the appropriate security measures defined in OMB and NIST guidance to the systems used in the RMA data mining effort, specifically, the development of a complete system security plan, a tested contingency plan, and regular testing and evaluation of the systems used in the effort. As we recommended, RMA developed a system security plan, developed and tested a contingency plan, and tested and evaluated its data mining system. The performance of system testing and evaluation against NIST guidance helps ensure that this data mining effort includes adequate privacy protections.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of Agriculture should direct the Administrator of the Risk Management Agency (RMA) to provide the required Privacy Act notices to individuals, including producers, insurance agents, and adjusters, when personal information is collected from them.

    Agency Affected: Department of Agriculture

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining efforts. Specifically, we reported that the Risk Management Agency (RMA) had not provided the required Privacy Act notice to all individuals who supplied personal information. We therefore recommended that RMA provide these required notices to individuals, including producers, insurance agents, and adjusters, when personal information is collected from them. As we recommended, RMA developed a Privacy Act statement to employ each time personally identifiable information is collected by an Approved Insurance Provider from an agent, loss adjuster, and policyholder. Based on our recommendation, in December 2008, RMA also issued a bulletin in which it requires Approved Insurance Providers to incorporate the statement each time they seek to obtain personal information from these individuals. By providing the Privacy Act notice to individuals when personal information is collected from them, RMA can help ensure that individual privacy rights are being appropriately protected.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of the Treasury should direct the Commissioner of the Internal Revenue Service to make the completed privacy impact assessment available to the public, as appropriate.

    Agency Affected: Department of the Treasury

    Status: Closed - Not Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of IRS's Reveal System, we recommended that the Secretary of the Treasury should direct the Commissioner of the Internal Revenue Service to make the completed privacy impact assessment available to the public, as appropriate. Since then, the Reveal system was migrated to Criminal Investigation System Domain (CI-1) General Support System (GSS). The C1-1 GSS PIA was completed and approved by the IRS Office of Privacy on April 27, 2006. The PIA includes Reveal as a component. The CI-1 PIA is not publicly posted due to the sensitive nature of the GSS.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Attorney General should direct the Director of the Federal Bureau of Investigation to apply the appropriate information security measures defined in OMB and NIST guidance to the systems used in the Foreign Terrorist Tracking Task Force data mining effort, including the development of tested contingency plans.

    Agency Affected: Department of Justice

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data-mining systems. In the case of the Department of Justice's Federal Bureau of Investigation's (FBI) Foreign Terrorist Tracking Task Force (FTTTF) data mining effort, we reported that the FBI had not demonstrated that they had tested contingency plans. We therefore recommended that the FBI develop these plans for its FTTTF data mining effort. As we recommended, in December 2008, the FBI developed an FTTTF Information Technology (IT) Contingency Plan and tested the plan in February 2009. By developing a tested contingency plan, the FBI can more effectively respond to and recover from damage following an unexpected interruption.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Administrator of the General Services Administration should publish a system of records notice for the purchase card program that specifies the name of the system, the categories of individuals and records in the system, the categories of information sources used by the system, the routine uses of the system, how the agency stores and maintains the system, the individual responsible for the effort, the process by which individuals can request notification that the system includes records about them, and the procedures individuals should use to review records pertaining to them.

    Agency Affected: General Services Administration

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data-mining systems. In the case of GSA's purchase card program, we recommended that the Administrator of the General Services Administration publish a system of records notice for the purchase card program that specifies the name of the system, the categories of individuals and records in the system, the categories of information sources used by the system, the routine uses of the system, how the agency stores and maintains the system, the individual responsible for the effort, the process by which individuals can request notification that the system includes records about them, and the procedures individuals should use to review records pertaining to them. In June 2006, the General Services Administration (GSA) published a governmentwide system of records notice for the GSA SmartPay Purchase Card Program. Consistent with our recommendation, the notice includes the name of the system; the categories of individuals covered by the system; the categories of records in the system; the routine uses of the system records, including categories of users and their purpose for using the system; the policies and practices for storing, retrieving, accessing, retaining, and disposing of system records; the system manager; the process by which individuals can request notification about their records from a purchase card manager; and the procedure by which individuals can request access to their records. As a result of publishing a record of systems notice, privacy protections are strengthened. Further, those whose information is used by the system can better understand the use of such information, as well as understanding how to get that information.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Administrator of the Small Business Administration should make the completed privacy impact assessment available to the public, as appropriate.

    Agency Affected: Small Business Administration

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of the Small Business Administration?s (SBA) Loan/Lender Monitoring System, we found that the system's privacy impact assessment did not address all of the information required under Office of Management and Budget (OMB) guidance. We therefore recommended that SBA revise the assessment to address the required elements, including analyses of the information to be collected, the purposes of the collection, the intended use of the information, how the information is to be secured, opportunities for impacted individuals to comment, and the choices made by the agency as a result of the assessment. In July 2009, SBA completed a revised assessment for its Loan /Lender Monitoring System and posted the notice on its public web site. Based on our recommendation, SBA assessed the information that is collected by the system, the purposes of collection of this information, and the intended use of the information. Further, SBA identified how the information is to be secured. As a result of this more comprehensive assessment, SBA should be better able to balance the operational needs of the program with individuals' rights to privacy.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Administrator of the Small Business Administration should complete a privacy impact assessment for the data mining effort that complies with OMB guidance, including analyses of the information to be collected, the purposes of the collection, the intended use of the information, how the information is to be secured, opportunities for impacted individuals to comment, and the choices made by the agency as a result of the assessment.

    Agency Affected: Small Business Administration

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data mining systems. In the case of the Small Business Administration?s (SBA) Loan/Lender Monitoring System, we found that the system's privacy impact assessment did not address all of the information required under Office of Management and Budget (OMB) guidance. We therefore recommended that SBA revise the assessment to address the required elements, including analyses of the information to be collected, the purposes of the collection, the intended use of the information, how the information is to be secured, opportunities for impacted individuals to comment, and the choices made by the agency as a result of the assessment. In July 2009, SBA completed a revised assessment for its Loan /Lender Monitoring System. Based on our recommendation, SBA assessed the information that is collected by the system, the purposes of collection of this information, and the intended use of the information. Further, SBA identified how the information is to be secured. As a result of this more comprehensive assessment, SBA should be better able to balance the operational needs of the program with individuals' rights to privacy.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Administrator of the Small Business Administration should amend the system of records notice regarding its data mining effort to clearly identify the individual responsible for the effort, the process by which individuals can request notification that the system includes records about them, and the procedures individuals should use to review records pertaining to them.

    Agency Affected: Small Business Administration

    Status: Closed - Implemented

    Comments: In August 2005, we reported on the privacy and security protections used in several federal data-mining systems. In the case of the Small Business Administration?s (SBA) Loan/Lender Monitoring System, we recommended that the Administrator of SBA amend its system of records notice regarding its data mining effort to clearly identify the individual responsible for the effort, the process by which individuals can request notification that the system includes records about them, and the procedures individuals should use to review records pertaining to them. In April 2009, SBA published a revised system of records notice for its Loan/Lender Monitoring System. Consistent with our recommendation, the revised notice identifies the system managers; the process by which individuals can request notification about their records from a systems manager; and the procedure by which individuals can request access to their records. By publishing a revised record of systems notice, individuals whose information is used by the system can better understand how to review that information.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Secretary of State should direct the Under Secretary for Management to notify purchase card participants of the legal basis under which the department collects their personal information, as required.

    Agency Affected: Department of State

    Status: Closed - Implemented

    Comments: In response to our recommendation, in February 2006, the Department of State modified its purchase card toolkit template and all other toolkit templates used to collect personal information to include a privacy notice which notifies individuals of the agency's legal authority to collect the requested information, the purpose(s) for collecting the information, and routine use(s) of the information.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Attorney General should direct the Director of the Federal Bureau of Investigation to make the completed privacy impact assessment available to the public, as appropriate.

    Agency Affected: Department of Justice

    Status: Closed - Implemented

    Comments: In October 2005, FBI completed a privacy impact assessment that was consistent with OMB guidance, addressing the information to be collected, why it was collected, the intended use of the information, with whom it will be shared, the opportunities for individuals to review information about themselves, and how the information will be secured. The assessment was approved by the FBI's Senior Privacy Official, in consultation with the agency's Privacy Council. The approval was contingent on FTTTF meeting several conditions which should strengthen the privacy of individuals whose information is used in the FTTTF systems. Consistent with OMB guidance, FBI does not plan to publicly release the assessment because it includes sensitive information.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Attorney General should direct the Director of the Federal Bureau of Investigation to have the completed privacy impact assessment approved by the chief information officer or equivalent official.

    Agency Affected: Department of Justice

    Status: Closed - Implemented

    Comments: In October 2005, FBI completed a privacy impact assessment that was consistent with OMB guidance, addressing the information to be collected, why it was collected, the intended use of the information, with whom it will be shared, the opportunities for individuals to review information about themselves, and how the information will be secured. The assessment was approved by the FBI's Senior Privacy Official, in consultation with the agency's Privacy Council. The approval was contingent on FTTTF meeting several conditions which should strengthen the privacy of individuals whose information is used in the FTTTF systems. Consistent with OMB guidance, FBI does not plan to publicly release the assessment because it includes sensitive information.

    Recommendation: To ensure that the data mining efforts reviewed include adequate privacy protections, the Attorney General should direct the Director of the Federal Bureau of Investigation to establish a date for the completion of a privacy impact assessment for its data mining effort that complies with OMB guidance, including analyses of the information to be collected, the purposes of the collection, the intended use of the information, with whom information will be shared, how the information is to be secured, opportunities for impacted individuals to comment, and the choices made by the agency as a result of the assessment.

    Agency Affected: Department of Justice

    Status: Closed - Implemented

    Comments: In October 2005, FBI completed a privacy impact assessment that was consistent with OMB guidance, addressing the information to be collected, why it was collected, the intended use of the information, with whom it will be shared, the opportunities for individuals to review information about themselves, and how the information will be secured. The assessment was approved by the FBI's Senior Privacy Official, in consultation with the agency's Privacy Council. The approval was contingent on FTTTF meeting several conditions which should strengthen the privacy of individuals whose information is used in the FTTTF systems. Consistent with OMB guidance, FBI does not plan to publicly release the assessment because it includes sensitive information.

    Jun 10, 2014

    May 22, 2014

    May 12, 2014

    May 8, 2014

    May 7, 2014

    Apr 2, 2014

    Feb 26, 2014

    Feb 12, 2014

    Looking for more? Browse all our products here