Runaway and Homeless Youth Grants: Improvements Needed in the Grant Award Process

In FY12, HHS reported that its grant reviewers are trained and instructed to review and score each application independently based on the published criteria. In FY13, the agency reported that ACF streamlined the grants process and standardized all training across the agency. Internal guidance was provided to program offices regarding the grant review process and program offices must ensure that reviewers do not use any templates other than the published funding opportunity announcement (FOA) for review criteria. In addition, as part of the grant review process, each panel is overseen by a federal staff member to ensure that comments and scores reflect the criteria in the published FOA. HHS reported in May 2014 that ACF continues to provide extensive training to program offices on how to draft FOA's, specifically where information is to be placed within the FOA. Peer reviewers are instructed to review all applications using the evaluation criteria located in Section V.1 of the FOA. Evaluation criteria are not included in any other sections of the FOA. ACF policy states that offices must ensure that reviewers do not use any templates other than the published FOA for review criteria as described in Section V.1 only. In addition, ACF uses the ARM system, a web-based tool used for managing, reviewing, and monitoring the review process for applications for each published FOA. Comments and or scores are not finalized until reviewed and approved by the federal staff member.

In 2010, HHS commented that the agency elected to accept reviewers who are knowledgeable of the risk factors faced by runaway and homeless youth, and that many professional disciplines often intersect with runaway and homeless youth. The agency plans to take steps to ensure that all reviewers possess the knowledge and expertise in the particular program for which they are reviewing grant applications. In the event of a shortage of reviewers, they will staff panels with at least one peer reviewer with extensive relevant knowledge, which would continue to differ from the current policy that grants must receive an objective, advisory review by a minimum of "three" unbiased reviewers with expertise in the programmatic area for which applications are submitted. GAO notes that because professionals in varied disciplines could have sufficient expertise to serve as reviewers and it may be difficult for the agency to find enough reviewers with expertise in a particular program, this recommendation includes those that have expertise in the programmatic area for which they are evaluating grant applications, and not a specific program. In FY12, HHS responded that it actively recruits reviewers from the Runaway and Homeless Youth (RHY) field. When that's not possible, they make a concerted effort to find individuals from professional disciplines who have expertise and a working knowledge of the field. Since the last update in FY 12, ACF has implemented and used the ARM system for reviews. The ARM system is a web-based tool used for managing, reviewing, and monitoring the review process for grant applications. The Application Review Module (ARM) greatly increases the number of potential reviewers than can be available for reviews since it gives program offices the flexibility to access and hold reviews with individuals across the country. The grant review logistics contractor ensures that cumulative professional experiences meet a baseline of knowledge required to effectively evaluate RHY grant applications. In addition, ACF continues to take steps to ensure that all reviewers possess the knowledge and expertise through orientation and continuing education of the subject matter prior to selection as a reviewer.

In 2010, HHS indicated that all meetings for peer reviewers and chairs are already mandatory but due to unforeseen factors, it is not always possible for all reviewers to attend. GAO observed that attendance at these meetings was not explicitly mandatory, but highly encouraged. Furthermore, attendance was not enforced and attendance records were not maintained. HHS plans to officially notify all reviewers and chairpersons participating in future reviews that all training is mandatory. In the event some reviewers and chairpersons are not able to attend the mandatory training sessions due to unforeseen circumstances, the agency intends to offer "make up" sessions. In FY12, HHS reported that all grant review meetings are mandatory for all review participants.

In FY12, HHS reported that it now uses the Application Review Module (ARM), a secure web-based system, to track the comments flowing between federal staff and the panels. ARM captures comments of federal staff on each application provided as part of the back and forth in their roles as panel managers, and electronically captures and maintains the reviewer comments and scores associated with a panel review.

In 2010, HHS reported its plans to include more details concerning geographic distribution in the letters to applicants who are denied grants for this reason. In FY12 and FY13, HHS reported that it documents the reasons for not funding high scoring applications as a part of the funding decision memo, which is maintained in the GrantSolutions system, ACF's official record keeping system. The reasons for not funding an applicant are documented in accordance with the HHS Awarding Agency Grants Administration Manual, which requires that "if the application's position in the list of applications approved for funding is different from its position in the ranking list, a statement of the reasons for the difference that influenced the judgment of the approval. This should include a justification for funding of the particular application." HHS reported in FY13 that it is beginning to note in the letters to unsuccessful grant applicants when a geographic exception resulted in their application being unfunded.

HHS reported in FY12 that it took steps to provide clear information to applicants. Applicants that are disqualified are screened and notified before the grant review; these applications are not reviewed and scored. ACF is standardizing the screen out letters to these applicants so they are clear that they did not meet the requirements. Starting in FY 2013, every applicant - whether funded or unfunded - receives the aggregated score of their application along with the compilation of reviewer comments outlining the strengths and weaknesses on all criteria as determined by the review. For those applicants who may not have been funded due to low scores, this will be helpful in clarifying how the applicants failed to meet the evaluation criteria. When the pages in an application are decreased due to formatting requirements (e.g., page limitations, single spacing, number of files uploaded), the applicant will receive a letter from ACF notifying them why their application has been reduced. In addition, staff are made available by phone and/or email to answer questions from concerned applicants.

In 2010, HHS commented that, as part of the grant application process, the agency plans to explain to applicants that final grant decisions depend on the results of the grant award negotiations between ACF and the prospective grantees. In FY12, HHS reported that immediately following the finalization of funding decisions, organizations whose applications were not selected for funding are notified by letter. In FY13, HHS reported that ACF continues to refine the grant negotiation process to provide timely notification to successful and unsuccessful applicants. Successful applicants will be notified via the GrantSolutions system, informing them how to access their Notice of Award and terms and conditions. Unsuccessful applicants will be notified either by email or hard copy mail. These applicants will also receive the average score and a compilation of the reviewers strengths and weakness statements on all criteria. If the application was reduced, the applicant will also receive a letter explaining the reason for the reduction.