Review Process

Reviewers will help decide which applications are fundable, and within those, which are highly competitive versus competitive. Another group, the Executive Committee, uses the information that you provide in the context of the existing portfolio of and program goals in order to make the final funding decisions. More on the process here. 

Reviewers will be sent login instructions in order to view the applications that have been assigned to you and link to the review form. They will complete an individual review and then be invited to a call to discuss the applications with other reviewers. 

Reviewers should follow the steps below to ensure the highest quality review:

• Allow adequate time to review applications.
• **Review the rubric** before evaluating applications, and review it if you take a break and return to reviewing. As a reminder, you may be asked to evaluate applications from ventures that are in very different stages of development. Answers and asks from very early-stage applicants should not be directly compared to those from ventures that are further along! Ventures in an idea stage are given as much consideration as those that are further along.
• Preview applications before providing in-depth scores and feedback on any one application. Disclose to the Managing Director any financial interest and/or any board positions in an applicant OR in any competitor to the applicant before proceeding. Reflect on your own biases that you may bring in evaluating them. 
• Take notes! This can help you assert well-grounded positions with specific details during the group discussion, even if it conflicts with other reviewers. You may also want to note to yourself when you have a strong first impression to an application, prior exposure to the applicant, or a strong impression based on how information is conveyed versus what is conveyed – all of these may unknowingly impact your impressions, so noting them helps bring them out of the subconscious.
• Review your scores and feedback at the end and ask yourself some questions:
  • What alternative views could be justified?
  • How could you play devil’s advocate on your decisions or feedback?
  • Would your review be different if something about the applicant were different?
  • Is the feedback constructive and point to specific examples?
  • If you had to read your feedback to the applicant out loud, in-person, would you change it?

Confidentiality and Conflict of Interest

Reviewers may not disclose, divulge, or reveal application information or any part of it to any person other than those employees of the Maryland Launch Fund, University System of Maryland, or State employees or agents who reasonably have a need for such information and who are similarly bound.

Reviewers must not have or must disclose any financial interest and/or any board positions in any applicant that is submitting for the opportunity. For those applications that a reviewer is asked to reviewer, they also must disclose any such relationships with competitors to the applicant as well.

The Importance of Recognizing the Role of Bias In Reviewing Applications

We all have biases. Human brains are wired to take mental shortcuts in order to make decisions. These include judgements based on previous experiences and knowledge. We are often not aware of these mental models; they can operate in our subconscious mind.

Although biases are natural and inevitable, individuals can consciously engage in strategies to change their biases over time or keep them in check in a specific situation to help reduce the impact it has on decision-making. 

Situations that foster biased thinking include ambiguity and uncertainty. As you know, early-stage entrepreneurship is full of ambiguity for entrepreneurs and decisions about them. However, through thoughtful program design and help from reviewers regarding biases, the program aims to optimize the review process to align with the core values of diversity, equity, and inclusion.

• What Are Examples of Relevant Biases?
  • Anchoring Bias – Placing too much weight in a first impression or early information
  • Attractiveness Bias – Affiliating attractive things and people with success
  • Language Bias – Favoring communication that is similar to how we communicate
  • Confirmation Bias – Favoring information that confirms previously existing beliefs
  • Affinity Bias – Favoring people who are similar to us
  • Familiarity Bias – Favoring what is familiar to us
  • Halo/Horn Bias – Letting a positive or negative opinion of one attribute of something or someone affect other attributes
  • Social Stereotype Bias – Letting social attributes of someone (e.g., age, ethnicity, race, nationality, gender, occupation) affect thoughts, feelings, or assumptions about them.
  • Conformity Bias – Aligning behavior with others rather than exercising independent judgment.
• How Might Bias Show Up in the Launch Fund Review Process?

• • A poorly- or well-written first application answer can affect how you perceive the rest of the application.
  • If you have a previous first impression (good or bad) of an applicant, it can shape how you view their application.
  • The first application you view may unintentionally set a bar for how you will score the rest. 
  • Great visuals can unintentionally lend credibility in other ways and vice versa. Answers that are well-written may hide lack of substance and vice versa.
  • Spelling, grammar, and word choice may unduly affect impressions of the contents of the application.
  • Videos of applicants showing appearance and speaking style, including accents, may unfairly alter the perception of an application.
  • In group discussion, well-grounded dissenting views could be undervalued.
• How Do Reviewers and Program Design Work to Reduce the Impact of Bias?
  • Follow the instructions at the top of the page closely; they are designed to help minimize the impact of bias. 
  • The rubric attempts to provide context for consistent evaluation.
  • Names and demographics are collected but not shared with reviewers.
  • The default written application reduces opportunity for visual and auditory bias. Flexibility in application format allows for applicants to communicate in the way(s) that they think is most favorable. 
  • Reviewers are asked about any previous perceptions of applicants.
  • We provide education to reviewers on bias, recruit a diverse pool of reviewers to bring different perspectives, and moderate the group discussion to attempt to reduce groupthink. This is all to reduce bias from individuals and reduce the amplification of any biases that remain. 

The information on this page currently draws on materials from two main sources, which provide funding in contexts that share some similarity with this program. We welcome other resource recommendations! Get in touch at launch@usmd.edu.

• The National Science Foundation – https://beta.nsf.gov/od/oia/merit-review-orientation. 
• Chicago Beyond – https://chicagobeyond.org/whatwedo/.