Dr. Andrew Holter
The war for talent has finally reached the shores of the fire service. Lateral transfers. Sign-on bonuses. Educational pay incentives. The fire service is sounding more and more like other service sectors. Many departments across the country have seen a steady and rapid decline in applicants for open positions. Promotional processes are becoming less competitive in many departments too. All the while, we are also working as a fire service to become more inclusive and equitable. Despite how long we’ve been doing it, for many of us, our hiring processes are still plagued with issues. A recent headline out of Vermont says several firefighters resigned following the controversial hiring of a new chief. Soon lawyers will be involved and asking questions about the interview process. We all likely need to create a more legally defensible, fair, and just process than our current one. So, as we collectively throw things at the wall to see what sticks, have we considered that the interview process might be keeping the best candidates out of those positions?
Setting A Foundation
Like building a house, creating an interview requires a solid foundation, and that foundation must come first in the process. A job analysis must be performed for all positions to understand the associated tasks, skills, and responsibilities. This is essential in building a structured interview because it establishes what elements are needed in the interview itself. Job descriptions are also created based on a job analysis. It may have been some time since the last job analysis was performed on positions in your organization, so it does not hurt to reexamine those roles for needed changes in the job description or for building an interview.
A job analysis generally uses observations and interviews to gather data on a position. Although interviews are often used and very helpful in the process, it is important to focus on the role rather than the individual to collect the pertinent information. A job analysis should show the skills and abilities needed, activities performed, working conditions, demonstrated behaviors, equipment used, physical demands, and other interactions. We can build an accurate job description and a structured interview with this information in hand.
Framing Your Structured Interview
There are many considerations when creating an interview, including the structure of it, instructions provided to candidates, how scoring will be done, the actual questions asked, and more. The first thing to consider is the overall structure of the interview process. Research has repeatedly proven that structured interviews are the best predictive measures of success in our toolbox despite seeming cold and heartless. What is a structured interview? A structured interview is an interview where all the questions are predetermined before the interview beings. The questions are all presented in the same way to each candidate and given in the same order. This approach ensures that every candidate is given a fair and equal interview, making this approach the most legally defensible. Unstructured and semi-structured interviews allow candidates and interviewers to have different experiences in each interview, creating a risk around the fairness and objectivity of the process.
Now comes the fun part, creating the interview questions that will largely determine who gets the job. Have you ever been asked what you would be if you were an animal? What does that question measure, and how does that apply to being a firefighter or an officer? What is the purpose of that question in an interview? When designing our interviews, we need to be critical of the questions we allow into the process. Often we are limited to less than 12 questions. Each question must have a purpose in the process and measure something related to the job analysis we performed. Usually, we ask about preparation for the job and associated traits and characteristics such as teamwork, problem-solving, communication, taking orders, and adherence to ethical standards. There are creative ways to let candidates express their ability to think critically and get creative, such as situational or scenario-based questions. But remember that these should have a specific purpose, such as measuring their conflict resolution abilities. With that said, think about removing the question about what type of animal they are and replacing it with something worth learning about your candidates. Candidates, especially those we see today in Generation Z (Born after 1996), will be quick to question the authenticity and justice present in an organization that judges someone based on questions in the interview that have no business being there. Ensure your process is fair and just, and ask purposeful and relevant questions.
Scoring the Interview
Far too many errors are made in this part of the process. These errors could be costing candidates their opportunities. Unfortunately, the most at-risk candidates are those diverse candidates the fire service has long struggled to reach and recruit. So, what is the best method for scoring interviews? First, we need to start with a scoring rubric or rating scale. Creating a rubric or scale will help eliminate bias in the interview process and help keep the process as objective as possible. Typically, a scale of five is used to rate responses. Each rating corresponds to a specific criterion that is spelled out to achieve a particular score. For example, the candidate answered the question completely by providing a specific example demonstrating the desired attributes. Each response should be rated independently. One response might get a five, and the next a one, which will happen. It is important not to let a grand slam in the first inning make you miss the rest of the interview and vice versa. While all attempts should be made to make the process as objective as possible, it is common to allow the interviewer to score the intangible overall performance of the candidate. Just be sure to spell out what the interviewer is grading.
So, the interview took place, and each question now has a corresponding score by each person on the interview panel. Now what? Spoiler alert. Please refrain from corrupting the process by deciding the overall score for that candidate as a group. Several departments in a class recently told me that this is what they are currently doing. A candidate leaves the interview, and immediately the group begins to discuss the performance in order to come to an agreement on a final overall score as a group. A whole new world of problems is introduced into the process by this type of consensus scoring or hiring. Suddenly that candidate’s performance can be decided by the loudest voice in the room or the most senior person. Think about the Olympics and picture the judges holding up those scorecards. They did not stop and talk about the scores amongst themselves, and together all decided on an eight. Instead, they all hold up numbers, having never shown the other judges. They are there for their expertise to judge the performance. Your interview panelists should be no different. Experts in the field are there to apply their perspectives through the objective lens of the scoring sheet or rubric. The most accurate thing to do is to collect the interviewers’ scoring sheets and allow them to be tallied by an independent third party who was not present in the interview process. Once scores are totaled, your organization can decide how best to move forward with that data. A best practice is to use descriptive statistics to understand the distribution of scores which can be used to set rating parameters such as “highly recommended, recommended, not recommended .”These parameters can be based on standard deviations from the mean, a more objective approach than a predetermined cutoff score.
Putting it Altogether
The last step that should happen before the interview is training the interview panelists. Some may have limited experience in interviewing, and it is a skill that must be taught. Potential legal pitfalls must be discussed, but you must also ensure that these panelists understand the scoring sheet and interview questions. It is best to do the training as close to the interview as possible so the training is fresh in their minds. Panelists should be given the rubric and an opportunity to understand the traits and attributes being assessed by the questions. A guide with definitions is beneficial so that all panelists have the same understanding of each attribute. Look to resources outside of your department as well for training if needed. There may be a Chief Equity and Inclusion Officer, or you may consider reaching out to local colleges or your HR department for help learning about unconscious bias and things like the halo effect. A little awareness goes a long way in preventing harmful influences on the interview process. Your panelists will feel far more confident in their abilities to interview candidates following this process, which will lead to more objective results that will provide the department with the best possible candidates for the job. It will also build a legally defensible interview process based on a foundational job analysis.
Dr. Andrew Holter is an organizational psychologist serving as the Administrative Services Manager of the Wilmington Fire Department. He has had an 18-year career in emergency services, starting as a volunteer firefighter in rural Pennsylvania and rising to the rank of Chief Paramedic in a regional ambulance service. Dr. Holter has worked as a firefighter, lieutenant, captain, and fire inspector and has served as a staff liaison for NFPA. He holds a B.S. in Fire Science and an MPA from Anna Maria College in Paxton, MA, and this past year successfully defended his dissertation earning his Ph.D. in organizational psychology from Capella University.
