Merit Matters, a group that opposes race-based hiring in the FDNY, is taking aim at the department’s new entrance exam, claiming it gives an unfair advantage to minorities and encourages lying on subjective questions in order to receive higher scores. The Vulcan Society, the fraternal organization of black firefighters, who helped create the exam, denies the accusation.
“This test was engineered to produce a pre-determined outcome and should be challenged by those with standing to do so,” said Deputy Chief Paul Mannix, in his position as an advocate and the founder of Merit Matters, adding, “To allow this test to pass without analytical commentary would be a disservice to everyone connected to the FDNY and also to the civilians who depend on us.”
Paul Washington, immediate past president of the Vulcan Society, who oversees the organization’s test prep program, said Mannix has accused the group of cheating before, but sent as many people as he could to the Vulcan courses.
“Why would you send your people to a dishonest program?” Washington asked, adding “He’s not even worth responding to. He’s an ineffective leader and his organization is weak.”
Controversy over the FDNY exam has been persistent over the last few years with the Vulcans, who have long lamented a lack of diversity in the department, joining a lawsuit launched by the Justice Department accusing the city of discrimination.
U.S. District Judge Nicholas Garaufis concluded the tests given in 1999, 2002 and 2007 were biased because of the small number of minorities who passed, and he ordered that a new test be created. It was administered in April.
Merit Matters takes aim at several elements of the test, which it considers problematic. Foremost is the way the exam was engineered, designed to minimize racial, ethnic and gender group score differences among other things, according to Merit Matters. That criteria “suggests that the test was engineered, or rigged, to achieve a pre-determined outcome,” Mannix said.
The group also takes issue with the cognitive and noncognitive questions being given equal weight, claiming it effectively dumbs down the test. Cognitive questions are objective and designed to test knowledge while noncognitive ones seek opinions and therefore should not be measured equally, the group argues.
Some of the 85 noncognitive questions include:
• How much time do you spend relaxing as compared to your friends?
• How many times have you said “thank you” this week?
• If someone is yelling at you, is it okay to yell back if you are right?
The premise for asking such questions, according to Merit Matters, is to be able to predict what kind of employee a person will be based on the person’s answers. However, honesty is key in being able to make such an assessment, Mannix said, so he is angered that candidates have been coached and given “suggestions” for how to respond.
A 2012 Vulcan Society tutorial cited by Mannix states, “Knowing what personality traits and characteristics are being sought will greatly improve your chances of scoring well on these questions.”
It also goes on to describe the characteristics and personality traits of a firefighter; advises test takers not to state that they prefer to work alone, as team work is preferred; and it recommends never answering a question with “not sure” because it will result in a lower test score.
“It blows apart the idea of honesty out of the water,” Mannix said. “It blows apart the premise of knowing what kind of employee you’re going to get, if people aren’t answering the questions truthfully. It’s very dangerous.”
Also a point of contention is the reading portion of the exam. While the skill was tested, it only accounts for 15 percent of the final grade, Mannix said. Video and audio sections of the test were added and prefaced related questions in an effort to equalize the differences between applicants who read well enough to do the job and those who exceeded the requirements, according to Mannix.
Other issues Merit Matters found with the test include: the scoring methodology, which was not made public before the test was given; the time period to qualify for a five-point residency credit had expired before the test was given; and blacks received home visits from the Vulcan Society to help them complete any missing paperwork, an opportunity which was not afforded to white candidates.
“The fact that they didn’t announce the scoring methodology before the test, and that the scoring methodology still hasn’t been made public, makes the test all the more ambiguous,” Mannix said.
Asked whether he thought the new FDNY exam was more fair than previous tests, Washington replied, “I didn’t take the test, so I really can’t say. You talk to different people and you hear different things.”