Illegal Procedure: Stroud blindsided by unverified, questionable S2 Cognition test
Pat Kirwan, former NFL team executive and now co-host of must-hear Sirius XM's Moving the Chains, says the week before the player draft is the "lying season." He insists that during this time anything that is said, or heard, is suspect.
Against that background, we point out that one of the top-rated quarterbacks in this draft became the target of some curious and damning allegations last Friday (April 21). NFLDraftScout.com has questions that must be answered, and so should you.
That quarterback is Ohio State's C.J. Stroud, rated first or second in the draft along with Alabama's Bryce Young, by most accounts.
On Friday, respected media draft analyst Bob McGinn, who has covered football and the draft for 40 years, posted a story that said Stroud scored only 18 percent on an S2 Cognition Test, compared to 98 percent by Young.
McGinn has a solid reputation as a deeply credible journalist, but this story was entirely one-sided, very damning and screams for either a named source or some journalistic balance. Here are are some examples used in the Stroud S2 story:
“Stroud scored 18,” an executive said. “That is like red alert, red alert—you can’t take a guy like that. That is why I have Stroud as a bust. That in conjunction with the fact...name one Ohio State quarterback that’s ever done it in the league.”
“The S2 people will say, 'Hey, guys that graded high on this test don’t always play well,'” another executive said. “'But, we’ve never had somebody grade low and play well.'”
Including a reference to the history of Ohio State quarterbacks was off point and piling on an already lopsided presentation.
McGinn said multiple sources told him that while Stroud's number was only 18 percent, Young's was at 98 percent, while Fresno State's Jake Haener was at 96 percent. Other top QB prospects, Kentucky's Will Levis and BYU's Jaren Hall, both finished at 93 percent.
Important questions remain
That same day a Pro Football Focus analyst said during a roundtable vlog that he was told by someone from S2 Cognition that sometimes the leaked test results are bogus. That's possible, but we cannot know unless somebody from S2 says so on the record.
We reached out via email to S2 Cognition to get further explanation and confirmation on the reported results. "We are unable to comment on the inappropriate leaking of scores being reported in the media,” S2 CEO Vaden Landers responded. “The S2 data gathered on NFL prospects is owned by, and proprietary to, the NFL teams we work with on an exclusive basis."
While an expected answer, and a legitimate one, it doesn’t confirm the accuracy of the reported grades. So, important questions went unanswered.
When asked about them on a Pro Football Focus NFL show, Brandon Ally, the co-founder of the S2 Cognition test, said he was "aware of scores being leaked" but that some of the results need to taken "with a grain of salt.”
“All the leaks that you’re seeing is one number," Ally said. "It is a mistake to value one number. We give nine different cognitive tests. If you score in the 90th, you could be in the 10th for object tracking which means you have tunnel vision.”
None of this is explained in the report. And during the "lying season," strong, unbalanced assertions against a single player demand to be well documented.
Maybe McGinn's source had a vested interest in saying such things, either to enhance Young's image or damage Stroud's. It happens. Without knowing the name of one source, at least, it is impossible to know.
The day before the S2 story became news, Mike Florio of Pro Football Talk discussed concerns over the credibility of reports:
It's odd to me that this stuff is coming out now. Is it that the media is finally catching up to it? Is it that there are agents and teams aggressively pushing it? You know how this goes...there could be a team out there that would love nothing more than to have C.J. Stroud fall into their laps, so let's go ahead and traffic in all the negativity because maybe he'll be there when we pick.
Deep breath here. Despite the veracity of sources, we have one key question: Does an S2 Cognition test account for disparities among the players, including social, economic and ethnic? The popular SAT college tests were criticized for failing to take such things into account. The Wonderlic Test, long a staple for draft research, was accused of using tests that were culturally and racially biased. Each made concessions and allowances for certain variables.
Stroud and Young: two very different backgrounds
The Stroud-Young comparison may be a textbook example of why such concerns exist. Let's consider the differences in background of Young and Stroud.
Young was an only child raised by his mother, Julie, and a father who works in psychology. Bryce was afforded first-class amenities while growing up, including special quarterback tutoring and football camps. Before his junior year of high school, it was apparent Bryce had a bright future in football. The family transferred him from Cathedral High, northeast of downtown Los Angeles, to football powerhouse Mater Dei High School in Santa Ana California.
It was far different for Stroud, who grew up as the youngest of four siblings in the Inland Empire region, a sun-baked area about an hour east of Los Angeles. Stroud's father Coleridge Bernard Stroud was a minister and introduced C.J. to sports and strong religious beliefs by the time he was five. But when C.J. was 13, his father was arrested on multiple felony charges. He is serving a 36-year sentence in Folsom Prison, with no chance of parole until 2040. C.J. visits his father often, and they are as close as they can be under the circumstances.
Stroud's mother, Kimberly, struggled to provide for the family, which moved often and for a time lived in a small apartment over a storage facility. Stroud attended Rancho Cucamonga High School, but had no outside training, and finding money for basic athletic equipment was difficult.
We can't say for sure that their respective backgrounds impacted the ability to take tests on a video screen. But it's worth knowing.
According to a discussion on a Pro Football Focus post, Young has been taking this S2 test since the 10th grade. We call that practice. But we do not know if Stroud had previous experience with the test. These are the types of things that must be known to understand the scope, viability and relative value of such test results.
Defining the undefinable
The S2 Cognition test is done on a video screen, or via virtual reality headgear, and involves tracking an ever-increasing bunch of dots and reacting. On its company website, this is how S2 describes itself:
At S2 Cognition, we see the game differently...Often referred to as instincts or ‘it’ factor, S2 has discovered a way to define the undefinable — to quantify the intangible. Our scientifically validated evaluation determines how athletes process information at game speed and react when the pressure is on. Equipped with this blueprint, athletes and coaches can tailor training and drills to maximize players’ strengths and develop strategies for their challenges.
Define the undefinable? Quantify the intangible? The terms "oxymoron" and "jabberwocky" come to mind. Nice creative writing, though.
NFLDraftScout.com talked to more than two dozen college and pro coaches and scouts about Stroud over the past year. They expressed concerns about his unwillingness to run and a habit of locking on to his primary receiver too long — common stuff. But none of them questioned his ability to, uh, think. To the contrary, they praised the way he stood in the pocket, went through his progressions and made good decisions. But that was on a football field, not a video screen.
By the way, Stroud is 6-3, 214 and Young is 5-10, 190-ish. That's probably of no consequence on a video screen or virtual reality test yet certainly should be a factor for playing football against 300-pound men.
But, like the soaring reliability on analytics, maybe these video tests are the latest next-level method of scouting. The intel we received was mainly from those studying hours of game tape — from actual games — and not derived from a mind-probing, abstract video that registers your fingers' reactions in milliseconds to what happens on a screen.
Wait, didn't Electronic Arts already do that? It is called Madden Football.
That's it. Let's have Stroud and Young square off in a game of Madden Football. The winner gets to pick which NFL team can draft him.