Thursday, January 7, 2016

Checking back on point differential predictions

I wrote an entry predicting final season outcomes after week 7, based on point differentials at that point in the season. Let's see how they fared, along with some retrospective notes and commentary.

AFC EastPredictedActualFinal diff
New England Patriots15-112-4+150
New York Jets12-410-6+73
Miami Dolphins9-76-10-79
Buffalo Bills8-88-8+20

All teams were within a game of their differential-based expectations, but they all fizzled in the latter half of the season for various reasons. The prediction of finish order in the division had no material errors, but the Jets did miss the playoff.

AFC NorthPredictedActualFinal diff
Cincinnati Bengals13-312-4+140
Pittsburgh Steelers10-610-6+104
Baltimore Ravens6-105-11-73
Cleveland Browns6-103-13-154

Well, there's a reason Mike Pettine is no longer employed. These predictions were materially spot on: Cinci won the division but missed out on the bye, and Pittsburgh is awarded a wild card berth. Based on differential, we'd normally expect the Steelers to reach 11 wins, but their highly bipolar games (with vs without Roethlisberger) explain that outcome.

AFC SouthPredictedActualFinal diff
Indianapolis Colts6-108-8-75
Tennessee Titans6-103-13-124
Houston Texans5-119-7+26
Jacksonville Jaguars4-125-11-72

The Texans turned their season around as effectively as they jettisoned Ryan Mallett. Clearly he was not the answer, and going to Brian Hoyer was the trick. The Colts stayed right on track in point differential, but managed to squeeze 2 unexpected wins out. The Titans and Jags underperformed their expectations by 1 game each. This division went completely against projections, though a statistician would argue that there was a lot of noise in the signal in Week 7. All the teams looked bad, there was no way to guess who would get their act together.

AFC WestPredictedActualFinal diff
Denver Broncos11-512-4+59
Oakland Raiders7-97-9-40
Kansas City Chiefs6-1011-5+118
San Diego Chargers6-104-12-78

Who saw the Chiefs coming back to win 9 of their last 10 without Jamaal Charles? Seriously, that's crazy. The really crazy thing is their point differential puts their expectation just slightly towards 12 wins! Denver, on the other hand, outperformed their differential by 2 wins. I'm proud of the projection though, it's meaningful to project that a 6-0 team will be barely over .500 the rest of the way. The Chargers battled hard and we'd normally expect to see 6 wins at -78 points, but they just couldn't close out games. The projections correctly identified the division winner, but missed on the wild card.

Overall in the AFC, we correctly predicted 3 division winners and 1 wildcard (though to be fair, none of those predictions was a stretch), and a total of 4 playoff teams correctly.

NFC EastPredictedActualFinal diff
Philadelphia Eagles10-67-9-53
New York Giants9-76-10-22
Washington Redskins7-99-7+9
Dallas Cowboys5-114-12-99

The two teams that looked to be the favorites imploded down the stretch and are now searching for new coaches, and somehow Kirk Cousins channeled his inner Drew Brees. Redords are all within a game of where differential expects; everyone but the Cowboys changed course in the back half. The Cowboys ... well. The projections missed on the division winner, but correctly predicted no wild card teams. That counts for something, right?

NFC NorthPredictedActualFinal diff
Green Bay Packers13-310-6+45
Minnesota Vikings10-611-5+63
Detroit Lions4-127-9-42
Chicago Bears3-136-10-62

Projections identified both playoff teams (though the Vikings were in the maybe bucket with Atlanta), just in reverse order. This wasn't a stretch either. The Bears and Lions turned in respectable second halves, and the Packers completely underperformed. Records are where we'd expect them to be based on differentials.

NFC SouthPredictedActualFinal diff
Carolina Panthers12-415-1+192
Atlanta Falcons11-58-8-6
New Orleans Saints6-107-9-68
Tampa Bay Buccaneers5-116-10-75

A clean sweep! Remember at the time that many were doubting the Panthers despite their perfect start. They actually outperformed expectations at the end, +192 would project to 14 wins; they made a really strong run at 16-0. Differential at the time correctly predicted Atlanta's decline, but even it wasn't detached enough to see a 2-7 finish and missing the playoffs coming.

NFC WestPredictedActualFinal diff
Arizona Cardinals14-213-3+176
Seattle Seahawks10-610-6+146
St Louis Rams7-97-9-50
San Francisco 49ers3-135-11-149

Another really good projection! We nailed the division winner and the wildcard. Records were right where final differentials would expect, except for the 49ers who really had the differential of a 3-13 team but pulled some miracle version of Blaine Gabbert from their bench.

Overall in the NFC, we hit 3 of the division winners and 1.5 wild cards (calling Falcons/Vikings a 50/50), but overall got 5.5 of the playoff teams right.

For the NFL as a whole, using point differential after week 7 to predict final outcome was reasonably good. The method picked 6 of the 8 division winners and 9 or 10 of the 12 playoff teams. The method of using point differentials isn't anything earth-shattering or super insightful. The interesting question here is if representative data is present after 7 weeks. The answer seems to clearly point to yes.

The appeal of the method is its simplicity. It requires no complex math, no models adjusting for strength of schedule or opponents; it's something anyone can use in their head or on a napkin. If I felt up to it, I'd do an analysis of how many playoff teams are correctly predicted as a function of number of games played. Any strong indicators could be useful, say, in Vegas.

Per differential, the Panthers are favored in the Super Bowl. The Cardinals are 2nd and the Seahawks barely trail the Patriots for 3rd. Adjusting for injuries and when teams are peaking (or flailing), and it really seems like the top 3 contenders are from the NFC.

Friday, January 1, 2016

College football bloat season

Oops, that should be "bowl season".

The 2015 season features 40 bowl games (not including the national championship game), meaning over 2/3 of teams get to finish their season in a bowl. This is even more egregious than the NBA playoffs, where each year 16 of 29 (55%) of teams make the postseason. Aside from diluting the meaning of "making the postseason", this leads to a giant collection of non-competitive games. There's no reasonable way to pair up teams to begin with, and conference affiliations on many bowls further limit possibilities. I mean, why should the 5th best team from one conference vs the 6th best from another be a good game? Even the 6-wins criteria is laughable since most teams collect 3 wins against FCS teams; or in the cases of Minnesota and Nebraska, a losing regular season record was good enough.

So, how many of the 40 games are remotely competitive? And how does that break down between games where both, one, or neither team is ranked?

Here are the games between each group (red indicates a 20+ point margin of victory, green is 8 or less):

#1 Clemson vs #4 Oklahoma, 37-17
#2 Alabama vs #3 Michigan State, 38-0
#5 Iowa vs #6 Stanford, 16-45
#7 Ohio State vs #8 Notre Dame, 44-28
#9 Florida State vs #18 Houston, 24-38
#10 North Carolina vs #17 Baylor, 38-49
#12 Ole Miss vs #16 Oklahoma State, 48-20
#13 Northwestern vs #23 Tennessee, 6-45
#14 Michigan vs #19 Florida, 41-7

#20 LSU vs Texas Tech, 56-27
#21 Navy vs Pittsburgh, 44-28
#22 Utah vs BYU, 35-28
#24 Temple vs Toledo, 17-32
#25 USC vs Wisconsin, 21-23

Arizona vs New Mexico, 45-37
Appalachian State vs Ohio, 31-29
San Jose State vs Georgia State, 27-16
Louisiana Tech vs Arkansas State, 47-28
Western Kentucky vs South Florida, 45-35
Akron vs Utah State, 23-31
Boise State vs Northern Illinois, 55-7
Georgia Southern vs Bowling Green , 58-27
Western Michigan vs Middle Tennessee, 45-31
San Diego State vs Cincinnati, 42-7
Marshall vs Connecticut, 16-10
Washington State vs Miami, 20-14
Washington vs Southern Mississippi, 44-31
Duke vs Indiana, 44-41
Virginia Tech vs Tulsa, 55-52
Nebraska vs UCLA, 37-29
Minnesota vs Central Michigan, 21-14
California vs Air Force, 55-36
Nevada vs Colorado State, 28-23
Auburn vs Memphis, 31-10
Mississippi State vs NC State, 51-28
Louisville vs Texas A&M, 27-21

Not yet played:
#11 TCU vs #15 Oregon
Penn State vs Georgia
Kansas State vs Arkansas
West Virginia vs Arizona State

Interestingly, the games between unranked teams had a lot more drama. There was literally not a single game between ranked teams that was decided by one score or less, and 6 out of 9 were by 20 or more. These are the top-tier games and they are blowouts. The other games with one ranked teams did barely better. Overall, the count in played games of close-normal-blowout for each group is:
2 ranked teams: 0-3-6
1 ranked team: 2-2-1
2 unranked teams: 11-6-5

The overall tally is 13-11-12. Put another way: practically random. The lack of quality games involving a ranked team (2-5-7) is particularly sad. I'm not sure what to do about this, or if this just means the comparisons between conferences are a crapshoot.


What if college football used my what-if system?

I've written several times about an alternate approach to the college football playoff, and even outlined how it would have been more relevant last year. Time to do the same again as Stanford is beating down Iowa. For the record, here are the 6 BCS bowls, as played:

#1 Clemson vs #4 Oklahoma, 37-17, in the Orange Bowl **
#2 Alabama vs #3 Michigan St, 38-0, in the Cotton Bowl **
#5 Stanford vs #6 Iowa, 45-16, in the Rose Bowl
#7 Ohio State vs #8 Notre Dame, 44-28, in the Fiesta Bowl
#9 Florida State vs #18 Houston, 24-38, in the Peach Bowl
#12 Ole Miss vs #16 Oklahoma St, 48-20, in the Sugar Bowl

** - National championship game semifinal

Between scheduling and playing, all but one of these games were either blowouts or featured teams well outside of the top 12 teams in the country. Ohio St vs Notre Dame was the only decent game featuring teams with legitimate claims to a BCS bowl. Why is Houston or Oklahoma more deserving than #10 North Carolina and #11 TCU? Flipping it around, why is Ole Miss more deserving than Michigan or Northwestern?

To look at the what-if games, let's remind ourselves of the traditional bowl affiliations:
Orange Bowl: ACC champ vs at-large
Cotton Bowl: Big-12 champ vs SEC
Rose Bowl: Pac-12 champ vs Big Ten champ
Peach Bowl: SEC vs ACC
Sugar Bowl: SEC champ vs at-large
Fiesta Bowl: Pac-12 vs at-large

And remind ourselves of the top teams and Power 5 conference champs (in bold):
1. Clemson (ACC) 13-0
2. Alabama (SEC) 12-1
3. Michigan St (Big Ten) 12-1
4. Oklahoma (Big 12) 11-1
5. Iowa (Big Ten) 12-1
6. Stanford (Pac-12) 11-2
7. Ohio St (Big Ten) 11-2
8. Notre Dame (Ind) 10-2
9. Florida St (ACC) 10-2
10. North Carolina (ACC) 11-2
11. TCU (Big 12) 10-2
12. Ole Miss (SEC) 9-3
13. Northwestern (Big Ten) 10-2
14. Michigan (Big Ten) 9-3
15. Oregon (Pac-12) 9-3
16. Oklahoma St (Big 12) 10-2
17. Baylor (Big 12) 9-3
18. Houston (American) 12-1

Unlike many previous years, there isn't a highly ranked true outsider (like Boise St, TCU and Hawaii in the latter part of the previous decade). Using the affiliations as guidelines, we get partial games:

#1 Clemson vs at-large in the Orange Bowl
#2 Alabama vs at-large in the Sugar Bowl
#3 Michigan St vs #6 Stanford in the Rose Bowl
#4 Oklahoma vs SEC in the Cotton Bowl
at-large vs #15 Oregon in the Fiesta Bowl
SEC vs #9 Florida St in the Peach Bowl

Oregon is a stretch to make a BCS game, so we should consider swapping in a significantly more deserving non-Pac-12 team. We also don't have 3 deserving SEC teams for their 3 traditional slots. We now have to dole out the remaining teams in the at-large pool while maintaining high-caliber matchups to help get a real feel of how these teams rank against each other. Clemson already played Notre Dame so we don't want a rematch. The simplest thing would be to have Iowa play Clemson and Ohio State play Alabama. This gives us a rare chance to evaluate three conference champs against clearly comparable teams:

#1 Clemson vs #5 Iowa in the Orange Bowl
#2 Alabama vs #6 Ohio St in the Sugar Bowl
#3 Michigan St vs #6 Stanford in the Rose Bowl
#4 Oklahoma vs SEC in the Cotton Bowl
at-large vs #15 Oregon in the Fiesta Bowl
SEC vs #9 Florida St in the Peach Bowl

The next teams on the list are Notre Dame, North Carolina and TCU. Ole Miss is the only SEC team left, and their overall ranking suggests they should go to the Peach Bowl instead of the Cotton Bowl. Thus, the SEC should lose their Cotton Bowl spot in favor of a more deserving team. Oklahoma is realistically very far on the outside looking in already, so "only" getting Notre Dame is ok. And thus, the BCS bowls become:

#1 Clemson vs #5 Iowa in the Orange Bowl
#2 Alabama vs #7 Ohio St in the Sugar Bowl
#3 Michigan St vs #6 Stanford in the Rose Bowl
#4 Oklahoma vs #8 Notre Dame in the Cotton Bowl
#10 North Carolina vs #15 Oregon in the Fiesta Bowl
#12 Ole Miss vs #9 Florida St in the Peach Bowl

It features 11 of the top 12 teams in theoretically competitive matchups (though of course we don't know how any of these would actually turn out). Oregon has clearly the weakest resume and I wouldn't really argue against swapping in another team (TCU, Northwestern or Michigan, or Houston), but it's at the tail anyways so it's not going to affect the championship game. If Clemson and Alabama both win, the decision is easy. If either (or both) lose, it gets progressively more complicated, of course. But let's remember: every game should count!

If we accept that only a conference champ should even be in the national championship game conversation, the above essentially plays like the playoff, but gives Stanford the slightest glimmer of hope if they win big and every other top seed loses. I think a non-zero claim to the championship game is legit in that case because the overall records differentiate less, and Stanford would have beaten the top Big Ten team (while Clemson and Alabama would have lost to comparable, but technically lesser, ones).

Realistically this year it all worked out the way it "should have" considering Alabama and Clemson were the clear top two. But, we'd hopefully see a few more competitive games along the way.