Tracked it down pretty far to where it's failing in XenForo (why I have no idea, it's native functionality obviously). Sent some details over to the XF team so hopefully we'll have a fix soon.
It's definitely not caching. Straight up incorrect information in DB because when a post is made, information that should be updated isn't getting updated. New threads update it, but new posts don't. Both are supposed to update it.