Data collection built into the TV shouldn't be your only concern (in fact, if you're paranoid enough you should automatically assume any 'smart' product does this). The firmware on these devices is notoriously poorly designed and many models of smart TVs are vulnerable to exploitation. You can make a safe assumption that different models from a single manufacturer will likely share code, so these issues potentially affect entire product lines.
The same issue lies in just about any embedded device - DVRs, wireless APs, cameras, phones, MP3 players, even SD cards. Often security checks are badly implemented, incomplete, or entirely omitted due to {hardware or time constraints, efficiency concerns, ineptitude, laziness}...