I made the same move about 18 months ago. Money is noticeably better - base salary alone wiped out the constant stress of chasing billable hours. Bonus structure is saner, too. No more "congratulations, you hit 110% utilisation, here's a pizza party" nonsense.
But the people part? Yeah. That lands.
In-house marketing teams often operate with a completely different pace and depth of understanding. Agency people are forced to learn fast across multiple clients - you see patterns, pick up technical nuances, know how to navigate crawlers, server logs, and content silos. In-house, you get colleagues who've spent five years running the same two Facebook ads and think that's the ceiling of digital strategy. The strategic conversations can be... shallow. It's not malice, it's just a narrower worldview.
That said, the trade-off is worth it for most people I know. Less grind, more actual strategy implementation. But you do have to accept that you'll be the technical conscience in the room. My advice: document everything, build a solid internal knowledge base, and don't expect them to ever match your speed on the technical side. You'll be fine as long as you treat the move as a long‑term play - not a paradise, just a healthier compromise.