I bet he got loads of film offers after season 1 of Bridgerton became such a big hit. Netflix only had him signed on for one season, probably because they didn't expect the show to become such a huge hit for them and didn't know that he would become the breakout star (a mistake they probably won't make for future seasons and future actors). His character also doesn't have a major role in the books beyond book 1 so they didn't see the need of locking him down for multiple seasons. They said that Daphne WILL be appearing so they probably did lock down the actors and actresses portraying the actual Bridgerton siblings.
Since he left so suddenly (and shortly before filming for season 2 started) Netflix probably weren't able to persuade him to stick around for the second season.
Personally I don't really mind. Yes, I'll miss seeing his bum on screen every few scenes, but there is plenty of other male (and female) eye candy to keep me amused.
The second season is gonna be Anthony's love story, so that's exciting. I already love the actress they cast as his love interest.