Hidden Side is the hidden gem of the recent sets. I'd recommend half of them. Dropping $100 on the good locations will set you up very nicely for running games, giving you a "GM's toolkit" of terrain and monsters that is more flexible and useful than any castle:
The Welcome to the Hidden Side set is an excellent value: For $20 (list price, usually a few bucks cheaper in most stores) you get three ghostly monsters and terrain that can be used in your game with minor adjustments. Just take off the plates with the English-language words, and separate the sections, and you are good to go. For a bit more customization, you can remove the black tree branches to make a standalone black tree with blue leaves, and take off the face opening from the brick doorway.
Newbury Subway, Lighthouse of Darkness, and Abandoned Prison are also great choices for the dungeon crawls and outdoor ruins that games usually take place in. All of them are filled with the kind of interesting and interactive terrain that makes a game fun. Once you make them, you'll want to split them apart into separate dungeon dressing or wall segments for flexibility and reusability.
The 'castle' set (that actually resembles a Victorian era mansion) is nice-looking and tempting, but has less play value than a collection of the smaller sets. It is a distinct place that you may use once or twice rather than generic terrain you'll use every week.
Piling on the riches, this summer's line of Ninjago sets is much closer to a traditional fantasy adventure feel. Most of them, as usual, are giant creatures or vehicles with limited value, but the location sets are perfect:
There is a third iteration of the Creator 3-in-1 dragon and monster sets that are always nice. If you have one of the first two, this doesn't add anything, but if you are starting out, you will want it.
Every game will run into giants eventually. One option is to go with Forbidden Forest: Umbridge's Encounter, which also gives you two centaurs, forest terrain, and three minifigs, a good deal for $30.
Another option is the sets sold as 'mech suits', which are a good way of getting a nice poseable giant for $10. You have to fix them a bit, usually by adding a head. Usually Ninjago has a couple that are serviceable, but this year I'd go with the Thanos Mech. With a bit of alteration, you can remove the 'infinity gauntlet' pieces so it looks like a normal giant hand, and if you swap out the helmet the minifigure will be a generic purple-skinned humanoid rather than anything unique to the Marvel universe.
Friends has a 'Jungle Rescue' mini-theme that, while not as good as the Elves sets, does add more character and terrain options to your GM's toolkit. The minidolls are still appropriate for elf characters, even without pointed ears. About half of them do not have any modern-looking vehicles. Variety is always nice, so I would recommend one of these $20-30 sets rather than buying a third or fourth set in one of the themes you already have.
You should have something to represent a town or storefront or manor house or other civilian location. If these kinds of adventures will not happen often, you can go with something small like Townhouse Toy Store or Emma's Fashion Shop. Either of these works well, especially if you replace the plate glass on the first floor with something more period-appropriate (or just use as is and add magic gizmos for a Diagon Alley type of place). If you are running a rustic or lower-magic setting, go with last year's Outback Cabin, while you can still get it.
But if you have the budget, or expect a lot of town adventures, go with Attack on the Burrow. While it's no Medieval Market Village, it gives enough terrain for a small historical-fantasy village, once you de-stack it, and also gives you eight minis that are good for townsfolk or low-level starting adventurers.
And finally, Creator has given us a wonderful pirate ship for $100. I don't necessarily recommend it for most people, because you will get more value out of the smaller location-type sets, but if you are running a nautical (or Spelljammer) campaign it could be the center of many adventures:
This is the best pirate ship Lego has ever produced, for the simple reason that it uses a brick-built hull and sails rather than special pieces. The whole point of Lego is that you should be able to make anything from basic bricks. The old pirate ships broke this rule, and trained people to believe that you had to buy the special pirate ship set in order to make a pirate ship. If you tried to make your own, it would feel like an inferior substitute. But with this set, Lego is sending the message that normal bricks can do anything. After you build this ship, you should be able to make more on a similar pattern using your own bricks.
The special bonus of a brick sail is that you can customize it extensively. The skull and crossbones can be replaced with the iconography of a faction or nation in the game you are running. And the ship itself is also flexible, with potential for alteration. One of the great things about introducing ships (or airships) into a campaign is that it gives the players a mobile base that they will always want to upgrade. With this ship, you can alter or upgrade almost anything, and even make it longer.