The gaming handheld market has been going strong over the last couple of years, with more and more releases across all price brackets. Budget handhelds are a great entry point to the retro gaming hobby, and may even be the ideal choice if all you want to play are the classics (SNES, GBA, PS1,..). On the other hand, there are definitely some outstanding bigger handhelds that can put a dent in your wallet, with certain premium options costing up to thousands of dollars.
The $150 to $200 price point is the sweet spot for a large majority of use cases when it comes to handheld gaming and retro emulation. At this range, all systems up to PSP will run smoothly even when upscaled to multiples of their original resolution in some instances, and you'll get enough power to dabble in some harder-to-emulate platforms like the PS2 and GameCube. You'll also most likely get active cooling components to help with heat and throttling. The displays are generally high-resolution and of good quality, and the controls are polished, well-thought-out, and comfortable to use.
These mid-range handhelds are the best option if you want to upgrade from a budget model and gain access to more games and features. While they're less powerful than pricier hardware, they're still very capable, offer a gigantic library of games to choose from, and are sure to provide many hours of fun and unique experiences.