A lot of it is trial and error, but there are ways to get a little more than a ballpark estimate.
Use the heat flux calculator in steam engine to figure out what builds to do for what wattage and vise versa. All you have to do is plug in your build and then start plugging in wattage levels in the little box in the table where it shows the wrap count and such until you get a suitable heat flux. The heat flux basically tells you how hot your coil will get at the specified wattage. This and the mass/surface area are going to be the main factors in what the performance limitations for your coils are.
You can look to the heat capacity shown on that same table to get a relative idea of how big your coil is, which is what generally determines the amount of flavor and vapor (a bigger coil has the potential to kick out more flavor and vapor,) as well as marking what the general operating temperature range will be. Coils with higher heat capacities will not perform as well with higher heat fluxes - they sometimes make for cooler vapes because they're harder to drive to high temperatures. The opposite can be said of smaller coils, which will peak-out more readily and easily give you a warmer vape with less power.
Though... ...just to confuse you, sometimes the opposite is true and large coils must be run with higher fluxes than a smaller coil could handle. Heat capacity also corresponds to how fast a build gains/loses temperature, which also plays a big role in how much wattage you can pump in and what heat flux is appropriate.
Ideally, you want to strike a balance between the heat capacity and heat flux. Too high of a capacity, and the coils will not heat up quickly enough, even at the wattage require to get a good flux. It will be able to produce a lot of vapor and give you great flavor, but will take a long time to do so and will make your device hotter with continuous use. Too low of a capacity, and the coils will have wicking problems from heating up too quickly and flavor/vapor production will suffer due to a lack of surface area.
For smaller, more closed-off systems with thinner wire, you generally want a flux in the green to yellow range. For more wide-open RDA's with thicker wire or large amounts of thinner wire, you want yellow to orange. You'll pretty much never want to go into the red or you will have wicking problems, hot legs, hot spots, etc.. I find that red is only vapable on mechs. Your coil can calculate out to something in the "very hot" range at its natural wattage if you calculate the wattage at 4.2v because the voltage naturally rolls off on a mech, meaning it runs cooler than the heat flux calculator thinks it would based on the raw numbers. The wattage is lower than what you wind up plugging in.
I'd toss numbers at you for all of that, but it is highly subjective. The right combinations are going to depend on the atty you're using, how you've got the airflow configured, and most importantly, how you vape.
Also don't be mislead by resistance on regulated mods. As long as your mod can power it, then the size of the coil will dictate your wattage. You sometimes have to run higher resistance coils at a higher wattage than low ones because a low-res coil can be smaller than a high-res one and thus require less power to heat them up.
It depends as much on the thickness of your wire, too. Thinner wire will require less wattage-per-ohm to reach greater temperatures. It's counterintuitive, seeing as the classic way of doing things is too look at ohms as an indicator of heat and power, but that's just a connection people make because lower-resistance coils pull more current. On mech mods, this means they naturally generate more power, but on regulated devices, it just means that they need less voltage.
Really, you just have to play around with it a bit more. Try not to make rules for it and just experiment with different coil masses and power levels until you find the sweet spot. Like the others said, start low and go up incrementally until it starts giving you the heat, flavor, and vapor that you want. By the time you work it out, you'll practically be able to look at the build and sorta know what the power range is gonna be for it. There's no real trick to knowing the right wattage.