There’s one place where affectionate nicknames aren’t going to get you in trouble, and that’s inside your own home. Everywhere else, from professional contexts to where you buy coffee, stick to “sir”, “mam”, “miss” or “mate”. Or, preferably, use someone’s name.
Calling someone “sweetie” is not acceptable in a professional context, for the simple reason that it is demeaning and presumptive. It’s a name you give something cute and harmless, and while not everyone minds, many do, and it’s not worth taking the risk.
Same goes for “darling” and pretty much any other gendered, affectionate term.
There’s one exception: when it’s deliver by older women to younger women. Coming from the same sex, in rather specific situations where the tone is unambiguously supportive, it’s more likely to be welcomed.
And yes, manners aren’t fair. If you want to offend for equality, it’s at your own risk.