Worldbuilding is fun, and you definitely need to think at least a little about how a setting Works for it to make any sense, but I sometimes get the impression that people on this website have forgotten the extent to which you can just kind of wing it.
You should probably have an explanation for why there are potatoes in your quasi-European setting, but you don't necessarily have to develop a five hundred year overview of the history of Fantasy Peru unless the story is actually set there. It's probably sufficient to say "imperial conquest that is a broad allegory for real world events" or "this world is filled with traveling bands of botanists" or "the world was a single supercontinent within living memory, and then some crazy shit happened, let me tell you about That" depending on the themes and ideas you are working with.
Much like using "said" instead of a more specific synonym, 90% of the time you can just have people use normal cusses and greetings and it's literally fine.
Good world building is about illusion. People don't need a 5 page essay on a particular throw away detail of the setting, but for the world to feel real they need to believe that you have that 5 page essay somewhere in your brain.
Its the secret to not wasting effort on world building.
It can be fun to figure out all the details, all the reasons why this or that happened, if you want to explore that rabbit burrow. But it's not needed for the worldbuilding to be complete. You don't have to dig out the whole burrow to know there's rabbits inside, or to walk on the ground above it. Knowing the potential is there to be explored is often enough.
My issue with this is, why does there have to be an explanation for potatoes? My fantasy world isn't Europe. It isn't Peru or China or the Philippines either. So why does it have to follow the rules of our world?