It makes a difference where I live, and I'm not in a particularly humid climate (California). When printing with wet filament, you'll sometimes hear it popping and see steam coming out of the extruder (it's usually only this extreme with nylon). With most other filaments, when they're wet, the extruded filament will have small bubbles in it and the surface finish of the parts will be rougher, with breaks in the layer lines. It can also lead to more oozing and stringing. Air print a few centimeters of filament and look at it closely to see if there's any bubbles, if not, it's probably dry enough. Whether the filament absorbs enough water to be noticeable in a few hours, in a day, or in a week depends a lot on the filament (and I assume the humidity too). I'm mostly noticed problems with nylon, ABS, and NinjaFlex, less with PLA and PETG (though I avoid leaving any filament out for more than a day).

If you're not seeing any difference between, then I wouldn't worry about it. Storing filament dry is a hassle.