a certain lack of entropy

Where we live one needs an AC, especially during summer. Our apartment is not that big, but we actually have two separate ACs, one for the ground floor and the other for the bedrooms. Unfortunately, they never manage to get to the same temperature (and humidity) - so when I walk up the stairway from the ground floor I can feel the temperature drop. And such temperature differences of about 1F can last for hours, long after the AC turned off.

But how is this possible? The air in our house is pretty much an ideal gas with some water molecules added and if I can walk through the stairway (there is no door btw) then the much smaller air molecules should be able to fly from one floor to the other too. The faster molecules in one direction, the slower ones in the other until equilibrium is restored, as we learn it in school. Furthermore, should the cooler air not quickly float down to the ground floor due to gravity or something?
In general, entropy seems to work fine in our house, e.g. when I spill coffee or drop something; but where is it when we need it?

Well, Sean Carroll explained to us that entropy is actually a cosmological effect and apparently it has to do with an "arrow of time". So I have to conclude that something is not right with the universe - and perhaps I should spend more time in the upper floor, assuming that the "arrow of time" is less pointy there? (*)

(*) added later: not really...