If you have a DSL or Cablemodem, this behavior is unfortunately quite common. Basically your ISP is putting your connection into a very low priority queue to better service non-idle connections. The problem is that some enduser's connections will actually be taken OFFLINE until some traffic from the user's DSL/Cablemodem connection awakens the ISP's hardware.
Some DSL installations can take an idle connection OFFLINE and only be checked for activity once every 30 seconds or so.
Some Cablemodem setups can set an idle connection into a low priority queue and only be checked for activity every minute or so.
What do I recommend to do? Ping your default gateway every 30 seconds. To do this, edit the /etc/rc.d/rc.local file and add the following to the bottom of the file:
ping -i 30 18.104.22.168 > /dev/null &
Replace the 22.214.171.124 with your default router (upstream router).