The engineer found himself locked out of a Terminator box[1]. He would ssh to the box, only to find that his prompt would never materialize. After much agonizing, he logged in as root on another terminal, and decided to truss the offending ssh daemon. To his amazement, he found that a shell was actually being forked off – but that it was waiting for one of its children to do something. The child in question (bastard, he swore under his breath) was quota. Quota was itself waiting for something else to happen.
What was quota waiting for? A few more minutes of poking around helped solve the problem. A badly written application that he had been running was hard-mounting NFS exports, and not umounting them when done. Consequently, quota was going mad trying to calculate how much space he’d eaten up. With several hundred stale hard mounts lying around, there was no way that quota would be done before kingdom come. The engineer heartily cursed the moron who’d written the ugly piece of code, and set about kicking out the unwanted mounts. A few minutes later, all was well with the world.
The moral of the story? I can’t think of anything snappy to say, so I’ll leave it at that.
—
[1] “The Terminator” is how I affectionately refer to the T-[12]000 offerings from Sun:
http://www.sun.com/products-n-solutions/hardware/docs/Servers/coolthreads/t2000/index.html. Lovely beasts.
