Apache IPC semaphore problems
Ugh… For some reason, Apache has been having issues with startup occasionally, which seemed to be fixed upon reboot. I was getting pretty ticked with this “fix” for whatever Apache’s issue was, as the error log was giving me no information other than no space left on device and apachectl configtest was returning Syntax OK. Finally, I was able to get apachectl to spit out an error…
Ouch! ap_mm_create(1048576, "/var/run/httpd.mm.5868";) failed Error: MM: mm:core: failed to acquire semaphore (No space left on device)
Well, that makes a little more sense. Apache seems to have processes which are hanging or dying and not properly cleaning up any IPC semaphores used (and the kernel can apparently only allocate a certain number of these).
The fix on our Linux server was pretty simple. After shutting down Apache, I ran /usr/bin/ipcrm sem $(/usr/bin/ipcs -s | grep apache | awk '{print$2}'), then started Apache again and life is good.