I recently solved a situation where Apache was unable to start as a service under SystemD using systemctl start httpd, but started no problem using httpd -k start. This issue took place on our fleet of CentOS servers, but only affected one of them.
One of our CentOS 7 servers has not been able to start Apache under SystemD for a while now, and we could not figure it out. This one server has the most vHosts of all of them, about 620. There is no limit on how many an Apache server can run, if the configuration is correct. We also knew that it was nothing to do with the Apache config, as Apache did start correctly by calling the binary directly with httpd -k start.
Whenever we would try to start it using systemctl start httpd or apachectl start we would get the error that it could not, and to check the journalctl -xe for details. There were no details.
In the Apache error log, there was 1 entry, which finally lead to a Eureka moment.
|
1 |
AH00015: Unable to open logs |
After Googling for a while, it looked like there might be a permission issue, or perhaps one of the vHost files had a typo in the log file. So I started with the base vHost. The service started fine. Then I added half. Still fine. I kept adding more of the vHost conf files back in until it did not start anymore.
When that happened I noticed:
- The number of child processes that
httpdstarted went down to 1 - The server could only open 1 file at a time, and was very slow
This was the Eureka moment: As a service, httpd can only open so many files!
After that I knew that I just needed to tell SystemD to not limit the number of files that the httpd service is allowed to open.
|
1 2 3 4 5 6 7 |
cat >"/etc/systemd/system/httpd.service.d/override.conf" <<EOF [Service] LimitNOFILE=infinity EOF systemctl --system daemon-reload httpd -k stop systemctl start httpd |
And there we have it, Apache now loads as a service using systemctl start httpd, which makes me very happy.