crond php server crash
-1
My CentOS server crashed with 100% memory with swapping, after running fine for a week. Investigating the /var/log/messages shows me the following: https://pastebin.com/CSF8DDzX Basically, alot of the following:
Feb 12 23:31:32 [localhost] kernel: [31186] 0 31186 85296 0 116 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31252] 0 31252 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31253] 0 31253 85296 0 112 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31315] 0 31315 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31316] 0 31316 85296 0 116 1107 0 php
Feb 12 23:31:32 [localhost] kernel: [31377] 0 31377 45588 1 44 230 0 crond
It looks like it is stuck in a loop. My crontab -e shows me this:
0 0,4,8,12,16,20 * * * php PeriodicallyKillWorkers.php
* * * * * php cheduleWorker.php
The scripts work fine, as does/did crontab. Where do I investigate further?
centos cron php
asked Feb 14 at 9:58
GudgipGudgip
1013
add a comment |
-1
My CentOS server crashed with 100% memory with swapping, after running fine for a week. Investigating the /var/log/messages shows me the following: https://pastebin.com/CSF8DDzX Basically, alot of the following:
Feb 12 23:31:32 [localhost] kernel: [31186] 0 31186 85296 0 116 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31252] 0 31252 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31253] 0 31253 85296 0 112 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31315] 0 31315 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31316] 0 31316 85296 0 116 1107 0 php
Feb 12 23:31:32 [localhost] kernel: [31377] 0 31377 45588 1 44 230 0 crond
It looks like it is stuck in a loop. My crontab -e shows me this:
0 0,4,8,12,16,20 * * * php PeriodicallyKillWorkers.php
* * * * * php cheduleWorker.php
The scripts work fine, as does/did crontab. Where do I investigate further?
centos cron php
asked Feb 14 at 9:58
GudgipGudgip
1013
Please if you downvote my question, tell me why so I can improve it.
– Gudgip
Feb 14 at 10:11
add a comment |
-1
-1
-1
My CentOS server crashed with 100% memory with swapping, after running fine for a week. Investigating the /var/log/messages shows me the following: https://pastebin.com/CSF8DDzX Basically, alot of the following:
Feb 12 23:31:32 [localhost] kernel: [31186] 0 31186 85296 0 116 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31252] 0 31252 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31253] 0 31253 85296 0 112 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31315] 0 31315 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31316] 0 31316 85296 0 116 1107 0 php
Feb 12 23:31:32 [localhost] kernel: [31377] 0 31377 45588 1 44 230 0 crond
It looks like it is stuck in a loop. My crontab -e shows me this:
0 0,4,8,12,16,20 * * * php PeriodicallyKillWorkers.php
* * * * * php cheduleWorker.php
The scripts work fine, as does/did crontab. Where do I investigate further?
centos cron php
asked Feb 14 at 9:58
GudgipGudgip
1013
My CentOS server crashed with 100% memory with swapping, after running fine for a week. Investigating the /var/log/messages shows me the following: https://pastebin.com/CSF8DDzX Basically, alot of the following:
Feb 12 23:31:32 [localhost] kernel: [31186] 0 31186 85296 0 116 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31252] 0 31252 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31253] 0 31253 85296 0 112 1108 0 php
Feb 12 23:31:32 [localhost] kernel: [31315] 0 31315 45588 1 44 230 0 crond
Feb 12 23:31:32 [localhost] kernel: [31316] 0 31316 85296 0 116 1107 0 php
Feb 12 23:31:32 [localhost] kernel: [31377] 0 31377 45588 1 44 230 0 crond
It looks like it is stuck in a loop. My crontab -e shows me this:
0 0,4,8,12,16,20 * * * php PeriodicallyKillWorkers.php
* * * * * php cheduleWorker.php
The scripts work fine, as does/did crontab. Where do I investigate further?
centos cron php
centos cron php
asked Feb 14 at 9:58
GudgipGudgip
1013
asked Feb 14 at 9:58
GudgipGudgip
1013
asked Feb 14 at 9:58
GudgipGudgip
1013
asked Feb 14 at 9:58
GudgipGudgip
1013
asked Feb 14 at 9:58
GudgipGudgip
1013
1013
Please if you downvote my question, tell me why so I can improve it.
– Gudgip
Feb 14 at 10:11
add a comment |
Please if you downvote my question, tell me why so I can improve it.
– Gudgip
Feb 14 at 10:11
Please if you downvote my question, tell me why so I can improve it.
– Gudgip
Feb 14 at 10:11
Please if you downvote my question, tell me why so I can improve it.
– Gudgip
Feb 14 at 10:11
add a comment |
1 Answer
1
active
oldest
votes
1
You run this every minute:
* * * * * php cheduleWorker.php
Check what is inside and how much time it take to exec this script. And add full paths to the program and script
Also check the other job (how long in run, how much memory it use and so on)
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
1
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f500581%2fcrond-php-server-crash%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
You run this every minute:
* * * * * php cheduleWorker.php
Check what is inside and how much time it take to exec this script. And add full paths to the program and script
Also check the other job (how long in run, how much memory it use and so on)
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
1
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
add a comment |
1
You run this every minute:
* * * * * php cheduleWorker.php
Check what is inside and how much time it take to exec this script. And add full paths to the program and script
Also check the other job (how long in run, how much memory it use and so on)
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
1
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
add a comment |
1
1
1
You run this every minute:
* * * * * php cheduleWorker.php
Check what is inside and how much time it take to exec this script. And add full paths to the program and script
Also check the other job (how long in run, how much memory it use and so on)
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
You run this every minute:
* * * * * php cheduleWorker.php
Check what is inside and how much time it take to exec this script. And add full paths to the program and script
Also check the other job (how long in run, how much memory it use and so on)
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
edited Feb 14 at 10:14
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
answered Feb 14 at 10:05
Romeo NinovRomeo Ninov
6,54132028
6,54132028
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
1
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
add a comment |
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
1
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
if I time this php script: real 0m0.053s What it does is loop over a few database records and publish a rabbitmq message. I don't think the real paths will give you any additional info. What might be important is that the files are located on an network drive which disconnected for a small period of time.
– Gudgip
Feb 14 at 10:09
1
1
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
@Gudgip, this (network drive) can cause the problem. Better wrap the php in shell checking if drive is mounted. Also check the other job (how long in run, how much memory it use and so on)
– Romeo Ninov
Feb 14 at 10:14
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
The other job time: real 0m0.260s. This job kills all supervisord (=rabbitmq worker) jobs every 4 hours. Wrapping in a shell is indeed a good idea. Do you have any idea why it produces so much memory?
– Gudgip
Feb 14 at 10:17
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
@Gudgip, it can be some deadlock when you deal with MQ/database. Better add to the script a lot of control prints and via shell script add dedicated for each cron instance log and check.
– Romeo Ninov
Feb 14 at 10:32
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
Seems reasonable. I'll try this. :)
– Gudgip
Feb 14 at 12:09
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f500581%2fcrond-php-server-crash%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Please if you downvote my question, tell me why so I can improve it.
– Gudgip
Feb 14 at 10:11