Trace number 3371446

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.33 1204.36

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
rbcl_xits_09_UNKNOWN.cnf
MD5SUM18ad1dee1e9d863458e75a20545ae68a
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1430
Number of clauses79453
Sum of the clauses size236162
Maximum clause length10
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22358
Number of clauses of size 377072
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 523

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371446-1305417745/watcher-3371446-1305417745 -o /tmp/evaluation-result-3371446-1305417745/solver-3371446-1305417745 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371446-1305417745.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.73 5.63 6.92 3/410 9380
/proc/meminfo: memFree=21324420/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=8728 CPUtime=0 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 215 0 0 0 0 0 0 0 21 0 1 0 732005245 8937472 146 33554432000 4194304 4417452 140736739125632 18446744073709551615 215988592384 0 0 4096 0 18446744071562362840 0 0 17 4 0 0 0
/proc/9380/statm: 2182 146 122 55 0 68 0

[startup+0.0576059 s]
/proc/loadavg: 3.73 5.63 6.92 3/410 9380
/proc/meminfo: memFree=21324420/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=13644 CPUtime=0.02 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) R 9378 9380 8585 0 -1 4202496 1553 1250 0 2 2 0 0 0 21 0 1 0 732005245 13971456 1435 33554432000 4194304 4417452 140736739125632 18446744073709551615 4209539 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9380/statm: 3411 1435 149 55 0 1297 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 13644

[startup+0.1006 s]
/proc/loadavg: 3.73 5.63 6.92 3/410 9380
/proc/meminfo: memFree=21324420/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=25932 CPUtime=0.06 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) R 9378 9380 8585 0 -1 4202496 4625 1250 0 2 6 0 0 0 21 0 1 0 732005245 26554368 4507 33554432000 4194304 4417452 140736739125632 18446744073709551615 4274378 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9380/statm: 6483 4507 149 55 0 4369 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 25932

[startup+0.300567 s]
/proc/loadavg: 3.73 5.63 6.92 3/410 9380
/proc/meminfo: memFree=21324420/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=67700 CPUtime=0.79 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 4990 1250 0 2 79 0 0 0 21 0 5 0 732005245 69324800 4738 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 16925 4738 174 55 0 14811 0
Current children cumulated CPU time (s) 0.79
Current children cumulated vsize (KiB) 67700

[startup+0.700497 s]
/proc/loadavg: 3.73 5.63 6.92 3/410 9380
/proc/meminfo: memFree=21324420/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=140568 CPUtime=2.39 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 9302 1250 0 2 239 0 0 0 21 0 5 0 732005245 143941632 6304 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 35142 6304 179 55 0 33028 0
Current children cumulated CPU time (s) 2.39
Current children cumulated vsize (KiB) 140568

[startup+1.50036 s]
/proc/loadavg: 3.59 5.57 6.90 6/415 9408
/proc/meminfo: memFree=21298264/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=148240 CPUtime=5.58 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 12620 1250 0 2 557 1 0 0 21 0 5 0 732005245 151797760 8477 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 37060 8477 179 55 0 34946 0
[pid=9380/tid=9398] ppid=9378 vsize=148240 CPUtime=1.38 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 1702 1250 0 2 138 0 0 0 25 0 5 0 732005257 151797760 8477 33554432000 4194304 4417452 140736739125632 18446744073709551615 4259826 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=148240 CPUtime=1.37 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 1432 1250 0 2 137 0 0 0 25 0 5 0 732005257 151797760 8477 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=148240 CPUtime=1.37 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 2568 1250 0 2 137 0 0 0 25 0 5 0 732005257 151797760 8477 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=148240 CPUtime=1.36 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 2281 1250 0 2 136 0 0 0 25 0 5 0 732005257 151797760 8477 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.58
Current children cumulated vsize (KiB) 148240

[startup+3.10108 s]
/proc/loadavg: 3.59 5.57 6.90 9/422 9441
/proc/meminfo: memFree=21258860/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=167084 CPUtime=11.97 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 19262 1250 0 2 1195 2 0 0 21 0 5 0 732005245 171094016 13188 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 41771 13188 179 55 0 39657 0
[pid=9380/tid=9398] ppid=9378 vsize=167084 CPUtime=2.98 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 3689 1250 0 2 298 0 0 0 25 0 5 0 732005257 171094016 13188 33554432000 4194304 4417452 140736739125632 18446744073709551615 4246343 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=167084 CPUtime=2.96 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 1944 1250 0 2 296 0 0 0 25 0 5 0 732005257 171094016 13188 33554432000 4194304 4417452 140736739125632 18446744073709551615 4366362 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=167084 CPUtime=2.97 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 3265 1250 0 2 297 0 0 0 25 0 5 0 732005257 171094016 13188 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=167084 CPUtime=2.94 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 5727 1250 0 2 294 0 0 0 25 0 5 0 732005257 171094016 13188 33554432000 4194304 4417452 140736739125632 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.97
Current children cumulated vsize (KiB) 167084

[startup+6.30052 s]
/proc/loadavg: 3.95 5.61 6.90 9/422 9441
/proc/meminfo: memFree=21235424/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=180280 CPUtime=24.72 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 26291 1250 0 2 2469 3 0 0 21 0 5 0 732005245 184606720 16488 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 45070 16488 180 55 0 42956 0
[pid=9380/tid=9398] ppid=9378 vsize=180280 CPUtime=6.17 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 4138 1250 0 2 617 0 0 0 25 0 5 0 732005257 184606720 16488 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=180280 CPUtime=6.16 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 3777 1250 0 2 616 0 0 0 25 0 5 0 732005257 184606720 16488 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=180280 CPUtime=6.17 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 4472 1250 0 2 617 0 0 0 25 0 5 0 732005257 184606720 16488 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=180280 CPUtime=6.1 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 9267 1250 0 2 609 1 0 0 25 0 5 0 732005257 184606720 16488 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.72
Current children cumulated vsize (KiB) 180280

[startup+12.7004 s]
/proc/loadavg: 4.35 5.66 6.91 9/422 9441
/proc/meminfo: memFree=21197868/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=212188 CPUtime=50.23 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 41668 1250 0 2 5017 6 0 0 21 0 5 0 732005245 217280512 24465 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 53047 24465 180 55 0 50933 0
[pid=9380/tid=9398] ppid=9378 vsize=212188 CPUtime=12.58 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 6591 1250 0 2 1257 1 0 0 25 0 5 0 732005257 217280512 24465 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=212188 CPUtime=12.56 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 5414 1250 0 2 1256 0 0 0 25 0 5 0 732005257 217280512 24465 33554432000 4194304 4417452 140736739125632 18446744073709551615 4365954 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=212188 CPUtime=12.57 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 9049 1250 0 2 1256 1 0 0 25 0 5 0 732005257 217280512 24465 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=212188 CPUtime=12.41 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 15977 1250 0 2 1239 2 0 0 25 0 5 0 732005257 217280512 24465 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.23
Current children cumulated vsize (KiB) 212188

[startup+25.5012 s]
/proc/loadavg: 5.16 5.78 6.93 9/422 9441
/proc/meminfo: memFree=21161924/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=230216 CPUtime=101.26 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 71661 1250 0 2 10115 11 0 0 21 0 5 0 732005245 235741184 29004 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 57554 29004 180 55 0 55440 0
[pid=9380/tid=9398] ppid=9378 vsize=230216 CPUtime=25.37 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 13453 1250 0 2 2535 2 0 0 25 0 5 0 732005257 235741184 29004 33554432000 4194304 4417452 140736739125632 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=230216 CPUtime=25.36 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 11310 1250 0 2 2535 1 0 0 25 0 5 0 732005257 235741184 29004 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=230216 CPUtime=25.37 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 16005 1250 0 2 2535 2 0 0 25 0 5 0 732005257 235741184 29004 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=230216 CPUtime=25.03 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 26256 1250 0 2 2499 4 0 0 25 0 5 0 732005257 235741184 29004 33554432000 4194304 4417452 140736739125632 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.26
Current children cumulated vsize (KiB) 230216

[startup+51.1007 s]
/proc/loadavg: 6.20 5.97 6.96 9/422 9442
/proc/meminfo: memFree=21046724/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=304384 CPUtime=203.31 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 124092 1250 0 2 20310 21 0 0 21 0 5 0 732005245 311689216 47514 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 76096 47514 180 55 0 73982 0
[pid=9380/tid=9398] ppid=9378 vsize=304384 CPUtime=50.98 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 25035 1250 0 2 5094 4 0 0 25 0 5 0 732005257 311689216 47514 33554432000 4194304 4417452 140736739125632 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=304384 CPUtime=50.96 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 27033 1250 0 2 5092 4 0 0 25 0 5 0 732005257 311689216 47514 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=304384 CPUtime=50.97 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 22602 1250 0 2 5094 3 0 0 25 0 5 0 732005257 311689216 47514 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=304384 CPUtime=50.29 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 44785 1250 0 2 5021 8 0 0 25 0 5 0 732005257 311689216 47514 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.31
Current children cumulated vsize (KiB) 304384

[startup+102.316 s]
/proc/loadavg: 7.38 6.31 7.02 9/422 9443
/proc/meminfo: memFree=20932388/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=361704 CPUtime=407.46 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 191209 1250 0 2 40714 32 0 0 21 0 5 0 732005245 370384896 61844 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 90426 61844 180 55 0 88312 0
[pid=9380/tid=9398] ppid=9378 vsize=361704 CPUtime=102.19 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 46965 1250 0 2 10211 8 0 0 25 0 5 0 732005257 370384896 61844 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=361704 CPUtime=102.16 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 47684 1250 0 2 10209 7 0 0 25 0 5 0 732005257 370384896 61844 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=361704 CPUtime=102.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 35805 1250 0 2 10214 5 0 0 25 0 5 0 732005257 370384896 61844 33554432000 4194304 4417452 140736739125632 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=361704 CPUtime=100.8 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 56118 1250 0 2 10070 10 0 0 25 0 5 0 732005257 370384896 61844 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.46
Current children cumulated vsize (KiB) 361704

[startup+162.3 s]
/proc/loadavg: 7.96 6.66 7.10 9/422 9445
/proc/meminfo: memFree=20812812/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=416268 CPUtime=646.58 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 260446 1250 0 2 64613 45 0 0 21 0 5 0 732005245 426258432 75051 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 104067 75051 180 55 0 101953 0
[pid=9380/tid=9398] ppid=9378 vsize=416268 CPUtime=162.18 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 67384 1250 0 2 16207 11 0 0 25 0 5 0 732005257 426258432 75051 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=416268 CPUtime=162.13 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 65943 1250 0 2 16203 10 0 0 25 0 5 0 732005257 426258432 75051 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364903 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=416268 CPUtime=162.17 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 42217 1250 0 2 16211 6 0 0 25 0 5 0 732005257 426258432 75051 33554432000 4194304 4417452 140736739125632 18446744073709551615 4366137 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=416268 CPUtime=159.97 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 80265 1250 0 2 15982 15 0 0 25 0 5 0 732005257 426258432 75051 33554432000 4194304 4417452 140736739125632 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 646.58
Current children cumulated vsize (KiB) 416268


################
# More data... #
################

[pid=9380/tid=9399] ppid=9378 vsize=709092 CPUtime=762.02 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 208226 1250 0 2 76170 32 0 0 25 0 5 0 732005257 726110208 148691 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=709092 CPUtime=762.2 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 106711 1250 0 2 76203 17 0 0 25 0 5 0 732005257 726110208 148691 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=709092 CPUtime=751.82 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 194581 1250 0 2 75145 37 0 0 25 0 5 0 732005257 726110208 148691 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3038.35
Current children cumulated vsize (KiB) 709092

[startup+822.301 s]
/proc/loadavg: 8.10 7.98 7.61 9/422 9463
/proc/meminfo: memFree=20139568/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=711904 CPUtime=3277.47 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 742415 1250 0 2 327620 127 0 0 21 0 5 0 732005245 728989696 149394 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 177976 149394 180 55 0 175862 0
[pid=9380/tid=9398] ppid=9378 vsize=711904 CPUtime=822.19 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 220413 1250 0 2 82180 39 0 0 25 0 5 0 732005257 728989696 149394 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=711904 CPUtime=822 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 215634 1250 0 2 82167 33 0 0 25 0 5 0 732005257 728989696 149394 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=711904 CPUtime=822.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 106711 1250 0 2 82202 17 0 0 25 0 5 0 732005257 728989696 149394 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=711904 CPUtime=811 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 195020 1250 0 2 81063 37 0 0 25 0 5 0 732005257 728989696 149394 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3277.47
Current children cumulated vsize (KiB) 711904

[startup+882.3 s]
/proc/loadavg: 8.09 7.99 7.64 9/422 9464
/proc/meminfo: memFree=20076908/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=744600 CPUtime=3516.64 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 760052 1250 0 2 351534 130 0 0 21 0 5 0 732005245 762470400 157568 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 186150 157568 180 55 0 184036 0
[pid=9380/tid=9398] ppid=9378 vsize=744600 CPUtime=882.19 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 226430 1250 0 2 88179 40 0 0 25 0 5 0 732005257 762470400 157568 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=744600 CPUtime=881.98 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 215634 1250 0 2 88165 33 0 0 25 0 5 0 732005257 762470400 157568 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=744600 CPUtime=882.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 113662 1250 0 2 88201 18 0 0 25 0 5 0 732005257 762470400 157568 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=744600 CPUtime=870.16 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 199689 1250 0 2 86979 37 0 0 25 0 5 0 732005257 762470400 157568 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3516.64
Current children cumulated vsize (KiB) 744600

[startup+942.301 s]
/proc/loadavg: 8.06 8.00 7.66 9/422 9466
/proc/meminfo: memFree=19960532/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=817388 CPUtime=3755.82 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 795835 1250 0 2 375448 134 0 0 21 0 5 0 732005245 837005312 175765 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 204347 175765 180 55 0 202233 0
[pid=9380/tid=9398] ppid=9378 vsize=817388 CPUtime=942.19 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 226430 1250 0 2 94179 40 0 0 25 0 5 0 732005257 837005312 175765 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=817388 CPUtime=941.96 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 228425 1250 0 2 94162 34 0 0 25 0 5 0 732005257 837005312 175765 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=817388 CPUtime=942.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 134802 1250 0 2 94199 20 0 0 25 0 5 0 732005257 837005312 175765 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364803 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=817388 CPUtime=929.36 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 201541 1250 0 2 92898 38 0 0 25 0 5 0 732005257 837005312 175765 33554432000 4194304 4417452 140736739125632 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3755.82
Current children cumulated vsize (KiB) 817388

[startup+1002.3 s]
/proc/loadavg: 8.15 8.03 7.69 9/422 9467
/proc/meminfo: memFree=19805376/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=856844 CPUtime=3994.99 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 805699 1250 0 2 399364 135 0 0 21 0 5 0 732005245 877408256 185629 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 214211 185629 180 55 0 212097 0
[pid=9380/tid=9398] ppid=9378 vsize=856844 CPUtime=1002.2 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 233045 1250 0 2 100179 41 0 0 25 0 5 0 732005257 877408256 185629 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=856844 CPUtime=1001.95 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 228425 1250 0 2 100161 34 0 0 25 0 5 0 732005257 877408256 185629 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=856844 CPUtime=1002.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 136294 1250 0 2 100199 20 0 0 25 0 5 0 732005257 877408256 185629 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=856844 CPUtime=988.54 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 203298 1250 0 2 98816 38 0 0 25 0 5 0 732005257 877408256 185629 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364657 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3994.99
Current children cumulated vsize (KiB) 856844

[startup+1062.3 s]
/proc/loadavg: 8.17 8.05 7.72 9/422 9469
/proc/meminfo: memFree=19711824/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=930304 CPUtime=4234.18 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 850836 1250 0 2 423273 145 0 0 21 0 5 0 732005245 952631296 203994 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 232576 203994 180 55 0 230462 0
[pid=9380/tid=9398] ppid=9378 vsize=930304 CPUtime=1062.19 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 241844 1250 0 2 106177 42 0 0 25 0 5 0 732005257 952631296 203994 33554432000 4194304 4417452 140736739125632 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=930304 CPUtime=1061.94 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 239988 1250 0 2 106158 36 0 0 25 0 5 0 732005257 952631296 203994 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=930304 CPUtime=1062.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 136294 1250 0 2 106199 20 0 0 25 0 5 0 732005257 952631296 203994 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=930304 CPUtime=1047.73 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 228073 1250 0 2 104729 44 0 0 25 0 5 0 732005257 952631296 203994 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4234.18
Current children cumulated vsize (KiB) 930304

[startup+1122.3 s]
/proc/loadavg: 8.16 8.07 7.74 9/422 9471
/proc/meminfo: memFree=19706808/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=1001052 CPUtime=4473.33 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 868523 1250 0 2 447184 149 0 0 21 0 5 0 732005245 1025077248 221681 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 250263 221681 180 55 0 248149 0
[pid=9380/tid=9398] ppid=9378 vsize=1001052 CPUtime=1122.19 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 241882 1250 0 2 112177 42 0 0 25 0 5 0 732005257 1025077248 221681 33554432000 4194304 4417452 140736739125632 18446744073709551615 4306923 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=1001052 CPUtime=1121.92 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 239988 1250 0 2 112156 36 0 0 25 0 5 0 732005257 1025077248 221681 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=1001052 CPUtime=1122.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 144566 1250 0 2 112197 22 0 0 25 0 5 0 732005257 1025077248 221681 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=1001052 CPUtime=1106.9 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 237450 1250 0 2 110644 46 0 0 25 0 5 0 732005257 1025077248 221681 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4473.33
Current children cumulated vsize (KiB) 1001052

[startup+1182.3 s]
/proc/loadavg: 8.20 8.10 7.77 9/422 9472
/proc/meminfo: memFree=19693624/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=957376 CPUtime=4712.52 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 892797 1250 0 2 471098 154 0 0 21 0 5 0 732005245 980353024 210762 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 239344 210762 180 55 0 237230 0
[pid=9380/tid=9398] ppid=9378 vsize=957376 CPUtime=1182.2 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 252899 1250 0 2 118175 45 0 0 25 0 5 0 732005257 980353024 210762 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=957376 CPUtime=1181.91 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 248410 1250 0 2 118154 37 0 0 25 0 5 0 732005257 980353024 210762 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364703 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=957376 CPUtime=1182.19 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 149401 1250 0 2 118196 23 0 0 25 0 5 0 732005257 980353024 210762 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=957376 CPUtime=1166.09 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 237450 1250 0 2 116563 46 0 0 25 0 5 0 732005257 980353024 210762 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4712.52
Current children cumulated vsize (KiB) 957376



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.3 s]
/proc/loadavg: 8.14 8.09 7.77 9/422 9473
/proc/meminfo: memFree=19711464/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=911992 CPUtime=4800.23 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 897835 1250 0 2 479868 155 0 0 21 0 5 0 732005245 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 227998 199416 180 55 0 225884 0
[pid=9380/tid=9398] ppid=9378 vsize=911992 CPUtime=1204.22 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 252899 1250 0 2 120377 45 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=911992 CPUtime=1203.92 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 248410 1250 0 2 120355 37 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=911992 CPUtime=1204.2 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 150926 1250 0 2 120397 23 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=911992 CPUtime=1187.79 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 240963 1250 0 2 118732 47 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.23
Current children cumulated vsize (KiB) 911992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.3 s]
/proc/loadavg: 8.14 8.09 7.77 9/422 9473
/proc/meminfo: memFree=19711464/32950940 swapFree=67111356/67111528
[pid=9380] ppid=9378 vsize=911992 CPUtime=4800.23 cores=0,2,4,6
/proc/9380/stat : 9380 (plingeling) S 9378 9380 8585 0 -1 4202496 897835 1250 0 2 479868 155 0 0 21 0 5 0 732005245 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/9380/statm: 227998 199416 180 55 0 225884 0
[pid=9380/tid=9398] ppid=9378 vsize=911992 CPUtime=1204.22 cores=0,2,4,6
/proc/9380/task/9398/stat : 9398 (plingeling) R 9378 9380 8585 0 -1 4202560 252899 1250 0 2 120377 45 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9380/tid=9399] ppid=9378 vsize=911992 CPUtime=1203.92 cores=0,2,4,6
/proc/9380/task/9399/stat : 9399 (plingeling) R 9378 9380 8585 0 -1 4202560 248410 1250 0 2 120355 37 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9380/tid=9400] ppid=9378 vsize=911992 CPUtime=1204.2 cores=0,2,4,6
/proc/9380/task/9400/stat : 9400 (plingeling) R 9378 9380 8585 0 -1 4202560 150926 1250 0 2 120397 23 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9380/tid=9401] ppid=9378 vsize=911992 CPUtime=1187.79 cores=0,2,4,6
/proc/9380/task/9401/stat : 9401 (plingeling) R 9378 9380 8585 0 -1 4202560 240963 1250 0 2 118732 47 0 0 25 0 5 0 732005257 933879808 199416 33554432000 4194304 4417452 140736739125632 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.23
Current children cumulated vsize (KiB) 911992

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1204.36
CPU time (s): 4800.33
CPU user time (s): 4798.73
CPU system time (s): 1.59576
CPU usage (%): 398.579
Max. virtual memory (cumulated for all children) (KiB): 1001052

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.73
system time used= 1.59576
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 899085
page faults= 2
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 113
involuntary context switches= 24119

runsolver used 4.64029 second user time and 11.8072 second system time

The end

Launcher Data 

Begin job on node117 at 2011-05-15 02:02:25
IDJOB=3371446
IDBENCH=71098
IDSOLVER=1848
FILE ID=node117/3371446-1305417745
RUNJOBID= node117-1305417744-9366
PBS_JOBID= 13324528
Free space on /tmp= 73416 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/rbcl_xits_09_UNKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371446-1305417745/watcher-3371446-1305417745 -o /tmp/evaluation-result-3371446-1305417745/solver-3371446-1305417745 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371446-1305417745.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 18ad1dee1e9d863458e75a20545ae68a
RANDOM SEED=90838112

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      21324708 kB
Buffers:        301612 kB
Cached:        3088012 kB
SwapCached:          0 kB
Active:        9383324 kB
Inactive:      1888364 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      21324708 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            2400 kB
Writeback:           0 kB
AnonPages:     7881632 kB
Mapped:          31496 kB
Slab:           243732 kB
PageTables:      43816 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 100410600 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73412 MiB
End job on node117 at 2011-05-15 02:22:31