Trace number 3249492

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.43 1201.98

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-17x30.cnf
MD5SUM3a506c7557d86d0d326e3b3cdee85b69
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2040
Number of clauses240210
Sum of the clauses size954720
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 23060
Number of clauses of size 30
Number of clauses of size 4237150
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249492-1304086822/watcher-3249492-1304086822 -o /tmp/evaluation-result-3249492-1304086822/solver-3249492-1304086822 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3249492-1304086822.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.97 7.08 7.68 5/181 18625
/proc/meminfo: memFree=30329416/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=9628 CPUtime=0 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) R 18623 18625 17576 0 -1 4202496 482 1261 0 0 0 0 0 0 25 0 1 0 520282581 9859072 412 33554432000 4194304 4417332 140736236227984 18446744073709551615 4231667 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/18625/statm: 2407 412 146 55 0 293 0

[startup+0.00969488 s]
/proc/loadavg: 4.97 7.08 7.68 5/181 18625
/proc/meminfo: memFree=30329416/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=10408 CPUtime=0 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) R 18623 18625 17576 0 -1 4202496 661 1261 0 0 0 0 0 0 25 0 1 0 520282581 10657792 591 33554432000 4194304 4417332 140736236227984 18446744073709551615 4231248 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/18625/statm: 2602 591 146 55 0 488 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 10408

[startup+0.100717 s]
/proc/loadavg: 4.97 7.08 7.68 5/181 18625
/proc/meminfo: memFree=30329416/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=34124 CPUtime=0.08 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) R 18623 18625 17576 0 -1 4202496 6711 1261 0 0 8 0 0 0 25 0 1 0 520282581 34942976 6517 33554432000 4194304 4417332 140736236227984 18446744073709551615 253640553565 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/18625/statm: 8531 6541 147 55 0 6417 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 34124

[startup+0.300644 s]
/proc/loadavg: 4.97 7.08 7.68 5/181 18625
/proc/meminfo: memFree=30329416/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=108928 CPUtime=0.29 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 17012 1261 0 0 28 1 0 0 25 0 5 0 520282581 111542272 15032 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 27232 15016 163 55 0 25118 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 108928

[startup+0.700572 s]
/proc/loadavg: 4.97 7.08 7.68 5/181 18625
/proc/meminfo: memFree=30329416/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=130840 CPUtime=1.89 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 24684 1261 0 0 186 3 0 0 25 0 5 0 520282581 133980160 20526 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 32710 20526 177 55 0 30596 0
Current children cumulated CPU time (s) 1.89
Current children cumulated vsize (KiB) 130840

[startup+1.50043 s]
/proc/loadavg: 4.97 7.08 7.68 9/186 18635
/proc/meminfo: memFree=30229440/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=137396 CPUtime=5.08 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 27885 1261 0 0 505 3 0 0 25 0 5 0 520282581 140693504 22139 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 34349 22139 179 55 0 32235 0
[pid=18625/tid=18632] ppid=18623 vsize=137396 CPUtime=1.19 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 2296 1261 0 0 119 0 0 0 25 0 5 0 520282611 140693504 22139 33554432000 4194304 4417332 140736236227984 18446744073709551615 4254862 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=137396 CPUtime=1.19 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 2305 1261 0 0 119 0 0 0 25 0 5 0 520282611 140693504 22139 33554432000 4194304 4417332 140736236227984 18446744073709551615 4255010 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=137396 CPUtime=1.19 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 3279 1261 0 0 119 0 0 0 25 0 5 0 520282611 140693504 22139 33554432000 4194304 4417332 140736236227984 18446744073709551615 4320926 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=137396 CPUtime=1.19 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 3006 1261 0 0 119 0 0 0 25 0 5 0 520282611 140693504 22139 33554432000 4194304 4417332 140736236227984 18446744073709551615 4250832 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.08
Current children cumulated vsize (KiB) 137396

[startup+3.10113 s]
/proc/loadavg: 4.97 7.08 7.68 9/186 18635
/proc/meminfo: memFree=30203400/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=177280 CPUtime=11.48 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 40560 1261 0 0 1144 4 0 0 25 0 5 0 520282581 181534720 32106 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 44320 32106 179 55 0 42206 0
[pid=18625/tid=18632] ppid=18623 vsize=177280 CPUtime=2.78 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 5770 1261 0 0 278 0 0 0 25 0 5 0 520282611 181534720 32106 33554432000 4194304 4417332 140736236227984 18446744073709551615 4239414 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=177280 CPUtime=2.79 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 5779 1261 0 0 279 0 0 0 25 0 5 0 520282611 181534720 32106 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=177280 CPUtime=2.79 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 5595 1261 0 0 279 0 0 0 25 0 5 0 520282611 181534720 32106 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=177280 CPUtime=2.79 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 6417 1261 0 0 279 0 0 0 25 0 5 0 520282611 181534720 32106 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.48
Current children cumulated vsize (KiB) 177280

[startup+6.30055 s]
/proc/loadavg: 5.22 7.10 7.68 9/186 18635
/proc/meminfo: memFree=30129560/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=191044 CPUtime=24.26 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 51933 1261 0 0 2421 5 0 0 25 0 5 0 520282581 195629056 35579 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 47761 35579 179 55 0 45647 0
[pid=18625/tid=18632] ppid=18623 vsize=191044 CPUtime=5.96 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 5770 1261 0 0 596 0 0 0 25 0 5 0 520282611 195629056 35579 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=191044 CPUtime=5.99 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 5779 1261 0 0 599 0 0 0 25 0 5 0 520282611 195629056 35579 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=191044 CPUtime=5.99 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 5595 1261 0 0 599 0 0 0 25 0 5 0 520282611 195629056 35579 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=191044 CPUtime=5.99 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 17790 1261 0 0 598 1 0 0 25 0 5 0 520282611 195629056 35579 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364355 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.26
Current children cumulated vsize (KiB) 191044

[startup+12.7004 s]
/proc/loadavg: 5.44 7.11 7.68 9/186 18635
/proc/meminfo: memFree=30085312/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=202724 CPUtime=49.83 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 76842 1261 0 0 4975 8 0 0 25 0 5 0 520282581 207589376 38468 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 50681 38468 180 55 0 48567 0
[pid=18625/tid=18632] ppid=18623 vsize=202724 CPUtime=12.33 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 6338 1261 0 0 1233 0 0 0 25 0 5 0 520282611 207589376 38468 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=202724 CPUtime=12.39 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 6141 1261 0 0 1239 0 0 0 25 0 5 0 520282611 207589376 38468 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=202724 CPUtime=12.39 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 9540 1261 0 0 1238 1 0 0 25 0 5 0 520282611 207589376 38468 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=202724 CPUtime=12.39 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 37824 1261 0 0 1235 4 0 0 25 0 5 0 520282611 207589376 38468 33554432000 4194304 4417332 140736236227984 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.83
Current children cumulated vsize (KiB) 202724

[startup+25.501 s]
/proc/loadavg: 6.01 7.16 7.68 9/186 18635
/proc/meminfo: memFree=29994792/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=239116 CPUtime=100.96 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 110986 1261 0 0 10084 12 0 0 25 0 5 0 520282581 244854784 47566 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 59779 47566 180 55 0 57665 0
[pid=18625/tid=18632] ppid=18623 vsize=239116 CPUtime=25.07 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 11526 1261 0 0 2506 1 0 0 25 0 5 0 520282611 244854784 47566 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=239116 CPUtime=25.2 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 12769 1261 0 0 2519 1 0 0 25 0 5 0 520282611 244854784 47566 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=239116 CPUtime=25.19 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 12247 1261 0 0 2518 1 0 0 25 0 5 0 520282611 244854784 47566 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=239116 CPUtime=25.19 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 57445 1261 0 0 2513 6 0 0 25 0 5 0 520282611 244854784 47566 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.96
Current children cumulated vsize (KiB) 239116

[startup+51.1004 s]
/proc/loadavg: 6.75 7.24 7.70 9/186 18636
/proc/meminfo: memFree=29794720/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=383476 CPUtime=203.24 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 199621 1261 0 0 20299 25 0 0 25 0 5 0 520282581 392679424 83656 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 95869 83656 180 55 0 93755 0
[pid=18625/tid=18632] ppid=18623 vsize=383476 CPUtime=50.54 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 26744 1261 0 0 5051 3 0 0 25 0 5 0 520282611 392679424 83656 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364631 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=383476 CPUtime=50.8 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 28217 1261 0 0 5077 3 0 0 25 0 5 0 520282611 392679424 83656 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=383476 CPUtime=50.79 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 27459 1261 0 0 5076 3 0 0 25 0 5 0 520282611 392679424 83656 33554432000 4194304 4417332 140736236227984 18446744073709551615 4249193 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=383476 CPUtime=50.79 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 100202 1261 0 0 5066 13 0 0 25 0 5 0 520282611 392679424 83656 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.24
Current children cumulated vsize (KiB) 383476

[startup+102.306 s]
/proc/loadavg: 7.53 7.37 7.72 9/186 18637
/proc/meminfo: memFree=29485948/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=502912 CPUtime=407.8 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 369962 1261 0 0 40733 47 0 0 25 0 5 0 520282581 514981888 97164 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 125728 97164 180 55 0 123614 0
[pid=18625/tid=18632] ppid=18623 vsize=502912 CPUtime=101.5 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 81215 1261 0 0 10139 11 0 0 25 0 5 0 520282611 514981888 97164 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=502912 CPUtime=101.99 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 54766 1261 0 0 10193 6 0 0 25 0 5 0 520282611 514981888 97164 33554432000 4194304 4417332 140736236227984 18446744073709551615 4354410 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=502912 CPUtime=101.99 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 71354 1261 0 0 10191 8 0 0 25 0 5 0 520282611 514981888 97164 33554432000 4194304 4417332 140736236227984 18446744073709551615 4353881 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=502912 CPUtime=101.98 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 145628 1261 0 0 10180 18 0 0 25 0 5 0 520282611 514981888 97164 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357218 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 407.8
Current children cumulated vsize (KiB) 502912

[startup+162.301 s]
/proc/loadavg: 7.90 7.50 7.74 9/186 18639
/proc/meminfo: memFree=29045356/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=743288 CPUtime=647.46 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 568216 1261 0 0 64673 73 0 0 25 0 5 0 520282581 761126912 157258 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 185822 157258 180 55 0 183708 0
[pid=18625/tid=18632] ppid=18623 vsize=743288 CPUtime=161.19 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 132578 1261 0 0 16101 18 0 0 25 0 5 0 520282611 761126912 157258 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=743288 CPUtime=161.99 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 117716 1261 0 0 16185 14 0 0 25 0 5 0 520282611 761126912 157258 33554432000 4194304 4417332 140736236227984 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=743288 CPUtime=161.99 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 124875 1261 0 0 16184 15 0 0 25 0 5 0 520282611 761126912 157258 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357164 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=743288 CPUtime=161.98 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 176048 1261 0 0 16174 24 0 0 25 0 5 0 520282611 761126912 157258 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 647.46
Current children cumulated vsize (KiB) 743288


################
# More data... #
################

[pid=18625/tid=18633] ppid=18623 vsize=1839064 CPUtime=762 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 206517 1261 0 0 76175 25 0 0 25 0 5 0 520282611 1883201536 431202 33554432000 4194304 4417332 140736236227984 18446744073709551615 4236482 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=1839064 CPUtime=762.01 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 476661 1261 0 0 76145 56 0 0 25 0 5 0 520282611 1883201536 431202 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=1839064 CPUtime=761.93 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 445109 1261 0 0 76126 67 0 0 25 0 5 0 520282611 1883201536 431202 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3044.3
Current children cumulated vsize (KiB) 1839064

[startup+822.3 s]
/proc/loadavg: 8.04 7.99 7.89 9/186 18656
/proc/meminfo: memFree=27421708/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=1737500 CPUtime=3283.96 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 1599991 1261 0 0 328181 215 0 0 25 0 5 0 520282581 1779200000 405811 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 434375 405811 180 55 0 432261 0
[pid=18625/tid=18632] ppid=18623 vsize=1737500 CPUtime=817.72 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 420817 1261 0 0 81714 58 0 0 25 0 5 0 520282611 1779200000 405811 33554432000 4194304 4417332 140736236227984 18446744073709551615 4270130 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=1737500 CPUtime=822 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 213844 1261 0 0 82173 27 0 0 25 0 5 0 520282611 1779200000 405811 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=1737500 CPUtime=822 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 503222 1261 0 0 82140 60 0 0 25 0 5 0 520282611 1779200000 405811 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=1737500 CPUtime=821.92 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 445109 1261 0 0 82125 67 0 0 25 0 5 0 520282611 1779200000 405811 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3283.96
Current children cumulated vsize (KiB) 1737500

[startup+882.301 s]
/proc/loadavg: 8.05 8.00 7.90 9/186 18658
/proc/meminfo: memFree=27222796/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=1659976 CPUtime=3523.64 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 1728434 1261 0 0 352128 236 0 0 25 0 5 0 520282581 1699815424 386430 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 414994 386430 180 55 0 412880 0
[pid=18625/tid=18632] ppid=18623 vsize=1659976 CPUtime=877.39 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 457356 1261 0 0 87673 66 0 0 25 0 5 0 520282611 1699815424 386430 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=1659976 CPUtime=882 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 232532 1261 0 0 88171 29 0 0 25 0 5 0 520282611 1699815424 386430 33554432000 4194304 4417332 140736236227984 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=1659976 CPUtime=882 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 530102 1261 0 0 88137 63 0 0 25 0 5 0 520282611 1699815424 386430 33554432000 4194304 4417332 140736236227984 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=1659976 CPUtime=881.91 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 491445 1261 0 0 88117 74 0 0 25 0 5 0 520282611 1699815424 386430 33554432000 4194304 4417332 140736236227984 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3523.64
Current children cumulated vsize (KiB) 1659976

[startup+942.301 s]
/proc/loadavg: 8.13 8.03 7.92 9/186 18660
/proc/meminfo: memFree=27234684/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=1585540 CPUtime=3763.31 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 1782903 1261 0 0 376089 242 0 0 25 0 5 0 520282581 1623592960 367821 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 396385 367821 180 55 0 394271 0
[pid=18625/tid=18632] ppid=18623 vsize=1585540 CPUtime=937.08 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 480204 1261 0 0 93638 70 0 0 25 0 5 0 520282611 1623592960 367821 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=1585540 CPUtime=942 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 240725 1261 0 0 94170 30 0 0 25 0 5 0 520282611 1623592960 367821 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=1585540 CPUtime=942 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 538294 1261 0 0 94136 64 0 0 25 0 5 0 520282611 1623592960 367821 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=1585540 CPUtime=941.91 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 506681 1261 0 0 94115 76 0 0 25 0 5 0 520282611 1623592960 367821 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357806 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3763.31
Current children cumulated vsize (KiB) 1585540

[startup+1002.3 s]
/proc/loadavg: 8.19 8.06 7.93 9/186 18661
/proc/meminfo: memFree=26978360/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=1713888 CPUtime=4002.99 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 1877750 1261 0 0 400045 254 0 0 25 0 5 0 520282581 1755021312 399908 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 428472 399908 180 55 0 426358 0
[pid=18625/tid=18632] ppid=18623 vsize=1713888 CPUtime=996.76 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 503052 1261 0 0 99604 72 0 0 25 0 5 0 520282611 1755021312 399908 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=1713888 CPUtime=1002.01 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 293143 1261 0 0 100164 37 0 0 25 0 5 0 520282611 1755021312 399908 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=1713888 CPUtime=1002 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 557875 1261 0 0 100134 66 0 0 25 0 5 0 520282611 1755021312 399908 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=1713888 CPUtime=1001.9 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 506681 1261 0 0 100114 76 0 0 25 0 5 0 520282611 1755021312 399908 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4002.99
Current children cumulated vsize (KiB) 1713888

[startup+1062.3 s]
/proc/loadavg: 8.20 8.08 7.94 9/186 18663
/proc/meminfo: memFree=27161120/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=1813364 CPUtime=4242.67 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 1941868 1261 0 0 424003 264 0 0 25 0 5 0 520282581 1856884736 424777 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 453341 424777 180 55 0 451227 0
[pid=18625/tid=18632] ppid=18623 vsize=1813364 CPUtime=1056.44 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 503052 1261 0 0 105572 72 0 0 25 0 5 0 520282611 1856884736 424777 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=1813364 CPUtime=1062.01 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 311953 1261 0 0 106162 39 0 0 25 0 5 0 520282611 1856884736 424777 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357169 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=1813364 CPUtime=1062 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 603183 1261 0 0 106126 74 0 0 25 0 5 0 520282611 1856884736 424777 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=1813364 CPUtime=1061.9 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 506681 1261 0 0 106114 76 0 0 25 0 5 0 520282611 1856884736 424777 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4242.67
Current children cumulated vsize (KiB) 1813364

[startup+1122.3 s]
/proc/loadavg: 8.19 8.09 7.95 9/186 18664
/proc/meminfo: memFree=27019996/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=2082700 CPUtime=4482.34 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 2038201 1261 0 0 447954 280 0 0 25 0 5 0 520282581 2132684800 492111 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 520675 492111 180 55 0 518561 0
[pid=18625/tid=18632] ppid=18623 vsize=2082700 CPUtime=1116.11 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 526708 1261 0 0 111535 76 0 0 25 0 5 0 520282611 2132684800 492111 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=2082700 CPUtime=1122.01 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 354157 1261 0 0 112157 44 0 0 25 0 5 0 520282611 2132684800 492111 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=2082700 CPUtime=1122 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 603183 1261 0 0 112126 74 0 0 25 0 5 0 520282611 2132684800 492111 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=2082700 CPUtime=1121.89 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 537154 1261 0 0 112107 82 0 0 25 0 5 0 520282611 2132684800 492111 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4482.34
Current children cumulated vsize (KiB) 2082700

[startup+1182.3 s]
/proc/loadavg: 8.07 8.07 7.95 9/186 18666
/proc/meminfo: memFree=26778056/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=2190112 CPUtime=4722.02 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 2249134 1261 0 0 471880 322 0 0 25 0 5 0 520282581 2242674688 518964 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 547528 518964 180 55 0 545414 0
[pid=18625/tid=18632] ppid=18623 vsize=2190112 CPUtime=1175.8 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 560692 1261 0 0 117497 83 0 0 25 0 5 0 520282611 2242674688 518964 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=2190112 CPUtime=1182.01 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 382876 1261 0 0 118151 50 0 0 25 0 5 0 520282611 2242674688 518964 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=2190112 CPUtime=1182 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 720578 1261 0 0 118103 97 0 0 25 0 5 0 520282611 2242674688 518964 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=2190112 CPUtime=1181.88 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 567989 1261 0 0 118099 89 0 0 25 0 5 0 520282611 2242674688 518964 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4722.02
Current children cumulated vsize (KiB) 2190112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.05 8.07 7.95 5/180 18668
/proc/meminfo: memFree=28309744/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=2155732 CPUtime=4800.33 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 2276430 1261 0 0 479705 328 0 0 25 0 5 0 520282581 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 538933 510369 180 55 0 536819 0
[pid=18625/tid=18632] ppid=18623 vsize=2155732 CPUtime=1195.29 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 560692 1261 0 0 119446 83 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=2155732 CPUtime=1201.62 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 403633 1261 0 0 120108 54 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4336603 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=2155732 CPUtime=1201.62 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 727117 1261 0 0 120063 99 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=2155732 CPUtime=1201.48 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 567989 1261 0 0 120059 89 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 2155732

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 8.05 8.07 7.95 5/180 18668
/proc/meminfo: memFree=28309744/32951124 swapFree=67111364/67111528
[pid=18625] ppid=18623 vsize=2155732 CPUtime=4800.33 cores=0,2,4,6
/proc/18625/stat : 18625 (plingeling) S 18623 18625 17576 0 -1 4202496 2276430 1261 0 0 479705 328 0 0 25 0 5 0 520282581 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/18625/statm: 538933 510369 180 55 0 536819 0
[pid=18625/tid=18632] ppid=18623 vsize=2155732 CPUtime=1195.29 cores=0,2,4,6
/proc/18625/task/18632/stat : 18632 (plingeling) R 18623 18625 17576 0 -1 4202560 560692 1261 0 0 119446 83 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=18625/tid=18633] ppid=18623 vsize=2155732 CPUtime=1201.62 cores=0,2,4,6
/proc/18625/task/18633/stat : 18633 (plingeling) R 18623 18625 17576 0 -1 4202560 403633 1261 0 0 120108 54 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4336603 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18625/tid=18634] ppid=18623 vsize=2155732 CPUtime=1201.62 cores=0,2,4,6
/proc/18625/task/18634/stat : 18634 (plingeling) R 18623 18625 17576 0 -1 4202560 727117 1261 0 0 120063 99 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18625/tid=18635] ppid=18623 vsize=2155732 CPUtime=1201.48 cores=0,2,4,6
/proc/18625/task/18635/stat : 18635 (plingeling) R 18623 18625 17576 0 -1 4202560 567989 1261 0 0 120059 89 0 0 25 0 5 0 520282611 2207469568 510369 33554432000 4194304 4417332 140736236227984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 2155732

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.98
CPU time (s): 4800.43
CPU user time (s): 4797.07
CPU system time (s): 3.35749
CPU usage (%): 399.375
Max. virtual memory (cumulated for all children) (KiB): 2392176

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.07
system time used= 3.35749
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2277691
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 122
involuntary context switches= 35361

runsolver used 1.9697 second user time and 4.38133 second system time

The end

Launcher Data 

Begin job on node106 at 2011-04-29 16:20:22
IDJOB=3249492
IDBENCH=83172
IDSOLVER=1605
FILE ID=node106/3249492-1304086822
RUNJOBID= node106-1304086821-18488
PBS_JOBID= 13170519
Free space on /tmp= 72152 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-17x30.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249492-1304086822/watcher-3249492-1304086822 -o /tmp/evaluation-result-3249492-1304086822/solver-3249492-1304086822 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3249492-1304086822.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3a506c7557d86d0d326e3b3cdee85b69
RANDOM SEED=798570630

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30329820 kB
Buffers:        247108 kB
Cached:        2070504 kB
SwapCached:         88 kB
Active:         812308 kB
Inactive:      1598256 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30329820 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            7512 kB
Writeback:           0 kB
AnonPages:       92020 kB
Mapped:          14872 kB
Slab:           146136 kB
PageTables:       4360 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   277092 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72156 MiB
End job on node106 at 2011-04-29 16:40:26