Trace number 3250372

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.11 1202.95

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-5-5_34309.cnf
MD5SUMe252fbf81407f3388187ab50f0786909
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables34307
Number of clauses708206
Sum of the clauses size3541030
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 5708206
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3250372-1304091889/watcher-3250372-1304091889 -o /tmp/evaluation-result-3250372-1304091889/solver-3250372-1304091889 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3250372-1304091889.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.96 7.90 7.76 6/181 2623
/proc/meminfo: memFree=20717308/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=9272 CPUtime=0 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) R 2621 2623 567 0 -1 4202496 405 1252 0 0 0 0 0 0 23 0 1 0 599419718 9494528 334 33554432000 4194304 4417332 140733672547136 18446744073709551615 231895490304 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/2623/statm: 2318 336 145 55 0 204 0

[startup+0.022067 s]
/proc/loadavg: 6.96 7.90 7.76 6/181 2623
/proc/meminfo: memFree=20717308/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=13960 CPUtime=0.01 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) R 2621 2623 567 0 -1 4202496 1615 1252 0 0 1 0 0 0 23 0 1 0 599419718 14295040 1512 33554432000 4194304 4417332 140733672547136 18446744073709551615 4323180 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/2623/statm: 3490 1512 147 55 0 1376 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 13960

[startup+0.101054 s]
/proc/loadavg: 6.96 7.90 7.76 6/181 2623
/proc/meminfo: memFree=20717308/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=30548 CPUtime=0.09 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) R 2621 2623 567 0 -1 4202496 5778 1252 0 0 9 0 0 0 23 0 1 0 599419718 31281152 5659 33554432000 4194304 4417332 140733672547136 18446744073709551615 4320765 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/2623/statm: 7637 5659 147 55 0 5523 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 30548

[startup+0.301005 s]
/proc/loadavg: 6.96 7.90 7.76 6/181 2623
/proc/meminfo: memFree=20717308/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=51684 CPUtime=0.29 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) R 2621 2623 567 0 -1 4202496 11062 1252 0 0 28 1 0 0 24 0 1 0 599419718 52924416 10943 33554432000 4194304 4417332 140733672547136 18446744073709551615 4259612 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/2623/statm: 12921 10943 147 55 0 10807 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 51684

[startup+0.700932 s]
/proc/loadavg: 6.96 7.90 7.76 6/181 2623
/proc/meminfo: memFree=20717308/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=84452 CPUtime=0.69 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) R 2621 2623 567 0 -1 4202496 19254 1252 0 0 67 2 0 0 25 0 1 0 599419718 86478848 19135 33554432000 4194304 4417332 140733672547136 18446744073709551615 231895490304 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/2623/statm: 21113 19135 147 55 0 18999 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 84452

[startup+1.5008 s]
/proc/loadavg: 6.96 7.90 7.76 6/182 2629
/proc/meminfo: memFree=20563540/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=159556 CPUtime=1.48 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) R 2621 2623 567 0 -1 4202496 38042 1252 0 0 144 4 0 0 25 0 1 0 599419718 163385344 37911 33554432000 4194304 4417332 140733672547136 18446744073709551615 4323155 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/2623/statm: 39889 37911 147 55 0 37775 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 159556

[startup+3.10042 s]
/proc/loadavg: 6.96 7.90 7.76 9/186 2633
/proc/meminfo: memFree=20497912/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=268620 CPUtime=7.21 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 74883 1252 0 0 710 11 0 0 25 0 5 0 599419718 275066880 54937 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 67155 54937 175 55 0 65041 0
[pid=2623/tid=2630] ppid=2621 vsize=268620 CPUtime=1.37 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 5334 1252 0 0 137 0 0 0 19 0 5 0 599419890 275066880 54937 33554432000 4194304 4417332 140733672547136 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=268620 CPUtime=1.36 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 5267 1252 0 0 135 1 0 0 25 0 5 0 599419890 275066880 54937 33554432000 4194304 4417332 140733672547136 18446744073709551615 4259501 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=268620 CPUtime=1.37 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 4918 1252 0 0 136 1 0 0 25 0 5 0 599419890 275066880 54937 33554432000 4194304 4417332 140733672547136 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=268620 CPUtime=1.37 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 4926 1252 0 0 136 1 0 0 25 0 5 0 599419890 275066880 54937 33554432000 4194304 4417332 140733672547136 18446744073709551615 4356771 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 7.21
Current children cumulated vsize (KiB) 268620

[startup+6.30082 s]
/proc/loadavg: 7.04 7.90 7.76 9/186 2633
/proc/meminfo: memFree=20487256/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=270076 CPUtime=19.99 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 77510 1252 0 0 1988 11 0 0 25 0 5 0 599419718 276557824 55304 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 67519 55304 178 55 0 65405 0
[pid=2623/tid=2630] ppid=2621 vsize=270076 CPUtime=4.57 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 6004 1252 0 0 457 0 0 0 25 0 5 0 599419890 276557824 55304 33554432000 4194304 4417332 140733672547136 18446744073709551615 4249407 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=270076 CPUtime=4.54 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 5767 1252 0 0 453 1 0 0 25 0 5 0 599419890 276557824 55304 33554432000 4194304 4417332 140733672547136 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=270076 CPUtime=4.57 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 5690 1252 0 0 456 1 0 0 25 0 5 0 599419890 276557824 55304 33554432000 4194304 4417332 140733672547136 18446744073709551615 4258641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=270076 CPUtime=4.57 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 5611 1252 0 0 456 1 0 0 25 0 5 0 599419890 276557824 55304 33554432000 4194304 4417332 140733672547136 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 19.99
Current children cumulated vsize (KiB) 270076

[startup+12.7006 s]
/proc/loadavg: 7.12 7.90 7.76 9/186 2633
/proc/meminfo: memFree=20464844/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=278256 CPUtime=45.55 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 82702 1252 0 0 4544 11 0 0 25 0 5 0 599419718 284934144 57349 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 69564 57349 178 55 0 67450 0
[pid=2623/tid=2630] ppid=2621 vsize=278256 CPUtime=10.97 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 6004 1252 0 0 1097 0 0 0 25 0 5 0 599419890 284934144 57349 33554432000 4194304 4417332 140733672547136 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=278256 CPUtime=10.91 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 5767 1252 0 0 1090 1 0 0 25 0 5 0 599419890 284934144 57349 33554432000 4194304 4417332 140733672547136 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=278256 CPUtime=10.97 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 6013 1252 0 0 1096 1 0 0 25 0 5 0 599419890 284934144 57349 33554432000 4194304 4417332 140733672547136 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=278256 CPUtime=10.97 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 10480 1252 0 0 1096 1 0 0 25 0 5 0 599419890 284934144 57349 33554432000 4194304 4417332 140733672547136 18446744073709551615 4357618 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 45.55
Current children cumulated vsize (KiB) 278256

[startup+25.5012 s]
/proc/loadavg: 7.38 7.92 7.77 9/186 2634
/proc/meminfo: memFree=20442772/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=356644 CPUtime=96.69 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 92517 1252 0 0 9656 13 0 0 25 0 5 0 599419718 365203456 60627 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 89161 60627 178 55 0 87047 0
[pid=2623/tid=2630] ppid=2621 vsize=356644 CPUtime=23.78 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 6434 1252 0 0 2377 1 0 0 25 0 5 0 599419890 365203456 60627 33554432000 4194304 4417332 140733672547136 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=356644 CPUtime=23.64 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 6693 1252 0 0 2363 1 0 0 25 0 5 0 599419890 365203456 60627 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=356644 CPUtime=23.77 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 6976 1252 0 0 2376 1 0 0 25 0 5 0 599419890 365203456 60627 33554432000 4194304 4417332 140733672547136 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=356644 CPUtime=23.77 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 17976 1252 0 0 2375 2 0 0 25 0 5 0 599419890 365203456 60627 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 96.69
Current children cumulated vsize (KiB) 356644

[startup+51.1004 s]
/proc/loadavg: 7.59 7.93 7.78 9/186 2634
/proc/meminfo: memFree=20392112/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=390392 CPUtime=198.95 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 120933 1252 0 0 19878 17 0 0 25 0 5 0 599419718 399761408 69479 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 97598 69479 179 55 0 95484 0
[pid=2623/tid=2630] ppid=2621 vsize=390392 CPUtime=49.38 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 8047 1252 0 0 4937 1 0 0 25 0 5 0 599419890 399761408 69479 33554432000 4194304 4417332 140733672547136 18446744073709551615 4356798 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=390392 CPUtime=49.09 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 10187 1252 0 0 4908 1 0 0 25 0 5 0 599419890 399761408 69479 33554432000 4194304 4417332 140733672547136 18446744073709551615 4319417 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=390392 CPUtime=49.37 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 10596 1252 0 0 4936 1 0 0 25 0 5 0 599419890 399761408 69479 33554432000 4194304 4417332 140733672547136 18446744073709551615 4211449 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=390392 CPUtime=49.37 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 37665 1252 0 0 4932 5 0 0 25 0 5 0 599419890 399761408 69479 33554432000 4194304 4417332 140733672547136 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 198.95
Current children cumulated vsize (KiB) 390392

[startup+102.307 s]
/proc/loadavg: 7.82 7.94 7.79 9/186 2636
/proc/meminfo: memFree=20328360/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=433976 CPUtime=403.5 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 190594 1252 0 0 40322 28 0 0 25 0 5 0 599419718 444391424 79929 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 108494 79929 179 55 0 106380 0
[pid=2623/tid=2630] ppid=2621 vsize=433976 CPUtime=100.58 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 17620 1252 0 0 10056 2 0 0 25 0 5 0 599419890 444391424 79929 33554432000 4194304 4417332 140733672547136 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=433976 CPUtime=100.03 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 20417 1252 0 0 10000 3 0 0 25 0 5 0 599419890 444391424 79929 33554432000 4194304 4417332 140733672547136 18446744073709551615 4348343 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=433976 CPUtime=100.57 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 23215 1252 0 0 10054 3 0 0 25 0 5 0 599419890 444391424 79929 33554432000 4194304 4417332 140733672547136 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=433976 CPUtime=100.58 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 74904 1252 0 0 10047 11 0 0 25 0 5 0 599419890 444391424 79929 33554432000 4194304 4417332 140733672547136 18446744073709551615 4357986 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 403.5
Current children cumulated vsize (KiB) 433976

[startup+162.301 s]
/proc/loadavg: 7.93 7.94 7.80 9/186 2637
/proc/meminfo: memFree=20309840/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=467832 CPUtime=643.16 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 285723 1252 0 0 64272 44 0 0 25 0 5 0 599419718 479059968 88418 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 116958 88418 179 55 0 114844 0
[pid=2623/tid=2630] ppid=2621 vsize=467832 CPUtime=160.58 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 31077 1252 0 0 16053 5 0 0 25 0 5 0 599419890 479059968 88418 33554432000 4194304 4417332 140733672547136 18446744073709551615 4357986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=467832 CPUtime=159.69 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 32937 1252 0 0 15964 5 0 0 25 0 5 0 599419890 479059968 88418 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=467832 CPUtime=160.57 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 46929 1252 0 0 16049 8 0 0 25 0 5 0 599419890 479059968 88418 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=467832 CPUtime=160.57 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 120342 1252 0 0 16039 18 0 0 25 0 5 0 599419890 479059968 88418 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 643.16
Current children cumulated vsize (KiB) 467832

[startup+222.3 s]
/proc/loadavg: 7.97 7.95 7.81 9/186 2639
/proc/meminfo: memFree=20190376/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=522312 CPUtime=882.84 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 382675 1252 0 0 88222 62 0 0 25 0 5 0 599419718 534847488 102013 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 130578 102013 179 55 0 128464 0
[pid=2623/tid=2630] ppid=2621 vsize=522312 CPUtime=220.58 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 52418 1252 0 0 22049 9 0 0 25 0 5 0 599419890 534847488 102013 33554432000 4194304 4417332 140733672547136 18446744073709551615 4357986 0 0 4096 17442 0 0 0 -1 1 0 0 0

################
# More data... #
################

[pid=2623/tid=2631] ppid=2621 vsize=864716 CPUtime=756.45 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 186119 1252 0 0 75618 27 0 0 25 0 5 0 599419890 885469184 187385 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=864716 CPUtime=760.6 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 271614 1252 0 0 76020 40 0 0 25 0 5 0 599419890 885469184 187385 33554432000 4194304 4417332 140733672547136 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=864716 CPUtime=760.61 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 373393 1252 0 0 76003 58 0 0 25 0 5 0 599419890 885469184 187385 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3040
Current children cumulated vsize (KiB) 864716

[startup+822.301 s]
/proc/loadavg: 8.15 8.08 7.93 9/186 2655
/proc/meminfo: memFree=19779364/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=903832 CPUtime=3279.65 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1140261 1252 0 0 327794 171 0 0 25 0 5 0 599419718 925523968 197393 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 225958 197393 179 55 0 223844 0
[pid=2623/tid=2630] ppid=2621 vsize=903832 CPUtime=820.6 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 215641 1252 0 0 82026 34 0 0 25 0 5 0 599419890 925523968 197393 33554432000 4194304 4417332 140733672547136 18446744073709551615 4365991 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=903832 CPUtime=816.12 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 198114 1252 0 0 81584 28 0 0 25 0 5 0 599419890 925523968 197393 33554432000 4194304 4417332 140733672547136 18446744073709551615 4249633 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=903832 CPUtime=820.6 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 290566 1252 0 0 82018 42 0 0 25 0 5 0 599419890 925523968 197393 33554432000 4194304 4417332 140733672547136 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=903832 CPUtime=820.6 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 381502 1252 0 0 82001 59 0 0 25 0 5 0 599419890 925523968 197393 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3279.65
Current children cumulated vsize (KiB) 903832

[startup+882.301 s]
/proc/loadavg: 8.16 8.10 7.94 9/186 2657
/proc/meminfo: memFree=19842192/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=795680 CPUtime=3519.33 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1224788 1252 0 0 351751 182 0 0 25 0 5 0 599419718 814776320 170355 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 198920 170355 179 55 0 196806 0
[pid=2623/tid=2630] ppid=2621 vsize=795680 CPUtime=880.6 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 230526 1252 0 0 88024 36 0 0 25 0 5 0 599419890 814776320 170355 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=795680 CPUtime=875.79 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 215693 1252 0 0 87549 30 0 0 25 0 5 0 599419890 814776320 170355 33554432000 4194304 4417332 140733672547136 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=795680 CPUtime=880.59 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 310079 1252 0 0 88015 44 0 0 25 0 5 0 599419890 814776320 170355 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=795680 CPUtime=880.6 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 414052 1252 0 0 87996 64 0 0 25 0 5 0 599419890 814776320 170355 33554432000 4194304 4417332 140733672547136 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3519.33
Current children cumulated vsize (KiB) 795680

[startup+942.3 s]
/proc/loadavg: 8.15 8.11 7.95 9/186 2658
/proc/meminfo: memFree=19812648/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=850988 CPUtime=3759.01 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1286464 1252 0 0 375711 190 0 0 25 0 5 0 599419718 871411712 184182 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 212747 184182 179 55 0 210633 0
[pid=2623/tid=2630] ppid=2621 vsize=850988 CPUtime=940.6 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 243536 1252 0 0 94022 38 0 0 25 0 5 0 599419890 871411712 184182 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364877 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=850988 CPUtime=935.47 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 226574 1252 0 0 93515 32 0 0 25 0 5 0 599419890 871411712 184182 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=850988 CPUtime=940.59 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 331077 1252 0 0 94013 46 0 0 25 0 5 0 599419890 871411712 184182 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=850988 CPUtime=940.61 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 430839 1252 0 0 93995 66 0 0 25 0 5 0 599419890 871411712 184182 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3759.01
Current children cumulated vsize (KiB) 850988

[startup+1002.3 s]
/proc/loadavg: 8.14 8.11 7.96 9/186 2669
/proc/meminfo: memFree=19766892/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=876668 CPUtime=3998.68 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1331664 1252 0 0 399672 196 0 0 25 0 5 0 599419718 897708032 190602 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 219167 190602 179 55 0 217053 0
[pid=2623/tid=2630] ppid=2621 vsize=876668 CPUtime=1000.61 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 256514 1252 0 0 100021 40 0 0 25 0 5 0 599419890 897708032 190602 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=876668 CPUtime=995.14 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 236087 1252 0 0 99481 33 0 0 25 0 5 0 599419890 897708032 190602 33554432000 4194304 4417332 140733672547136 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=876668 CPUtime=1000.6 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 348511 1252 0 0 100011 49 0 0 25 0 5 0 599419890 897708032 190602 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=876668 CPUtime=1000.59 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 436114 1252 0 0 99993 66 0 0 25 0 5 0 599419890 897708032 190602 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3998.68
Current children cumulated vsize (KiB) 876668

[startup+1062.3 s]
/proc/loadavg: 8.13 8.12 7.97 9/186 2670
/proc/meminfo: memFree=19738464/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=896552 CPUtime=4238.36 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1385325 1252 0 0 423634 202 0 0 25 0 5 0 599419718 918069248 195573 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 224138 195573 179 55 0 222024 0
[pid=2623/tid=2630] ppid=2621 vsize=896552 CPUtime=1060.61 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 273449 1252 0 0 106019 42 0 0 25 0 5 0 599419890 918069248 195573 33554432000 4194304 4417332 140733672547136 18446744073709551615 4366038 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=896552 CPUtime=1054.82 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 250863 1252 0 0 105447 35 0 0 25 0 5 0 599419890 918069248 195573 33554432000 4194304 4417332 140733672547136 18446744073709551615 4357218 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=896552 CPUtime=1060.6 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 370461 1252 0 0 106009 51 0 0 25 0 5 0 599419890 918069248 195573 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=896552 CPUtime=1060.59 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 436114 1252 0 0 105993 66 0 0 25 0 5 0 599419890 918069248 195573 33554432000 4194304 4417332 140733672547136 18446744073709551615 4354322 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4238.36
Current children cumulated vsize (KiB) 896552

[startup+1122.3 s]
/proc/loadavg: 8.12 8.12 7.98 9/186 2672
/proc/meminfo: memFree=19579344/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=1037632 CPUtime=4478.04 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1464969 1252 0 0 447590 214 0 0 25 0 5 0 599419718 1062535168 230843 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 259408 230843 179 55 0 257294 0
[pid=2623/tid=2630] ppid=2621 vsize=1037632 CPUtime=1120.61 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 291208 1252 0 0 112016 45 0 0 25 0 5 0 599419890 1062535168 230843 33554432000 4194304 4417332 140733672547136 18446744073709551615 4251076 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=1037632 CPUtime=1114.49 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 259977 1252 0 0 111414 35 0 0 25 0 5 0 599419890 1062535168 230843 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=1037632 CPUtime=1120.61 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 384661 1252 0 0 112008 53 0 0 25 0 5 0 599419890 1062535168 230843 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=1037632 CPUtime=1120.59 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 474685 1252 0 0 111986 73 0 0 25 0 5 0 599419890 1062535168 230843 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4478.04
Current children cumulated vsize (KiB) 1037632

[startup+1182.3 s]
/proc/loadavg: 8.08 8.11 7.98 9/186 2674
/proc/meminfo: memFree=19565056/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=1044156 CPUtime=4717.73 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1509300 1252 0 0 471551 222 0 0 25 0 5 0 599419718 1069215744 232474 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 261039 232474 179 55 0 258925 0
[pid=2623/tid=2630] ppid=2621 vsize=1044156 CPUtime=1180.61 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 305696 1252 0 0 118014 47 0 0 25 0 5 0 599419890 1069215744 232474 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=1044156 CPUtime=1174.17 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 272977 1252 0 0 117380 37 0 0 25 0 5 0 599419890 1069215744 232474 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=1044156 CPUtime=1180.61 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 401504 1252 0 0 118005 56 0 0 25 0 5 0 599419890 1069215744 232474 33554432000 4194304 4417332 140733672547136 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=1044156 CPUtime=1180.59 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 474685 1252 0 0 117986 73 0 0 25 0 5 0 599419890 1069215744 232474 33554432000 4194304 4417332 140733672547136 18446744073709551615 4365997 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4717.73
Current children cumulated vsize (KiB) 1044156



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.9 s]
/proc/loadavg: 7.79 8.05 7.97 5/177 2692
/proc/meminfo: memFree=19784692/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=1057308 CPUtime=4800.03 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1528759 1252 0 0 479777 226 0 0 25 0 5 0 599419718 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 264327 235762 179 55 0 262213 0
[pid=2623/tid=2630] ppid=2621 vsize=1057308 CPUtime=1201.22 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 309230 1252 0 0 120074 48 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=1057308 CPUtime=1194.66 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 275766 1252 0 0 119428 38 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4355764 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=1057308 CPUtime=1201.21 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 410173 1252 0 0 120064 57 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=1057308 CPUtime=1201.2 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 479152 1252 0 0 120046 74 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 1057308

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.9 s]
/proc/loadavg: 7.79 8.05 7.97 5/177 2692
/proc/meminfo: memFree=19784692/32950928 swapFree=67111524/67111528
[pid=2623] ppid=2621 vsize=1057308 CPUtime=4800.03 cores=1,3,5,7
/proc/2623/stat : 2623 (plingeling) S 2621 2623 567 0 -1 4202496 1528759 1252 0 0 479777 226 0 0 25 0 5 0 599419718 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/2623/statm: 264327 235762 179 55 0 262213 0
[pid=2623/tid=2630] ppid=2621 vsize=1057308 CPUtime=1201.22 cores=1,3,5,7
/proc/2623/task/2630/stat : 2630 (plingeling) R 2621 2623 567 0 -1 4202560 309230 1252 0 0 120074 48 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=2623/tid=2631] ppid=2621 vsize=1057308 CPUtime=1194.66 cores=1,3,5,7
/proc/2623/task/2631/stat : 2631 (plingeling) R 2621 2623 567 0 -1 4202560 275766 1252 0 0 119428 38 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4355764 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=2623/tid=2632] ppid=2621 vsize=1057308 CPUtime=1201.21 cores=1,3,5,7
/proc/2623/task/2632/stat : 2632 (plingeling) R 2621 2623 567 0 -1 4202560 410173 1252 0 0 120064 57 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=2623/tid=2633] ppid=2621 vsize=1057308 CPUtime=1201.2 cores=1,3,5,7
/proc/2623/task/2633/stat : 2633 (plingeling) R 2621 2623 567 0 -1 4202560 479152 1252 0 0 120046 74 0 0 25 0 5 0 599419890 1082683392 235762 33554432000 4194304 4417332 140733672547136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 1057308

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.95
CPU time (s): 4800.11
CPU user time (s): 4797.8
CPU system time (s): 2.31065
CPU usage (%): 399.027
Max. virtual memory (cumulated for all children) (KiB): 1068316

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.8
system time used= 2.31065
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1530011
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 210
involuntary context switches= 34929

runsolver used 1.9557 second user time and 4.6143 second system time

The end

Launcher Data 

Begin job on node114 at 2011-04-29 17:44:50
IDJOB=3250372
IDBENCH=83251
IDSOLVER=1605
FILE ID=node114/3250372-1304091889
RUNJOBID= node114-1304091889-2595
PBS_JOBID= 13170509
Free space on /tmp= 71780 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-5-5_34309.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3250372-1304091889/watcher-3250372-1304091889 -o /tmp/evaluation-result-3250372-1304091889/solver-3250372-1304091889 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3250372-1304091889.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e252fbf81407f3388187ab50f0786909
RANDOM SEED=1087611736

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20717464 kB
Buffers:       2224180 kB
Cached:        9352432 kB
SwapCached:          4 kB
Active:        2866772 kB
Inactive:      8786336 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20717464 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           23376 kB
Writeback:           0 kB
AnonPages:       75732 kB
Mapped:          15264 kB
Slab:           516912 kB
PageTables:       4652 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   259944 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71780 MiB
End job on node114 at 2011-04-29 18:04:55