Trace number 3249756

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.17 1202.28

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-21x39.cnf
MD5SUMe7679ca49bb432c9b90305d907365c4e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables3276
Number of clauses628173
Sum of the clauses size2502864
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 24914
Number of clauses of size 30
Number of clauses of size 4623259
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249756-1304087471/watcher-3249756-1304087471 -o /tmp/evaluation-result-3249756-1304087471/solver-3249756-1304087471 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3249756-1304087471.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.46 7.98 8.01 6/181 16594
/proc/meminfo: memFree=30617972/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=10200 CPUtime=0 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) R 16592 16594 15826 0 -1 4202496 611 1251 0 0 0 0 0 0 25 0 1 0 520346964 10444800 539 33554432000 4194304 4417332 140735377798480 18446744073709551615 248089698048 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/16594/statm: 2550 539 146 55 0 436 0

[startup+0.00691207 s]
/proc/loadavg: 7.46 7.98 8.01 6/181 16594
/proc/meminfo: memFree=30617972/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=10200 CPUtime=0 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) R 16592 16594 15826 0 -1 4202496 627 1251 0 0 0 0 0 0 25 0 1 0 520346964 10444800 555 33554432000 4194304 4417332 140735377798480 18446744073709551615 4231537 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/16594/statm: 2550 555 146 55 0 436 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 10200

[startup+0.100897 s]
/proc/loadavg: 7.46 7.98 8.01 6/181 16594
/proc/meminfo: memFree=30617972/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=26616 CPUtime=0.09 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) R 16592 16594 15826 0 -1 4202496 4844 1251 0 0 9 0 0 0 25 0 1 0 520346964 27254784 4676 33554432000 4194304 4417332 140735377798480 18446744073709551615 248089316864 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/16594/statm: 6654 4676 147 55 0 4540 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26616

[startup+0.300862 s]
/proc/loadavg: 7.46 7.98 8.01 6/181 16594
/proc/meminfo: memFree=30617972/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=75768 CPUtime=0.28 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) R 16592 16594 15826 0 -1 4202496 17132 1251 0 0 27 1 0 0 25 0 1 0 520346964 77586432 16964 33554432000 4194304 4417332 140735377798480 18446744073709551615 4231631 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/16594/statm: 18942 16964 147 55 0 16828 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 75768

[startup+0.70079 s]
/proc/loadavg: 7.46 7.98 8.01 6/181 16594
/proc/meminfo: memFree=30617972/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=141304 CPUtime=0.69 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) R 16592 16594 15826 0 -1 4202496 33516 1251 0 0 65 4 0 0 25 0 1 0 520346964 144695296 33348 33554432000 4194304 4417332 140735377798480 18446744073709551615 248089698048 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/16594/statm: 35326 33348 147 55 0 33212 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 141304

[startup+1.50066 s]
/proc/loadavg: 7.26 7.93 8.00 9/186 16604
/proc/meminfo: memFree=30464304/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=223264 CPUtime=3.74 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 49222 1251 0 0 367 7 0 0 25 0 5 0 520346964 228622336 43624 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 55816 43624 174 55 0 53702 0
[pid=16594/tid=16601] ppid=16592 vsize=223264 CPUtime=0.74 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 4722 1251 0 0 74 0 0 0 25 0 5 0 520347039 228622336 43624 33554432000 4194304 4417332 140735377798480 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=223264 CPUtime=0.73 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 4751 1251 0 0 73 0 0 0 25 0 5 0 520347039 228622336 43624 33554432000 4194304 4417332 140735377798480 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=223264 CPUtime=0.74 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 4135 1251 0 0 74 0 0 0 25 0 5 0 520347039 228622336 43624 33554432000 4194304 4417332 140735377798480 18446744073709551615 4278585 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=223264 CPUtime=0.74 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 2086 1251 0 0 74 0 0 0 25 0 5 0 520347039 228622336 43624 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.74
Current children cumulated vsize (KiB) 223264

[startup+3.10037 s]
/proc/loadavg: 7.26 7.93 8.00 9/186 16604
/proc/meminfo: memFree=30409768/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=256664 CPUtime=10.13 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 63803 1251 0 0 1004 9 0 0 25 0 5 0 520346964 262823936 51952 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 64166 51952 179 55 0 62052 0
[pid=16594/tid=16601] ppid=16592 vsize=256664 CPUtime=2.35 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 7765 1251 0 0 234 1 0 0 25 0 5 0 520347039 262823936 51952 33554432000 4194304 4417332 140735377798480 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=256664 CPUtime=2.33 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 7794 1251 0 0 232 1 0 0 25 0 5 0 520347039 262823936 51952 33554432000 4194304 4417332 140735377798480 18446744073709551615 4239605 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=256664 CPUtime=2.34 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 7183 1251 0 0 233 1 0 0 25 0 5 0 520347039 262823936 51952 33554432000 4194304 4417332 140735377798480 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=256664 CPUtime=2.34 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 7533 1251 0 0 233 1 0 0 25 0 5 0 520347039 262823936 51952 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.13
Current children cumulated vsize (KiB) 256664

[startup+6.30081 s]
/proc/loadavg: 7.26 7.93 8.00 9/186 16604
/proc/meminfo: memFree=30327844/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=356572 CPUtime=22.91 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 94138 1251 0 0 2278 13 0 0 25 0 5 0 520346964 365129728 76929 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 89143 76929 179 55 0 87029 0
[pid=16594/tid=16601] ppid=16592 vsize=356572 CPUtime=5.54 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 13852 1251 0 0 553 1 0 0 25 0 5 0 520347039 365129728 76929 33554432000 4194304 4417332 140735377798480 18446744073709551615 4239414 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=356572 CPUtime=5.52 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 13881 1251 0 0 550 2 0 0 25 0 5 0 520347039 365129728 76929 33554432000 4194304 4417332 140735377798480 18446744073709551615 4239418 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=356572 CPUtime=5.54 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 13270 1251 0 0 552 2 0 0 25 0 5 0 520347039 365129728 76929 33554432000 4194304 4417332 140735377798480 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=356572 CPUtime=5.55 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 19607 1251 0 0 552 3 0 0 25 0 5 0 520347039 365129728 76929 33554432000 4194304 4417332 140735377798480 18446744073709551615 4365094 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 22.91
Current children cumulated vsize (KiB) 356572

[startup+12.7007 s]
/proc/loadavg: 7.45 7.95 8.00 9/186 16604
/proc/meminfo: memFree=30226064/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=357896 CPUtime=48.48 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 111464 1251 0 0 4833 15 0 0 25 0 5 0 520346964 366485504 77260 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 89474 77260 179 55 0 87360 0
[pid=16594/tid=16601] ppid=16592 vsize=357896 CPUtime=11.94 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 13852 1251 0 0 1193 1 0 0 25 0 5 0 520347039 366485504 77260 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=357896 CPUtime=11.88 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 13881 1251 0 0 1186 2 0 0 25 0 5 0 520347039 366485504 77260 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=357896 CPUtime=11.94 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 13270 1251 0 0 1192 2 0 0 25 0 5 0 520347039 366485504 77260 33554432000 4194304 4417332 140735377798480 18446744073709551615 4248238 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=357896 CPUtime=11.94 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 36933 1251 0 0 1190 4 0 0 25 0 5 0 520347039 366485504 77260 33554432000 4194304 4417332 140735377798480 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 48.48
Current children cumulated vsize (KiB) 357896

[startup+25.5004 s]
/proc/loadavg: 7.53 7.95 8.00 9/186 16605
/proc/meminfo: memFree=30140048/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=390956 CPUtime=99.6 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 137053 1251 0 0 9941 19 0 0 25 0 5 0 520346964 400338944 85557 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 97739 85557 179 55 0 95625 0
[pid=16594/tid=16601] ppid=16592 vsize=390956 CPUtime=24.74 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 13852 1251 0 0 2473 1 0 0 25 0 5 0 520347039 400338944 85557 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=390956 CPUtime=24.61 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 13881 1251 0 0 2459 2 0 0 25 0 5 0 520347039 400338944 85557 33554432000 4194304 4417332 140735377798480 18446744073709551615 4250010 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=390956 CPUtime=24.74 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 13270 1251 0 0 2472 2 0 0 25 0 5 0 520347039 400338944 85557 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=390956 CPUtime=24.74 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 62522 1251 0 0 2465 9 0 0 25 0 5 0 520347039 400338944 85557 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 99.6
Current children cumulated vsize (KiB) 390956

[startup+51.1009 s]
/proc/loadavg: 7.71 7.95 8.00 9/186 16605
/proc/meminfo: memFree=29958176/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=548568 CPUtime=201.87 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 247158 1251 0 0 20151 36 0 0 25 0 5 0 520346964 561733632 108577 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 137142 108577 179 55 0 135028 0
[pid=16594/tid=16601] ppid=16592 vsize=548568 CPUtime=50.35 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 56291 1251 0 0 5027 8 0 0 25 0 5 0 520347039 561733632 108577 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=548568 CPUtime=50.07 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 20863 1251 0 0 5004 3 0 0 25 0 5 0 520347039 561733632 108577 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=548568 CPUtime=50.34 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 38563 1251 0 0 5028 6 0 0 25 0 5 0 520347039 561733632 108577 33554432000 4194304 4417332 140735377798480 18446744073709551615 4365007 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=548568 CPUtime=50.34 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 97913 1251 0 0 5020 14 0 0 25 0 5 0 520347039 561733632 108577 33554432000 4194304 4417332 140735377798480 18446744073709551615 18446744073699065971 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 201.87
Current children cumulated vsize (KiB) 548568

[startup+102.307 s]
/proc/loadavg: 7.92 7.97 8.00 9/186 16607
/proc/meminfo: memFree=29511032/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=763620 CPUtime=406.4 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 455591 1251 0 0 40569 71 0 0 25 0 5 0 520346964 781946880 162340 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 190905 162340 179 55 0 188791 0
[pid=16594/tid=16601] ppid=16592 vsize=763620 CPUtime=101.54 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 111596 1251 0 0 10138 16 0 0 25 0 5 0 520347039 781946880 162340 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=763620 CPUtime=100.99 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 65755 1251 0 0 10090 9 0 0 25 0 5 0 520347039 781946880 162340 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=763620 CPUtime=101.54 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 87778 1251 0 0 10140 14 0 0 25 0 5 0 520347039 781946880 162340 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=763620 CPUtime=101.55 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 156934 1251 0 0 10129 26 0 0 25 0 5 0 520347039 781946880 162340 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 406.4
Current children cumulated vsize (KiB) 763620

[startup+162.3 s]
/proc/loadavg: 8.00 7.99 8.00 9/186 16608
/proc/meminfo: memFree=29116296/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=929776 CPUtime=646.06 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 608587 1251 0 0 64508 98 0 0 25 0 5 0 520346964 952090624 203879 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 232444 203879 179 55 0 230330 0
[pid=16594/tid=16601] ppid=16592 vsize=929776 CPUtime=161.54 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 142368 1251 0 0 16131 23 0 0 25 0 5 0 520347039 952090624 203879 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=929776 CPUtime=160.66 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 96167 1251 0 0 16053 13 0 0 25 0 5 0 520347039 952090624 203879 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249919 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=929776 CPUtime=161.54 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 145607 1251 0 0 16129 25 0 0 25 0 5 0 520347039 952090624 203879 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=929776 CPUtime=161.55 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 190917 1251 0 0 16123 32 0 0 25 0 5 0 520347039 952090624 203879 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 646.06
Current children cumulated vsize (KiB) 929776


################
# More data... #
################

[pid=16594/tid=16602] ppid=16592 vsize=1652140 CPUtime=697.72 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 375016 1251 0 0 69719 53 0 0 25 0 5 0 520347039 1691791360 384470 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1652140 CPUtime=701.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 545858 1251 0 0 70073 82 0 0 25 0 5 0 520347039 1691791360 384470 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1652140 CPUtime=701.56 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 413593 1251 0 0 70085 71 0 0 25 0 5 0 520347039 1691791360 384470 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2803.13
Current children cumulated vsize (KiB) 1652140

[startup+762.306 s]
/proc/loadavg: 8.09 8.08 8.02 9/186 16624
/proc/meminfo: memFree=27490560/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=1829196 CPUtime=3042.84 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 2366501 1251 0 0 303912 372 0 0 25 0 5 0 520346964 1873096704 428734 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 457299 428734 179 55 0 455185 0
[pid=16594/tid=16601] ppid=16592 vsize=1829196 CPUtime=761.54 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 925801 1251 0 0 76005 149 0 0 25 0 5 0 520347039 1873096704 428734 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=1829196 CPUtime=757.4 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 385003 1251 0 0 75686 54 0 0 25 0 5 0 520347039 1873096704 428734 33554432000 4194304 4417332 140735377798480 18446744073709551615 4250977 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1829196 CPUtime=761.56 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 595528 1251 0 0 76067 89 0 0 25 0 5 0 520347039 1873096704 428734 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1829196 CPUtime=761.57 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 426641 1251 0 0 76083 74 0 0 25 0 5 0 520347039 1873096704 428734 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3042.84
Current children cumulated vsize (KiB) 1829196

[startup+822.301 s]
/proc/loadavg: 8.08 8.08 8.02 9/186 16626
/proc/meminfo: memFree=27585132/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=1954580 CPUtime=3282.49 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 2612156 1251 0 0 327834 415 0 0 25 0 5 0 520346964 2001489920 460080 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 488645 460080 179 55 0 486531 0
[pid=16594/tid=16601] ppid=16592 vsize=1954580 CPUtime=821.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1063176 1251 0 0 81980 173 0 0 25 0 5 0 520347039 2001489920 460080 33554432000 4194304 4417332 140735377798480 18446744073709551615 4263206 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=1954580 CPUtime=817.07 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 448335 1251 0 0 81642 65 0 0 25 0 5 0 520347039 2001489920 460080 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1954580 CPUtime=821.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 623249 1251 0 0 82061 94 0 0 25 0 5 0 520347039 2001489920 460080 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1954580 CPUtime=821.56 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 443868 1251 0 0 82079 77 0 0 25 0 5 0 520347039 2001489920 460080 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364482 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3282.49
Current children cumulated vsize (KiB) 1954580

[startup+882.301 s]
/proc/loadavg: 8.12 8.09 8.02 9/186 16627
/proc/meminfo: memFree=27663572/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=1738344 CPUtime=3522.17 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 2696177 1251 0 0 351787 430 0 0 25 0 5 0 520346964 1780064256 406021 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 434586 406021 179 55 0 432472 0
[pid=16594/tid=16601] ppid=16592 vsize=1738344 CPUtime=881.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1069501 1251 0 0 87977 176 0 0 25 0 5 0 520347039 1780064256 406021 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=1738344 CPUtime=876.74 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 487183 1251 0 0 87604 70 0 0 25 0 5 0 520347039 1780064256 406021 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1738344 CPUtime=881.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 624401 1251 0 0 88061 94 0 0 25 0 5 0 520347039 1780064256 406021 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1738344 CPUtime=881.56 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 481564 1251 0 0 88072 84 0 0 25 0 5 0 520347039 1780064256 406021 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3522.17
Current children cumulated vsize (KiB) 1738344

[startup+942.301 s]
/proc/loadavg: 8.12 8.10 8.03 9/186 16629
/proc/meminfo: memFree=27382168/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=2034492 CPUtime=3761.85 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 2891443 1251 0 0 375731 454 0 0 25 0 5 0 520346964 2083319808 480058 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 508623 480058 179 55 0 506509 0
[pid=16594/tid=16601] ppid=16592 vsize=2034492 CPUtime=941.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1167933 1251 0 0 93964 189 0 0 25 0 5 0 520347039 2083319808 480058 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=2034492 CPUtime=936.42 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 521807 1251 0 0 93567 75 0 0 25 0 5 0 520347039 2083319808 480058 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=2034492 CPUtime=941.56 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 662657 1251 0 0 94057 99 0 0 25 0 5 0 520347039 2083319808 480058 33554432000 4194304 4417332 140735377798480 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=2034492 CPUtime=941.56 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 505518 1251 0 0 94070 86 0 0 25 0 5 0 520347039 2083319808 480058 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3761.85
Current children cumulated vsize (KiB) 2034492

[startup+1002.3 s]
/proc/loadavg: 8.12 8.11 8.03 9/186 16631
/proc/meminfo: memFree=27329048/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=2002712 CPUtime=4001.52 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 2937005 1251 0 0 399691 461 0 0 25 0 5 0 520346964 2050777088 472113 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 500678 472113 179 55 0 498564 0
[pid=16594/tid=16601] ppid=16592 vsize=2002712 CPUtime=1001.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1167933 1251 0 0 99964 189 0 0 25 0 5 0 520347039 2050777088 472113 33554432000 4194304 4417332 140735377798480 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=2002712 CPUtime=996.09 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 541642 1251 0 0 99531 78 0 0 25 0 5 0 520347039 2050777088 472113 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=2002712 CPUtime=1001.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 688384 1251 0 0 100054 101 0 0 25 0 5 0 520347039 2050777088 472113 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=2002712 CPUtime=1001.57 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 505518 1251 0 0 100071 86 0 0 25 0 5 0 520347039 2050777088 472113 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4001.52
Current children cumulated vsize (KiB) 2002712

[startup+1062.3 s]
/proc/loadavg: 8.08 8.10 8.03 9/186 16632
/proc/meminfo: memFree=27504828/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=1914612 CPUtime=4241.19 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 3041965 1251 0 0 423641 478 0 0 25 0 5 0 520346964 1960562688 450120 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 478653 450120 179 55 0 476539 0
[pid=16594/tid=16601] ppid=16592 vsize=1914612 CPUtime=1061.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1223784 1251 0 0 105954 199 0 0 25 0 5 0 520347039 1960562688 450120 33554432000 4194304 4417332 140735377798480 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=1914612 CPUtime=1055.77 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 564343 1251 0 0 105495 82 0 0 25 0 5 0 520347039 1960562688 450120 33554432000 4194304 4417332 140735377798480 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1914612 CPUtime=1061.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 714792 1251 0 0 106050 105 0 0 25 0 5 0 520347039 1960562688 450120 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1914612 CPUtime=1061.57 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 505518 1251 0 0 106071 86 0 0 25 0 5 0 520347039 1960562688 450120 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4241.19
Current children cumulated vsize (KiB) 1914612

[startup+1122.3 s]
/proc/loadavg: 8.06 8.09 8.03 9/186 16634
/proc/meminfo: memFree=27182880/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=2235348 CPUtime=4480.87 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 3160543 1251 0 0 447592 495 0 0 25 0 5 0 520346964 2288996352 530304 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 558837 530304 179 55 0 556723 0
[pid=16594/tid=16601] ppid=16592 vsize=2235348 CPUtime=1121.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1281070 1251 0 0 111946 207 0 0 25 0 5 0 520347039 2288996352 530304 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=2235348 CPUtime=1115.44 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 586055 1251 0 0 111459 85 0 0 25 0 5 0 520347039 2288996352 530304 33554432000 4194304 4417332 140735377798480 18446744073709551615 4253524 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=2235348 CPUtime=1121.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 745801 1251 0 0 112046 109 0 0 25 0 5 0 520347039 2288996352 530304 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364800 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=2235348 CPUtime=1121.57 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 514089 1251 0 0 112069 88 0 0 25 0 5 0 520347039 2288996352 530304 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4480.87
Current children cumulated vsize (KiB) 2235348

[startup+1182.3 s]
/proc/loadavg: 8.10 8.09 8.03 9/186 16635
/proc/meminfo: memFree=27754596/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=1862396 CPUtime=4720.55 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 3332635 1251 0 0 471533 522 0 0 25 0 5 0 520346964 1907093504 437066 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 465599 437066 179 55 0 463485 0
[pid=16594/tid=16601] ppid=16592 vsize=1862396 CPUtime=1181.53 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1335720 1251 0 0 117936 217 0 0 25 0 5 0 520347039 1907093504 437066 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=1862396 CPUtime=1175.12 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 658392 1251 0 0 117416 96 0 0 25 0 5 0 520347039 1907093504 437066 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1862396 CPUtime=1181.55 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 790906 1251 0 0 118040 115 0 0 25 0 5 0 520347039 1907093504 437066 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1862396 CPUtime=1181.57 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 514089 1251 0 0 118069 88 0 0 25 0 5 0 520347039 1907093504 437066 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4720.55
Current children cumulated vsize (KiB) 1862396



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.2 s]
/proc/loadavg: 8.07 8.08 8.03 5/180 16637
/proc/meminfo: memFree=29222408/32950928 swapFree=67111428/67111528
[pid=16594] ppid=16592 vsize=1579200 CPUtime=4800.07 cores=1,3,5,7
/proc/16594/stat : 16594 (plingeling) S 16592 16594 15826 0 -1 4202496 3429360 1251 0 0 479469 538 0 0 25 0 5 0 520346964 1617100800 366235 33554432000 4194304 4417332 140735377798480 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/16594/statm: 394800 366235 179 55 0 392686 0
[pid=16594/tid=16601] ppid=16592 vsize=1579200 CPUtime=1201.43 cores=1,3,5,7
/proc/16594/task/16601/stat : 16601 (plingeling) R 16592 16594 15826 0 -1 4202560 1352895 1251 0 0 119924 219 0 0 25 0 5 0 520347039 1617100800 366235 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16594/tid=16602] ppid=16592 vsize=1579200 CPUtime=1194.91 cores=1,3,5,7
/proc/16594/task/16602/stat : 16602 (plingeling) R 16592 16594 15826 0 -1 4202560 662208 1251 0 0 119395 96 0 0 25 0 5 0 520347039 1617100800 366235 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16594/tid=16603] ppid=16592 vsize=1579200 CPUtime=1201.47 cores=1,3,5,7
/proc/16594/task/16603/stat : 16603 (plingeling) R 16592 16594 15826 0 -1 4202560 798675 1251 0 0 120031 116 0 0 25 0 5 0 520347039 1617100800 366235 33554432000 4194304 4417332 140735377798480 18446744073709551615 4260519 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16594/tid=16604] ppid=16592 vsize=1579200 CPUtime=1201.48 cores=1,3,5,7
/proc/16594/task/16604/stat : 16604 (plingeling) R 16592 16594 15826 0 -1 4202560 582054 1251 0 0 120048 100 0 0 25 0 5 0 520347039 1617100800 366235 33554432000 4194304 4417332 140735377798480 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.07
Current children cumulated vsize (KiB) 1579200

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.28
CPU time (s): 4800.17
CPU user time (s): 4794.72
CPU system time (s): 5.44917
CPU usage (%): 399.255
Max. virtual memory (cumulated for all children) (KiB): 2444176

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4794.72
system time used= 5.44917
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3430611
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 75
involuntary context switches= 32775

runsolver used 1.9567 second user time and 4.6113 second system time

The end

Launcher Data 

Begin job on node123 at 2011-04-29 16:31:12
IDJOB=3249756
IDBENCH=83184
IDSOLVER=1605
FILE ID=node123/3249756-1304087471
RUNJOBID= node123-1304087470-16494
PBS_JOBID= 13170535
Free space on /tmp= 72800 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-21x39.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249756-1304087471/watcher-3249756-1304087471 -o /tmp/evaluation-result-3249756-1304087471/solver-3249756-1304087471 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3249756-1304087471.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e7679ca49bb432c9b90305d907365c4e
RANDOM SEED=1697101155

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30618004 kB
Buffers:        555044 kB
Cached:        1276620 kB
SwapCached:        100 kB
Active:         204352 kB
Inactive:      1766416 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30618004 kB
SwapTotal:    67111528 kB
SwapFree:     67111428 kB
Dirty:           20732 kB
Writeback:           0 kB
AnonPages:      138768 kB
Mapped:          15248 kB
Slab:           298924 kB
PageTables:       4892 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   325120 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72804 MiB
End job on node123 at 2011-04-29 16:51:16