Trace number 3309366

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.8 5003.95

General information on the benchmark

NameAPPLICATIONS/bitverif/countbitsarray/
countbitsarray04_32.cnf
MD5SUM2fe3650d4a602f63086e937176360ebb
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables8750
Number of clauses25865
Sum of the clauses size60349
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 217242
Number of clauses of size 38621
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309366-1304002282/watcher-3309366-1304002282 -o /tmp/evaluation-result-3309366-1304002282/solver-3309366-1304002282 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309366-1304002282.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.43 7.90 7.96 1/170 17849
/proc/meminfo: memFree=13925604/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=9456 CPUtime=0 cores=0-7
/proc/17849/stat : 17849 (plingeling) R 17847 17849 17823 0 -1 4202496 454 1250 0 0 0 0 0 0 25 0 1 0 511829139 9682944 382 33554432000 4194304 4417332 140735308424624 18446744073709551615 270612804559 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/17849/statm: 2364 382 147 55 0 250 0

[startup+0.0221379 s]
/proc/loadavg: 7.43 7.90 7.96 1/170 17849
/proc/meminfo: memFree=13925604/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=13512 CPUtime=0.01 cores=0-7
/proc/17849/stat : 17849 (plingeling) R 17847 17849 17823 0 -1 4202496 1473 1250 0 0 1 0 0 0 25 0 1 0 511829139 13836288 1401 33554432000 4194304 4417332 140735308424624 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/17849/statm: 3378 1401 148 55 0 1264 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 13512

[startup+0.108118 s]
/proc/loadavg: 7.43 7.90 7.96 1/170 17849
/proc/meminfo: memFree=13925604/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=168304 CPUtime=0.34 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 5597 1250 0 0 34 0 0 0 18 0 9 0 511829139 172343296 3471 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 42076 3471 172 55 0 39962 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 168304

[startup+0.301047 s]
/proc/loadavg: 7.43 7.90 7.96 1/170 17849
/proc/meminfo: memFree=13925604/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=171552 CPUtime=1.88 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 7278 1250 0 0 187 1 0 0 18 0 9 0 511829139 175669248 4329 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 42888 4329 176 55 0 40774 0
Current children cumulated CPU time (s) 1.88
Current children cumulated vsize (KiB) 171552

[startup+0.700974 s]
/proc/loadavg: 7.43 7.90 7.96 1/170 17849
/proc/meminfo: memFree=13925604/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=172248 CPUtime=5.08 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 7599 1250 0 0 507 1 0 0 18 0 9 0 511829139 176381952 4625 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 43062 4625 180 55 0 40948 0
Current children cumulated CPU time (s) 5.08
Current children cumulated vsize (KiB) 172248

[startup+1.50083 s]
/proc/loadavg: 7.43 7.90 7.96 9/179 17863
/proc/meminfo: memFree=13906320/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=179980 CPUtime=11.47 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 11607 1250 0 0 1145 2 0 0 18 0 9 0 511829139 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 44995 6581 181 55 0 42881 0
[pid=17849/tid=17856] ppid=17847 vsize=179980 CPUtime=1.43 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 966 1250 0 0 143 0 0 0 19 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=179980 CPUtime=1.43 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 1027 1250 0 0 143 0 0 0 20 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=179980 CPUtime=1.43 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 715 1250 0 0 143 0 0 0 20 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=179980 CPUtime=1.41 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 605 1250 0 0 141 0 0 0 25 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=179980 CPUtime=1.37 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 671 1250 0 0 137 0 0 0 25 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4310641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=179980 CPUtime=1.43 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 888 1250 0 0 143 0 0 0 19 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4307396 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=179980 CPUtime=1.43 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1280 1250 0 0 143 0 0 0 20 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4275936 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=179980 CPUtime=1.43 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1645 1250 0 0 143 0 0 0 19 0 9 0 511829145 184299520 6581 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.47
Current children cumulated vsize (KiB) 179980

[startup+3.10055 s]
/proc/loadavg: 7.47 7.91 7.96 9/179 17863
/proc/meminfo: memFree=13892308/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=195336 CPUtime=24.26 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 26638 1250 0 0 2421 5 0 0 18 0 9 0 511829139 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 48834 10360 181 55 0 46720 0
[pid=17849/tid=17856] ppid=17847 vsize=195336 CPUtime=3.03 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 3290 1250 0 0 303 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4252349 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=195336 CPUtime=3.03 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 3187 1250 0 0 303 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=195336 CPUtime=3.03 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 2378 1250 0 0 303 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=195336 CPUtime=3.01 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 2304 1250 0 0 301 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=195336 CPUtime=2.96 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 2455 1250 0 0 296 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4236008 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=195336 CPUtime=3.03 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 2685 1250 0 0 303 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4251081 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=195336 CPUtime=3.02 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 2796 1250 0 0 302 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=195336 CPUtime=3.03 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 3733 1250 0 0 303 0 0 0 25 0 9 0 511829145 200024064 10360 33554432000 4194304 4417332 140735308424624 18446744073709551615 18446744073699065949 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.26
Current children cumulated vsize (KiB) 195336

[startup+6.30099 s]
/proc/loadavg: 7.47 7.91 7.96 9/179 17863
/proc/meminfo: memFree=13866936/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=213340 CPUtime=49.85 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 59312 1250 0 0 4972 13 0 0 18 0 9 0 511829139 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 53335 15260 181 55 0 51221 0
[pid=17849/tid=17856] ppid=17847 vsize=213340 CPUtime=6.23 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 7209 1250 0 0 621 2 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4250432 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=213340 CPUtime=6.23 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 7843 1250 0 0 622 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=213340 CPUtime=6.23 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 5701 1250 0 0 622 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=213340 CPUtime=6.21 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 6197 1250 0 0 620 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364575 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=213340 CPUtime=6.15 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 6990 1250 0 0 614 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=213340 CPUtime=6.22 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 7784 1250 0 0 621 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364695 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=213340 CPUtime=6.22 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 6222 1250 0 0 621 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4235999 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=213340 CPUtime=6.23 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 7556 1250 0 0 622 1 0 0 25 0 9 0 511829145 218460160 15260 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 49.85
Current children cumulated vsize (KiB) 213340

[startup+12.7009 s]
/proc/loadavg: 7.56 7.91 7.96 9/179 17863
/proc/meminfo: memFree=13830020/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=244328 CPUtime=101.01 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 117195 1250 0 0 10076 25 0 0 18 0 9 0 511829139 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 61082 23215 181 55 0 58968 0
[pid=17849/tid=17856] ppid=17847 vsize=244328 CPUtime=12.63 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 15469 1250 0 0 1259 4 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4357760 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=244328 CPUtime=12.63 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 15150 1250 0 0 1260 3 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4354410 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=244328 CPUtime=12.63 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 12557 1250 0 0 1261 2 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4358010 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=244328 CPUtime=12.61 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 13463 1250 0 0 1258 3 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=244328 CPUtime=12.52 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 15719 1250 0 0 1248 4 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=244328 CPUtime=12.62 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 13933 1250 0 0 1260 2 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4251034 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=244328 CPUtime=12.63 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 12641 1250 0 0 1261 2 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4357072 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=244328 CPUtime=12.62 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 14453 1250 0 0 1260 2 0 0 25 0 9 0 511829145 250191872 23215 33554432000 4194304 4417332 140735308424624 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.01
Current children cumulated vsize (KiB) 244328

[startup+25.5006 s]
/proc/loadavg: 7.62 7.91 7.96 9/179 17863
/proc/meminfo: memFree=13797088/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=295168 CPUtime=203.35 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 225697 1250 0 0 20285 50 0 0 18 0 9 0 511829139 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 73792 35470 181 55 0 71678 0
[pid=17849/tid=17856] ppid=17847 vsize=295168 CPUtime=25.43 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 28850 1250 0 0 2537 6 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364957 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=295168 CPUtime=25.43 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 26375 1250 0 0 2538 5 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4365026 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=295168 CPUtime=25.43 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 22884 1250 0 0 2540 3 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=295168 CPUtime=25.4 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 26714 1250 0 0 2534 6 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=295168 CPUtime=25.26 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 29285 1250 0 0 2519 7 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=295168 CPUtime=25.42 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 29130 1250 0 0 2536 6 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4354841 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=295168 CPUtime=25.42 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 29056 1250 0 0 2537 5 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4250602 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=295168 CPUtime=25.43 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 29593 1250 0 0 2535 8 0 0 25 0 9 0 511829145 302252032 35470 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.35
Current children cumulated vsize (KiB) 295168

[startup+51.1011 s]
/proc/loadavg: 7.75 7.92 7.96 9/179 17864
/proc/meminfo: memFree=13680152/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=378284 CPUtime=408.02 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 376616 1250 0 0 40718 84 0 0 18 0 9 0 511829139 387362816 56374 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 94571 56374 181 55 0 92457 0
[pid=17849/tid=17856] ppid=17847 vsize=378284 CPUtime=51.03 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 42787 1250 0 0 5094 9 0 0 25 0 9 0 511829145 387362816 56374 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=378284 CPUtime=51.03 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 35955 1250 0 0 5096 7 0 0 25 0 9 0 511829145 387362816 56374 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=378284 CPUtime=51.03 cores=0-7

################
# More data... #
################

[pid=17849/tid=17858] ppid=17847 vsize=4809904 CPUtime=4722.3 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 1043401 1250 0 0 472014 216 0 0 25 0 9 0 511829145 4925341696 1164044 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=4809904 CPUtime=4722.29 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 1619924 1250 0 0 471986 243 0 0 25 0 9 0 511829145 4925341696 1164044 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=4809904 CPUtime=4696.59 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 740125 1250 0 0 469505 154 0 0 25 0 9 0 511829145 4925341696 1164044 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=4809904 CPUtime=4720.81 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 1032718 1250 0 0 471849 232 0 0 25 0 9 0 511829145 4925341696 1164044 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=4809904 CPUtime=4722.34 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1187517 1250 0 0 471987 247 0 0 25 0 9 0 511829145 4925341696 1164044 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=4809904 CPUtime=4722.33 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1058888 1250 0 0 472043 190 0 0 25 0 9 0 511829145 4925341696 1164044 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37750.8
Current children cumulated vsize (KiB) 4809904

[startup+4782.3 s]
/proc/loadavg: 8.01 8.03 8.00 9/179 18014
/proc/meminfo: memFree=9234288/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=4838052 CPUtime=38230.4 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 8556414 1250 0 0 3821369 1675 0 0 18 0 9 0 511829139 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 1209513 1171081 181 55 0 1207399 0
[pid=17849/tid=17856] ppid=17847 vsize=4838052 CPUtime=4782.35 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 756090 1250 0 0 478065 170 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4309203 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=4838052 CPUtime=4781.66 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 1106904 1250 0 0 477949 217 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=4838052 CPUtime=4782.3 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 1043401 1250 0 0 478013 217 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=4838052 CPUtime=4782.29 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 1626961 1250 0 0 477985 244 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=4838052 CPUtime=4756.26 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 740125 1250 0 0 475472 154 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=4838052 CPUtime=4780.79 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 1032718 1250 0 0 477847 232 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=4838052 CPUtime=4782.34 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1187517 1250 0 0 477987 247 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=4838052 CPUtime=4782.33 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1058888 1250 0 0 478043 190 0 0 25 0 9 0 511829145 4954165248 1171081 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38230.4
Current children cumulated vsize (KiB) 4838052

[startup+4842.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/179 18016
/proc/meminfo: memFree=8949092/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=5123228 CPUtime=38710.1 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 8713738 1250 0 0 3869297 1714 0 0 18 0 9 0 511829139 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 1280807 1242375 181 55 0 1278693 0
[pid=17849/tid=17856] ppid=17847 vsize=5123228 CPUtime=4842.35 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 843781 1250 0 0 484045 190 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=5123228 CPUtime=4841.67 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 1106904 1250 0 0 483949 218 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=5123228 CPUtime=4842.3 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 1095665 1250 0 0 483997 233 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=5123228 CPUtime=4842.29 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 1644330 1250 0 0 483983 246 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=5123228 CPUtime=4815.93 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 740125 1250 0 0 481439 154 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=5123228 CPUtime=4840.79 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 1032718 1250 0 0 483847 232 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=5123228 CPUtime=4842.34 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1187517 1250 0 0 483987 247 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=5123228 CPUtime=4842.33 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1058888 1250 0 0 484043 190 0 0 25 0 9 0 511829145 5246185472 1242375 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38710.1
Current children cumulated vsize (KiB) 5123228

[startup+4902.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/179 18017
/proc/meminfo: memFree=8765184/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=5307048 CPUtime=39189.8 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 8759693 1250 0 0 3917261 1719 0 0 18 0 9 0 511829139 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 1326762 1288330 181 55 0 1324648 0
[pid=17849/tid=17856] ppid=17847 vsize=5307048 CPUtime=4902.35 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 843781 1250 0 0 490045 190 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=5307048 CPUtime=4901.67 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 1139672 1250 0 0 489946 221 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=5307048 CPUtime=4902.3 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 1095665 1250 0 0 489997 233 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=5307048 CPUtime=4902.29 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 1657517 1250 0 0 489982 247 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=5307048 CPUtime=4875.61 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 740125 1250 0 0 487407 154 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=5307048 CPUtime=4900.78 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 1032718 1250 0 0 489846 232 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=5307048 CPUtime=4902.34 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1187517 1250 0 0 489987 247 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=5307048 CPUtime=4902.33 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1058888 1250 0 0 490043 190 0 0 25 0 9 0 511829145 5434417152 1288330 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39189.8
Current children cumulated vsize (KiB) 5307048

[startup+4962.3 s]
/proc/loadavg: 8.07 8.02 8.00 9/179 18019
/proc/meminfo: memFree=8735772/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=5336284 CPUtime=39669.5 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 8794971 1250 0 0 3965220 1727 0 0 18 0 9 0 511829139 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 1334071 1295639 181 55 0 1331957 0
[pid=17849/tid=17856] ppid=17847 vsize=5336284 CPUtime=4962.35 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 843781 1250 0 0 496045 190 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=5336284 CPUtime=4961.67 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 1139672 1250 0 0 495946 221 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=5336284 CPUtime=4962.3 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 1095665 1250 0 0 495997 233 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=5336284 CPUtime=4962.29 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 1672496 1250 0 0 495980 249 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=5336284 CPUtime=4935.28 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 740125 1250 0 0 493374 154 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=5336284 CPUtime=4960.78 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 1032718 1250 0 0 495846 232 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=5336284 CPUtime=4962.34 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1207816 1250 0 0 495981 253 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=5336284 CPUtime=4962.33 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1058888 1250 0 0 496043 190 0 0 25 0 9 0 511829145 5464354816 1295639 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39669.5
Current children cumulated vsize (KiB) 5336284



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.7 s]
/proc/loadavg: 8.03 8.02 8.00 9/179 18020
/proc/meminfo: memFree=8704024/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=5367908 CPUtime=40000.5 cores=0-7
/proc/17849/stat : 17849 (plingeling) S 17847 17849 17823 0 -1 4202496 8802877 1250 0 0 3998320 1729 0 0 18 0 9 0 511829139 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 1341977 1303545 181 55 0 1339863 0
[pid=17849/tid=17856] ppid=17847 vsize=5367908 CPUtime=5003.76 cores=0-7
/proc/17849/task/17856/stat : 17856 (plingeling) R 17847 17849 17823 0 -1 4202560 843781 1250 0 0 500186 190 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364957 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17849/tid=17857] ppid=17847 vsize=5367908 CPUtime=5003.07 cores=0-7
/proc/17849/task/17857/stat : 17857 (plingeling) R 17847 17849 17823 0 -1 4202560 1139672 1250 0 0 500086 221 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=17849/tid=17858] ppid=17847 vsize=5367908 CPUtime=5003.71 cores=0-7
/proc/17849/task/17858/stat : 17858 (plingeling) R 17847 17849 17823 0 -1 4202560 1095665 1250 0 0 500138 233 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=17849/tid=17859] ppid=17847 vsize=5367908 CPUtime=5003.7 cores=0-7
/proc/17849/task/17859/stat : 17859 (plingeling) R 17847 17849 17823 0 -1 4202560 1680402 1250 0 0 500118 252 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=5367908 CPUtime=4976.46 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4202560 740125 1250 0 0 497492 154 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17849/tid=17861] ppid=17847 vsize=5367908 CPUtime=5002.18 cores=0-7
/proc/17849/task/17861/stat : 17861 (plingeling) R 17847 17849 17823 0 -1 4202560 1032718 1250 0 0 499986 232 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17849/tid=17862] ppid=17847 vsize=5367908 CPUtime=5003.75 cores=0-7
/proc/17849/task/17862/stat : 17862 (plingeling) R 17847 17849 17823 0 -1 4202560 1207816 1250 0 0 500122 253 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17849/tid=17863] ppid=17847 vsize=5367908 CPUtime=5003.74 cores=0-7
/proc/17849/task/17863/stat : 17863 (plingeling) R 17847 17849 17823 0 -1 4202560 1058888 1250 0 0 500184 190 0 0 25 0 9 0 511829145 5496737792 1303545 33554432000 4194304 4417332 140735308424624 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 5367908

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.81 s]
/proc/loadavg: 8.03 8.02 8.00 2/173 18021
/proc/meminfo: memFree=10993484/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=0 CPUtime=40000.6 cores=0-7
/proc/17849/stat : 17849 (plingeling) Z 17847 17849 17823 0 -1 4203548 8802877 1250 0 0 3998324 1739 0 0 18 0 2 0 511829139 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 0 0 0 0 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=0 CPUtime=4976.57 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4203588 740125 1250 0 0 497493 164 0 0 25 0 2 0 511829145 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

[startup+5003.91 s]
/proc/loadavg: 8.03 8.02 8.00 2/173 18021
/proc/meminfo: memFree=10993484/32951124 swapFree=67111528/67111528
[pid=17849] ppid=17847 vsize=0 CPUtime=40000.7 cores=0-7
/proc/17849/stat : 17849 (plingeling) Z 17847 17849 17823 0 -1 4203548 8802877 1250 0 0 3998324 1748 0 0 18 0 2 0 511829139 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/17849/statm: 0 0 0 0 0 0 0
[pid=17849/tid=17860] ppid=17847 vsize=0 CPUtime=4976.66 cores=0-7
/proc/17849/task/17860/stat : 17860 (plingeling) R 17847 17849 17823 0 -1 4203588 740125 1250 0 0 497493 173 0 0 25 0 2 0 511829145 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5003.95
CPU time (s): 40000.8
CPU user time (s): 39983.2
CPU system time (s): 17.5313
CPU usage (%): 799.385
Max. virtual memory (cumulated for all children) (KiB): 5388244

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39983.2
system time used= 17.5313
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 8804127
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 362
involuntary context switches= 186417

runsolver used 7.79781 second user time and 19.444 second system time

The end

Launcher Data 

Begin job on node108 at 2011-04-28 16:51:22
IDJOB=3309366
IDBENCH=70810
IDSOLVER=1695
FILE ID=node108/3309366-1304002282
RUNJOBID= node108-1304002282-17837
PBS_JOBID= 13168790
Free space on /tmp= 72120 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT09/APPLICATIONS/bitverif/countbitsarray/countbitsarray04_32.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309366-1304002282/watcher-3309366-1304002282 -o /tmp/evaluation-result-3309366-1304002282/solver-3309366-1304002282 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309366-1304002282.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 2fe3650d4a602f63086e937176360ebb
RANDOM SEED=384606383

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      13926132 kB
Buffers:       2291416 kB
Cached:       16088112 kB
SwapCached:          0 kB
Active:       10191696 kB
Inactive:      8251228 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      13926132 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:             392 kB
Writeback:           0 kB
AnonPages:       62224 kB
Mapped:          14104 kB
Slab:           517896 kB
PageTables:       4204 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   182552 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72120 MiB
End job on node108 at 2011-04-28 18:14:48