Trace number 3309792

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.9 5004.04

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom25.cnf
MD5SUMc40bd543dad558d19995fef4fd742233
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables67777
Number of clauses227761
Sum of the clauses size582177
Maximum clause length2084
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2115213
Number of clauses of size 3111398
Number of clauses of size 4324
Number of clauses of size 523
Number of clauses of size over 5802

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309792-1304008915/watcher-3309792-1304008915 -o /tmp/evaluation-result-3309792-1304008915/solver-3309792-1304008915 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309792-1304008915.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 7.68 7.74 2/170 23914
/proc/meminfo: memFree=14407168/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=15688 CPUtime=0 cores=0-7
/proc/23914/stat : 23914 (plingeling) R 23912 23914 23888 0 -1 4202496 1853 1250 0 0 0 0 0 0 21 0 1 0 493306958 16064512 1782 33554432000 4194304 4417332 140736944147072 18446744073709551615 228095145053 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/23914/statm: 3922 1794 140 55 0 1808 0

[startup+0.0226299 s]
/proc/loadavg: 7.47 7.68 7.74 2/170 23914
/proc/meminfo: memFree=14407168/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=29152 CPUtime=0.01 cores=0-7
/proc/23914/stat : 23914 (plingeling) R 23912 23914 23888 0 -1 4202496 5381 1250 0 0 1 0 0 0 21 0 1 0 493306958 29851648 5310 33554432000 4194304 4417332 140736944147072 18446744073709551615 4260021 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/23914/statm: 7288 5310 147 55 0 5174 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 29152

[startup+0.10061 s]
/proc/loadavg: 7.47 7.68 7.74 2/170 23914
/proc/meminfo: memFree=14407168/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=45364 CPUtime=0.09 cores=0-7
/proc/23914/stat : 23914 (plingeling) R 23912 23914 23888 0 -1 4202496 9467 1250 0 0 8 1 0 0 21 0 1 0 493306958 46452736 9364 33554432000 4194304 4417332 140736944147072 18446744073709551615 228095508906 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/23914/statm: 11405 9396 148 55 0 9291 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 45364

[startup+0.300576 s]
/proc/loadavg: 7.47 7.68 7.74 2/170 23914
/proc/meminfo: memFree=14407168/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=89700 CPUtime=0.29 cores=0-7
/proc/23914/stat : 23914 (plingeling) R 23912 23914 23888 0 -1 4202496 20551 1250 0 0 27 2 0 0 22 0 1 0 493306958 91852800 20448 33554432000 4194304 4417332 140736944147072 18446744073709551615 4231423 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/23914/statm: 22425 20448 148 55 0 20311 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 89700

[startup+0.700438 s]
/proc/loadavg: 7.47 7.68 7.74 2/170 23914
/proc/meminfo: memFree=14407168/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=349228 CPUtime=1.35 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 48377 1250 0 0 130 5 0 0 24 0 9 0 493306958 357609472 34674 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 87307 34674 172 55 0 85193 0
Current children cumulated CPU time (s) 1.35
Current children cumulated vsize (KiB) 349228

[startup+1.50129 s]
/proc/loadavg: 7.47 7.68 7.74 9/179 23928
/proc/meminfo: memFree=14263264/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=337828 CPUtime=7.7 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 65125 1250 0 0 763 7 0 0 24 0 9 0 493306958 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 84457 34999 175 55 0 82343 0
[pid=23914/tid=23921] ppid=23912 vsize=337828 CPUtime=0.91 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 1805 1250 0 0 91 0 0 0 18 0 9 0 493307015 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4284976 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=337828 CPUtime=0.92 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 1815 1250 0 0 92 0 0 0 25 0 9 0 493307015 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=337828 CPUtime=0.91 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 4533 1250 0 0 91 0 0 0 19 0 9 0 493307015 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4292063 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=337828 CPUtime=0.89 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 1095 1250 0 0 89 0 0 0 25 0 9 0 493307015 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4237310 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=337828 CPUtime=0.89 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 4778 1250 0 0 89 0 0 0 19 0 9 0 493307015 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=337828 CPUtime=0.9 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 5317 1250 0 0 90 0 0 0 19 0 9 0 493307017 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4302131 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=337828 CPUtime=0.89 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 4231 1250 0 0 89 0 0 0 19 0 9 0 493307017 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4301955 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=337828 CPUtime=0.78 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 5492 1250 0 0 77 1 0 0 25 0 9 0 493307017 345935872 34999 33554432000 4194304 4417332 140736944147072 18446744073709551615 4281470 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7.7
Current children cumulated vsize (KiB) 337828

[startup+3.10099 s]
/proc/loadavg: 7.47 7.68 7.74 9/179 23928
/proc/meminfo: memFree=14266860/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=361836 CPUtime=20.5 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 93362 1250 0 0 2039 11 0 0 24 0 9 0 493306958 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 90459 39652 178 55 0 88345 0
[pid=23914/tid=23921] ppid=23912 vsize=361836 CPUtime=2.5 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 6993 1250 0 0 250 0 0 0 22 0 9 0 493307015 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4346552 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=361836 CPUtime=2.52 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 3302 1250 0 0 252 0 0 0 25 0 9 0 493307015 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4346375 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=361836 CPUtime=2.51 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 8321 1250 0 0 251 0 0 0 22 0 9 0 493307015 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4339793 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=361836 CPUtime=2.49 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 1095 1250 0 0 249 0 0 0 25 0 9 0 493307015 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=361836 CPUtime=2.48 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 7292 1250 0 0 248 0 0 0 22 0 9 0 493307015 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4234178 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=361836 CPUtime=2.49 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 8439 1250 0 0 249 0 0 0 22 0 9 0 493307017 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4346215 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=361836 CPUtime=2.49 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 8404 1250 0 0 248 1 0 0 22 0 9 0 493307017 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4346390 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=361836 CPUtime=2.38 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 13457 1250 0 0 236 2 0 0 25 0 9 0 493307017 370520064 39652 33554432000 4194304 4417332 140736944147072 18446744073709551615 4346401 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 20.5
Current children cumulated vsize (KiB) 361836

[startup+6.3004 s]
/proc/loadavg: 7.51 7.69 7.74 9/179 23928
/proc/meminfo: memFree=14226568/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=394120 CPUtime=46.07 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 106294 1250 0 0 4594 13 0 0 24 0 9 0 493306958 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 98530 45082 181 55 0 96416 0
[pid=23914/tid=23921] ppid=23912 vsize=394120 CPUtime=5.7 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 8535 1250 0 0 569 1 0 0 25 0 9 0 493307015 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=394120 CPUtime=5.71 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 4844 1250 0 0 571 0 0 0 25 0 9 0 493307015 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=394120 CPUtime=5.71 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 8833 1250 0 0 571 0 0 0 25 0 9 0 493307015 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4232730 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=394120 CPUtime=5.69 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 1095 1250 0 0 569 0 0 0 25 0 9 0 493307015 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=394120 CPUtime=5.67 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 9734 1250 0 0 566 1 0 0 25 0 9 0 493307015 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4258589 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=394120 CPUtime=5.7 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 10880 1250 0 0 569 1 0 0 25 0 9 0 493307017 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4258379 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=394120 CPUtime=5.69 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 10192 1250 0 0 568 1 0 0 25 0 9 0 493307017 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4365936 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=394120 CPUtime=5.57 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 16122 1250 0 0 555 2 0 0 21 0 9 0 493307017 403578880 45082 33554432000 4194304 4417332 140736944147072 18446744073709551615 4258389 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 46.07
Current children cumulated vsize (KiB) 394120

[startup+12.7012 s]
/proc/loadavg: 7.55 7.69 7.75 9/179 23928
/proc/meminfo: memFree=14229196/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=393232 CPUtime=97.25 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 107020 1250 0 0 9712 13 0 0 24 0 9 0 493306958 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 98308 44218 181 55 0 96194 0
[pid=23914/tid=23921] ppid=23912 vsize=393232 CPUtime=12.11 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 8554 1250 0 0 1210 1 0 0 25 0 9 0 493307015 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4232612 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=393232 CPUtime=12.11 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 4863 1250 0 0 1211 0 0 0 25 0 9 0 493307015 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=393232 CPUtime=12.11 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 8833 1250 0 0 1211 0 0 0 25 0 9 0 493307015 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4357167 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=393232 CPUtime=12.09 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 1095 1250 0 0 1209 0 0 0 25 0 9 0 493307015 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4250945 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=393232 CPUtime=12.04 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 9753 1250 0 0 1203 1 0 0 25 0 9 0 493307015 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=393232 CPUtime=12.1 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 11501 1250 0 0 1209 1 0 0 25 0 9 0 493307017 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4295500 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=393232 CPUtime=12.09 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 10221 1250 0 0 1208 1 0 0 25 0 9 0 493307017 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=393232 CPUtime=11.97 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 16141 1250 0 0 1195 2 0 0 25 0 9 0 493307017 402669568 44218 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236870 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 97.25
Current children cumulated vsize (KiB) 393232

[startup+25.5009 s]
/proc/loadavg: 7.65 7.71 7.75 9/179 23928
/proc/meminfo: memFree=14215724/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=409644 CPUtime=199.57 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 111384 1250 0 0 19944 13 0 0 24 0 9 0 493306958 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 102411 48333 181 55 0 100297 0
[pid=23914/tid=23921] ppid=23912 vsize=409644 CPUtime=24.9 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 8590 1250 0 0 2489 1 0 0 25 0 9 0 493307015 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=409644 CPUtime=24.91 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 4897 1250 0 0 2491 0 0 0 25 0 9 0 493307015 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4277495 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=409644 CPUtime=24.91 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 9909 1250 0 0 2491 0 0 0 25 0 9 0 493307015 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=409644 CPUtime=24.89 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 2631 1250 0 0 2489 0 0 0 25 0 9 0 493307015 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4320811 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=409644 CPUtime=24.77 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 9788 1250 0 0 2476 1 0 0 25 0 9 0 493307015 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=409644 CPUtime=24.9 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 11537 1250 0 0 2489 1 0 0 25 0 9 0 493307017 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=409644 CPUtime=24.89 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 11797 1250 0 0 2488 1 0 0 25 0 9 0 493307017 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=409644 CPUtime=24.77 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 16176 1250 0 0 2475 2 0 0 25 0 9 0 493307017 419475456 48333 33554432000 4194304 4417332 140736944147072 18446744073709551615 4365414 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 199.57
Current children cumulated vsize (KiB) 409644

[startup+51.1012 s]
/proc/loadavg: 7.84 7.75 7.76 9/179 23929
/proc/meminfo: memFree=14167920/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=458728 CPUtime=404.26 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 129440 1250 0 0 40410 16 0 0 24 0 9 0 493306958 469737472 60618 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 114682 60618 181 55 0 112568 0
[pid=23914/tid=23921] ppid=23912 vsize=458728 CPUtime=50.5 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 8608 1250 0 0 5049 1 0 0 25 0 9 0 493307015 469737472 60618 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=458728 CPUtime=50.51 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 7090 1250 0 0 5051 0 0 0 25 0 9 0 493307015 469737472 60618 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=458728 CPUtime=50.51 cores=0-7

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.02 8.03 8.00 9/179 24066
/proc/meminfo: memFree=11701496/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=2876172 CPUtime=37749.4 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 3652697 1250 0 0 3774328 613 0 0 24 0 9 0 493306958 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 719043 676074 181 55 0 716929 0
[pid=23914/tid=23921] ppid=23912 vsize=2876172 CPUtime=4721.38 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 573926 1250 0 0 472045 93 0 0 25 0 9 0 493307015 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=2876172 CPUtime=4721.72 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 422682 1250 0 0 472108 64 0 0 25 0 9 0 493307015 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=2876172 CPUtime=4721.82 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 397970 1250 0 0 472117 65 0 0 25 0 9 0 493307015 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=2876172 CPUtime=4721.82 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 694018 1250 0 0 472049 133 0 0 25 0 9 0 493307015 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=2876172 CPUtime=4696.74 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 478720 1250 0 0 469579 95 0 0 25 0 9 0 493307015 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=2876172 CPUtime=4721.81 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 269376 1250 0 0 472140 41 0 0 25 0 9 0 493307017 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=2876172 CPUtime=4721.79 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 283163 1250 0 0 472135 44 0 0 25 0 9 0 493307017 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=2876172 CPUtime=4721.7 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 496783 1250 0 0 472099 71 0 0 25 0 9 0 493307017 2945200128 676074 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37749.4
Current children cumulated vsize (KiB) 2876172

[startup+4782.3 s]
/proc/loadavg: 8.01 8.02 8.00 9/179 24076
/proc/meminfo: memFree=11446460/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=3155708 CPUtime=38229 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 3730988 1250 0 0 3822278 625 0 0 24 0 9 0 493306958 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 788927 739589 181 55 0 786813 0
[pid=23914/tid=23921] ppid=23912 vsize=3155708 CPUtime=4781.36 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 573926 1250 0 0 478043 93 0 0 25 0 9 0 493307015 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=3155708 CPUtime=4781.71 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 430874 1250 0 0 478106 65 0 0 25 0 9 0 493307015 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4358020 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=3155708 CPUtime=4781.8 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 407314 1250 0 0 478115 65 0 0 25 0 9 0 493307015 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=3155708 CPUtime=4781.82 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 715404 1250 0 0 478047 135 0 0 25 0 9 0 493307015 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=3155708 CPUtime=4756.41 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 509897 1250 0 0 475539 102 0 0 25 0 9 0 493307015 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=3155708 CPUtime=4781.81 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 277568 1250 0 0 478139 42 0 0 25 0 9 0 493307017 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=3155708 CPUtime=4781.78 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 283163 1250 0 0 478134 44 0 0 25 0 9 0 493307017 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=3155708 CPUtime=4781.69 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 496783 1250 0 0 478098 71 0 0 25 0 9 0 493307017 3231444992 739589 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38229
Current children cumulated vsize (KiB) 3155708

[startup+4842.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/179 24078
/proc/meminfo: memFree=11061128/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=3539836 CPUtime=38708.7 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 3847863 1250 0 0 3870222 650 0 0 24 0 9 0 493306958 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 884959 835678 181 55 0 882845 0
[pid=23914/tid=23921] ppid=23912 vsize=3539836 CPUtime=4841.36 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 615412 1250 0 0 484036 100 0 0 25 0 9 0 493307015 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=3539836 CPUtime=4841.71 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 477242 1250 0 0 484092 79 0 0 25 0 9 0 493307015 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=3539836 CPUtime=4841.8 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 407314 1250 0 0 484115 65 0 0 25 0 9 0 493307015 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4365994 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=3539836 CPUtime=4841.82 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 715404 1250 0 0 484047 135 0 0 25 0 9 0 493307015 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=3539836 CPUtime=4816.09 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 526283 1250 0 0 481505 104 0 0 25 0 9 0 493307015 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=3539836 CPUtime=4841.81 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 277568 1250 0 0 484139 42 0 0 25 0 9 0 493307017 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=3539836 CPUtime=4841.78 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 283189 1250 0 0 484134 44 0 0 25 0 9 0 493307017 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=3539836 CPUtime=4841.7 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 509392 1250 0 0 484097 73 0 0 25 0 9 0 493307017 3624792064 835678 33554432000 4194304 4417332 140736944147072 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38708.7
Current children cumulated vsize (KiB) 3539836

[startup+4902.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/179 24079
/proc/meminfo: memFree=11410664/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=3192104 CPUtime=39188.4 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 3881060 1250 0 0 3918184 658 0 0 24 0 9 0 493306958 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 798026 748745 181 55 0 795912 0
[pid=23914/tid=23921] ppid=23912 vsize=3192104 CPUtime=4901.35 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 615412 1250 0 0 490035 100 0 0 25 0 9 0 493307015 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=3192104 CPUtime=4901.71 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 480646 1250 0 0 490090 81 0 0 25 0 9 0 493307015 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4250795 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=3192104 CPUtime=4901.8 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 407314 1250 0 0 490115 65 0 0 25 0 9 0 493307015 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=3192104 CPUtime=4901.82 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 723596 1250 0 0 490045 137 0 0 25 0 9 0 493307015 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364362 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=3192104 CPUtime=4875.78 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 530251 1250 0 0 487473 105 0 0 25 0 9 0 493307015 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236830 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=3192104 CPUtime=4901.81 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 277568 1250 0 0 490139 42 0 0 25 0 9 0 493307017 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=3192104 CPUtime=4901.78 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 292630 1250 0 0 490133 45 0 0 25 0 9 0 493307017 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=3192104 CPUtime=4901.7 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 517584 1250 0 0 490096 74 0 0 25 0 9 0 493307017 3268714496 748745 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39188.4
Current children cumulated vsize (KiB) 3192104

[startup+4962.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/179 24081
/proc/meminfo: memFree=11793560/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=2828144 CPUtime=39668.1 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 3909279 1250 0 0 3966147 663 0 0 24 0 9 0 493306958 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 707036 653109 181 55 0 704922 0
[pid=23914/tid=23921] ppid=23912 vsize=2828144 CPUtime=4961.35 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 615414 1250 0 0 496034 101 0 0 25 0 9 0 493307015 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=2828144 CPUtime=4961.71 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 490873 1250 0 0 496089 82 0 0 25 0 9 0 493307015 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=2828144 CPUtime=4961.81 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 424940 1250 0 0 496113 68 0 0 25 0 9 0 493307015 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=2828144 CPUtime=4961.82 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 723596 1250 0 0 496045 137 0 0 25 0 9 0 493307015 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4251172 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=2828144 CPUtime=4935.46 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 530615 1250 0 0 493440 106 0 0 25 0 9 0 493307015 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=2828144 CPUtime=4961.82 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 277568 1250 0 0 496140 42 0 0 25 0 9 0 493307017 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=2828144 CPUtime=4961.79 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 292630 1250 0 0 496134 45 0 0 25 0 9 0 493307017 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=2828144 CPUtime=4961.7 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 517584 1250 0 0 496096 74 0 0 25 0 9 0 493307017 2896019456 653109 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39668.1
Current children cumulated vsize (KiB) 2828144



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.91 s]
/proc/loadavg: 8.04 8.02 8.00 9/179 24082
/proc/meminfo: memFree=11685300/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=2937448 CPUtime=40000.8 cores=0-7
/proc/23914/stat : 23914 (plingeling) S 23912 23914 23888 0 -1 4202496 3975319 1250 0 0 3999399 678 0 0 24 0 9 0 493306958 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 228107253733 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 734362 680235 181 55 0 732248 0
[pid=23914/tid=23921] ppid=23912 vsize=2937448 CPUtime=5002.96 cores=0-7
/proc/23914/task/23921/stat : 23921 (plingeling) R 23912 23914 23888 0 -1 4202560 623606 1250 0 0 500194 102 0 0 25 0 9 0 493307015 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23914/tid=23922] ppid=23912 vsize=2937448 CPUtime=5003.32 cores=0-7
/proc/23914/task/23922/stat : 23922 (plingeling) R 23912 23914 23888 0 -1 4202560 490873 1250 0 0 500250 82 0 0 25 0 9 0 493307015 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23914/tid=23923] ppid=23912 vsize=2937448 CPUtime=5003.42 cores=0-7
/proc/23914/task/23923/stat : 23923 (plingeling) R 23912 23914 23888 0 -1 4202560 432143 1250 0 0 500273 69 0 0 25 0 9 0 493307015 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23914/tid=23924] ppid=23912 vsize=2937448 CPUtime=5003.44 cores=0-7
/proc/23914/task/23924/stat : 23924 (plingeling) R 23912 23914 23888 0 -1 4202560 747692 1250 0 0 500199 145 0 0 25 0 9 0 493307015 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=2937448 CPUtime=4976.85 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4202560 533110 1250 0 0 497578 107 0 0 25 0 9 0 493307015 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23914/tid=23926] ppid=23912 vsize=2937448 CPUtime=5003.43 cores=0-7
/proc/23914/task/23926/stat : 23926 (plingeling) R 23912 23914 23888 0 -1 4202560 277568 1250 0 0 500301 42 0 0 25 0 9 0 493307017 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23914/tid=23927] ppid=23912 vsize=2937448 CPUtime=5003.4 cores=0-7
/proc/23914/task/23927/stat : 23927 (plingeling) R 23912 23914 23888 0 -1 4202560 302070 1250 0 0 500292 48 0 0 25 0 9 0 493307017 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23914/tid=23928] ppid=23912 vsize=2937448 CPUtime=5003.31 cores=0-7
/proc/23914/task/23928/stat : 23928 (plingeling) R 23912 23914 23888 0 -1 4202560 532198 1250 0 0 500256 75 0 0 25 0 9 0 493307017 3007946752 680235 33554432000 4194304 4417332 140736944147072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 2937448

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5004.01 s]
/proc/loadavg: 8.04 8.02 8.00 9/179 24082
/proc/meminfo: memFree=11685300/32951124 swapFree=67111528/67111528
[pid=23914] ppid=23912 vsize=0 CPUtime=40000.9 cores=0-7
/proc/23914/stat : 23914 (plingeling) Z 23912 23914 23888 0 -1 4203548 3975319 1250 0 0 3999403 687 0 0 18 0 2 0 493306958 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/23914/statm: 0 0 0 0 0 0 0
[pid=23914/tid=23925] ppid=23912 vsize=0 CPUtime=4976.95 cores=0-7
/proc/23914/task/23925/stat : 23925 (plingeling) R 23912 23914 23888 0 -1 4203588 533110 1250 0 0 497579 116 0 0 25 0 2 0 493307015 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5004.04
CPU time (s): 40000.9
CPU user time (s): 39994
CPU system time (s): 6.90895
CPU usage (%): 799.373
Max. virtual memory (cumulated for all children) (KiB): 3588476

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39994
system time used= 6.90895
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3976569
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 724
involuntary context switches= 192858

runsolver used 7.61484 second user time and 18.1642 second system time

The end

Launcher Data 

Begin job on node142 at 2011-04-28 18:41:55
IDJOB=3309792
IDBENCH=83109
IDSOLVER=1695
FILE ID=node142/3309792-1304008915
RUNJOBID= node142-1304008915-23902
PBS_JOBID= 13168845
Free space on /tmp= 71312 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom25.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309792-1304008915/watcher-3309792-1304008915 -o /tmp/evaluation-result-3309792-1304008915/solver-3309792-1304008915 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309792-1304008915.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c40bd543dad558d19995fef4fd742233
RANDOM SEED=663019149

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      14407448 kB
Buffers:       2099572 kB
Cached:       15799260 kB
SwapCached:          0 kB
Active:        7816132 kB
Inactive:     10146796 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      14407448 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            4288 kB
Writeback:           4 kB
AnonPages:       63828 kB
Mapped:          14440 kB
Slab:           513788 kB
PageTables:       4980 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   179576 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71308 MiB
End job on node142 at 2011-04-28 20:05:21