Trace number 3372449

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.9 5029.45

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.050-SAT.cnf
MD5SUM48f03e5fbe31fc85b9a464480fc8aa5a
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark745.077
Satisfiable
(Un)Satisfiability was proved
Number of variables1575206
Number of clauses8464620
Sum of the clauses size20118920
Maximum clause length982
Minimum clause length1
Number of clauses of size 1720
Number of clauses of size 28393300
Number of clauses of size 3400
Number of clauses of size 41600
Number of clauses of size 54000
Number of clauses of size over 564600

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372449-1305437558/watcher-3372449-1305437558 -o /tmp/evaluation-result-3372449-1305437558/solver-3372449-1305437558 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372449-1305437558.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.08 6.28 5.90 3/391 5968
/proc/meminfo: memFree=23933104/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=21040 CPUtime=0 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 3079 1253 0 0 0 0 0 0 22 0 1 0 655354997 21544960 3010 33554432000 4194304 4417452 140736527988592 18446744073709551615 242970244189 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 5260 3021 139 55 0 3146 0

[startup+0.0257291 s]
/proc/loadavg: 5.08 6.28 5.90 3/391 5968
/proc/meminfo: memFree=23933104/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=33980 CPUtime=0.01 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 6587 1253 0 0 1 0 0 0 22 0 1 0 655354997 34795520 6517 33554432000 4194304 4417452 140736527988592 18446744073709551615 242970168926 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 8495 6517 147 55 0 6381 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 33980

[startup+0.100716 s]
/proc/loadavg: 5.08 6.28 5.90 3/391 5968
/proc/meminfo: memFree=23933104/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=41592 CPUtime=0.08 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 8492 1253 0 0 8 0 0 0 22 0 1 0 655354997 42590208 8421 33554432000 4194304 4417452 140736527988592 18446744073709551615 4203110 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 10398 8421 148 55 0 8284 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 41592

[startup+0.300695 s]
/proc/loadavg: 5.08 6.28 5.90 3/391 5968
/proc/meminfo: memFree=23933104/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=72412 CPUtime=0.29 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 16198 1253 0 0 27 2 0 0 23 0 1 0 655354997 74149888 16127 33554432000 4194304 4417452 140736527988592 18446744073709551615 242970550016 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 18103 16127 149 55 0 15989 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 72412

[startup+0.700625 s]
/proc/loadavg: 5.08 6.28 5.90 3/391 5968
/proc/meminfo: memFree=23933104/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=117880 CPUtime=0.69 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 27565 1253 0 0 66 3 0 0 25 0 1 0 655354997 120709120 27494 33554432000 4194304 4417452 140736527988592 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 29470 27494 149 55 0 27356 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 117880

[startup+1.50049 s]
/proc/loadavg: 4.91 6.22 5.88 2/392 5975
/proc/meminfo: memFree=23724988/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=235424 CPUtime=1.49 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 57017 1253 0 0 144 5 0 0 25 0 1 0 655354997 241074176 56880 33554432000 4194304 4417452 140736527988592 18446744073709551615 242970168876 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 58856 56880 149 55 0 56742 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 235424

[startup+3.10021 s]
/proc/loadavg: 4.91 6.22 5.88 2/392 5975
/proc/meminfo: memFree=23667096/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=495148 CPUtime=3.09 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 122172 1253 0 0 296 13 0 0 25 0 1 0 655354997 507031552 121811 33554432000 4194304 4417452 140736527988592 18446744073709551615 4262041 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 123787 121811 149 55 0 121673 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 495148

[startup+6.30066 s]
/proc/loadavg: 4.91 6.22 5.88 2/392 5975
/proc/meminfo: memFree=22996328/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=941616 CPUtime=6.28 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 233917 1253 0 0 602 26 0 0 25 0 1 0 655354997 964214784 233428 33554432000 4194304 4417452 140736527988592 18446744073709551615 4209566 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 235404 233428 149 55 0 233290 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 941616

[startup+12.7006 s]
/proc/loadavg: 4.77 6.15 5.86 2/392 5975
/proc/meminfo: memFree=22103016/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=1900080 CPUtime=12.69 cores=0-7
/proc/5968/stat : 5968 (plingeling) R 5965 5968 5779 0 -1 4202496 473533 1253 0 0 1216 53 0 0 25 0 1 0 655354997 1945681920 473044 33554432000 4194304 4417452 140736527988592 18446744073709551615 242970168926 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5968/statm: 475020 473044 149 55 0 472906 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 1900080

[startup+25.5013 s]
/proc/loadavg: 5.21 6.19 5.88 10/400 5984
/proc/meminfo: memFree=21655544/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=2526288 CPUtime=67.87 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 954940 1253 0 0 6581 206 0 0 25 0 9 0 655354997 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 631572 577183 177 55 0 629458 0
[pid=5968/tid=5976] ppid=5965 vsize=2526288 CPUtime=4.63 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 37314 1253 0 0 419 44 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=2526288 CPUtime=7.07 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 43382 1253 0 0 694 13 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=2526288 CPUtime=7 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 61956 1253 0 0 669 31 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4261313 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=2526288 CPUtime=5.47 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 35903 1253 0 0 538 9 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=2526288 CPUtime=6.99 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 44877 1253 0 0 689 10 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4266901 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=2526288 CPUtime=6.93 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 43537 1253 0 0 683 10 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4215574 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=2526288 CPUtime=6.14 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 42704 1253 0 0 605 9 0 0 25 0 9 0 655356770 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4267654 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=2526288 CPUtime=5.84 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 40646 1253 0 0 577 7 0 0 25 0 9 0 655356775 2586918912 577183 33554432000 4194304 4417452 140736527988592 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 67.87
Current children cumulated vsize (KiB) 2526288

[startup+51.1009 s]
/proc/loadavg: 7.17 6.58 6.02 9/400 5985
/proc/meminfo: memFree=21421424/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=2811464 CPUtime=249.48 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 1989024 1253 0 0 24457 491 0 0 25 0 9 0 655354997 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 702866 627449 179 55 0 700752 0
[pid=5968/tid=5976] ppid=5965 vsize=2811464 CPUtime=27.1 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 160842 1253 0 0 2644 66 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254468 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=2811464 CPUtime=31.99 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 184280 1253 0 0 3161 38 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254990 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=2811464 CPUtime=32.21 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 192149 1253 0 0 3164 57 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254964 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=2811464 CPUtime=27.93 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 43220 1253 0 0 2778 15 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=2811464 CPUtime=28.18 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 212522 1253 0 0 2766 52 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254810 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=2811464 CPUtime=28.45 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 183844 1253 0 0 2774 71 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254425 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=2811464 CPUtime=25.98 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 203021 1253 0 0 2552 46 0 0 25 0 9 0 655356770 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254955 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=2811464 CPUtime=29.83 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 204525 1253 0 0 2910 73 0 0 25 0 9 0 655356775 2878939136 627449 33554432000 4194304 4417452 140736527988592 18446744073709551615 4254437 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 249.48
Current children cumulated vsize (KiB) 2811464

[startup+102.315 s]
/proc/loadavg: 7.73 6.83 6.13 9/400 5986
/proc/meminfo: memFree=21238052/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=2944336 CPUtime=658.47 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 2183827 1253 0 0 65313 534 0 0 25 0 9 0 655354997 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 736084 675002 180 55 0 733970 0
[pid=5968/tid=5976] ppid=5965 vsize=2944336 CPUtime=78.32 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 171689 1253 0 0 7764 68 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=2944336 CPUtime=83.21 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 197175 1253 0 0 8281 40 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=2944336 CPUtime=83.41 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 231293 1253 0 0 8278 63 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=2944336 CPUtime=79.15 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 90196 1253 0 0 7893 22 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=2944336 CPUtime=79.36 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 230879 1253 0 0 7869 67 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232666 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=2944336 CPUtime=78.99 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 194691 1253 0 0 7826 73 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=2944336 CPUtime=77.19 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 240434 1253 0 0 7667 52 0 0 25 0 9 0 655356770 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=2944336 CPUtime=81.04 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 222849 1253 0 0 8027 77 0 0 25 0 9 0 655356775 3015000064 675002 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 658.47
Current children cumulated vsize (KiB) 2944336

[startup+162.301 s]
/proc/loadavg: 7.96 7.06 6.25 9/400 5988
/proc/meminfo: memFree=21009424/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=3152780 CPUtime=1137.49 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 2278455 1253 0 0 113202 547 0 0 25 0 9 0 655354997 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 788195 731824 180 55 0 786081 0
[pid=5968/tid=5976] ppid=5965 vsize=3152780 CPUtime=138.3 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 182930 1253 0 0 13761 69 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=3152780 CPUtime=143.18 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 206962 1253 0 0 14276 42 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4264916 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=3152780 CPUtime=143.38 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 243107 1253 0 0 14273 65 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=3152780 CPUtime=139.13 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 113294 1253 0 0 13888 25 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4357314 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=3152780 CPUtime=139.34 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 244208 1253 0 0 13866 68 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=3152780 CPUtime=138.16 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 194691 1253 0 0 13743 73 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=3152780 CPUtime=137.18 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 248941 1253 0 0 13665 53 0 0 25 0 9 0 655356770 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4237002 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=3152780 CPUtime=141 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 239701 1253 0 0 14021 79 0 0 25 0 9 0 655356775 3228446720 731824 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232566 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1137.49
Current children cumulated vsize (KiB) 3152780


################
# More data... #
################

[pid=5968/tid=5979] ppid=5965 vsize=14527852 CPUtime=4762.47 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 2666863 1253 0 0 475669 578 0 0 25 0 9 0 655356770 14876520448 3565397 33554432000 4194304 4417452 140736527988592 18446744073709551615 4281649 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=14527852 CPUtime=4735.25 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 6691597 1253 0 0 471928 1597 0 0 25 0 9 0 655356770 14876520448 3565397 33554432000 4194304 4417452 140736527988592 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=14527852 CPUtime=4747.2 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 3489967 1253 0 0 473898 822 0 0 25 0 9 0 655356770 14876520448 3565397 33554432000 4194304 4417452 140736527988592 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=14527852 CPUtime=4738.32 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 1675774 1253 0 0 473441 391 0 0 25 0 9 0 655356770 14876520448 3565397 33554432000 4194304 4417452 140736527988592 18446744073709551615 4311298 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=14527852 CPUtime=4757.08 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 2952106 1253 0 0 474991 717 0 0 25 0 9 0 655356770 14876520448 3565397 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232666 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=14527852 CPUtime=4760.02 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 1923733 1253 0 0 475562 440 0 0 25 0 9 0 655356775 14876520448 3565397 33554432000 4194304 4417452 140736527988592 18446744073709551615 4309920 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38030.9
Current children cumulated vsize (KiB) 14527852

[startup+4842.3 s]
/proc/loadavg: 8.13 8.07 8.01 9/400 6132
/proc/meminfo: memFree=9287132/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=14891588 CPUtime=38510.1 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 24686569 1253 0 0 3845212 5794 0 0 25 0 9 0 655354997 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 3722897 3656331 180 55 0 3720783 0
[pid=5968/tid=5976] ppid=5965 vsize=14891588 CPUtime=4810.44 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 2391987 1253 0 0 480390 654 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4366234 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=14891588 CPUtime=4822.34 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 2104441 1253 0 0 481758 476 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=14891588 CPUtime=4822.47 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 2666863 1253 0 0 481669 578 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=14891588 CPUtime=4795.2 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 6877077 1253 0 0 477878 1642 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4265408 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=14891588 CPUtime=4806.4 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 3489967 1253 0 0 479818 822 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=14891588 CPUtime=4798.32 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 1675774 1253 0 0 479441 391 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=14891588 CPUtime=4817.08 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 2952106 1253 0 0 480991 717 0 0 25 0 9 0 655356770 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=14891588 CPUtime=4819.99 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 1923733 1253 0 0 481559 440 0 0 25 0 9 0 655356775 15248986112 3656331 33554432000 4194304 4417452 140736527988592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38510.1
Current children cumulated vsize (KiB) 14891588

[startup+4902.3 s]
/proc/loadavg: 8.05 8.06 8.00 9/400 6133
/proc/meminfo: memFree=10408124/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=13774444 CPUtime=38989.2 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 25056770 1253 0 0 3893035 5887 0 0 25 0 9 0 655354997 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 3443611 3377045 180 55 0 3441497 0
[pid=5968/tid=5976] ppid=5965 vsize=13774444 CPUtime=4870.44 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 2397112 1253 0 0 486381 663 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232566 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=13774444 CPUtime=4882.34 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 2365550 1253 0 0 487696 538 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4366500 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=13774444 CPUtime=4882.47 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 2666863 1253 0 0 487669 578 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=13774444 CPUtime=4855.17 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 6877077 1253 0 0 483874 1643 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=13774444 CPUtime=4865.61 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 3489967 1253 0 0 485739 822 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=13774444 CPUtime=4858.32 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 1675774 1253 0 0 485441 391 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4252349 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=13774444 CPUtime=4877.08 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 2952106 1253 0 0 486990 718 0 0 25 0 9 0 655356770 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=13774444 CPUtime=4879.98 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 2027700 1253 0 0 487537 461 0 0 25 0 9 0 655356775 14105030656 3377045 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38989.2
Current children cumulated vsize (KiB) 13774444

[startup+4962.3 s]
/proc/loadavg: 8.10 8.06 8.01 9/400 6135
/proc/meminfo: memFree=11089780/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=13104484 CPUtime=39468.4 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 25384157 1253 0 0 3940866 5972 0 0 25 0 9 0 655354997 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 3276121 3209555 180 55 0 3274007 0
[pid=5968/tid=5976] ppid=5965 vsize=13104484 CPUtime=4930.44 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 2412584 1253 0 0 492379 665 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232612 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=13104484 CPUtime=4942.34 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 2365550 1253 0 0 493696 538 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=13104484 CPUtime=4942.47 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 2742636 1253 0 0 493645 602 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=13104484 CPUtime=4915.14 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 6911972 1253 0 0 489858 1656 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=13104484 CPUtime=4924.8 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 3542345 1253 0 0 491645 835 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=13104484 CPUtime=4918.32 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 1824643 1253 0 0 491407 425 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4365579 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=13104484 CPUtime=4937.08 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 2952106 1253 0 0 492990 718 0 0 25 0 9 0 655356770 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232702 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=13104484 CPUtime=4939.97 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 2027700 1253 0 0 493536 461 0 0 25 0 9 0 655356775 13418991616 3209555 33554432000 4194304 4417452 140736527988592 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39468.4
Current children cumulated vsize (KiB) 13104484

[startup+5022.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/400 6137
/proc/meminfo: memFree=10165152/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=14009176 CPUtime=39947.5 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 25698088 1253 0 0 3988695 6057 0 0 25 0 9 0 655354997 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 3502294 3437744 180 55 0 3500180 0
[pid=5968/tid=5976] ppid=5965 vsize=14009176 CPUtime=4990.44 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 2526703 1253 0 0 498362 682 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4263232 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=14009176 CPUtime=5002.35 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 2365550 1253 0 0 499697 538 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=14009176 CPUtime=5002.47 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 2808781 1253 0 0 499622 625 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=14009176 CPUtime=4975.09 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 7035900 1253 0 0 495811 1698 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=14009176 CPUtime=4984.01 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 3552084 1253 0 0 497564 837 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=14009176 CPUtime=4978.32 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 1824643 1253 0 0 497407 425 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=14009176 CPUtime=4997.08 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 2952106 1253 0 0 498990 718 0 0 25 0 9 0 655356770 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4250986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=14009176 CPUtime=4999.96 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 2027700 1253 0 0 499535 461 0 0 25 0 9 0 655356775 14345396224 3437744 33554432000 4194304 4417452 140736527988592 18446744073709551615 4264959 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39947.5
Current children cumulated vsize (KiB) 14009176



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5028.9 s]
/proc/loadavg: 8.03 8.05 8.00 9/400 6137
/proc/meminfo: memFree=10048844/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=14125288 CPUtime=40000.3 cores=0-7
/proc/5968/stat : 5968 (plingeling) S 5965 5968 5779 0 -1 4202496 25727116 1253 0 0 3993961 6068 0 0 25 0 9 0 655354997 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 3531322 3466772 180 55 0 3529208 0
[pid=5968/tid=5976] ppid=5965 vsize=14125288 CPUtime=4997.05 cores=0-7
/proc/5968/task/5976/stat : 5976 (plingeling) R 5965 5968 5779 0 -1 4202560 2526703 1253 0 0 499023 682 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5968/tid=5978] ppid=5965 vsize=14125288 CPUtime=5008.96 cores=0-7
/proc/5968/task/5978/stat : 5978 (plingeling) R 5965 5968 5779 0 -1 4202560 2365550 1253 0 0 500358 538 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5968/tid=5979] ppid=5965 vsize=14125288 CPUtime=5009.09 cores=0-7
/proc/5968/task/5979/stat : 5979 (plingeling) R 5965 5968 5779 0 -1 4202560 2808781 1253 0 0 500284 625 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=5968/tid=5980] ppid=5965 vsize=14125288 CPUtime=4981.69 cores=0-7
/proc/5968/task/5980/stat : 5980 (plingeling) R 5965 5968 5779 0 -1 4202560 7064928 1253 0 0 496461 1708 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=14125288 CPUtime=4990.53 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4202560 3552084 1253 0 0 498216 837 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5968/tid=5982] ppid=5965 vsize=14125288 CPUtime=4984.94 cores=0-7
/proc/5968/task/5982/stat : 5982 (plingeling) R 5965 5968 5779 0 -1 4202560 1824643 1253 0 0 498069 425 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5968/tid=5983] ppid=5965 vsize=14125288 CPUtime=5003.7 cores=0-7
/proc/5968/task/5983/stat : 5983 (plingeling) R 5965 5968 5779 0 -1 4202560 2952106 1253 0 0 499652 718 0 0 25 0 9 0 655356770 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4265499 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5968/tid=5984] ppid=5965 vsize=14125288 CPUtime=5006.57 cores=0-7
/proc/5968/task/5984/stat : 5984 (plingeling) R 5965 5968 5779 0 -1 4202560 2027700 1253 0 0 500196 461 0 0 25 0 9 0 655356775 14464294912 3466772 33554432000 4194304 4417452 140736527988592 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 14125288

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5029.02 s]
/proc/loadavg: 8.03 8.05 8.00 9/400 6137
/proc/meminfo: memFree=10048844/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=0 CPUtime=40000.5 cores=0-7
/proc/5968/stat : 5968 (plingeling) Z 5965 5968 5779 0 -1 4203548 25727117 1253 0 0 3993970 6078 0 0 18 0 2 0 655354997 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 0 0 0 0 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=0 CPUtime=4990.63 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4203588 3552084 1253 0 0 498216 847 0 0 25 0 2 0 655356770 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

[startup+5029.4 s]
/proc/loadavg: 8.03 8.05 8.00 2/394 6138
/proc/meminfo: memFree=14569064/32951132 swapFree=67093844/67111528
[pid=5968] ppid=5965 vsize=0 CPUtime=40000.9 cores=0-7
/proc/5968/stat : 5968 (plingeling) Z 5965 5968 5779 0 -1 4203548 25727117 1253 0 0 3993970 6116 0 0 18 0 2 0 655354997 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/5968/statm: 0 0 0 0 0 0 0
[pid=5968/tid=5981] ppid=5965 vsize=0 CPUtime=4991.01 cores=0-7
/proc/5968/task/5981/stat : 5981 (plingeling) R 5965 5968 5779 0 -1 4203588 3552084 1253 0 0 498216 885 0 0 25 0 2 0 655356770 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5029.45
CPU time (s): 40000.9
CPU user time (s): 39939.7
CPU system time (s): 61.2197
CPU usage (%): 795.333
Max. virtual memory (cumulated for all children) (KiB): 15779576

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39939.7
system time used= 61.2197
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 25728370
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 693
involuntary context switches= 193287

runsolver used 17.1594 second user time and 48.8806 second system time

The end

Launcher Data 

Begin job on node128 at 2011-05-15 07:32:43
IDJOB=3372449
IDBENCH=82388
IDSOLVER=1851
FILE ID=node128/3372449-1305437558
RUNJOBID= node128-1305436581-5794
PBS_JOBID= 13324753
Free space on /tmp= 73452 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.050-SAT.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372449-1305437558/watcher-3372449-1305437558 -o /tmp/evaluation-result-3372449-1305437558/solver-3372449-1305437558 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372449-1305437558.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 48f03e5fbe31fc85b9a464480fc8aa5a
RANDOM SEED=1490839825

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      23933624 kB
Buffers:        278236 kB
Cached:        3168652 kB
SwapCached:       1440 kB
Active:        6661492 kB
Inactive:      2054448 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      23933624 kB
SwapTotal:    67111528 kB
SwapFree:     67093844 kB
Dirty:          171252 kB
Writeback:           0 kB
AnonPages:     5267840 kB
Mapped:          31460 kB
Slab:           195648 kB
PageTables:      37796 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 100685284 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73452 MiB
End job on node128 at 2011-05-15 08:56:34