Trace number 3394928

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.7 5011.66

General information on the benchmark

NameSAT02/handmade/simon/satex-challenges/
par32-3.shuffled.cnf
MD5SUM0115a0e26236f3f01aa829c2004452c1
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.638902
Satisfiable
(Un)Satisfiability was proved
Number of variables3176
Number of clauses10297
Sum of the clauses size27581
Maximum clause length3
Minimum clause length1
Number of clauses of size 1141
Number of clauses of size 23028
Number of clauses of size 37128
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394928-1306109483/watcher-3394928-1306109483 -o /tmp/evaluation-result-3394928-1306109483/solver-3394928-1306109483 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394928-1306109483.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.45 7.94 7.98 1/547 3047
/proc/meminfo: memFree=17286660/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=9892 CPUtime=0 cores=0-7
/proc/3047/stat : 3047 (plingeling) R 3045 3047 3020 0 -1 4202496 542 1251 0 0 0 0 0 0 25 0 1 0 722545403 10129408 472 33554432000 4194304 4417452 140734920107120 18446744073709551615 239476694784 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/3047/statm: 2473 472 148 55 0 359 0

[startup+0.030101 s]
/proc/loadavg: 7.45 7.94 7.98 1/547 3047
/proc/meminfo: memFree=17286660/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=15728 CPUtime=0.02 cores=0-7
/proc/3047/stat : 3047 (plingeling) R 3045 3047 3020 0 -1 4202496 2090 1251 0 0 2 0 0 0 25 0 1 0 722545403 16105472 1956 33554432000 4194304 4417452 140734920107120 18446744073709551615 4231248 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/3047/statm: 3932 1956 149 55 0 1818 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 15728

[startup+0.939845 s]
/proc/loadavg: 7.45 7.94 7.98 1/547 3047
/proc/meminfo: memFree=17286660/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=169600 CPUtime=7.25 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6403 1251 0 0 725 0 0 0 25 0 9 0 722545403 173670400 3710 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 42400 3710 180 55 0 40286 0
Current children cumulated CPU time (s) 7.25
Current children cumulated vsize (KiB) 169600

[startup+1.03983 s]
/proc/loadavg: 7.45 7.94 7.98 1/547 3047
/proc/meminfo: memFree=17286660/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=170912 CPUtime=8.04 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 7055 1251 0 0 804 0 0 0 25 0 9 0 722545403 175013888 4038 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 42728 4038 180 55 0 40614 0
Current children cumulated CPU time (s) 8.04
Current children cumulated vsize (KiB) 170912

[startup+1.13981 s]
/proc/loadavg: 7.45 7.94 7.98 1/547 3047
/proc/meminfo: memFree=17286660/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=170300 CPUtime=8.84 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 7685 1251 0 0 884 0 0 0 25 0 9 0 722545403 174387200 3885 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 42575 3885 180 55 0 40461 0
Current children cumulated CPU time (s) 8.84
Current children cumulated vsize (KiB) 170300

[startup+1.50074 s]
/proc/loadavg: 7.45 7.94 7.98 1/547 3047
/proc/meminfo: memFree=17286660/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=174720 CPUtime=11.74 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 10190 1251 0 0 1173 1 0 0 25 0 9 0 722545403 178913280 5007 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 43680 5007 180 55 0 41566 0
Current children cumulated CPU time (s) 11.74
Current children cumulated vsize (KiB) 174720

[startup+3.10043 s]
/proc/loadavg: 7.45 7.94 7.98 9/556 3062
/proc/meminfo: memFree=17261160/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=179064 CPUtime=24.48 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 17960 1251 0 0 2446 2 0 0 25 0 9 0 722545403 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 44766 6096 180 55 0 42652 0
[pid=3047/tid=3054] ppid=3045 vsize=179064 CPUtime=3.06 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 2330 1251 0 0 306 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4366680 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=179064 CPUtime=3.02 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 2043 1251 0 0 302 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=179064 CPUtime=3.06 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 2102 1251 0 0 306 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4250602 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=179064 CPUtime=3.05 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 975 1251 0 0 305 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=179064 CPUtime=3.06 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 2230 1251 0 0 306 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=179064 CPUtime=3.06 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 2414 1251 0 0 306 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=179064 CPUtime=3.06 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 1954 1251 0 0 306 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=179064 CPUtime=3.05 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 1806 1251 0 0 305 0 0 0 25 0 9 0 722545406 183361536 6096 33554432000 4194304 4417452 140734920107120 18446744073709551615 4250571 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.48
Current children cumulated vsize (KiB) 179064

[startup+6.32082 s]
/proc/loadavg: 7.49 7.94 7.98 9/556 3062
/proc/meminfo: memFree=17253628/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=189588 CPUtime=50.18 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 35016 1251 0 0 5013 5 0 0 25 0 9 0 722545403 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 47397 8708 180 55 0 45283 0
[pid=3047/tid=3054] ppid=3045 vsize=189588 CPUtime=6.27 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 4868 1251 0 0 627 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=189588 CPUtime=6.18 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 4164 1251 0 0 618 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=189588 CPUtime=6.27 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 4201 1251 0 0 627 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=189588 CPUtime=6.27 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 2252 1251 0 0 627 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=189588 CPUtime=6.28 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 4895 1251 0 0 627 1 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=189588 CPUtime=6.27 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 4947 1251 0 0 627 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=189588 CPUtime=6.28 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 3338 1251 0 0 628 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=189588 CPUtime=6.26 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 4245 1251 0 0 626 0 0 0 25 0 9 0 722545406 194138112 8708 33554432000 4194304 4417452 140734920107120 18446744073709551615 4250807 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.18
Current children cumulated vsize (KiB) 189588

[startup+12.7006 s]
/proc/loadavg: 7.53 7.94 7.98 9/556 3062
/proc/meminfo: memFree=17237256/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=202704 CPUtime=101.13 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 65151 1251 0 0 10103 10 0 0 25 0 9 0 722545403 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 50676 11990 180 55 0 48562 0
[pid=3047/tid=3054] ppid=3045 vsize=202704 CPUtime=12.66 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 8396 1251 0 0 1265 1 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=202704 CPUtime=12.46 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 8712 1251 0 0 1245 1 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=202704 CPUtime=12.66 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 7688 1251 0 0 1265 1 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=202704 CPUtime=12.64 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 4271 1251 0 0 1264 0 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=202704 CPUtime=12.66 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 8023 1251 0 0 1265 1 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=202704 CPUtime=12.66 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 10256 1251 0 0 1265 1 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=202704 CPUtime=12.65 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 7228 1251 0 0 1265 0 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=202704 CPUtime=12.65 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 8471 1251 0 0 1264 1 0 0 25 0 9 0 722545406 207568896 11990 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.13
Current children cumulated vsize (KiB) 202704

[startup+25.5011 s]
/proc/loadavg: 7.63 7.94 7.98 9/556 3063
/proc/meminfo: memFree=17217964/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=223672 CPUtime=203.29 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 120237 1251 0 0 20309 20 0 0 25 0 9 0 722545403 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 55918 17237 180 55 0 53804 0
[pid=3047/tid=3054] ppid=3045 vsize=223672 CPUtime=25.46 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 14690 1251 0 0 2544 2 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=223672 CPUtime=25.02 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 17043 1251 0 0 2500 2 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4236812 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=223672 CPUtime=25.45 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 14667 1251 0 0 2543 2 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=223672 CPUtime=25.44 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 7482 1251 0 0 2544 0 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=223672 CPUtime=25.46 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 16871 1251 0 0 2543 3 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=223672 CPUtime=25.46 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 18769 1251 0 0 2544 2 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364495 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=223672 CPUtime=25.46 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 13411 1251 0 0 2544 2 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=223672 CPUtime=25.44 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 15198 1251 0 0 2542 2 0 0 25 0 9 0 722545406 229040128 17237 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.29
Current children cumulated vsize (KiB) 223672

[startup+51.1012 s]
/proc/loadavg: 7.83 7.96 7.98 9/556 3063
/proc/meminfo: memFree=17206852/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=237984 CPUtime=407.63 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 201172 1251 0 0 40730 33 0 0 25 0 9 0 722545403 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 59496 20835 180 55 0 57382 0
[pid=3047/tid=3054] ppid=3045 vsize=237984 CPUtime=51.06 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 25143 1251 0 0 5102 4 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=237984 CPUtime=50.16 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 30594 1251 0 0 5012 4 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=237984 CPUtime=51.06 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 23650 1251 0 0 5102 4 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=237984 CPUtime=51.04 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 13496 1251 0 0 5103 1 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=237984 CPUtime=51.06 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 26861 1251 0 0 5100 6 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=237984 CPUtime=51.06 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 31976 1251 0 0 5102 4 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=237984 CPUtime=51.06 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 21807 1251 0 0 5103 3 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4356843 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=237984 CPUtime=51.03 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 25539 1251 0 0 5099 4 0 0 25 0 9 0 722545406 243695616 20835 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.63
Current children cumulated vsize (KiB) 237984

[startup+102.3 s]
/proc/loadavg: 8.00 7.98 7.99 9/556 3065

################
# More data... #
################

/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 611639 1251 0 0 466147 95 0 0 25 0 9 0 722545406 782684160 152408 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=764340 CPUtime=4662.43 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 859678 1251 0 0 466110 133 0 0 25 0 9 0 722545406 782684160 152408 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=764340 CPUtime=4661.62 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 705069 1251 0 0 466050 112 0 0 25 0 9 0 722545406 782684160 152408 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=764340 CPUtime=4662.44 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 922870 1251 0 0 466112 132 0 0 25 0 9 0 722545406 782684160 152408 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=764340 CPUtime=4661.03 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 783863 1251 0 0 465975 128 0 0 25 0 9 0 722545406 782684160 152408 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37212.5
Current children cumulated vsize (KiB) 764340

[startup+4722.3 s]
/proc/loadavg: 8.10 8.08 8.01 9/556 3218
/proc/meminfo: memFree=16677136/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=789368 CPUtime=37691.4 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6309185 1251 0 0 3768176 962 0 0 25 0 9 0 722545403 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 197342 158665 180 55 0 195228 0
[pid=3047/tid=3054] ppid=3045 vsize=789368 CPUtime=4722.28 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 750369 1251 0 0 472113 115 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=789368 CPUtime=4636.61 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 851018 1251 0 0 463538 123 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=789368 CPUtime=4722.44 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 784715 1251 0 0 472129 115 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=789368 CPUtime=4722.43 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 616089 1251 0 0 472147 96 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=789368 CPUtime=4722.44 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 868658 1251 0 0 472109 135 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=789368 CPUtime=4721.62 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 711728 1251 0 0 472049 113 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=789368 CPUtime=4722.44 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 929939 1251 0 0 472112 132 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=789368 CPUtime=4721.01 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 794563 1251 0 0 471972 129 0 0 25 0 9 0 722545406 808312832 158665 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37691.4
Current children cumulated vsize (KiB) 789368

[startup+4782.3 s]
/proc/loadavg: 8.04 8.06 8.01 9/556 3219
/proc/meminfo: memFree=16675076/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=743520 CPUtime=38170.3 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6358290 1251 0 0 3816060 968 0 0 25 0 9 0 722545403 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 185880 147203 180 55 0 183766 0
[pid=3047/tid=3054] ppid=3045 vsize=743520 CPUtime=4782.29 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 753933 1251 0 0 478113 116 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=743520 CPUtime=4695.53 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 860406 1251 0 0 469429 124 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=743520 CPUtime=4782.44 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 788106 1251 0 0 478129 115 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=743520 CPUtime=4782.43 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 619316 1251 0 0 478147 96 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=743520 CPUtime=4782.44 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 879407 1251 0 0 478108 136 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4366447 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=743520 CPUtime=4781.63 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 718423 1251 0 0 478049 114 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=743520 CPUtime=4782.44 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 934875 1251 0 0 478111 133 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=743520 CPUtime=4780.99 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 801718 1251 0 0 477969 130 0 0 25 0 9 0 722545406 761364480 147203 33554432000 4194304 4417452 140734920107120 18446744073709551615 4242818 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38170.3
Current children cumulated vsize (KiB) 743520

[startup+4842.3 s]
/proc/loadavg: 8.01 8.05 8.00 9/556 3221
/proc/meminfo: memFree=16672756/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=770556 CPUtime=38649.2 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6412449 1251 0 0 3863944 975 0 0 25 0 9 0 722545403 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 192639 153965 180 55 0 190525 0
[pid=3047/tid=3054] ppid=3045 vsize=770556 CPUtime=4842.28 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 757786 1251 0 0 484112 116 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=770556 CPUtime=4754.43 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 869844 1251 0 0 475318 125 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=770556 CPUtime=4842.44 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 798266 1251 0 0 484128 116 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=770556 CPUtime=4842.43 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 623763 1251 0 0 484146 97 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=770556 CPUtime=4842.44 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 885254 1251 0 0 484108 136 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=770556 CPUtime=4841.63 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 723269 1251 0 0 484048 115 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=770556 CPUtime=4842.44 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 945975 1251 0 0 484110 134 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=770556 CPUtime=4840.98 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 806186 1251 0 0 483967 131 0 0 25 0 9 0 722545406 789049344 153965 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38649.2
Current children cumulated vsize (KiB) 770556

[startup+4902.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/556 3223
/proc/meminfo: memFree=16718020/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=725176 CPUtime=39128.1 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6456354 1251 0 0 3911833 981 0 0 25 0 9 0 722545403 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 181294 142615 180 55 0 179180 0
[pid=3047/tid=3054] ppid=3045 vsize=725176 CPUtime=4902.31 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 761902 1251 0 0 490114 117 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=725176 CPUtime=4813.35 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 870417 1251 0 0 481210 125 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=725176 CPUtime=4902.46 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 807620 1251 0 0 490129 117 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=725176 CPUtime=4902.45 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 627297 1251 0 0 490148 97 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=725176 CPUtime=4902.46 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 890203 1251 0 0 490109 137 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=725176 CPUtime=4901.65 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 730212 1251 0 0 490050 115 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=725176 CPUtime=4902.47 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 956683 1251 0 0 490111 136 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=725176 CPUtime=4900.99 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 809914 1251 0 0 489967 132 0 0 25 0 9 0 722545406 742580224 142615 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39128.1
Current children cumulated vsize (KiB) 725176

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/556 3224
/proc/meminfo: memFree=16655128/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=812412 CPUtime=39607 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6527978 1251 0 0 3959706 995 0 0 25 0 9 0 722545403 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 203103 164425 180 55 0 200989 0
[pid=3047/tid=3054] ppid=3045 vsize=812412 CPUtime=4962.29 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 776679 1251 0 0 496110 119 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=812412 CPUtime=4872.26 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 879340 1251 0 0 487100 126 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=812412 CPUtime=4962.44 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 819937 1251 0 0 496124 120 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=812412 CPUtime=4962.44 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 629411 1251 0 0 496146 98 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=812412 CPUtime=4962.44 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 902776 1251 0 0 496105 139 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=812412 CPUtime=4961.63 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 734356 1251 0 0 496047 116 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=812412 CPUtime=4962.45 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 963355 1251 0 0 496107 138 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=812412 CPUtime=4960.94 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 820018 1251 0 0 495960 134 0 0 25 0 9 0 722545406 831909888 164425 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39607
Current children cumulated vsize (KiB) 812412



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.6 s]
/proc/loadavg: 8.07 8.06 8.00 9/556 3226
/proc/meminfo: memFree=16667024/32950928 swapFree=67056576/67111528
[pid=3047] ppid=3045 vsize=775436 CPUtime=40000.5 cores=0-7
/proc/3047/stat : 3047 (plingeling) S 3045 3047 3020 0 -1 4202496 6574651 1251 0 0 3999053 1001 0 0 25 0 9 0 722545403 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/3047/statm: 193859 155182 180 55 0 191745 0
[pid=3047/tid=3054] ppid=3045 vsize=775436 CPUtime=5011.6 cores=0-7
/proc/3047/task/3054/stat : 3054 (plingeling) R 3045 3047 3020 0 -1 4202560 783321 1251 0 0 501040 120 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3047/tid=3055] ppid=3045 vsize=775436 CPUtime=4920.66 cores=0-7
/proc/3047/task/3055/stat : 3055 (plingeling) R 3045 3047 3020 0 -1 4202560 888317 1251 0 0 491939 127 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3047/tid=3056] ppid=3045 vsize=775436 CPUtime=5011.77 cores=0-7
/proc/3047/task/3056/stat : 3056 (plingeling) R 3045 3047 3020 0 -1 4202560 820655 1251 0 0 501057 120 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=3047/tid=3057] ppid=3045 vsize=775436 CPUtime=5011.76 cores=0-7
/proc/3047/task/3057/stat : 3057 (plingeling) R 3045 3047 3020 0 -1 4202560 635960 1251 0 0 501078 98 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3047/tid=3059] ppid=3045 vsize=775436 CPUtime=5011.77 cores=0-7
/proc/3047/task/3059/stat : 3059 (plingeling) R 3045 3047 3020 0 -1 4202560 906620 1251 0 0 501037 140 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=3047/tid=3060] ppid=3045 vsize=775436 CPUtime=5010.95 cores=0-7
/proc/3047/task/3060/stat : 3060 (plingeling) R 3045 3047 3020 0 -1 4202560 738754 1251 0 0 500979 116 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3047/tid=3061] ppid=3045 vsize=775436 CPUtime=5011.78 cores=0-7
/proc/3047/task/3061/stat : 3061 (plingeling) R 3045 3047 3020 0 -1 4202560 967881 1251 0 0 501039 139 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3047/tid=3062] ppid=3045 vsize=775436 CPUtime=5010.26 cores=0-7
/proc/3047/task/3062/stat : 3062 (plingeling) R 3045 3047 3020 0 -1 4202560 831037 1251 0 0 500890 136 0 0 25 0 9 0 722545406 794046464 155182 33554432000 4194304 4417452 140734920107120 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 775436

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.66
CPU time (s): 40000.7
CPU user time (s): 39990.7
CPU system time (s): 10.0445
CPU usage (%): 798.154
Max. virtual memory (cumulated for all children) (KiB): 851096

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39990.7
system time used= 10.0445
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6575902
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 215
involuntary context switches= 192486

runsolver used 25.8061 second user time and 65.3271 second system time

The end

Launcher Data 

Begin job on node148 at 2011-05-23 02:11:23
IDJOB=3394928
IDBENCH=87739
IDSOLVER=1851
FILE ID=node148/3394928-1306109483
RUNJOBID= node148-1306109483-3035
PBS_JOBID= 13388038
Free space on /tmp= 73160 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT02/handmade/simon/satex-challenges/par32-3.shuffled.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394928-1306109483/watcher-3394928-1306109483 -o /tmp/evaluation-result-3394928-1306109483/solver-3394928-1306109483 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394928-1306109483.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 0115a0e26236f3f01aa829c2004452c1
RANDOM SEED=1651279500

node148.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17287064 kB
Buffers:        376280 kB
Cached:        8080488 kB
SwapCached:       8480 kB
Active:        9663112 kB
Inactive:      5621088 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17287064 kB
SwapTotal:    67111528 kB
SwapFree:     67056576 kB
Dirty:             396 kB
Writeback:           0 kB
AnonPages:     6824872 kB
Mapped:          43016 kB
Slab:           249052 kB
PageTables:      57476 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 170358656 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73160 MiB
End job on node148 at 2011-05-23 03:34:57