Trace number 3371493

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.27 1210.44

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
MD5SUM6c2d25eda70423643946d76935ceb950
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1512832
Number of clauses8233086
Sum of the clauses size19530286
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 28165680
Number of clauses of size 31280
Number of clauses of size 41600
Number of clauses of size 51600
Number of clauses of size over 562080

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371493-1305421701/watcher-3371493-1305421701 -o /tmp/evaluation-result-3371493-1305421701/solver-3371493-1305421701 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371493-1305421701.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.58 5.40 7.00 3/279 6370
/proc/meminfo: memFree=18592016/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=11796 CPUtime=0 cores=0,2,4,6
/proc/6370/stat : 6370 (runsolver) R 6368 6370 5706 0 -1 4202560 40 0 0 0 0 0 0 0 19 0 1 0 653770638 12079104 49 33554432000 4194304 5432360 140736444978528 18446744073709551615 4881623 0 2147483391 4096 24578 0 0 0 17 6 0 0 0
/proc/6370/statm: 2949 49 30 303 0 2643 0

[startup+0.0325901 s]
/proc/loadavg: 3.58 5.40 7.00 3/279 6370
/proc/meminfo: memFree=18592016/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=31832 CPUtime=0.01 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 6047 1261 0 0 1 0 0 0 19 0 1 0 653770638 32595968 5978 33554432000 4194304 4417452 140734196230592 18446744073709551615 274664427100 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 7958 5978 145 55 0 5844 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 31832

[startup+0.100618 s]
/proc/loadavg: 3.58 5.40 7.00 3/279 6370
/proc/meminfo: memFree=18592016/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=40652 CPUtime=0.08 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 8260 1261 0 0 7 1 0 0 19 0 1 0 653770638 41627648 8186 33554432000 4194304 4417452 140734196230592 18446744073709551615 4209539 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 10163 8186 148 55 0 8049 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 40652

[startup+0.300582 s]
/proc/loadavg: 3.58 5.40 7.00 3/279 6370
/proc/meminfo: memFree=18592016/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=70252 CPUtime=0.28 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 15669 1261 0 0 26 2 0 0 20 0 1 0 653770638 71938048 15587 33554432000 4194304 4417452 140734196230592 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 17563 15587 149 55 0 15449 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 70252

[startup+0.70051 s]
/proc/loadavg: 3.58 5.40 7.00 3/279 6370
/proc/meminfo: memFree=18592016/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=116140 CPUtime=0.68 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 27258 1261 0 0 64 4 0 0 20 0 1 0 653770638 118927360 27059 33554432000 4194304 4417452 140734196230592 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 29035 27059 149 55 0 26921 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 116140

[startup+1.50034 s]
/proc/loadavg: 3.58 5.40 7.00 3/278 6376
/proc/meminfo: memFree=18441324/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=223744 CPUtime=1.48 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 54207 1261 0 0 141 7 0 0 23 0 1 0 653770638 229113856 53960 33554432000 4194304 4417452 140734196230592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 55936 53960 149 55 0 53822 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 223744

[startup+3.10108 s]
/proc/loadavg: 3.58 5.40 7.00 3/283 6387
/proc/meminfo: memFree=18277960/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=483840 CPUtime=3.08 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 119231 1261 0 0 292 16 0 0 25 0 1 0 653770638 495452160 118984 33554432000 4194304 4417452 140734196230592 18446744073709551615 4259193 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 120960 118984 149 55 0 118846 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 483840

[startup+6.30053 s]
/proc/loadavg: 3.70 5.39 6.99 3/283 6387
/proc/meminfo: memFree=17560732/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=696832 CPUtime=6.28 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) R 6368 6370 5706 0 -1 4202496 172479 1261 0 0 604 24 0 0 25 0 1 0 653770638 713555968 172232 33554432000 4194304 4417452 140734196230592 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6370/statm: 174208 172232 149 55 0 172094 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 696832

[startup+12.7003 s]
/proc/loadavg: 3.80 5.39 6.98 11/291 6395
/proc/meminfo: memFree=15885612/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1363560 CPUtime=21.3 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 448816 1261 0 0 2072 58 0 0 25 0 5 0 653770638 1396285440 328665 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 340890 328665 168 55 0 338776 0
[pid=6370/tid=6388] ppid=6368 vsize=1363560 CPUtime=2.81 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 38436 1261 0 0 277 4 0 0 23 0 5 0 653771602 1396285440 328665 33554432000 4194304 4417452 140734196230592 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=1363560 CPUtime=3.01 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 38445 1261 0 0 296 5 0 0 25 0 5 0 653771602 1396285440 328665 33554432000 4194304 4417452 140734196230592 18446744073709551615 4287606 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=1363560 CPUtime=3.03 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 38402 1261 0 0 299 4 0 0 22 0 5 0 653771602 1396285440 328665 33554432000 4194304 4417452 140734196230592 18446744073709551615 4287554 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=1363560 CPUtime=2.8 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 28946 1261 0 0 277 3 0 0 24 0 5 0 653771602 1396285440 328665 33554432000 4194304 4417452 140734196230592 18446744073709551615 274664866218 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 21.3
Current children cumulated vsize (KiB) 1363560

[startup+25.5011 s]
/proc/loadavg: 5.99 5.79 7.08 12/291 6395
/proc/meminfo: memFree=15824412/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1346552 CPUtime=72.38 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 903402 1261 0 0 7111 127 0 0 25 0 5 0 653770638 1378869248 308693 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 336638 308693 179 55 0 334524 0
[pid=6370/tid=6388] ppid=6368 vsize=1346552 CPUtime=15.6 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 169224 1261 0 0 1537 23 0 0 25 0 5 0 653771602 1378869248 308693 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331837 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=1346552 CPUtime=15.68 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 211056 1261 0 0 1537 31 0 0 25 0 5 0 653771602 1378869248 308693 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=1346552 CPUtime=15.82 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 181394 1261 0 0 1557 25 0 0 25 0 5 0 653771602 1378869248 308693 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=1346552 CPUtime=15.59 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 37141 1261 0 0 1554 5 0 0 25 0 5 0 653771602 1378869248 308693 33554432000 4194304 4417452 140734196230592 18446744073709551615 4260027 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 72.38
Current children cumulated vsize (KiB) 1346552

[startup+51.1007 s]
/proc/loadavg: 8.58 6.42 7.26 11/291 6396
/proc/meminfo: memFree=15695032/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1581548 CPUtime=174.53 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 976922 1261 0 0 17317 136 0 0 25 0 5 0 653770638 1619505152 371551 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 395387 371551 180 55 0 393273 0
[pid=6370/tid=6388] ppid=6368 vsize=1581548 CPUtime=41.2 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 175928 1261 0 0 4096 24 0 0 25 0 5 0 653771602 1619505152 371551 33554432000 4194304 4417452 140734196230592 18446744073709551615 4253405 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=1581548 CPUtime=41.05 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 217762 1261 0 0 4073 32 0 0 25 0 5 0 653771602 1619505152 371551 33554432000 4194304 4417452 140734196230592 18446744073709551615 4249233 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=1581548 CPUtime=41.43 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 203785 1261 0 0 4115 28 0 0 25 0 5 0 653771602 1619505152 371551 33554432000 4194304 4417452 140734196230592 18446744073709551615 4235424 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=1581548 CPUtime=41.17 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 74860 1261 0 0 4108 9 0 0 25 0 5 0 653771602 1619505152 371551 33554432000 4194304 4417452 140734196230592 18446744073709551615 4365538 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 174.53
Current children cumulated vsize (KiB) 1581548

[startup+102.311 s]
/proc/loadavg: 8.70 6.84 7.36 5/283 6413
/proc/meminfo: memFree=17326132/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1505104 CPUtime=378.89 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 1030436 1261 0 0 37745 144 0 0 25 0 5 0 653770638 1541226496 352440 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 376276 352440 180 55 0 374162 0
[pid=6370/tid=6388] ppid=6368 vsize=1505104 CPUtime=92.42 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 189844 1261 0 0 9215 27 0 0 25 0 5 0 653771602 1541226496 352440 33554432000 4194304 4417452 140734196230592 18446744073709551615 4281633 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=1505104 CPUtime=91.8 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 222626 1261 0 0 9147 33 0 0 25 0 5 0 653771602 1541226496 352440 33554432000 4194304 4417452 140734196230592 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=1505104 CPUtime=92.64 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 207625 1261 0 0 9235 29 0 0 25 0 5 0 653771602 1541226496 352440 33554432000 4194304 4417452 140734196230592 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=1505104 CPUtime=92.37 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 105754 1261 0 0 9224 13 0 0 25 0 5 0 653771602 1541226496 352440 33554432000 4194304 4417452 140734196230592 18446744073709551615 4357865 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 378.89
Current children cumulated vsize (KiB) 1505104

[startup+162.3 s]
/proc/loadavg: 10.11 7.55 7.57 9/291 6435
/proc/meminfo: memFree=15824140/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1812460 CPUtime=618.31 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 1273596 1261 0 0 61655 176 0 0 25 0 5 0 653770638 1855959040 429279 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 453115 429279 180 55 0 451001 0
[pid=6370/tid=6388] ppid=6368 vsize=1812460 CPUtime=152.41 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 272060 1261 0 0 15205 36 0 0 25 0 5 0 653771602 1855959040 429279 33554432000 4194304 4417452 140734196230592 18446744073709551615 4357970 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=1812460 CPUtime=151.25 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 297018 1261 0 0 15082 43 0 0 25 0 5 0 653771602 1855959040 429279 33554432000 4194304 4417452 140734196230592 18446744073709551615 4274913 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=1812460 CPUtime=152.62 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 273970 1261 0 0 15224 38 0 0 25 0 5 0 653771602 1855959040 429279 33554432000 4194304 4417452 140734196230592 18446744073709551615 4245664 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=1812460 CPUtime=152.37 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 125961 1261 0 0 15220 17 0 0 25 0 5 0 653771602 1855959040 429279 33554432000 4194304 4417452 140734196230592 18446744073709551615 4302940 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 618.31
Current children cumulated vsize (KiB) 1812460

[startup+222.301 s]
/proc/loadavg: 8.81 7.64 7.60 9/291 6437
/proc/meminfo: memFree=15584932/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1759428 CPUtime=857.75 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 1303757 1261 0 0 85595 180 0 0 25 0 5 0 653770638 1801654272 416021 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 439857 416021 180 55 0 437743 0
[pid=6370/tid=6388] ppid=6368 vsize=1759428 CPUtime=212.4 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 277442 1261 0 0 21203 37 0 0 25 0 5 0 653771602 1801654272 416021 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=1759428 CPUtime=210.69 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 297018 1261 0 0 21026 43 0 0 25 0 5 0 653771602 1801654272 416021 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=1759428 CPUtime=212.62 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 290528 1261 0 0 21223 39 0 0 25 0 5 0 653771602 1801654272 416021 33554432000 4194304 4417452 140734196230592 18446744073709551615 4254977 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=1759428 CPUtime=212.36 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 134182 1261 0 0 21218 18 0 0 25 0 5 0 653771602 1801654272 416021 33554432000 4194304 4417452 140734196230592 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 857.75
Current children cumulated vsize (KiB) 1759428

[startup+282.301 s]
/proc/loadavg: 8.30 7.70 7.62 9/291 6438
/proc/meminfo: memFree=15508660/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=1709992 CPUtime=1097.19 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 1337112 1261 0 0 109534 185 0 0 25 0 5 0 653770638 1751031808 403662 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 427498 403662 180 55 0 425384 0
[pid=6370/tid=6388] ppid=6368 vsize=1709992 CPUtime=272.37 cores=0,2,4,6

################
# More data... #
################

/proc/meminfo: memFree=14237844/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2498700 CPUtime=3012.51 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 2179675 1261 0 0 300946 305 0 0 25 0 5 0 653770638 2558668800 600717 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 624675 600717 180 55 0 622561 0
[pid=6370/tid=6388] ppid=6368 vsize=2498700 CPUtime=752.02 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 444532 1261 0 0 75142 60 0 0 25 0 5 0 653771602 2558668800 600717 33554432000 4194304 4417452 140734196230592 18446744073709551615 4365584 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2498700 CPUtime=747.84 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 387813 1261 0 0 74727 57 0 0 25 0 5 0 653771602 2558668800 600717 33554432000 4194304 4417452 140734196230592 18446744073709551615 4358947 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2498700 CPUtime=750.6 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 510195 1261 0 0 74987 73 0 0 25 0 5 0 653771602 2558668800 600717 33554432000 4194304 4417452 140734196230592 18446744073709551615 4281201 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2498700 CPUtime=752.39 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 532548 1261 0 0 75166 73 0 0 25 0 5 0 653771602 2558668800 600717 33554432000 4194304 4417452 140734196230592 18446744073709551615 4357204 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3012.51
Current children cumulated vsize (KiB) 2498700

[startup+822.301 s]
/proc/loadavg: 8.12 8.04 7.82 9/291 6453
/proc/meminfo: memFree=13965284/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2650476 CPUtime=3251.89 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 2447149 1261 0 0 324845 344 0 0 25 0 5 0 653770638 2714087424 638594 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 662619 638594 180 55 0 660505 0
[pid=6370/tid=6388] ppid=6368 vsize=2650476 CPUtime=811.97 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 495006 1261 0 0 81132 65 0 0 25 0 5 0 653771602 2714087424 638594 33554432000 4194304 4417452 140734196230592 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2650476 CPUtime=807.84 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 479001 1261 0 0 80712 72 0 0 25 0 5 0 653771602 2714087424 638594 33554432000 4194304 4417452 140734196230592 18446744073709551615 4320947 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2650476 CPUtime=810.04 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 551677 1261 0 0 80925 79 0 0 25 0 5 0 653771602 2714087424 638594 33554432000 4194304 4417452 140734196230592 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2650476 CPUtime=812.37 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 616878 1261 0 0 81152 85 0 0 25 0 5 0 653771602 2714087424 638594 33554432000 4194304 4417452 140734196230592 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3251.89
Current children cumulated vsize (KiB) 2650476

[startup+882.301 s]
/proc/loadavg: 8.09 8.05 7.83 9/291 6454
/proc/meminfo: memFree=14372752/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2461808 CPUtime=3491.3 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 2517218 1261 0 0 348773 357 0 0 25 0 5 0 653770638 2520891392 591488 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 615452 591488 180 55 0 613338 0
[pid=6370/tid=6388] ppid=6368 vsize=2461808 CPUtime=871.93 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 518283 1261 0 0 87123 70 0 0 25 0 5 0 653771602 2520891392 591488 33554432000 4194304 4417452 140734196230592 18446744073709551615 4366523 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2461808 CPUtime=867.84 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 479001 1261 0 0 86712 72 0 0 25 0 5 0 653771602 2520891392 591488 33554432000 4194304 4417452 140734196230592 18446744073709551615 4236961 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2461808 CPUtime=869.5 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 570427 1261 0 0 86867 83 0 0 25 0 5 0 653771602 2520891392 591488 33554432000 4194304 4417452 140734196230592 18446744073709551615 4309183 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2461808 CPUtime=872.38 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 644920 1261 0 0 87148 90 0 0 25 0 5 0 653771602 2520891392 591488 33554432000 4194304 4417452 140734196230592 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3491.3
Current children cumulated vsize (KiB) 2461808

[startup+942.3 s]
/proc/loadavg: 8.03 8.04 7.83 9/291 6456
/proc/meminfo: memFree=14123624/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2612620 CPUtime=3730.71 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 2563182 1261 0 0 372708 363 0 0 25 0 5 0 653770638 2675322880 629264 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 653155 629264 180 55 0 651041 0
[pid=6370/tid=6388] ppid=6368 vsize=2612620 CPUtime=931.88 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 527155 1261 0 0 93116 72 0 0 25 0 5 0 653771602 2675322880 629264 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2612620 CPUtime=927.84 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 479001 1261 0 0 92712 72 0 0 25 0 5 0 653771602 2675322880 629264 33554432000 4194304 4417452 140734196230592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2612620 CPUtime=928.96 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 570433 1261 0 0 92813 83 0 0 25 0 5 0 653771602 2675322880 629264 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2612620 CPUtime=932.38 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 682006 1261 0 0 93143 95 0 0 25 0 5 0 653771602 2675322880 629264 33554432000 4194304 4417452 140734196230592 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3730.71
Current children cumulated vsize (KiB) 2612620

[startup+1002.3 s]
/proc/loadavg: 8.05 8.04 7.85 9/291 6457
/proc/meminfo: memFree=13607908/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2627500 CPUtime=3970.13 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 2699508 1261 0 0 396630 383 0 0 25 0 5 0 653770638 2690560000 633012 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 656875 633012 180 55 0 654761 0
[pid=6370/tid=6388] ppid=6368 vsize=2627500 CPUtime=991.84 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 527155 1261 0 0 99111 73 0 0 25 0 5 0 653771602 2690560000 633012 33554432000 4194304 4417452 140734196230592 18446744073709551615 4252336 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2627500 CPUtime=987.83 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 544693 1261 0 0 98704 79 0 0 25 0 5 0 653771602 2690560000 633012 33554432000 4194304 4417452 140734196230592 18446744073709551615 4281649 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2627500 CPUtime=988.41 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 585682 1261 0 0 98756 85 0 0 25 0 5 0 653771602 2690560000 633012 33554432000 4194304 4417452 140734196230592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2627500 CPUtime=992.38 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 737391 1261 0 0 99134 104 0 0 25 0 5 0 653771602 2690560000 633012 33554432000 4194304 4417452 140734196230592 18446744073709551615 4250633 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3970.13
Current children cumulated vsize (KiB) 2627500

[startup+1062.3 s]
/proc/loadavg: 8.02 8.03 7.85 9/291 6459
/proc/meminfo: memFree=13432920/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2823612 CPUtime=4209.53 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 2816010 1261 0 0 420552 401 0 0 25 0 5 0 653770638 2891378688 682047 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 705903 682047 180 55 0 703789 0
[pid=6370/tid=6388] ppid=6368 vsize=2823612 CPUtime=1051.78 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 537190 1261 0 0 105104 74 0 0 25 0 5 0 653771602 2891378688 682047 33554432000 4194304 4417452 140734196230592 18446744073709551615 4237157 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2823612 CPUtime=1047.83 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 544693 1261 0 0 104704 79 0 0 25 0 5 0 653771602 2891378688 682047 33554432000 4194304 4417452 140734196230592 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2823612 CPUtime=1047.87 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 619231 1261 0 0 104698 89 0 0 25 0 5 0 653771602 2891378688 682047 33554432000 4194304 4417452 140734196230592 18446744073709551615 4317305 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2823612 CPUtime=1052.38 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 810309 1261 0 0 105122 116 0 0 25 0 5 0 653771602 2891378688 682047 33554432000 4194304 4417452 140734196230592 18446744073709551615 4355841 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4209.53
Current children cumulated vsize (KiB) 2823612

[startup+1122.3 s]
/proc/loadavg: 8.15 8.07 7.87 9/291 6461
/proc/meminfo: memFree=12690772/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=3554644 CPUtime=4448.95 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 3101076 1261 0 0 444454 441 0 0 25 0 5 0 653770638 3639955456 864805 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 888661 864805 180 55 0 886547 0
[pid=6370/tid=6388] ppid=6368 vsize=3554644 CPUtime=1111.74 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 641670 1261 0 0 111087 87 0 0 25 0 5 0 653771602 3639955456 864805 33554432000 4194304 4417452 140734196230592 18446744073709551615 4342557 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=3554644 CPUtime=1107.84 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 544693 1261 0 0 110704 80 0 0 25 0 5 0 653771602 3639955456 864805 33554432000 4194304 4417452 140734196230592 18446744073709551615 4232791 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=3554644 CPUtime=1107.32 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 688917 1261 0 0 110632 100 0 0 25 0 5 0 653771602 3639955456 864805 33554432000 4194304 4417452 140734196230592 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=3554644 CPUtime=1112.38 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 921209 1261 0 0 111106 132 0 0 25 0 5 0 653771602 3639955456 864805 33554432000 4194304 4417452 140734196230592 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4448.95
Current children cumulated vsize (KiB) 3554644

[startup+1182.3 s]
/proc/loadavg: 8.21 8.09 7.89 9/291 6462
/proc/meminfo: memFree=13320552/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=2663088 CPUtime=4688.37 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 3159390 1261 0 0 468383 454 0 0 25 0 5 0 653770638 2727002112 641916 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 665772 641916 180 55 0 663658 0
[pid=6370/tid=6388] ppid=6368 vsize=2663088 CPUtime=1171.7 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 659550 1261 0 0 117080 90 0 0 25 0 5 0 653771602 2727002112 641916 33554432000 4194304 4417452 140734196230592 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=2663088 CPUtime=1167.84 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 544693 1261 0 0 116704 80 0 0 25 0 5 0 653771602 2727002112 641916 33554432000 4194304 4417452 140734196230592 18446744073709551615 4236634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=2663088 CPUtime=1166.78 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 692883 1261 0 0 116577 101 0 0 25 0 5 0 653771602 2727002112 641916 33554432000 4194304 4417452 140734196230592 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=2663088 CPUtime=1172.38 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 957677 1261 0 0 117097 141 0 0 25 0 5 0 653771602 2727002112 641916 33554432000 4194304 4417452 140734196230592 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4688.37
Current children cumulated vsize (KiB) 2663088



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1210.3 s]
/proc/loadavg: 8.18 8.09 7.90 9/291 6463
/proc/meminfo: memFree=13289556/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=3028884 CPUtime=4800.1 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) S 6368 6370 5706 0 -1 4202496 3284402 1261 0 0 479536 474 0 0 25 0 5 0 653770638 3101577216 733365 33554432000 4194304 4417452 140734196230592 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 757221 733365 180 55 0 755107 0
[pid=6370/tid=6388] ppid=6368 vsize=3028884 CPUtime=1199.68 cores=0,2,4,6
/proc/6370/task/6388/stat : 6388 (plingeling) R 6368 6370 5706 0 -1 4202560 706741 1261 0 0 119871 97 0 0 25 0 5 0 653771602 3101577216 733365 33554432000 4194304 4417452 140734196230592 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6370/tid=6389] ppid=6368 vsize=3028884 CPUtime=1195.85 cores=0,2,4,6
/proc/6370/task/6389/stat : 6389 (plingeling) R 6368 6370 5706 0 -1 4202560 611020 1261 0 0 119494 91 0 0 25 0 5 0 653771602 3101577216 733365 33554432000 4194304 4417452 140734196230592 18446744073709551615 4255615 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=3028884 CPUtime=1194.53 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4202560 696855 1261 0 0 119351 102 0 0 25 0 5 0 653771602 3101577216 733365 33554432000 4194304 4417452 140734196230592 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6370/tid=6391] ppid=6368 vsize=3028884 CPUtime=1200.39 cores=0,2,4,6
/proc/6370/task/6391/stat : 6391 (plingeling) R 6368 6370 5706 0 -1 4202560 965199 1261 0 0 119897 142 0 0 25 0 5 0 653771602 3101577216 733365 33554432000 4194304 4417452 140734196230592 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.1
Current children cumulated vsize (KiB) 3028884

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1210.42 s]
/proc/loadavg: 8.18 8.09 7.90 9/291 6463
/proc/meminfo: memFree=13289556/32951132 swapFree=67093824/67111528
[pid=6370] ppid=6368 vsize=0 CPUtime=4800.24 cores=0,2,4,6
/proc/6370/stat : 6370 (plingeling) Z 6368 6370 5706 0 -1 4203548 3284402 1261 0 0 479540 484 0 0 18 0 2 0 653770638 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/6370/statm: 0 0 0 0 0 0 0
[pid=6370/tid=6390] ppid=6368 vsize=0 CPUtime=1194.63 cores=0,2,4,6
/proc/6370/task/6390/stat : 6390 (plingeling) R 6368 6370 5706 0 -1 4203588 696855 1261 0 0 119351 112 0 0 25 0 2 0 653771602 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1210.44
CPU time (s): 4800.27
CPU user time (s): 4795.4
CPU system time (s): 4.86826
CPU usage (%): 396.574
Max. virtual memory (cumulated for all children) (KiB): 3652084

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.4
system time used= 4.86826
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3285663
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 155
involuntary context switches= 20357

runsolver used 3.01654 second user time and 7.99878 second system time

The end

Launcher Data 

Begin job on node131 at 2011-05-15 03:08:30
IDJOB=3371493
IDBENCH=82404
IDSOLVER=1848
FILE ID=node131/3371493-1305421701
RUNJOBID= node131-1305421701-6355
PBS_JOBID= 13324564
Free space on /tmp= 73352 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371493-1305421701/watcher-3371493-1305421701 -o /tmp/evaluation-result-3371493-1305421701/solver-3371493-1305421701 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371493-1305421701.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6c2d25eda70423643946d76935ceb950
RANDOM SEED=1393790238

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      18592056 kB
Buffers:        278140 kB
Cached:        4090544 kB
SwapCached:       1504 kB
Active:       10741000 kB
Inactive:      3348744 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      18592056 kB
SwapTotal:    67111528 kB
SwapFree:     67093824 kB
Dirty:          345660 kB
Writeback:           0 kB
AnonPages:     9719372 kB
Mapped:          26340 kB
Slab:           176460 kB
PageTables:      31028 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 27997460 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73408 MiB
End job on node131 at 2011-05-15 03:28:42