Trace number 3371262

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.21 1207.49

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
MD5SUMfce318b20c1a474ab81b77594d3f6f99
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark583.061
Satisfiable
(Un)Satisfiability was proved
Number of variables945406
Number of clauses5079060
Sum of the clauses size12071640
Maximum clause length982
Minimum clause length1
Number of clauses of size 1720
Number of clauses of size 25035980
Number of clauses of size 3240
Number of clauses of size 4960
Number of clauses of size 52400
Number of clauses of size over 538760

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371262-1305420821/watcher-3371262-1305420821 -o /tmp/evaluation-result-3371262-1305420821/solver-3371262-1305420821 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371262-1305420821.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.65 8.01 7.97 6/328 25697
/proc/meminfo: memFree=24915068/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=22608 CPUtime=0 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 3710 1256 0 0 0 0 0 0 23 0 1 0 653682893 23150592 3640 33554432000 4194304 4417452 140735562825680 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 5652 3640 144 55 0 3538 0

[startup+0.057553 s]
/proc/loadavg: 7.65 8.01 7.97 6/328 25697
/proc/meminfo: memFree=24915068/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=27088 CPUtime=0.05 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 4866 1256 0 0 5 0 0 0 23 0 1 0 653682893 27738112 4796 33554432000 4194304 4417452 140735562825680 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 6772 4796 148 55 0 4658 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 27088

[startup+0.100545 s]
/proc/loadavg: 7.65 8.01 7.97 6/328 25697
/proc/meminfo: memFree=24915068/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=32544 CPUtime=0.09 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 6236 1256 0 0 9 0 0 0 23 0 1 0 653682893 33325056 6161 33554432000 4194304 4417452 140735562825680 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 8136 6161 149 55 0 6022 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 32544

[startup+0.300508 s]
/proc/loadavg: 7.65 8.01 7.97 6/328 25697
/proc/meminfo: memFree=24915068/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=65212 CPUtime=0.29 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 14403 1256 0 0 28 1 0 0 24 0 1 0 653682893 66777088 14328 33554432000 4194304 4417452 140735562825680 18446744073709551615 4264959 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 16303 14328 149 55 0 14189 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 65212

[startup+0.700433 s]
/proc/loadavg: 7.65 8.01 7.97 6/328 25697
/proc/meminfo: memFree=24915068/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=122944 CPUtime=0.69 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 28873 1256 0 0 66 3 0 0 25 0 1 0 653682893 125894656 28761 33554432000 4194304 4417452 140735562825680 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 30736 28761 149 55 0 28622 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 122944

[startup+1.50028 s]
/proc/loadavg: 7.65 8.01 7.97 6/329 25703
/proc/meminfo: memFree=24777372/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=253956 CPUtime=1.49 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 61771 1256 0 0 141 8 0 0 25 0 1 0 653682893 260050944 61514 33554432000 4194304 4417452 140735562825680 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 63489 61514 149 55 0 61375 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 253956

[startup+3.101 s]
/proc/loadavg: 7.65 8.01 7.97 6/329 25703
/proc/meminfo: memFree=24597076/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=477188 CPUtime=3.08 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) R 25695 25697 25544 0 -1 4202496 117579 1256 0 0 294 14 0 0 25 0 1 0 653682893 488640512 117322 33554432000 4194304 4417452 140735562825680 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/25697/statm: 119297 117322 149 55 0 117183 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 477188

[startup+6.30036 s]
/proc/loadavg: 7.52 7.98 7.96 9/333 25707
/proc/meminfo: memFree=24322672/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=830472 CPUtime=9.46 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 241247 1256 0 0 911 35 0 0 25 0 5 0 653682893 850403328 180674 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 207618 180674 168 55 0 205504 0
[pid=25697/tid=25704] ppid=25695 vsize=830472 CPUtime=1.05 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 23274 1256 0 0 102 3 0 0 25 0 5 0 653683416 850403328 180674 33554432000 4194304 4417452 140735562825680 18446744073709551615 4288250 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=830472 CPUtime=1.05 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 23290 1256 0 0 102 3 0 0 18 0 5 0 653683416 850403328 180674 33554432000 4194304 4417452 140735562825680 18446744073709551615 4288209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=830472 CPUtime=1.04 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 23238 1256 0 0 100 4 0 0 19 0 5 0 653683416 850403328 180674 33554432000 4194304 4417452 140735562825680 18446744073709551615 4286467 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=830472 CPUtime=1.05 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 16990 1256 0 0 102 3 0 0 25 0 5 0 653683416 850403328 180674 33554432000 4194304 4417452 140735562825680 18446744073709551615 4365579 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 9.46
Current children cumulated vsize (KiB) 830472

[startup+12.7012 s]
/proc/loadavg: 7.56 7.98 7.96 9/333 25708
/proc/meminfo: memFree=24161708/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=810644 CPUtime=34.96 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 418451 1256 0 0 3437 59 0 0 25 0 5 0 653682893 830099456 180642 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 202661 180642 179 55 0 200547 0
[pid=25697/tid=25704] ppid=25695 vsize=810644 CPUtime=7.45 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 73036 1256 0 0 736 9 0 0 25 0 5 0 653683416 830099456 180642 33554432000 4194304 4417452 140735562825680 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=810644 CPUtime=7.46 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 85530 1256 0 0 733 13 0 0 25 0 5 0 653683416 830099456 180642 33554432000 4194304 4417452 140735562825680 18446744073709551615 4302058 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=810644 CPUtime=7.39 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 85877 1256 0 0 727 12 0 0 25 0 5 0 653683416 830099456 180642 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331715 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=810644 CPUtime=7.41 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 19553 1256 0 0 738 3 0 0 25 0 5 0 653683416 830099456 180642 33554432000 4194304 4417452 140735562825680 18446744073709551615 4357314 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34.96
Current children cumulated vsize (KiB) 810644

[startup+25.5008 s]
/proc/loadavg: 7.66 7.98 7.96 9/333 25708
/proc/meminfo: memFree=24141232/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=847496 CPUtime=86.02 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 427664 1256 0 0 8542 60 0 0 25 0 5 0 653682893 867835904 183333 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 211874 183333 179 55 0 209760 0
[pid=25697/tid=25704] ppid=25695 vsize=847496 CPUtime=20.25 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 73036 1256 0 0 2016 9 0 0 25 0 5 0 653683416 867835904 183333 33554432000 4194304 4417452 140735562825680 18446744073709551615 4254396 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=847496 CPUtime=20.25 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 85530 1256 0 0 2012 13 0 0 25 0 5 0 653683416 867835904 183333 33554432000 4194304 4417452 140735562825680 18446744073709551615 4253528 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=847496 CPUtime=20.19 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 85877 1256 0 0 2007 12 0 0 25 0 5 0 653683416 867835904 183333 33554432000 4194304 4417452 140735562825680 18446744073709551615 4253513 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=847496 CPUtime=20.07 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 28766 1256 0 0 2003 4 0 0 25 0 5 0 653683416 867835904 183333 33554432000 4194304 4417452 140735562825680 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 86.02
Current children cumulated vsize (KiB) 847496

[startup+51.101 s]
/proc/loadavg: 7.83 7.99 7.97 9/333 25708
/proc/meminfo: memFree=23971688/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=974184 CPUtime=188.14 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 489355 1256 0 0 18747 67 0 0 25 0 5 0 653682893 997564416 219083 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 243546 219083 180 55 0 241432 0
[pid=25697/tid=25704] ppid=25695 vsize=974184 CPUtime=45.85 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 80602 1256 0 0 4575 10 0 0 25 0 5 0 653683416 997564416 219083 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=974184 CPUtime=45.85 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 95176 1256 0 0 4571 14 0 0 25 0 5 0 653683416 997564416 219083 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=974184 CPUtime=45.77 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 100698 1256 0 0 4564 13 0 0 25 0 5 0 653683416 997564416 219083 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=974184 CPUtime=45.4 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 58424 1256 0 0 4533 7 0 0 25 0 5 0 653683416 997564416 219083 33554432000 4194304 4417452 140735562825680 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 188.14
Current children cumulated vsize (KiB) 974184

[startup+102.312 s]
/proc/loadavg: 8.03 8.02 7.98 9/333 25710
/proc/meminfo: memFree=23745612/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=1133096 CPUtime=392.43 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 529088 1256 0 0 39172 71 0 0 25 0 5 0 653682893 1160290304 258811 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 283274 258811 180 55 0 281160 0
[pid=25697/tid=25704] ppid=25695 vsize=1133096 CPUtime=97.06 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 80602 1256 0 0 9696 10 0 0 25 0 5 0 653683416 1160290304 258811 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=1133096 CPUtime=97.06 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 95176 1256 0 0 9692 14 0 0 25 0 5 0 653683416 1160290304 258811 33554432000 4194304 4417452 140735562825680 18446744073709551615 4357227 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=1133096 CPUtime=96.99 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 100698 1256 0 0 9686 13 0 0 25 0 5 0 653683416 1160290304 258811 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236554 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=1133096 CPUtime=96.05 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 98157 1256 0 0 9594 11 0 0 25 0 5 0 653683416 1160290304 258811 33554432000 4194304 4417452 140735562825680 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 392.43
Current children cumulated vsize (KiB) 1133096

[startup+162.3 s]
/proc/loadavg: 8.07 8.03 7.99 9/333 25712
/proc/meminfo: memFree=23653204/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=1177664 CPUtime=631.74 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 656015 1256 0 0 63086 88 0 0 25 0 5 0 653682893 1205927936 273950 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 294416 273950 180 55 0 292302 0
[pid=25697/tid=25704] ppid=25695 vsize=1177664 CPUtime=157.05 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 129844 1256 0 0 15689 16 0 0 25 0 5 0 653683416 1205927936 273950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=1177664 CPUtime=157.05 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 129706 1256 0 0 15686 19 0 0 25 0 5 0 653683416 1205927936 273950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=1177664 CPUtime=156.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 133969 1256 0 0 15680 18 0 0 25 0 5 0 653683416 1205927936 273950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4254425 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=1177664 CPUtime=155.4 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 108041 1256 0 0 15527 13 0 0 25 0 5 0 653683416 1205927936 273950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4358116 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 631.74
Current children cumulated vsize (KiB) 1177664

[startup+222.301 s]
/proc/loadavg: 8.08 8.04 7.99 9/333 25713
/proc/meminfo: memFree=23561416/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=1200832 CPUtime=871.1 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 663615 1256 0 0 87021 89 0 0 25 0 5 0 653682893 1229651968 275777 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 300208 275777 180 55 0 298094 0
[pid=25697/tid=25704] ppid=25695 vsize=1200832 CPUtime=217.05 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 129844 1256 0 0 21689 16 0 0 25 0 5 0 653683416 1229651968 275777 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236672 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=1200832 CPUtime=217.06 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 129706 1256 0 0 21687 19 0 0 25 0 5 0 653683416 1229651968 275777 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236961 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=1200832 CPUtime=216.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 133969 1256 0 0 21680 18 0 0 25 0 5 0 653683416 1229651968 275777 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=1200832 CPUtime=214.74 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 115641 1256 0 0 21461 13 0 0 25 0 5 0 653683416 1229651968 275777 33554432000 4194304 4417452 140735562825680 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 871.1
Current children cumulated vsize (KiB) 1200832

################
# More data... #
################

[pid=25697/tid=25705] ppid=25695 vsize=1730132 CPUtime=696.21 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 196452 1256 0 0 69594 27 0 0 25 0 5 0 653683416 1771655168 406950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4320859 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=1730132 CPUtime=696.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 226824 1256 0 0 69668 30 0 0 25 0 5 0 653683416 1771655168 406950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=1730132 CPUtime=689.55 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 411511 1256 0 0 68904 51 0 0 25 0 5 0 653683416 1771655168 406950 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364453 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2785.08
Current children cumulated vsize (KiB) 1730132

[startup+762.312 s]
/proc/loadavg: 8.12 8.15 8.05 9/333 25740
/proc/meminfo: memFree=22710080/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=1764644 CPUtime=3024.47 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 1448696 1256 0 0 302258 189 0 0 25 0 5 0 653682893 1806995456 413414 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 441161 413414 180 55 0 439047 0
[pid=25697/tid=25704] ppid=25695 vsize=1764644 CPUtime=757.09 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 360258 1256 0 0 75663 46 0 0 25 0 5 0 653683416 1806995456 413414 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331765 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=1764644 CPUtime=756.23 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 196452 1256 0 0 75596 27 0 0 25 0 5 0 653683416 1806995456 413414 33554432000 4194304 4417452 140735562825680 18446744073709551615 4254955 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=1764644 CPUtime=756.99 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 275369 1256 0 0 75662 37 0 0 25 0 5 0 653683416 1806995456 413414 33554432000 4194304 4417452 140735562825680 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=1764644 CPUtime=748.91 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 462162 1256 0 0 74834 57 0 0 25 0 5 0 653683416 1806995456 413414 33554432000 4194304 4417452 140735562825680 18446744073709551615 4353568 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3024.47
Current children cumulated vsize (KiB) 1764644

[startup+822.301 s]
/proc/loadavg: 8.12 8.15 8.05 9/333 25741
/proc/meminfo: memFree=21810416/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=1962256 CPUtime=3263.8 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 1536004 1256 0 0 326181 199 0 0 25 0 5 0 653682893 2009350144 460610 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 490564 460610 180 55 0 488450 0
[pid=25697/tid=25704] ppid=25695 vsize=1962256 CPUtime=817.08 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 360258 1256 0 0 81662 46 0 0 25 0 5 0 653683416 2009350144 460610 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=1962256 CPUtime=816.22 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 196452 1256 0 0 81595 27 0 0 25 0 5 0 653683416 2009350144 460610 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236975 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=1962256 CPUtime=816.97 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 323859 1256 0 0 81655 42 0 0 25 0 5 0 653683416 2009350144 460610 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=1962256 CPUtime=808.26 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 500980 1256 0 0 80765 61 0 0 25 0 5 0 653683416 2009350144 460610 33554432000 4194304 4417452 140735562825680 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3263.8
Current children cumulated vsize (KiB) 1962256

[startup+882.301 s]
/proc/loadavg: 8.08 8.13 8.05 9/333 25743
/proc/meminfo: memFree=22355356/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2030376 CPUtime=3503.15 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 1649568 1256 0 0 350101 214 0 0 25 0 5 0 653682893 2079105024 476915 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 507594 476915 180 55 0 505480 0
[pid=25697/tid=25704] ppid=25695 vsize=2030376 CPUtime=877.07 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 415096 1256 0 0 87655 52 0 0 25 0 5 0 653683416 2079105024 476915 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2030376 CPUtime=876.22 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 228206 1256 0 0 87590 32 0 0 25 0 5 0 653683416 2079105024 476915 33554432000 4194304 4417452 140735562825680 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2030376 CPUtime=876.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 329403 1256 0 0 87655 43 0 0 25 0 5 0 653683416 2079105024 476915 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232695 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2030376 CPUtime=867.61 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 522408 1256 0 0 86697 64 0 0 25 0 5 0 653683416 2079105024 476915 33554432000 4194304 4417452 140735562825680 18446744073709551615 4281694 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3503.15
Current children cumulated vsize (KiB) 2030376

[startup+942.301 s]
/proc/loadavg: 8.03 8.11 8.05 9/333 25744
/proc/meminfo: memFree=22262432/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2160004 CPUtime=3742.5 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 1712227 1256 0 0 374027 223 0 0 25 0 5 0 653682893 2211844096 509330 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 540001 509330 180 55 0 537887 0
[pid=25697/tid=25704] ppid=25695 vsize=2160004 CPUtime=937.08 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 415096 1256 0 0 93656 52 0 0 25 0 5 0 653683416 2211844096 509330 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2160004 CPUtime=936.23 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 233861 1256 0 0 93590 33 0 0 25 0 5 0 653683416 2211844096 509330 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2160004 CPUtime=936.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 355174 1256 0 0 93651 47 0 0 25 0 5 0 653683416 2211844096 509330 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236735 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2160004 CPUtime=926.96 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 553641 1256 0 0 92627 69 0 0 25 0 5 0 653683416 2211844096 509330 33554432000 4194304 4417452 140735562825680 18446744073709551615 4237221 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3742.5
Current children cumulated vsize (KiB) 2160004

[startup+1002.3 s]
/proc/loadavg: 8.05 8.10 8.05 9/333 25746
/proc/meminfo: memFree=22275048/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2156972 CPUtime=3981.85 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 1945513 1256 0 0 397930 255 0 0 25 0 5 0 653682893 2208739328 503724 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 539243 503724 180 55 0 537129 0
[pid=25697/tid=25704] ppid=25695 vsize=2156972 CPUtime=997.08 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 499115 1256 0 0 99646 62 0 0 25 0 5 0 653683416 2208739328 503724 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2156972 CPUtime=996.23 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 239893 1256 0 0 99589 34 0 0 25 0 5 0 653683416 2208739328 503724 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232629 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2156972 CPUtime=996.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 446267 1256 0 0 99637 61 0 0 25 0 5 0 653683416 2208739328 503724 33554432000 4194304 4417452 140735562825680 18446744073709551615 4237148 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2156972 CPUtime=986.3 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 605783 1256 0 0 98555 75 0 0 25 0 5 0 653683416 2208739328 503724 33554432000 4194304 4417452 140735562825680 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3981.85
Current children cumulated vsize (KiB) 2156972

[startup+1062.3 s]
/proc/loadavg: 8.02 8.08 8.04 9/333 25748
/proc/meminfo: memFree=21850796/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2618804 CPUtime=4221.21 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 2065783 1256 0 0 421850 271 0 0 25 0 5 0 653682893 2681655296 619182 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 654701 619182 180 55 0 652587 0
[pid=25697/tid=25704] ppid=25695 vsize=2618804 CPUtime=1057.08 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 527177 1256 0 0 105642 66 0 0 25 0 5 0 653683416 2681655296 619182 33554432000 4194304 4417452 140735562825680 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2618804 CPUtime=1056.23 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 286628 1256 0 0 105583 40 0 0 25 0 5 0 653683416 2681655296 619182 33554432000 4194304 4417452 140735562825680 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2618804 CPUtime=1056.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 446267 1256 0 0 105637 61 0 0 25 0 5 0 653683416 2681655296 619182 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2618804 CPUtime=1045.65 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 651256 1256 0 0 104484 81 0 0 25 0 5 0 653683416 2681655296 619182 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4221.21
Current children cumulated vsize (KiB) 2618804

[startup+1122.3 s]
/proc/loadavg: 8.00 8.06 8.03 9/333 25749
/proc/meminfo: memFree=22062924/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2243816 CPUtime=4460.56 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 2210302 1256 0 0 445760 296 0 0 25 0 5 0 653682893 2297667584 537246 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 560954 537246 180 55 0 558840 0
[pid=25697/tid=25704] ppid=25695 vsize=2243816 CPUtime=1117.09 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 561066 1256 0 0 111638 71 0 0 25 0 5 0 653683416 2297667584 537246 33554432000 4194304 4417452 140735562825680 18446744073709551615 4236544 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2243816 CPUtime=1116.23 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 297702 1256 0 0 111580 43 0 0 25 0 5 0 653683416 2297667584 537246 33554432000 4194304 4417452 140735562825680 18446744073709551615 4286414 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2243816 CPUtime=1116.98 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 506381 1256 0 0 111628 70 0 0 25 0 5 0 653683416 2297667584 537246 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2243816 CPUtime=1105 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 690698 1256 0 0 110410 90 0 0 25 0 5 0 653683416 2297667584 537246 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4460.56
Current children cumulated vsize (KiB) 2243816

[startup+1182.3 s]
/proc/loadavg: 8.11 8.08 8.04 9/333 25751
/proc/meminfo: memFree=22104796/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2170312 CPUtime=4699.93 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 2385555 1256 0 0 469672 321 0 0 25 0 5 0 653682893 2222399488 518047 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 542578 518047 180 55 0 540464 0
[pid=25697/tid=25704] ppid=25695 vsize=2170312 CPUtime=1177.09 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 629324 1256 0 0 117628 81 0 0 25 0 5 0 653683416 2222399488 518047 33554432000 4194304 4417452 140735562825680 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2170312 CPUtime=1176.24 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 330472 1256 0 0 117577 47 0 0 25 0 5 0 653683416 2222399488 518047 33554432000 4194304 4417452 140735562825680 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2170312 CPUtime=1176.99 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 531687 1256 0 0 117624 75 0 0 25 0 5 0 653683416 2222399488 518047 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2170312 CPUtime=1164.37 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 739617 1256 0 0 116340 97 0 0 25 0 5 0 653683416 2222399488 518047 33554432000 4194304 4417452 140735562825680 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4699.93
Current children cumulated vsize (KiB) 2170312



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1207.4 s]
/proc/loadavg: 7.83 8.02 8.02 9/333 25788
/proc/meminfo: memFree=22934384/32951124 swapFree=44485764/67111528
[pid=25697] ppid=25695 vsize=2235364 CPUtime=4800.09 cores=1,3,5,7
/proc/25697/stat : 25697 (plingeling) S 25695 25697 25544 0 -1 4202496 2401845 1256 0 0 479686 323 0 0 25 0 5 0 653682893 2289012736 534310 33554432000 4194304 4417452 140735562825680 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/25697/statm: 558841 534310 180 55 0 556727 0
[pid=25697/tid=25704] ppid=25695 vsize=2235364 CPUtime=1202.2 cores=1,3,5,7
/proc/25697/task/25704/stat : 25704 (plingeling) R 25695 25697 25544 0 -1 4202560 629324 1256 0 0 120139 81 0 0 25 0 5 0 653683416 2289012736 534310 33554432000 4194304 4417452 140735562825680 18446744073709551615 4254850 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25697/tid=25705] ppid=25695 vsize=2235364 CPUtime=1201.35 cores=1,3,5,7
/proc/25697/task/25705/stat : 25705 (plingeling) R 25695 25697 25544 0 -1 4202560 336861 1256 0 0 120087 48 0 0 25 0 5 0 653683416 2289012736 534310 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232666 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25697/tid=25706] ppid=25695 vsize=2235364 CPUtime=1202.1 cores=1,3,5,7
/proc/25697/task/25706/stat : 25706 (plingeling) R 25695 25697 25544 0 -1 4202560 531687 1256 0 0 120135 75 0 0 25 0 5 0 653683416 2289012736 534310 33554432000 4194304 4417452 140735562825680 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25697/tid=25707] ppid=25695 vsize=2235364 CPUtime=1189.19 cores=1,3,5,7
/proc/25697/task/25707/stat : 25707 (plingeling) R 25695 25697 25544 0 -1 4202560 749518 1256 0 0 118822 97 0 0 25 0 5 0 653683416 2289012736 534310 33554432000 4194304 4417452 140735562825680 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 2235364

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1207.49
CPU time (s): 4800.21
CPU user time (s): 4796.89
CPU system time (s): 3.3145
CPU usage (%): 397.535
Max. virtual memory (cumulated for all children) (KiB): 2618804

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.89
system time used= 3.3145
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2403101
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 133
involuntary context switches= 23397

runsolver used 3.58245 second user time and 9.52355 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-15 02:53:45
IDJOB=3371262
IDBENCH=82367
IDSOLVER=1848
FILE ID=node105/3371262-1305420821
RUNJOBID= node105-1305419616-25563
PBS_JOBID= 13324579
Free space on /tmp= 73460 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371262-1305420821/watcher-3371262-1305420821 -o /tmp/evaluation-result-3371262-1305420821/solver-3371262-1305420821 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371262-1305420821.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= fce318b20c1a474ab81b77594d3f6f99
RANDOM SEED=412631759

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24915348 kB
Buffers:        282240 kB
Cached:        6072552 kB
SwapCached:      39656 kB
Active:         906056 kB
Inactive:      6861496 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24915348 kB
SwapTotal:    67111528 kB
SwapFree:     44485764 kB
Dirty:          134868 kB
Writeback:           0 kB
AnonPages:     1380360 kB
Mapped:          10728 kB
Slab:           137408 kB
PageTables:      66752 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 65584188 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73488 MiB
End job on node105 at 2011-05-15 03:13:54