Trace number 3394534

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.5 5011.26

General information on the benchmark

Namecrafted/skvortsov/battleship/
battleship-16-16-unsat.cnf
MD5SUMc07743bac800f91e403382f26717b1d0
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables256
Number of clauses2176
Sum of the clauses size7936
Maximum clause length16
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21920
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5256

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394534-1306099096/watcher-3394534-1306099096 -o /tmp/evaluation-result-3394534-1306099096/solver-3394534-1306099096 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394534-1306099096.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.51 7.95 7.90 1/507 4232
/proc/meminfo: memFree=12388520/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=9420 CPUtime=0 cores=0-7
/proc/4232/stat : 4232 (plingeling) R 4230 4232 3846 0 -1 4202496 420 1255 0 0 0 0 0 0 25 0 1 0 702325244 9646080 351 33554432000 4194304 4417452 140736578853776 18446744073709551615 264726101771 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4232/statm: 2355 351 147 55 0 241 0
[pid=4232/tid=4238] ppid=4230 vsize=157544 CPUtime=0.01 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 34 1255 0 0 1 0 0 0 18 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 4233495 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1 1255 0 0 0 0 0 0 25 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 264726510062 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1 1255 0 0 0 0 0 0 25 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 264726510062 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 58 1255 0 0 0 0 0 0 25 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1 1255 0 0 0 0 0 0 18 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 4346677 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 17 1255 0 0 0 0 0 0 25 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 4258194 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 34 1255 0 0 0 0 0 0 18 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 4235985 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=157544 CPUtime=0 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 22 1255 0 0 0 0 0 0 18 0 9 0 702325244 161325056 591 33554432000 4194304 4417452 140736578853776 18446744073709551615 4235411 0 0 4096 17442 0 0 0 -1 3 0 0 0

[startup+0.117084 s]
/proc/loadavg: 7.51 7.95 7.90 1/507 4232
/proc/meminfo: memFree=12388520/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=165372 CPUtime=0.81 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 3152 1255 0 0 81 0 0 0 25 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 41343 2571 180 55 0 39229 0
[pid=4232/tid=4238] ppid=4230 vsize=165372 CPUtime=0.1 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 412 1255 0 0 10 0 0 0 18 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 4311296 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=165372 CPUtime=0.09 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 222 1255 0 0 9 0 0 0 18 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 4311150 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=165372 CPUtime=0.07 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 146 1255 0 0 7 0 0 0 25 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 264726510062 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=165372 CPUtime=0.1 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 672 1255 0 0 10 0 0 0 25 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364669 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=165372 CPUtime=0.1 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 308 1255 0 0 10 0 0 0 18 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 4284752 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=165372 CPUtime=0.1 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 280 1255 0 0 10 0 0 0 25 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 4307824 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=165372 CPUtime=0.1 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 395 1255 0 0 10 0 0 0 18 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 4278192 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=165372 CPUtime=0.1 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 221 1255 0 0 10 0 0 0 18 0 9 0 702325244 169340928 2571 33554432000 4194304 4417452 140736578853776 18446744073709551615 264726510111 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.81
Current children cumulated vsize (KiB) 165372

[startup+0.200827 s]
/proc/loadavg: 7.51 7.95 7.90 1/507 4232
/proc/meminfo: memFree=12388520/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=167392 CPUtime=1.48 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 4236 1255 0 0 148 0 0 0 25 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 41848 3089 180 55 0 39734 0
[pid=4232/tid=4238] ppid=4230 vsize=167392 CPUtime=0.19 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 569 1255 0 0 19 0 0 0 18 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=167392 CPUtime=0.18 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 222 1255 0 0 18 0 0 0 18 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4307858 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=167392 CPUtime=0.16 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 370 1255 0 0 16 0 0 0 25 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=167392 CPUtime=0.18 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1001 1255 0 0 18 0 0 0 25 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=167392 CPUtime=0.18 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 308 1255 0 0 18 0 0 0 18 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4309888 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=167392 CPUtime=0.19 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 280 1255 0 0 19 0 0 0 25 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4309862 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=167392 CPUtime=0.18 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 769 1255 0 0 18 0 0 0 18 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4250571 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=167392 CPUtime=0.19 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 221 1255 0 0 19 0 0 0 18 0 9 0 702325244 171409408 3089 33554432000 4194304 4417452 140736578853776 18446744073709551615 4310334 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 167392

[startup+0.300809 s]
/proc/loadavg: 7.51 7.95 7.90 1/507 4232
/proc/meminfo: memFree=12388520/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=169076 CPUtime=2.29 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 6918 1255 0 0 228 1 0 0 25 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 42269 3544 180 55 0 40155 0
[pid=4232/tid=4238] ppid=4230 vsize=169076 CPUtime=0.29 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 934 1255 0 0 29 0 0 0 18 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=169076 CPUtime=0.28 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 485 1255 0 0 28 0 0 0 18 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4358116 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=169076 CPUtime=0.25 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 607 1255 0 0 25 0 0 0 25 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4311058 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=169076 CPUtime=0.28 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1800 1255 0 0 28 0 0 0 25 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=169076 CPUtime=0.28 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 584 1255 0 0 28 0 0 0 18 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=169076 CPUtime=0.29 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 429 1255 0 0 29 0 0 0 25 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=169076 CPUtime=0.28 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 948 1255 0 0 28 0 0 0 18 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 264726097616 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=169076 CPUtime=0.29 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 635 1255 0 0 29 0 0 0 18 0 9 0 702325244 173133824 3544 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365538 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.29
Current children cumulated vsize (KiB) 169076

[startup+0.700736 s]
/proc/loadavg: 7.51 7.95 7.90 1/507 4232
/proc/meminfo: memFree=12388520/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=178328 CPUtime=5.48 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 18197 1255 0 0 545 3 0 0 25 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 44582 5841 180 55 0 42468 0
[pid=4232/tid=4238] ppid=4230 vsize=178328 CPUtime=0.68 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 2472 1255 0 0 68 0 0 0 18 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4250633 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=178328 CPUtime=0.67 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1743 1255 0 0 67 0 0 0 18 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=178328 CPUtime=0.65 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 2282 1255 0 0 65 0 0 0 25 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=178328 CPUtime=0.68 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 2891 1255 0 0 68 0 0 0 25 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=178328 CPUtime=0.68 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 2216 1255 0 0 68 0 0 0 18 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=178328 CPUtime=0.68 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1559 1255 0 0 68 0 0 0 25 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365103 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=178328 CPUtime=0.68 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 2523 1255 0 0 68 0 0 0 18 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4356874 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=178328 CPUtime=0.68 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 2015 1255 0 0 68 0 0 0 18 0 9 0 702325244 182607872 5841 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.48
Current children cumulated vsize (KiB) 178328

[startup+1.50059 s]
/proc/loadavg: 7.51 7.95 7.90 9/516 4246
/proc/meminfo: memFree=12360044/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=186556 CPUtime=11.86 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 42391 1255 0 0 1180 6 0 0 25 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 46639 7866 180 55 0 44525 0
[pid=4232/tid=4238] ppid=4230 vsize=186556 CPUtime=1.48 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 5585 1255 0 0 148 0 0 0 20 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=186556 CPUtime=1.47 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 4715 1255 0 0 147 0 0 0 20 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=186556 CPUtime=1.43 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 5303 1255 0 0 142 1 0 0 25 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=186556 CPUtime=1.48 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 5770 1255 0 0 147 1 0 0 25 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=186556 CPUtime=1.48 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 5147 1255 0 0 148 0 0 0 19 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=186556 CPUtime=1.48 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 4558 1255 0 0 148 0 0 0 25 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=186556 CPUtime=1.48 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 5969 1255 0 0 148 0 0 0 20 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=186556 CPUtime=1.48 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 4848 1255 0 0 148 0 0 0 20 0 9 0 702325244 191033344 7866 33554432000 4194304 4417452 140736578853776 18446744073709551615 4321595 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.86
Current children cumulated vsize (KiB) 186556

[startup+3.10128 s]
/proc/loadavg: 7.51 7.95 7.90 9/516 4246
/proc/meminfo: memFree=12345484/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=202100 CPUtime=24.64 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 100161 1255 0 0 2449 15 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 50525 11755 180 55 0 48411 0
[pid=4232/tid=4238] ppid=4230 vsize=202100 CPUtime=3.09 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 12288 1255 0 0 308 1 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4357204 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=202100 CPUtime=3.07 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 12368 1255 0 0 306 1 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4310191 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=202100 CPUtime=3.01 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 12239 1255 0 0 299 2 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=202100 CPUtime=3.07 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 10057 1255 0 0 306 1 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=202100 CPUtime=3.07 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 14493 1255 0 0 306 1 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4307448 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=202100 CPUtime=3.08 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 11601 1255 0 0 307 1 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4307580 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=202100 CPUtime=3.08 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 13438 1255 0 0 307 1 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366391 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=202100 CPUtime=3.09 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 13181 1255 0 0 307 2 0 0 25 0 9 0 702325244 206950400 11755 33554432000 4194304 4417452 140736578853776 18446744073709551615 4309928 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.64
Current children cumulated vsize (KiB) 202100

[startup+6.30068 s]
/proc/loadavg: 7.55 7.95 7.90 9/516 4246
/proc/meminfo: memFree=12330484/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=216160 CPUtime=50.18 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 232771 1255 0 0 4984 34 0 0 25 0 9 0 702325244 221347840 15282 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 54040 15282 180 55 0 51926 0

################
# More data... #
################


[startup+4722.3 s]
/proc/loadavg: 8.11 8.08 8.02 9/516 4387
/proc/meminfo: memFree=8671692/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=3853260 CPUtime=37694.8 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 11802332 1255 0 0 3767545 1937 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 963315 927188 180 55 0 961201 0
[pid=4232/tid=4238] ppid=4230 vsize=3853260 CPUtime=4691.93 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 1387564 1255 0 0 468971 222 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=3853260 CPUtime=4670.13 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1314774 1255 0 0 466806 207 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=3853260 CPUtime=4720.68 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1944778 1255 0 0 471759 309 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=3853260 CPUtime=4722.46 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1076221 1255 0 0 472032 214 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=3853260 CPUtime=4722.44 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1383434 1255 0 0 472011 233 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4259618 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=3853260 CPUtime=4722.32 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1297789 1255 0 0 472031 201 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=3853260 CPUtime=4722.4 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 1587756 1255 0 0 471979 261 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4259618 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=3853260 CPUtime=4722.47 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 1809520 1255 0 0 471960 287 0 0 25 0 9 0 702325244 3945738240 927188 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37694.8
Current children cumulated vsize (KiB) 3853260

[startup+4782.3 s]
/proc/loadavg: 8.04 8.06 8.02 9/516 4388
/proc/meminfo: memFree=8917556/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=3609048 CPUtime=38173.7 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 11894102 1255 0 0 3815423 1950 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 902262 866135 180 55 0 900148 0
[pid=4232/tid=4238] ppid=4230 vsize=3609048 CPUtime=4750.89 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 1387564 1255 0 0 474867 222 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=3609048 CPUtime=4730.1 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1314774 1255 0 0 472803 207 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=3609048 CPUtime=4780.66 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1944778 1255 0 0 477757 309 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4343764 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=3609048 CPUtime=4782.44 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1143415 1255 0 0 478020 224 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=3609048 CPUtime=4782.42 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1408010 1255 0 0 478007 235 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=3609048 CPUtime=4782.3 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1297789 1255 0 0 478029 201 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=3609048 CPUtime=4782.39 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 1587756 1255 0 0 477978 261 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=3609048 CPUtime=4782.45 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 1809520 1255 0 0 477958 287 0 0 25 0 9 0 702325244 3695665152 866135 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38173.7
Current children cumulated vsize (KiB) 3609048

[startup+4842.32 s]
/proc/loadavg: 8.05 8.06 8.02 9/516 4390
/proc/meminfo: memFree=8825716/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=3701196 CPUtime=38652.8 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 11925331 1255 0 0 3863327 1957 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 925299 889172 180 55 0 923185 0
[pid=4232/tid=4238] ppid=4230 vsize=3701196 CPUtime=4809.87 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 1403948 1255 0 0 480760 227 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=3701196 CPUtime=4790.11 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1314774 1255 0 0 478804 207 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=3701196 CPUtime=4840.68 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1944778 1255 0 0 483759 309 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=3701196 CPUtime=4842.47 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1147772 1255 0 0 484022 225 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=3701196 CPUtime=4842.44 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1416202 1255 0 0 484008 236 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=3701196 CPUtime=4842.32 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1297789 1255 0 0 484031 201 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=3701196 CPUtime=4842.41 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 1590052 1255 0 0 483979 262 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4344176 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=3701196 CPUtime=4842.48 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 1809520 1255 0 0 483961 287 0 0 25 0 9 0 702325244 3790024704 889172 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38652.8
Current children cumulated vsize (KiB) 3701196

[startup+4902.3 s]
/proc/loadavg: 8.02 8.05 8.01 9/516 4392
/proc/meminfo: memFree=8841192/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=3684544 CPUtime=39131.7 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 11977675 1255 0 0 3911196 1970 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 921136 885009 180 55 0 919022 0
[pid=4232/tid=4238] ppid=4230 vsize=3684544 CPUtime=4868.82 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 1403948 1255 0 0 486655 227 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=3684544 CPUtime=4850.07 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1314774 1255 0 0 484800 207 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=3684544 CPUtime=4900.66 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1944778 1255 0 0 489757 309 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=3684544 CPUtime=4902.45 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1147772 1255 0 0 490020 225 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=3684544 CPUtime=4902.43 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1416202 1255 0 0 490007 236 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=3684544 CPUtime=4902.3 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1297789 1255 0 0 490029 201 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=3684544 CPUtime=4902.39 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 1642396 1255 0 0 489965 274 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=3684544 CPUtime=4902.46 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 1809520 1255 0 0 489959 287 0 0 25 0 9 0 702325244 3772973056 885009 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39131.7
Current children cumulated vsize (KiB) 3684544

[startup+4962.3 s]
/proc/loadavg: 8.04 8.05 8.01 9/516 4393
/proc/meminfo: memFree=8840652/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=3685832 CPUtime=39610.6 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 11977997 1255 0 0 3959092 1970 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 921458 885331 180 55 0 919344 0
[pid=4232/tid=4238] ppid=4230 vsize=3685832 CPUtime=4927.77 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 1403948 1255 0 0 492550 227 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=3685832 CPUtime=4910.06 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1314774 1255 0 0 490799 207 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=3685832 CPUtime=4960.67 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1944778 1255 0 0 495758 309 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=3685832 CPUtime=4962.45 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1147772 1255 0 0 496020 225 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=3685832 CPUtime=4962.43 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1416524 1255 0 0 496007 236 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=3685832 CPUtime=4962.3 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1297789 1255 0 0 496029 201 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=3685832 CPUtime=4962.39 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 1642396 1255 0 0 495965 274 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=3685832 CPUtime=4962.46 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 1809520 1255 0 0 495959 287 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39610.6
Current children cumulated vsize (KiB) 3685832



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.1 s]
/proc/loadavg: 8.05 8.06 8.01 9/516 4395
/proc/meminfo: memFree=8840752/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=3685832 CPUtime=40000.2 cores=0-7
/proc/4232/stat : 4232 (plingeling) S 4230 4232 3846 0 -1 4202496 11977997 1255 0 0 3998052 1970 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 264738207717 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 921458 885331 180 55 0 919344 0
[pid=4232/tid=4238] ppid=4230 vsize=3685832 CPUtime=4975.74 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4202560 1403948 1255 0 0 497347 227 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4232/tid=4239] ppid=4230 vsize=3685832 CPUtime=4958.87 cores=0-7
/proc/4232/task/4239/stat : 4239 (plingeling) R 4230 4232 3846 0 -1 4202560 1314774 1255 0 0 495680 207 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4232/tid=4240] ppid=4230 vsize=3685832 CPUtime=5009.49 cores=0-7
/proc/4232/task/4240/stat : 4240 (plingeling) R 4230 4232 3846 0 -1 4202560 1944778 1255 0 0 500640 309 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4232/tid=4241] ppid=4230 vsize=3685832 CPUtime=5011.27 cores=0-7
/proc/4232/task/4241/stat : 4241 (plingeling) R 4230 4232 3846 0 -1 4202560 1147772 1255 0 0 500902 225 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4232/tid=4242] ppid=4230 vsize=3685832 CPUtime=5011.25 cores=0-7
/proc/4232/task/4242/stat : 4242 (plingeling) R 4230 4232 3846 0 -1 4202560 1416524 1255 0 0 500889 236 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4343748 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4232/tid=4243] ppid=4230 vsize=3685832 CPUtime=5011.12 cores=0-7
/proc/4232/task/4243/stat : 4243 (plingeling) R 4230 4232 3846 0 -1 4202560 1297789 1255 0 0 500911 201 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4232/tid=4244] ppid=4230 vsize=3685832 CPUtime=5011.21 cores=0-7
/proc/4232/task/4244/stat : 4244 (plingeling) R 4230 4232 3846 0 -1 4202560 1642396 1255 0 0 500847 274 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4232/tid=4245] ppid=4230 vsize=3685832 CPUtime=5011.28 cores=0-7
/proc/4232/task/4245/stat : 4245 (plingeling) R 4230 4232 3846 0 -1 4202560 1809520 1255 0 0 500841 287 0 0 25 0 9 0 702325244 3774291968 885331 33554432000 4194304 4417452 140736578853776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 3685832

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.23 s]
/proc/loadavg: 8.05 8.06 8.01 9/516 4395
/proc/meminfo: memFree=8840752/32951124 swapFree=63083240/67111528
[pid=4232] ppid=4230 vsize=0 CPUtime=40000.5 cores=0-7
/proc/4232/stat : 4232 (plingeling) Z 4230 4232 3846 0 -1 4203548 11977997 1255 0 0 3998066 1980 0 0 18 0 2 0 702325244 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/4232/statm: 0 0 0 0 0 0 0
[pid=4232/tid=4238] ppid=4230 vsize=0 CPUtime=4975.84 cores=0-7
/proc/4232/task/4238/stat : 4238 (plingeling) R 4230 4232 3846 0 -1 4203588 1403948 1255 0 0 497347 237 0 0 25 0 2 0 702325244 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.26
CPU time (s): 40000.5
CPU user time (s): 39980.7
CPU system time (s): 19.838
CPU usage (%): 798.212
Max. virtual memory (cumulated for all children) (KiB): 4355536

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39980.7
system time used= 19.838
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11979252
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1022
involuntary context switches= 143658

runsolver used 23.4154 second user time and 63.1884 second system time

The end

Launcher Data 

Begin job on node141 at 2011-05-22 23:18:16
IDJOB=3394534
IDBENCH=83223
IDSOLVER=1851
FILE ID=node141/3394534-1306099096
RUNJOBID= node141-1306094835-3860
PBS_JOBID= 13387960
Free space on /tmp= 73372 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/skvortsov/battleship/battleship-16-16-unsat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394534-1306099096/watcher-3394534-1306099096 -o /tmp/evaluation-result-3394534-1306099096/solver-3394534-1306099096 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394534-1306099096.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c07743bac800f91e403382f26717b1d0
RANDOM SEED=1851898273

node141.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5362.96
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12388924 kB
Buffers:        337596 kB
Cached:        9846140 kB
SwapCached:      15808 kB
Active:       12662924 kB
Inactive:      7596816 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12388924 kB
SwapTotal:    67111528 kB
SwapFree:     63083240 kB
Dirty:             424 kB
Writeback:           0 kB
AnonPages:    10067496 kB
Mapped:          25928 kB
Slab:           162352 kB
PageTables:      69344 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 150649512 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73368 MiB
End job on node141 at 2011-05-23 00:41:49