Trace number 3394104

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.6 5011.05

General information on the benchmark

NameCRAFTED/rbsat/crafted/unforced/
rbsat-v2400c266431g9.cnf
MD5SUM35278b3ce28129761cf8ae7994f37d3f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394104-1306089833/watcher-3394104-1306089833 -o /tmp/evaluation-result-3394104-1306089833/solver-3394104-1306089833 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394104-1306089833.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.36 7.91 7.97 2/491 10567
/proc/meminfo: memFree=14107056/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=9740 CPUtime=0 cores=0-7
/proc/10567/stat : 10567 (plingeling) R 10565 10567 10351 0 -1 4202496 495 1259 0 0 0 0 0 0 25 0 1 0 799214000 9973760 425 33554432000 4194304 4417452 140733819885392 18446744073709551615 251367546624 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/10567/statm: 2435 425 147 55 0 321 0

[startup+0.0359449 s]
/proc/loadavg: 7.36 7.91 7.97 2/491 10567
/proc/meminfo: memFree=14107056/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=12248 CPUtime=0.03 cores=0-7
/proc/10567/stat : 10567 (plingeling) R 10565 10567 10351 0 -1 4202496 1131 1259 0 0 3 0 0 0 25 0 1 0 799214000 12541952 1061 33554432000 4194304 4417452 140733819885392 18446744073709551615 4262083 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/10567/statm: 3062 1061 148 55 0 948 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 12248

[startup+0.100921 s]
/proc/loadavg: 7.36 7.91 7.97 2/491 10567
/proc/meminfo: memFree=14107056/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=27796 CPUtime=0.09 cores=0-7
/proc/10567/stat : 10567 (plingeling) R 10565 10567 10351 0 -1 4202496 5026 1259 0 0 9 0 0 0 25 0 1 0 799214000 28463104 4956 33554432000 4194304 4417452 140733819885392 18446744073709551615 251367546624 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/10567/statm: 6949 4956 148 55 0 4835 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 27796

[startup+0.300889 s]
/proc/loadavg: 7.36 7.91 7.97 2/491 10567
/proc/meminfo: memFree=14107056/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=76948 CPUtime=0.28 cores=0-7
/proc/10567/stat : 10567 (plingeling) R 10565 10567 10351 0 -1 4202496 17314 1259 0 0 27 1 0 0 25 0 1 0 799214000 78794752 17244 33554432000 4194304 4417452 140733819885392 18446744073709551615 4262052 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/10567/statm: 19237 17244 148 55 0 17123 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 76948

[startup+0.700733 s]
/proc/loadavg: 7.36 7.91 7.97 2/491 10567
/proc/meminfo: memFree=14107056/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=298608 CPUtime=3.13 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 33028 1259 0 0 308 5 0 0 25 0 9 0 799214000 305774592 19598 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 74652 19598 177 55 0 72538 0
Current children cumulated CPU time (s) 3.13
Current children cumulated vsize (KiB) 298608

[startup+1.50058 s]
/proc/loadavg: 7.36 7.91 7.97 9/500 10581
/proc/meminfo: memFree=14005064/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=335832 CPUtime=9.51 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 44362 1259 0 0 944 7 0 0 25 0 9 0 799214000 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 83958 28875 180 55 0 81844 0
[pid=10567/tid=10574] ppid=10565 vsize=335832 CPUtime=1.14 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 3273 1259 0 0 114 0 0 0 19 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=335832 CPUtime=1.12 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 2054 1259 0 0 112 0 0 0 25 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=335832 CPUtime=1.14 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 4029 1259 0 0 114 0 0 0 19 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=335832 CPUtime=1.14 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 8530 1259 0 0 113 1 0 0 25 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=335832 CPUtime=1.14 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 1060 1259 0 0 114 0 0 0 25 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249501 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=335832 CPUtime=1.15 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 2396 1259 0 0 115 0 0 0 25 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=335832 CPUtime=1.14 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 2945 1259 0 0 114 0 0 0 19 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4258525 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=335832 CPUtime=1.14 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 2744 1259 0 0 114 0 0 0 19 0 9 0 799214034 343891968 28875 33554432000 4194304 4417452 140733819885392 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.51
Current children cumulated vsize (KiB) 335832

[startup+3.10029 s]
/proc/loadavg: 7.36 7.91 7.97 9/500 10581
/proc/meminfo: memFree=13993496/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=359880 CPUtime=22.29 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 58636 1259 0 0 2220 9 0 0 25 0 9 0 799214000 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 89970 34887 180 55 0 87856 0
[pid=10567/tid=10574] ppid=10565 vsize=359880 CPUtime=2.74 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 3273 1259 0 0 274 0 0 0 23 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=359880 CPUtime=2.7 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 3292 1259 0 0 270 0 0 0 25 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=359880 CPUtime=2.74 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 4029 1259 0 0 274 0 0 0 23 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4258201 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=359880 CPUtime=2.73 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 17852 1259 0 0 271 2 0 0 25 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=359880 CPUtime=2.73 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 2298 1259 0 0 273 0 0 0 25 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=359880 CPUtime=2.74 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 3634 1259 0 0 274 0 0 0 25 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4235999 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=359880 CPUtime=2.74 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 2945 1259 0 0 274 0 0 0 23 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249519 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=359880 CPUtime=2.74 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 3982 1259 0 0 274 0 0 0 22 0 9 0 799214034 368517120 34887 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 22.29
Current children cumulated vsize (KiB) 359880

[startup+6.30069 s]
/proc/loadavg: 7.42 7.91 7.97 9/500 10581
/proc/meminfo: memFree=13967108/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=378060 CPUtime=47.84 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 75434 1259 0 0 4772 12 0 0 25 0 9 0 799214000 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 94515 39220 180 55 0 92401 0
[pid=10567/tid=10574] ppid=10565 vsize=378060 CPUtime=5.94 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 3273 1259 0 0 594 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4258359 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=378060 CPUtime=5.85 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 3292 1259 0 0 585 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=378060 CPUtime=5.94 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 4212 1259 0 0 594 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4264920 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=378060 CPUtime=5.94 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 34168 1259 0 0 588 6 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=378060 CPUtime=5.93 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 2298 1259 0 0 593 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=378060 CPUtime=5.95 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 3634 1259 0 0 595 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=378060 CPUtime=5.94 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 3244 1259 0 0 594 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4343993 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=378060 CPUtime=5.94 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 3982 1259 0 0 594 0 0 0 25 0 9 0 799214034 387133440 39220 33554432000 4194304 4417452 140733819885392 18446744073709551615 4235968 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 47.84
Current children cumulated vsize (KiB) 378060

[startup+12.7005 s]
/proc/loadavg: 7.46 7.91 7.97 9/500 10581
/proc/meminfo: memFree=13951864/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=416748 CPUtime=98.94 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 112211 1259 0 0 9876 18 0 0 25 0 9 0 799214000 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 104187 49104 180 55 0 102073 0
[pid=10567/tid=10574] ppid=10565 vsize=416748 CPUtime=12.34 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 5749 1259 0 0 1234 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249563 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=416748 CPUtime=12.14 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 3292 1259 0 0 1214 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=416748 CPUtime=12.34 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 4212 1259 0 0 1234 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=416748 CPUtime=12.34 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 64755 1259 0 0 1224 10 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4354510 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=416748 CPUtime=12.33 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 3536 1259 0 0 1233 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=416748 CPUtime=12.35 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 4872 1259 0 0 1235 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4335984 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=416748 CPUtime=12.34 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 3244 1259 0 0 1234 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=416748 CPUtime=12.34 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 5220 1259 0 0 1234 0 0 0 25 0 9 0 799214034 426749952 49104 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 98.94
Current children cumulated vsize (KiB) 416748

[startup+25.5011 s]
/proc/loadavg: 7.58 7.92 7.97 9/500 10582
/proc/meminfo: memFree=13859656/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=473512 CPUtime=201.11 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 172960 1259 0 0 20082 29 0 0 25 0 9 0 799214000 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 118378 63327 180 55 0 116264 0
[pid=10567/tid=10574] ppid=10565 vsize=473512 CPUtime=25.15 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 6987 1259 0 0 2514 1 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364912 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=473512 CPUtime=24.73 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 4530 1259 0 0 2472 1 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366197 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=473512 CPUtime=25.15 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 7102 1259 0 0 2514 1 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=473512 CPUtime=25.14 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 112942 1259 0 0 2494 20 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=473512 CPUtime=25.14 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 4561 1259 0 0 2513 1 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=473512 CPUtime=25.14 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 5760 1259 0 0 2514 0 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4358074 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=473512 CPUtime=25.14 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 7273 1259 0 0 2513 1 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4248256 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=473512 CPUtime=25.14 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 6474 1259 0 0 2513 1 0 0 25 0 9 0 799214034 484876288 63327 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 201.11
Current children cumulated vsize (KiB) 473512

[startup+51.1004 s]
/proc/loadavg: 7.79 7.94 7.97 9/500 10582
/proc/meminfo: memFree=13700532/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=623636 CPUtime=405.49 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 251122 1259 0 0 40507 42 0 0 25 0 9 0 799214000 638603264 100826 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 155909 100826 180 55 0 153795 0
[pid=10567/tid=10574] ppid=10565 vsize=623636 CPUtime=50.74 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 9139 1259 0 0 5073 1 0 0 25 0 9 0 799214034 638603264 100826 33554432000 4194304 4417452 140733819885392 18446744073709551615 4365050 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=623636 CPUtime=49.92 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 5129 1259 0 0 4991 1 0 0 25 0 9 0 799214034 638603264 100826 33554432000 4194304 4417452 140733819885392 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=623636 CPUtime=50.74 cores=0-7

################
# More data... #
################


[startup+4722.32 s]
/proc/loadavg: 8.12 8.10 8.02 9/500 10728
/proc/meminfo: memFree=7658456/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=6656020 CPUtime=37697.1 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 6343355 1259 0 0 3768563 1149 0 0 25 0 9 0 799214000 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 1664005 1608846 180 55 0 1661891 0
[pid=10567/tid=10574] ppid=10565 vsize=6656020 CPUtime=4646.47 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 831590 1259 0 0 464520 127 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=6656020 CPUtime=4719.94 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 184967 1259 0 0 471967 27 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=6656020 CPUtime=4722.01 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 625623 1259 0 0 472083 118 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=6656020 CPUtime=4722.08 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 2145620 1259 0 0 471741 467 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=6656020 CPUtime=4721.66 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 534000 1259 0 0 472070 96 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=6656020 CPUtime=4721.82 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 555887 1259 0 0 472091 91 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=6656020 CPUtime=4720.77 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 630066 1259 0 0 471976 101 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=6656020 CPUtime=4721.93 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 818271 1259 0 0 472079 114 0 0 25 0 9 0 799214034 6815764480 1608846 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37697.1
Current children cumulated vsize (KiB) 6656020

[startup+4782.3 s]
/proc/loadavg: 8.09 8.09 8.02 9/500 10729
/proc/meminfo: memFree=7451368/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=6863568 CPUtime=38176 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 6451669 1259 0 0 3816433 1166 0 0 25 0 9 0 799214000 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 1715892 1660733 180 55 0 1713778 0
[pid=10567/tid=10574] ppid=10565 vsize=6863568 CPUtime=4705.49 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 859681 1259 0 0 470417 132 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=6863568 CPUtime=4779.92 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 184967 1259 0 0 477965 27 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4356980 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=6863568 CPUtime=4782 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 633815 1259 0 0 478080 120 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=6863568 CPUtime=4782.07 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 2145620 1259 0 0 477740 467 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=6863568 CPUtime=4781.64 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 546752 1259 0 0 478066 98 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=6863568 CPUtime=4781.81 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 585052 1259 0 0 478083 98 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=6863568 CPUtime=4780.74 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 660180 1259 0 0 477970 104 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=6863568 CPUtime=4781.91 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 818271 1259 0 0 478077 114 0 0 25 0 9 0 799214034 7028293632 1660733 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38176
Current children cumulated vsize (KiB) 6863568

[startup+4842.3 s]
/proc/loadavg: 8.08 8.09 8.02 9/500 10731
/proc/meminfo: memFree=7777116/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=6538620 CPUtime=38655 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 6514188 1259 0 0 3864318 1179 0 0 25 0 9 0 799214000 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 1634655 1579527 180 55 0 1632541 0
[pid=10567/tid=10574] ppid=10565 vsize=6538620 CPUtime=4764.48 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 869885 1259 0 0 476312 136 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=6538620 CPUtime=4839.92 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 184994 1259 0 0 483965 27 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=6538620 CPUtime=4842.01 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 633815 1259 0 0 484081 120 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=6538620 CPUtime=4842.07 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 2145620 1259 0 0 483740 467 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=6538620 CPUtime=4841.64 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 572481 1259 0 0 484062 102 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=6538620 CPUtime=4841.81 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 594167 1259 0 0 484082 99 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=6538620 CPUtime=4840.71 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 663364 1259 0 0 483967 104 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=6538620 CPUtime=4841.91 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 832531 1259 0 0 484074 117 0 0 25 0 9 0 799214034 6695546880 1579527 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38655
Current children cumulated vsize (KiB) 6538620

[startup+4902.3 s]
/proc/loadavg: 8.07 8.08 8.02 9/500 10733
/proc/meminfo: memFree=7732968/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=6583140 CPUtime=39134 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 6646442 1259 0 0 3912188 1211 0 0 25 0 9 0 799214000 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 1645785 1590657 180 55 0 1643671 0
[pid=10567/tid=10574] ppid=10565 vsize=6583140 CPUtime=4823.51 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 876678 1259 0 0 482213 138 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=6583140 CPUtime=4899.93 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 189090 1259 0 0 489964 29 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4320831 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=6583140 CPUtime=4902 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 643733 1259 0 0 490079 121 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=6583140 CPUtime=4902.07 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 2211156 1259 0 0 489721 486 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=6583140 CPUtime=4901.65 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 602803 1259 0 0 490057 108 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=6583140 CPUtime=4901.81 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 603389 1259 0 0 490081 100 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4236928 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=6583140 CPUtime=4900.71 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 669731 1259 0 0 489964 107 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=6583140 CPUtime=4901.91 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 832531 1259 0 0 490074 117 0 0 25 0 9 0 799214034 6741135360 1590657 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39134
Current children cumulated vsize (KiB) 6583140

[startup+4962.3 s]
/proc/loadavg: 8.02 8.07 8.01 9/500 10734
/proc/meminfo: memFree=7650008/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=6665560 CPUtime=39613 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 6740955 1259 0 0 3960078 1223 0 0 25 0 9 0 799214000 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 1666390 1611262 180 55 0 1664276 0
[pid=10567/tid=10574] ppid=10565 vsize=6665560 CPUtime=4883.4 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 887282 1259 0 0 488200 140 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=6665560 CPUtime=4959.06 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 189090 1259 0 0 495877 29 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=6665560 CPUtime=4962 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 684842 1259 0 0 496074 126 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=6665560 CPUtime=4962.08 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 2211156 1259 0 0 495722 486 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=6665560 CPUtime=4961.65 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 637057 1259 0 0 496053 112 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=6665560 CPUtime=4961.81 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 603389 1259 0 0 496081 100 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=6665560 CPUtime=4960.7 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 678277 1259 0 0 495962 108 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=6665560 CPUtime=4961.91 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 832531 1259 0 0 496074 117 0 0 25 0 9 0 799214034 6825533440 1611262 33554432000 4194304 4417452 140733819885392 18446744073709551615 4346508 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39613
Current children cumulated vsize (KiB) 6665560



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.8 s]
/proc/loadavg: 8.05 8.07 8.01 9/500 10736
/proc/meminfo: memFree=7375960/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=6939056 CPUtime=40000.3 cores=0-7
/proc/10567/stat : 10567 (plingeling) S 10565 10567 10351 0 -1 4202496 6854048 1259 0 0 3998785 1242 0 0 25 0 9 0 799214000 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 1734764 1679636 180 55 0 1732650 0
[pid=10567/tid=10574] ppid=10565 vsize=6939056 CPUtime=4931.91 cores=0-7
/proc/10567/task/10574/stat : 10574 (plingeling) R 10565 10567 10351 0 -1 4202560 936814 1259 0 0 493044 147 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4263247 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=6939056 CPUtime=5006.77 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4202560 189090 1259 0 0 500648 29 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4250633 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10567/tid=10576] ppid=10565 vsize=6939056 CPUtime=5010.53 cores=0-7
/proc/10567/task/10576/stat : 10576 (plingeling) R 10565 10567 10351 0 -1 4202560 689480 1259 0 0 500926 127 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4366146 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10567/tid=10577] ppid=10565 vsize=6939056 CPUtime=5010.6 cores=0-7
/proc/10567/task/10577/stat : 10577 (plingeling) R 10565 10567 10351 0 -1 4202560 2211156 1259 0 0 500574 486 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10567/tid=10578] ppid=10565 vsize=6939056 CPUtime=5010.17 cores=0-7
/proc/10567/task/10578/stat : 10578 (plingeling) R 10565 10567 10351 0 -1 4202560 637057 1259 0 0 500905 112 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10567/tid=10579] ppid=10565 vsize=6939056 CPUtime=5010.33 cores=0-7
/proc/10567/task/10579/stat : 10579 (plingeling) R 10565 10567 10351 0 -1 4202560 603389 1259 0 0 500933 100 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4357959 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10567/tid=10580] ppid=10565 vsize=6939056 CPUtime=5009.21 cores=0-7
/proc/10567/task/10580/stat : 10580 (plingeling) R 10565 10567 10351 0 -1 4202560 678277 1259 0 0 500813 108 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4263067 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10567/tid=10581] ppid=10565 vsize=6939056 CPUtime=5010.43 cores=0-7
/proc/10567/task/10581/stat : 10581 (plingeling) R 10565 10567 10351 0 -1 4202560 891454 1259 0 0 500915 128 0 0 25 0 9 0 799214034 7105593344 1679636 33554432000 4194304 4417452 140733819885392 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 6939056

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011 s]
/proc/loadavg: 8.05 8.07 8.01 9/500 10736
/proc/meminfo: memFree=7375960/32950928 swapFree=38710952/67111528
[pid=10567] ppid=10565 vsize=0 CPUtime=40000.6 cores=0-7
/proc/10567/stat : 10567 (plingeling) Z 10565 10567 10351 0 -1 4203548 6854049 1259 0 0 3998797 1259 0 0 18 0 2 0 799214000 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10567/statm: 0 0 0 0 0 0 0
[pid=10567/tid=10575] ppid=10565 vsize=0 CPUtime=5006.94 cores=0-7
/proc/10567/task/10575/stat : 10575 (plingeling) R 10565 10567 10351 0 -1 4203588 189090 1259 0 0 500648 46 0 0 25 0 2 0 799214034 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.05
CPU time (s): 40000.6
CPU user time (s): 39988
CPU system time (s): 12.6431
CPU usage (%): 798.249
Max. virtual memory (cumulated for all children) (KiB): 7091336

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988
system time used= 12.6431
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6855308
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 959
involuntary context switches= 193942

runsolver used 23.2575 second user time and 58.8211 second system time

The end

Launcher Data 

Begin job on node113 at 2011-05-22 20:43:53
IDJOB=3394104
IDBENCH=71492
IDSOLVER=1851
FILE ID=node113/3394104-1306089833
RUNJOBID= node113-1306084820-10365
PBS_JOBID= 13387920
Free space on /tmp= 73312 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/unforced/rbsat-v2400c266431g9.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394104-1306089833/watcher-3394104-1306089833 -o /tmp/evaluation-result-3394104-1306089833/solver-3394104-1306089833 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394104-1306089833.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 35278b3ce28129761cf8ae7994f37d3f
RANDOM SEED=667402418

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14107336 kB
Buffers:        332928 kB
Cached:        8774556 kB
SwapCached:      93732 kB
Active:        8058804 kB
Inactive:     10445384 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14107336 kB
SwapTotal:    67111528 kB
SwapFree:     38710952 kB
Dirty:            3532 kB
Writeback:           0 kB
AnonPages:     9314320 kB
Mapped:          25424 kB
Slab:           151880 kB
PageTables:     116244 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 154763412 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73312 MiB
End job on node113 at 2011-05-22 22:07:26