Trace number 3372719

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.8 5007.59

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom22.cnf
MD5SUM808b4b06b3dea960876e6772f0b9c207
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables53118
Number of clauses178062
Sum of the clauses size454634
Maximum clause length1670
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 290658
Number of clauses of size 386491
Number of clauses of size 4252
Number of clauses of size 520
Number of clauses of size over 5640

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372719-1305441204/watcher-3372719-1305441204 -o /tmp/evaluation-result-3372719-1305441204/solver-3372719-1305441204 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372719-1305441204.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.11 8.06 7.93 1/345 28077
/proc/meminfo: memFree=24694912/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=18880 CPUtime=0 cores=0-7
/proc/28077/stat : 28077 (plingeling) R 28075 28077 27815 0 -1 4202496 2813 1261 0 0 0 0 0 0 24 0 1 0 655717459 19333120 2743 33554432000 4194304 4417452 140736466190576 18446744073709551615 239476694784 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/28077/statm: 4720 2743 148 55 0 2606 0

[startup+0.0801259 s]
/proc/loadavg: 8.11 8.06 7.93 1/345 28077
/proc/meminfo: memFree=24694912/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=29788 CPUtime=0.06 cores=0-7
/proc/28077/stat : 28077 (plingeling) R 28075 28077 27815 0 -1 4202496 5581 1261 0 0 6 0 0 0 24 0 1 0 655717459 30502912 5455 33554432000 4194304 4417452 140736466190576 18446744073709551615 239476694784 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/28077/statm: 7447 5455 149 55 0 5333 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 29788

[startup+0.101116 s]
/proc/loadavg: 8.11 8.06 7.93 1/345 28077
/proc/meminfo: memFree=24694912/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=33884 CPUtime=0.08 cores=0-7
/proc/28077/stat : 28077 (plingeling) R 28075 28077 27815 0 -1 4202496 6605 1261 0 0 8 0 0 0 24 0 1 0 655717459 34697216 6479 33554432000 4194304 4417452 140736466190576 18446744073709551615 4274358 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/28077/statm: 8471 6479 149 55 0 6357 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 33884

[startup+0.301083 s]
/proc/loadavg: 8.11 8.06 7.93 1/345 28077
/proc/meminfo: memFree=24694912/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=81112 CPUtime=0.29 cores=0-7
/proc/28077/stat : 28077 (plingeling) R 28075 28077 27815 0 -1 4202496 18428 1261 0 0 27 2 0 0 25 0 1 0 655717459 83058688 18302 33554432000 4194304 4417452 140736466190576 18446744073709551615 4209679 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/28077/statm: 20278 18302 149 55 0 18164 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 81112

[startup+0.700935 s]
/proc/loadavg: 8.11 8.06 7.93 1/345 28077
/proc/meminfo: memFree=24694912/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=304568 CPUtime=2.21 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 32832 1261 0 0 217 4 0 0 25 0 9 0 655717459 311877632 22458 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 76142 22458 174 55 0 74028 0
Current children cumulated CPU time (s) 2.21
Current children cumulated vsize (KiB) 304568

[startup+1.50078 s]
/proc/loadavg: 8.11 8.06 7.93 9/354 28091
/proc/meminfo: memFree=24601228/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=306164 CPUtime=8.6 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 38307 1261 0 0 855 5 0 0 25 0 9 0 655717459 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 76541 22858 175 55 0 74427 0
[pid=28077/tid=28084] ppid=28075 vsize=306164 CPUtime=1.02 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 1524 1261 0 0 102 0 0 0 19 0 9 0 655717506 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4270729 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=306164 CPUtime=1.02 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 2069 1261 0 0 102 0 0 0 19 0 9 0 655717506 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4286960 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=306164 CPUtime=1.01 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 1411 1261 0 0 101 0 0 0 25 0 9 0 655717506 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4306634 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=306164 CPUtime=1.02 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 1166 1261 0 0 102 0 0 0 19 0 9 0 655717506 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=306164 CPUtime=1.01 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 2060 1261 0 0 101 0 0 0 25 0 9 0 655717506 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4288920 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=306164 CPUtime=1 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 5210 1261 0 0 100 0 0 0 19 0 9 0 655717507 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4284595 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=306164 CPUtime=0.99 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 2310 1261 0 0 99 0 0 0 25 0 9 0 655717507 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4284572 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=306164 CPUtime=1 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 4113 1261 0 0 100 0 0 0 19 0 9 0 655717507 313511936 22858 33554432000 4194304 4417452 140736466190576 18446744073709551615 4284289 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 8.6
Current children cumulated vsize (KiB) 306164

[startup+3.10047 s]
/proc/loadavg: 8.10 8.06 7.93 9/354 28091
/proc/meminfo: memFree=24600004/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=343020 CPUtime=21.38 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 58625 1261 0 0 2130 8 0 0 25 0 9 0 655717459 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 85755 31307 180 55 0 83641 0
[pid=28077/tid=28084] ppid=28075 vsize=343020 CPUtime=2.61 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 4511 1261 0 0 261 0 0 0 23 0 9 0 655717506 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=343020 CPUtime=2.62 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 4176 1261 0 0 262 0 0 0 22 0 9 0 655717506 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4234242 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=343020 CPUtime=2.61 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 4545 1261 0 0 260 1 0 0 25 0 9 0 655717506 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4236302 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=343020 CPUtime=2.62 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 2498 1261 0 0 262 0 0 0 23 0 9 0 655717506 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=343020 CPUtime=2.61 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 4062 1261 0 0 261 0 0 0 25 0 9 0 655717506 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4270935 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=343020 CPUtime=2.6 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 10080 1261 0 0 259 1 0 0 22 0 9 0 655717507 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4346482 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=343020 CPUtime=2.58 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 4216 1261 0 0 257 1 0 0 25 0 9 0 655717507 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4330445 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=343020 CPUtime=2.6 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 6093 1261 0 0 260 0 0 0 22 0 9 0 655717507 351252480 31307 33554432000 4194304 4417452 140736466190576 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 21.38
Current children cumulated vsize (KiB) 343020

[startup+6.30086 s]
/proc/loadavg: 8.10 8.06 7.93 9/354 28091
/proc/meminfo: memFree=24577436/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=329332 CPUtime=46.96 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 62793 1261 0 0 4686 10 0 0 25 0 9 0 655717459 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 82333 27915 180 55 0 80219 0
[pid=28077/tid=28084] ppid=28075 vsize=329332 CPUtime=5.81 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 5385 1261 0 0 581 0 0 0 25 0 9 0 655717506 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4365984 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=329332 CPUtime=5.82 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 4940 1261 0 0 582 0 0 0 25 0 9 0 655717506 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4236941 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=329332 CPUtime=5.82 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 4564 1261 0 0 581 1 0 0 25 0 9 0 655717506 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4232693 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=329332 CPUtime=5.82 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 2498 1261 0 0 582 0 0 0 25 0 9 0 655717506 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=329332 CPUtime=5.82 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 4580 1261 0 0 581 1 0 0 25 0 9 0 655717506 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=329332 CPUtime=5.8 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 10953 1261 0 0 579 1 0 0 25 0 9 0 655717507 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4320831 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=329332 CPUtime=5.74 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 4976 1261 0 0 573 1 0 0 25 0 9 0 655717507 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=329332 CPUtime=5.8 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 6453 1261 0 0 580 0 0 0 25 0 9 0 655717507 337235968 27915 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364420 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 46.96
Current children cumulated vsize (KiB) 329332

[startup+12.7006 s]
/proc/loadavg: 8.08 8.06 7.93 9/354 28091
/proc/meminfo: memFree=24578684/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=330920 CPUtime=98.08 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 63316 1261 0 0 9798 10 0 0 25 0 9 0 655717459 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 82730 28312 180 55 0 80616 0
[pid=28077/tid=28084] ppid=28075 vsize=330920 CPUtime=12.21 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 5413 1261 0 0 1221 0 0 0 25 0 9 0 655717506 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4246991 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=330920 CPUtime=12.22 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 4940 1261 0 0 1222 0 0 0 25 0 9 0 655717506 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4325965 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=330920 CPUtime=12.22 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 4622 1261 0 0 1221 1 0 0 25 0 9 0 655717506 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4365103 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=330920 CPUtime=12.22 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 2498 1261 0 0 1222 0 0 0 25 0 9 0 655717506 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=330920 CPUtime=12.22 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 4607 1261 0 0 1221 1 0 0 25 0 9 0 655717506 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364446 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=330920 CPUtime=12.2 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 10981 1261 0 0 1219 1 0 0 25 0 9 0 655717507 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4302940 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=330920 CPUtime=12.07 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 4996 1261 0 0 1206 1 0 0 25 0 9 0 655717507 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=330920 CPUtime=12.2 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 6815 1261 0 0 1220 0 0 0 25 0 9 0 655717507 338862080 28312 33554432000 4194304 4417452 140736466190576 18446744073709551615 4237310 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 98.08
Current children cumulated vsize (KiB) 330920

[startup+25.5012 s]
/proc/loadavg: 8.14 8.07 7.94 9/354 28092
/proc/meminfo: memFree=24532552/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=372604 CPUtime=200.34 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 79642 1261 0 0 20022 12 0 0 25 0 9 0 655717459 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 93151 38777 180 55 0 91037 0
[pid=28077/tid=28084] ppid=28075 vsize=372604 CPUtime=25.02 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 7619 1261 0 0 2501 1 0 0 25 0 9 0 655717506 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=372604 CPUtime=25.01 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 8212 1261 0 0 2501 0 0 0 25 0 9 0 655717506 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=372604 CPUtime=25.01 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 5914 1261 0 0 2500 1 0 0 25 0 9 0 655717506 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4278249 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=372604 CPUtime=25.01 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 6523 1261 0 0 2501 0 0 0 25 0 9 0 655717506 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=372604 CPUtime=25.01 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 6186 1261 0 0 2500 1 0 0 25 0 9 0 655717506 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=372604 CPUtime=25 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 12021 1261 0 0 2499 1 0 0 25 0 9 0 655717507 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364690 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=372604 CPUtime=24.73 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 6696 1261 0 0 2472 1 0 0 25 0 9 0 655717507 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=372604 CPUtime=25 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 8027 1261 0 0 2500 0 0 0 25 0 9 0 655717507 381546496 38777 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 200.34
Current children cumulated vsize (KiB) 372604

[startup+51.1013 s]
/proc/loadavg: 8.09 8.06 7.94 9/354 28093
/proc/meminfo: memFree=24478456/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=436424 CPUtime=404.86 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 105120 1261 0 0 40471 15 0 0 25 0 9 0 655717459 446898176 54700 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 109106 54700 180 55 0 106992 0
[pid=28077/tid=28084] ppid=28075 vsize=436424 CPUtime=50.62 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 9500 1261 0 0 5061 1 0 0 25 0 9 0 655717506 446898176 54700 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=436424 CPUtime=50.61 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 8212 1261 0 0 5061 0 0 0 25 0 9 0 655717506 446898176 54700 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=436424 CPUtime=50.62 cores=0-7

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.06 8.06 8.00 9/354 28229
/proc/meminfo: memFree=21138428/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=3768532 CPUtime=37722.9 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 4219565 1261 0 0 3771531 756 0 0 25 0 9 0 655717459 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 942133 887653 180 55 0 940019 0
[pid=28077/tid=28084] ppid=28075 vsize=3768532 CPUtime=4721.98 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 583203 1261 0 0 472092 106 0 0 25 0 9 0 655717506 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=3768532 CPUtime=4721.7 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 522577 1261 0 0 472071 99 0 0 25 0 9 0 655717506 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=3768532 CPUtime=4683.93 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 333150 1261 0 0 468341 52 0 0 25 0 9 0 655717506 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=3768532 CPUtime=4714.95 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 511749 1261 0 0 471414 81 0 0 25 0 9 0 655717506 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364420 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=3768532 CPUtime=4721.12 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 672656 1261 0 0 471974 138 0 0 25 0 9 0 655717506 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=3768532 CPUtime=4721.97 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 338898 1261 0 0 472141 56 0 0 25 0 9 0 655717507 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=3768532 CPUtime=4714.68 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 607987 1261 0 0 471372 96 0 0 25 0 9 0 655717507 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=3768532 CPUtime=4721.98 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 630901 1261 0 0 472077 121 0 0 25 0 9 0 655717507 3858976768 887653 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37722.9
Current children cumulated vsize (KiB) 3768532

[startup+4782.3 s]
/proc/loadavg: 8.05 8.06 8.00 9/354 28231
/proc/meminfo: memFree=21094524/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=3812724 CPUtime=38202.1 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 4230613 1261 0 0 3819454 758 0 0 25 0 9 0 655717459 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 953181 898701 180 55 0 951067 0
[pid=28077/tid=28084] ppid=28075 vsize=3812724 CPUtime=4781.97 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 583203 1261 0 0 478091 106 0 0 25 0 9 0 655717506 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=3812724 CPUtime=4781.69 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 522577 1261 0 0 478070 99 0 0 25 0 9 0 655717506 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4236956 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=3812724 CPUtime=4743.92 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 333150 1261 0 0 474340 52 0 0 25 0 9 0 655717506 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=3812724 CPUtime=4774.3 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 516765 1261 0 0 477348 82 0 0 25 0 9 0 655717506 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=3812724 CPUtime=4781.1 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 675232 1261 0 0 477971 139 0 0 25 0 9 0 655717506 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=3812724 CPUtime=4781.96 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 338898 1261 0 0 478140 56 0 0 25 0 9 0 655717507 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=3812724 CPUtime=4774.67 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 607987 1261 0 0 477371 96 0 0 25 0 9 0 655717507 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=3812724 CPUtime=4781.97 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 634357 1261 0 0 478075 122 0 0 25 0 9 0 655717507 3904229376 898701 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38202.1
Current children cumulated vsize (KiB) 3812724

[startup+4842.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/354 28232
/proc/meminfo: memFree=21382012/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=3526108 CPUtime=38681.5 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 4290565 1261 0 0 3867375 771 0 0 25 0 9 0 655717459 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 881527 827047 180 55 0 879413 0
[pid=28077/tid=28084] ppid=28075 vsize=3526108 CPUtime=4841.97 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 588234 1261 0 0 484089 108 0 0 25 0 9 0 655717506 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=3526108 CPUtime=4841.69 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 534673 1261 0 0 484066 103 0 0 25 0 9 0 655717506 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=3526108 CPUtime=4803.92 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 333150 1261 0 0 480340 52 0 0 25 0 9 0 655717506 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=3526108 CPUtime=4833.64 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 526821 1261 0 0 483279 85 0 0 25 0 9 0 655717506 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=3526108 CPUtime=4841.09 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 675232 1261 0 0 483970 139 0 0 25 0 9 0 655717506 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=3526108 CPUtime=4841.96 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 338898 1261 0 0 484140 56 0 0 25 0 9 0 655717507 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=3526108 CPUtime=4834.67 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 624371 1261 0 0 483370 97 0 0 25 0 9 0 655717507 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=3526108 CPUtime=4841.98 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 650742 1261 0 0 484074 124 0 0 25 0 9 0 655717507 3610734592 827047 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38681.5
Current children cumulated vsize (KiB) 3526108

[startup+4902.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/354 28234
/proc/meminfo: memFree=21278960/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=3631496 CPUtime=39160.8 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 4333297 1261 0 0 3915304 777 0 0 25 0 9 0 655717459 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 907874 853394 180 55 0 905760 0
[pid=28077/tid=28084] ppid=28075 vsize=3631496 CPUtime=4901.98 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 589644 1261 0 0 490089 109 0 0 25 0 9 0 655717506 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=3631496 CPUtime=4901.69 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 551057 1261 0 0 490065 104 0 0 25 0 9 0 655717506 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=3631496 CPUtime=4863.93 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 333162 1261 0 0 486341 52 0 0 25 0 9 0 655717506 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=3631496 CPUtime=4892.99 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 532413 1261 0 0 489213 86 0 0 25 0 9 0 655717506 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=3631496 CPUtime=4901.08 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 681286 1261 0 0 489968 140 0 0 25 0 9 0 655717506 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=3631496 CPUtime=4901.96 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 338898 1261 0 0 490140 56 0 0 25 0 9 0 655717507 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=3631496 CPUtime=4894.67 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 624371 1261 0 0 489370 97 0 0 25 0 9 0 655717507 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=3631496 CPUtime=4901.97 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 664022 1261 0 0 490071 126 0 0 25 0 9 0 655717507 3718651904 853394 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39160.8
Current children cumulated vsize (KiB) 3631496

[startup+4962.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/354 28235
/proc/meminfo: memFree=21182540/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=3725124 CPUtime=39640.1 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 4408185 1261 0 0 3963221 794 0 0 25 0 9 0 655717459 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 931281 876801 180 55 0 929167 0
[pid=28077/tid=28084] ppid=28075 vsize=3725124 CPUtime=4961.98 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 591148 1261 0 0 496089 109 0 0 25 0 9 0 655717506 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=3725124 CPUtime=4961.69 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 556877 1261 0 0 496062 107 0 0 25 0 9 0 655717506 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4353531 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=3725124 CPUtime=4923.93 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 349546 1261 0 0 492336 57 0 0 25 0 9 0 655717506 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=3725124 CPUtime=4952.32 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 539869 1261 0 0 495145 87 0 0 25 0 9 0 655717506 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=3725124 CPUtime=4961.07 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 681286 1261 0 0 495967 140 0 0 25 0 9 0 655717506 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=3725124 CPUtime=4961.97 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 366237 1261 0 0 496134 63 0 0 25 0 9 0 655717507 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=3725124 CPUtime=4954.67 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 624371 1261 0 0 495370 97 0 0 25 0 9 0 655717507 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=3725124 CPUtime=4961.97 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 680407 1261 0 0 496069 128 0 0 25 0 9 0 655717507 3814526976 876801 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39640.2
Current children cumulated vsize (KiB) 3725124



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.4 s]
/proc/loadavg: 8.00 8.02 8.00 9/354 28237
/proc/meminfo: memFree=21191460/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=3716144 CPUtime=40000.6 cores=0-7
/proc/28077/stat : 28077 (plingeling) S 28075 28077 27815 0 -1 4202496 4509999 1261 0 0 3999238 817 0 0 25 0 9 0 655717459 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 929036 874556 180 55 0 926922 0
[pid=28077/tid=28084] ppid=28075 vsize=3716144 CPUtime=5007.09 cores=0-7
/proc/28077/task/28084/stat : 28084 (plingeling) R 28075 28077 27815 0 -1 4202560 591148 1261 0 0 500600 109 0 0 25 0 9 0 655717506 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28077/tid=28085] ppid=28075 vsize=3716144 CPUtime=5006.8 cores=0-7
/proc/28077/task/28085/stat : 28085 (plingeling) R 28075 28077 27815 0 -1 4202560 562685 1261 0 0 500572 108 0 0 25 0 9 0 655717506 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28077/tid=28086] ppid=28075 vsize=3716144 CPUtime=4969.04 cores=0-7
/proc/28077/task/28086/stat : 28086 (plingeling) R 28075 28077 27815 0 -1 4202560 349546 1261 0 0 496847 57 0 0 25 0 9 0 655717506 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=3716144 CPUtime=4996.93 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4202560 542801 1261 0 0 499606 87 0 0 25 0 9 0 655717506 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28077/tid=28088] ppid=28075 vsize=3716144 CPUtime=5006.18 cores=0-7
/proc/28077/task/28088/stat : 28088 (plingeling) R 28075 28077 27815 0 -1 4202560 697670 1261 0 0 500476 142 0 0 25 0 9 0 655717506 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28077/tid=28089] ppid=28075 vsize=3716144 CPUtime=5007.08 cores=0-7
/proc/28077/task/28089/stat : 28089 (plingeling) R 28075 28077 27815 0 -1 4202560 394954 1261 0 0 500639 69 0 0 25 0 9 0 655717507 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28077/tid=28090] ppid=28075 vsize=3716144 CPUtime=4999.8 cores=0-7
/proc/28077/task/28090/stat : 28090 (plingeling) R 28075 28077 27815 0 -1 4202560 624371 1261 0 0 499882 98 0 0 25 0 9 0 655717507 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28077/tid=28091] ppid=28075 vsize=3716144 CPUtime=5007.09 cores=0-7
/proc/28077/task/28091/stat : 28091 (plingeling) R 28075 28077 27815 0 -1 4202560 728380 1261 0 0 500569 140 0 0 25 0 9 0 655717507 3805331456 874556 33554432000 4194304 4417452 140736466190576 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 3716144

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.52 s]
/proc/loadavg: 8.00 8.02 8.00 9/354 28237
/proc/meminfo: memFree=21191460/32950928 swapFree=67056568/67111528
[pid=28077] ppid=28075 vsize=0 CPUtime=40000.7 cores=0-7
/proc/28077/stat : 28077 (plingeling) Z 28075 28077 27815 0 -1 4203548 4509999 1261 0 0 3999243 827 0 0 18 0 2 0 655717459 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28077/statm: 0 0 0 0 0 0 0
[pid=28077/tid=28087] ppid=28075 vsize=0 CPUtime=4997.03 cores=0-7
/proc/28077/task/28087/stat : 28087 (plingeling) R 28075 28077 27815 0 -1 4203588 542801 1261 0 0 499606 97 0 0 25 0 2 0 655717506 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5007.59
CPU time (s): 40000.8
CPU user time (s): 39992.4
CPU system time (s): 8.34473
CPU usage (%): 798.803
Max. virtual memory (cumulated for all children) (KiB): 4245384

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39992.4
system time used= 8.34473
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4511260
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 775
involuntary context switches= 193682

runsolver used 15.6126 second user time and 39.228 second system time

The end

Launcher Data 

Begin job on node148 at 2011-05-15 08:33:24
IDJOB=3372719
IDBENCH=83133
IDSOLVER=1851
FILE ID=node148/3372719-1305441204
RUNJOBID= node148-1305437299-27829
PBS_JOBID= 13324768
Free space on /tmp= 72896 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom22.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372719-1305441204/watcher-3372719-1305441204 -o /tmp/evaluation-result-3372719-1305441204/solver-3372719-1305441204 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372719-1305441204.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 808b4b06b3dea960876e6772f0b9c207
RANDOM SEED=1133954710

node148.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24695192 kB
Buffers:        325852 kB
Cached:        4648052 kB
SwapCached:       8488 kB
Active:        4692972 kB
Inactive:      3215352 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24695192 kB
SwapTotal:    67111528 kB
SwapFree:     67056568 kB
Dirty:            3500 kB
Writeback:           0 kB
AnonPages:     2933420 kB
Mapped:          27192 kB
Slab:           252820 kB
PageTables:      29336 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 84644004 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72896 MiB
End job on node148 at 2011-05-15 09:56:53