Trace number 3257238

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.27 1201.72

General information on the benchmark

Namecrafted/spence/
hard/s113-100.cnf
MD5SUM346cb413e25976ab9ce2952e1d53c865
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark250.158
Satisfiable
(Un)Satisfiability was proved
Number of variables113
Number of clauses236
Sum of the clauses size708
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3236
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3257238-1304038330/watcher-3257238-1304038330 -o /tmp/evaluation-result-3257238-1304038330/solver-3257238-1304038330 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3257238-1304038330.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.96 7.98 7.95 6/181 26653
/proc/meminfo: memFree=19612752/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=49844 CPUtime=0 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 321 1258 0 0 0 0 0 0 25 0 5 0 594063664 51040256 247 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 12461 247 173 55 0 10347 0
[pid=26653/tid=26659] ppid=26651 vsize=49844 CPUtime=0 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 30 1258 0 0 0 0 0 0 25 0 5 0 594063664 51040256 247 33554432000 4194304 4417332 140735809527200 18446744073709551615 4253381 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=49844 CPUtime=0 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 0 1258 0 0 0 0 0 0 25 0 5 0 594063664 51040256 247 33554432000 4194304 4417332 140735809527200 18446744073709551615 251367603185 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=49844 CPUtime=0 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 0 1258 0 0 0 0 0 0 25 0 5 0 594063664 51040256 247 33554432000 4194304 4417332 140735809527200 18446744073709551615 251367603185 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=49844 CPUtime=0 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 6 1258 0 0 0 0 0 0 25 0 5 0 594063664 51040256 247 33554432000 4194304 4417332 140735809527200 18446744073709551615 251367240680 0 0 4096 17442 0 0 0 -1 1 0 0 0

[startup+0.156457 s]
/proc/loadavg: 7.96 7.98 7.95 6/181 26653
/proc/meminfo: memFree=19612752/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=119428 CPUtime=0.6 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1762 1258 0 0 60 0 0 0 25 0 5 0 594063664 122294272 1312 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 29857 1312 179 55 0 27743 0
[pid=26653/tid=26659] ppid=26651 vsize=119428 CPUtime=0.15 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 312 1258 0 0 15 0 0 0 25 0 5 0 594063664 122294272 1312 33554432000 4194304 4417332 140735809527200 18446744073709551615 4353800 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=119428 CPUtime=0.14 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 327 1258 0 0 14 0 0 0 25 0 5 0 594063664 122294272 1312 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364776 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=119428 CPUtime=0.14 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 580 1258 0 0 14 0 0 0 25 0 5 0 594063664 122294272 1312 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=119428 CPUtime=0.15 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 258 1258 0 0 15 0 0 0 25 0 5 0 594063664 122294272 1312 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 119428

[startup+0.20045 s]
/proc/loadavg: 7.96 7.98 7.95 6/181 26653
/proc/meminfo: memFree=19612752/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=119196 CPUtime=0.77 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 2186 1258 0 0 77 0 0 0 25 0 5 0 594063664 122056704 1254 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 29799 1254 179 55 0 27685 0
[pid=26653/tid=26659] ppid=26651 vsize=119196 CPUtime=0.19 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 427 1258 0 0 19 0 0 0 25 0 5 0 594063664 122056704 1254 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=119196 CPUtime=0.19 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 494 1258 0 0 19 0 0 0 25 0 5 0 594063664 122056704 1254 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=119196 CPUtime=0.19 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 656 1258 0 0 19 0 0 0 25 0 5 0 594063664 122056704 1254 33554432000 4194304 4417332 140735809527200 18446744073709551615 4251045 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=119196 CPUtime=0.19 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 324 1258 0 0 19 0 0 0 25 0 5 0 594063664 122056704 1254 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.77
Current children cumulated vsize (KiB) 119196

[startup+0.300431 s]
/proc/loadavg: 7.96 7.98 7.95 6/181 26653
/proc/meminfo: memFree=19612752/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=121360 CPUtime=1.17 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 3188 1258 0 0 117 0 0 0 25 0 5 0 594063664 124272640 1795 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 30340 1795 179 55 0 28226 0
[pid=26653/tid=26659] ppid=26651 vsize=121360 CPUtime=0.29 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 761 1258 0 0 29 0 0 0 25 0 5 0 594063664 124272640 1795 33554432000 4194304 4417332 140735809527200 18446744073709551615 4250731 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=121360 CPUtime=0.29 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 725 1258 0 0 29 0 0 0 25 0 5 0 594063664 124272640 1795 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=121360 CPUtime=0.29 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 943 1258 0 0 29 0 0 0 25 0 5 0 594063664 124272640 1795 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=121360 CPUtime=0.29 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 474 1258 0 0 29 0 0 0 25 0 5 0 594063664 124272640 1795 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 121360

[startup+0.700358 s]
/proc/loadavg: 7.96 7.98 7.95 6/181 26653
/proc/meminfo: memFree=19612752/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=122436 CPUtime=2.77 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 5965 1258 0 0 276 1 0 0 25 0 5 0 594063664 125374464 2064 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 30609 2064 179 55 0 28495 0
[pid=26653/tid=26659] ppid=26651 vsize=122436 CPUtime=0.69 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 1395 1258 0 0 69 0 0 0 25 0 5 0 594063664 125374464 2064 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=122436 CPUtime=0.68 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 1727 1258 0 0 68 0 0 0 25 0 5 0 594063664 125374464 2064 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=122436 CPUtime=0.69 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 1725 1258 0 0 69 0 0 0 25 0 5 0 594063664 125374464 2064 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=122436 CPUtime=0.69 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 833 1258 0 0 69 0 0 0 25 0 5 0 594063664 125374464 2064 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 122436

[startup+1.50021 s]
/proc/loadavg: 7.96 7.98 7.95 9/186 26663
/proc/meminfo: memFree=19577016/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=127704 CPUtime=5.96 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 10916 1258 0 0 594 2 0 0 25 0 5 0 594063664 130768896 3381 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 31926 3381 179 55 0 29812 0
[pid=26653/tid=26659] ppid=26651 vsize=127704 CPUtime=1.48 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 2567 1258 0 0 148 0 0 0 25 0 5 0 594063664 130768896 3381 33554432000 4194304 4417332 140735809527200 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=127704 CPUtime=1.48 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 2922 1258 0 0 148 0 0 0 25 0 5 0 594063664 130768896 3381 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=127704 CPUtime=1.48 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 2954 1258 0 0 148 0 0 0 25 0 5 0 594063664 130768896 3381 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=127704 CPUtime=1.49 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 2188 1258 0 0 149 0 0 0 25 0 5 0 594063664 130768896 3381 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.96
Current children cumulated vsize (KiB) 127704

[startup+3.10092 s]
/proc/loadavg: 7.96 7.98 7.95 9/186 26663
/proc/meminfo: memFree=19592288/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=130964 CPUtime=12.36 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 18235 1258 0 0 1233 3 0 0 25 0 5 0 594063664 134107136 4196 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 32741 4196 179 55 0 30627 0
[pid=26653/tid=26659] ppid=26651 vsize=130964 CPUtime=3.09 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 4642 1258 0 0 308 1 0 0 25 0 5 0 594063664 134107136 4196 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=130964 CPUtime=3.07 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 4746 1258 0 0 307 0 0 0 25 0 5 0 594063664 134107136 4196 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=130964 CPUtime=3.08 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 5071 1258 0 0 308 0 0 0 25 0 5 0 594063664 134107136 4196 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=130964 CPUtime=3.08 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 3491 1258 0 0 308 0 0 0 25 0 5 0 594063664 134107136 4196 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.36
Current children cumulated vsize (KiB) 130964

[startup+6.30032 s]
/proc/loadavg: 7.96 7.98 7.95 9/186 26664
/proc/meminfo: memFree=19571580/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=133396 CPUtime=25.14 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 32761 1258 0 0 2509 5 0 0 25 0 5 0 594063664 136597504 4797 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 33349 4797 179 55 0 31235 0
[pid=26653/tid=26659] ppid=26651 vsize=133396 CPUtime=6.29 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 8113 1258 0 0 628 1 0 0 25 0 5 0 594063664 136597504 4797 33554432000 4194304 4417332 140735809527200 18446744073709551615 4356768 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=133396 CPUtime=6.26 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 8366 1258 0 0 625 1 0 0 25 0 5 0 594063664 136597504 4797 33554432000 4194304 4417332 140735809527200 18446744073709551615 4251066 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=133396 CPUtime=6.28 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 9255 1258 0 0 627 1 0 0 25 0 5 0 594063664 136597504 4797 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=133396 CPUtime=6.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 6742 1258 0 0 628 0 0 0 25 0 5 0 594063664 136597504 4797 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 25.14
Current children cumulated vsize (KiB) 133396

[startup+12.7011 s]
/proc/loadavg: 7.96 7.98 7.95 9/186 26664
/proc/meminfo: memFree=19587208/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=141952 CPUtime=50.7 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 52017 1258 0 0 5062 8 0 0 25 0 5 0 594063664 145358848 6942 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 35488 6942 179 55 0 33374 0
[pid=26653/tid=26659] ppid=26651 vsize=141952 CPUtime=12.69 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 12822 1258 0 0 1267 2 0 0 25 0 5 0 594063664 145358848 6942 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=141952 CPUtime=12.62 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 13595 1258 0 0 1260 2 0 0 25 0 5 0 594063664 145358848 6942 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=141952 CPUtime=12.68 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 14430 1258 0 0 1266 2 0 0 25 0 5 0 594063664 145358848 6942 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=141952 CPUtime=12.69 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 10885 1258 0 0 1268 1 0 0 25 0 5 0 594063664 145358848 6942 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 50.7
Current children cumulated vsize (KiB) 141952

[startup+25.5008 s]
/proc/loadavg: 7.97 7.98 7.95 9/186 26664
/proc/meminfo: memFree=19573960/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=155620 CPUtime=101.83 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 87344 1258 0 0 10168 15 0 0 25 0 5 0 594063664 159354880 10353 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 38905 10353 179 55 0 36791 0
[pid=26653/tid=26659] ppid=26651 vsize=155620 CPUtime=25.48 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 20709 1258 0 0 2545 3 0 0 25 0 5 0 594063664 159354880 10353 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=155620 CPUtime=25.34 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 22415 1258 0 0 2531 3 0 0 25 0 5 0 594063664 159354880 10353 33554432000 4194304 4417332 140735809527200 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=155620 CPUtime=25.49 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 24942 1258 0 0 2544 5 0 0 25 0 5 0 594063664 159354880 10353 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=155620 CPUtime=25.49 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 18993 1258 0 0 2546 3 0 0 25 0 5 0 594063664 159354880 10353 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 155620

[startup+51.1011 s]
/proc/loadavg: 7.98 7.98 7.95 9/186 26665
/proc/meminfo: memFree=19542092/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=165416 CPUtime=204.09 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 140476 1258 0 0 20386 23 0 0 25 0 5 0 594063664 169385984 12800 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 41354 12800 179 55 0 39240 0
[pid=26653/tid=26659] ppid=26651 vsize=165416 CPUtime=51.08 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 32690 1258 0 0 5103 5 0 0 25 0 5 0 594063664 169385984 12800 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=165416 CPUtime=50.81 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 37724 1258 0 0 5075 6 0 0 25 0 5 0 594063664 169385984 12800 33554432000 4194304 4417332 140735809527200 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=165416 CPUtime=51.09 cores=1,3,5,7

################
# More data... #
################

[pid=26653/tid=26660] ppid=26651 vsize=238236 CPUtime=698.48 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 233645 1258 0 0 69811 37 0 0 25 0 5 0 594063664 243953664 30982 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=238236 CPUtime=702.31 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 244847 1258 0 0 70192 39 0 0 25 0 5 0 594063664 243953664 30982 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=238236 CPUtime=702.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 220716 1258 0 0 70193 35 0 0 25 0 5 0 594063664 243953664 30982 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2805.33
Current children cumulated vsize (KiB) 238236

[startup+762.301 s]
/proc/loadavg: 8.07 8.05 8.00 9/186 26730
/proc/meminfo: memFree=19187540/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=269076 CPUtime=3045.02 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 983878 1258 0 0 304343 159 0 0 25 0 5 0 594063664 275533824 38692 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 67269 38692 179 55 0 65155 0
[pid=26653/tid=26659] ppid=26651 vsize=269076 CPUtime=762.24 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 243587 1258 0 0 76183 41 0 0 25 0 5 0 594063664 275533824 38692 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=269076 CPUtime=758.15 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 249740 1258 0 0 75776 39 0 0 25 0 5 0 594063664 275533824 38692 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=269076 CPUtime=762.31 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 256512 1258 0 0 76191 40 0 0 25 0 5 0 594063664 275533824 38692 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=269076 CPUtime=762.29 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 233754 1258 0 0 76192 37 0 0 25 0 5 0 594063664 275533824 38692 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3045.02
Current children cumulated vsize (KiB) 269076

[startup+822.301 s]
/proc/loadavg: 8.08 8.05 8.00 9/186 26731
/proc/meminfo: memFree=19256848/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=246888 CPUtime=3284.68 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1040362 1258 0 0 328300 168 0 0 25 0 5 0 594063664 252813312 33145 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 61722 33145 179 55 0 59608 0
[pid=26653/tid=26659] ppid=26651 vsize=246888 CPUtime=822.23 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 258873 1258 0 0 82179 44 0 0 25 0 5 0 594063664 252813312 33145 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=246888 CPUtime=817.83 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 261241 1258 0 0 81742 41 0 0 25 0 5 0 594063664 252813312 33145 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=246888 CPUtime=822.3 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 271975 1258 0 0 82188 42 0 0 25 0 5 0 594063664 252813312 33145 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=246888 CPUtime=822.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 247988 1258 0 0 82189 39 0 0 25 0 5 0 594063664 252813312 33145 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3284.68
Current children cumulated vsize (KiB) 246888

[startup+882.306 s]
/proc/loadavg: 8.12 8.07 8.01 9/186 26733
/proc/meminfo: memFree=19265272/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=246160 CPUtime=3524.37 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1099364 1258 0 0 352259 178 0 0 25 0 5 0 594063664 252067840 32963 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 61540 32963 179 55 0 59426 0
[pid=26653/tid=26659] ppid=26651 vsize=246160 CPUtime=882.24 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 275313 1258 0 0 88178 46 0 0 25 0 5 0 594063664 252067840 32963 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364887 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=246160 CPUtime=877.51 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 273998 1258 0 0 87708 43 0 0 25 0 5 0 594063664 252067840 32963 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=246160 CPUtime=882.31 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 287893 1258 0 0 88185 46 0 0 25 0 5 0 594063664 252067840 32963 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=246160 CPUtime=882.29 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 261875 1258 0 0 88187 42 0 0 25 0 5 0 594063664 252067840 32963 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3524.37
Current children cumulated vsize (KiB) 246160

[startup+942.3 s]
/proc/loadavg: 8.08 8.07 8.01 9/186 26734
/proc/meminfo: memFree=19276176/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=240848 CPUtime=3764.03 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1155525 1258 0 0 376217 186 0 0 25 0 5 0 594063664 246628352 31635 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 60212 31635 179 55 0 58098 0
[pid=26653/tid=26659] ppid=26651 vsize=240848 CPUtime=942.22 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 289360 1258 0 0 94175 47 0 0 25 0 5 0 594063664 246628352 31635 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=240848 CPUtime=937.18 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 288034 1258 0 0 93673 45 0 0 25 0 5 0 594063664 246628352 31635 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=240848 CPUtime=942.31 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 302371 1258 0 0 94183 48 0 0 25 0 5 0 594063664 246628352 31635 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=240848 CPUtime=942.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 275475 1258 0 0 94184 44 0 0 25 0 5 0 594063664 246628352 31635 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3764.03
Current children cumulated vsize (KiB) 240848

[startup+1002.3 s]
/proc/loadavg: 8.07 8.07 8.01 9/186 26736
/proc/meminfo: memFree=19216152/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=245144 CPUtime=4003.7 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1214571 1258 0 0 400174 196 0 0 25 0 5 0 594063664 251027456 32729 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 61286 32729 179 55 0 59172 0
[pid=26653/tid=26659] ppid=26651 vsize=245144 CPUtime=1002.22 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 308472 1258 0 0 100172 50 0 0 25 0 5 0 594063664 251027456 32729 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=245144 CPUtime=996.85 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 301078 1258 0 0 99637 48 0 0 25 0 5 0 594063664 251027456 32729 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=245144 CPUtime=1002.32 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 317752 1258 0 0 100181 51 0 0 25 0 5 0 594063664 251027456 32729 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=245144 CPUtime=1002.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 286984 1258 0 0 100183 45 0 0 25 0 5 0 594063664 251027456 32729 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4003.7
Current children cumulated vsize (KiB) 245144

[startup+1062.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/186 26738
/proc/meminfo: memFree=19160092/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=250376 CPUtime=4243.37 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1268537 1258 0 0 424132 205 0 0 25 0 5 0 594063664 256385024 34037 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 62594 34037 179 55 0 60480 0
[pid=26653/tid=26659] ppid=26651 vsize=250376 CPUtime=1062.22 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 323676 1258 0 0 106169 53 0 0 25 0 5 0 594063664 256385024 34037 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=250376 CPUtime=1056.53 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 315049 1258 0 0 105603 50 0 0 25 0 5 0 594063664 256385024 34037 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=250376 CPUtime=1062.31 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 331534 1258 0 0 106178 53 0 0 25 0 5 0 594063664 256385024 34037 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=250376 CPUtime=1062.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 297993 1258 0 0 106181 47 0 0 25 0 5 0 594063664 256385024 34037 33554432000 4194304 4417332 140735809527200 18446744073709551615 4366002 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4243.37
Current children cumulated vsize (KiB) 250376

[startup+1122.3 s]
/proc/loadavg: 8.17 8.10 8.02 9/186 26739
/proc/meminfo: memFree=19242048/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=260756 CPUtime=4483.05 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1320399 1258 0 0 448092 213 0 0 25 0 5 0 594063664 267014144 36612 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 65189 36612 179 55 0 63075 0
[pid=26653/tid=26659] ppid=26651 vsize=260756 CPUtime=1122.21 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 339800 1258 0 0 112166 55 0 0 25 0 5 0 594063664 267014144 36612 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=260756 CPUtime=1116.2 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 327430 1258 0 0 111568 52 0 0 25 0 5 0 594063664 267014144 36612 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=260756 CPUtime=1122.32 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 343885 1258 0 0 112177 55 0 0 25 0 5 0 594063664 267014144 36612 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=260756 CPUtime=1122.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 308999 1258 0 0 112179 49 0 0 25 0 5 0 594063664 267014144 36612 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4483.05
Current children cumulated vsize (KiB) 260756

[startup+1182.3 s]
/proc/loadavg: 8.17 8.11 8.02 9/186 26741
/proc/meminfo: memFree=19247240/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=260684 CPUtime=4722.72 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1374466 1258 0 0 472051 221 0 0 25 0 5 0 594063664 266940416 36618 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 65171 36618 179 55 0 63057 0
[pid=26653/tid=26659] ppid=26651 vsize=260684 CPUtime=1182.21 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 347369 1258 0 0 118164 57 0 0 25 0 5 0 594063664 266940416 36618 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364498 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=260684 CPUtime=1175.88 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 346203 1258 0 0 117533 55 0 0 25 0 5 0 594063664 266940416 36618 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=260684 CPUtime=1182.32 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 359064 1258 0 0 118175 57 0 0 25 0 5 0 594063664 266940416 36618 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=260684 CPUtime=1182.28 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 321545 1258 0 0 118177 51 0 0 25 0 5 0 594063664 266940416 36618 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4722.72
Current children cumulated vsize (KiB) 260684



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.19 8.12 8.02 9/186 26741
/proc/meminfo: memFree=19244756/32950928 swapFree=67111364/67111528
[pid=26653] ppid=26651 vsize=246152 CPUtime=4800.23 cores=1,3,5,7
/proc/26653/stat : 26653 (plingeling) S 26651 26653 26261 0 -1 4202496 1388466 1258 0 0 479800 223 0 0 25 0 5 0 594063664 252059648 32985 33554432000 4194304 4417332 140735809527200 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/26653/statm: 61538 32985 179 55 0 59424 0
[pid=26653/tid=26659] ppid=26651 vsize=246152 CPUtime=1201.61 cores=1,3,5,7
/proc/26653/task/26659/stat : 26659 (plingeling) R 26651 26653 26261 0 -1 4202560 352088 1258 0 0 120104 57 0 0 25 0 5 0 594063664 252059648 32985 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26653/tid=26660] ppid=26651 vsize=246152 CPUtime=1195.17 cores=1,3,5,7
/proc/26653/task/26660/stat : 26660 (plingeling) R 26651 26653 26261 0 -1 4202560 348288 1258 0 0 119462 55 0 0 25 0 5 0 594063664 252059648 32985 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26653/tid=26661] ppid=26651 vsize=246152 CPUtime=1201.72 cores=1,3,5,7
/proc/26653/task/26661/stat : 26661 (plingeling) R 26651 26653 26261 0 -1 4202560 362667 1258 0 0 120115 57 0 0 25 0 5 0 594063664 252059648 32985 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26653/tid=26662] ppid=26651 vsize=246152 CPUtime=1201.68 cores=1,3,5,7
/proc/26653/task/26662/stat : 26662 (plingeling) R 26651 26653 26261 0 -1 4202560 325138 1258 0 0 120117 51 0 0 25 0 5 0 594063664 252059648 32985 33554432000 4194304 4417332 140735809527200 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.23
Current children cumulated vsize (KiB) 246152

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.72
CPU time (s): 4800.27
CPU user time (s): 4798.03
CPU system time (s): 2.24166
CPU usage (%): 399.45
Max. virtual memory (cumulated for all children) (KiB): 302740

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.03
system time used= 2.24166
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1389724
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 73
involuntary context switches= 34289

runsolver used 2.03069 second user time and 4.53431 second system time

The end

Launcher Data 

Begin job on node113 at 2011-04-29 02:52:10
IDJOB=3257238
IDBENCH=20442
IDSOLVER=1605
FILE ID=node113/3257238-1304038330
RUNJOBID= node113-1304036318-26280
PBS_JOBID= 13170352
Free space on /tmp= 71168 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT07/crafted/spence/hard/s113-100.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3257238-1304038330/watcher-3257238-1304038330 -o /tmp/evaluation-result-3257238-1304038330/solver-3257238-1304038330 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3257238-1304038330.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 346cb413e25976ab9ce2952e1d53c865
RANDOM SEED=1303500422

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      19613032 kB
Buffers:       1733060 kB
Cached:       10787828 kB
SwapCached:        164 kB
Active:        6443980 kB
Inactive:      6304256 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      19613032 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:             364 kB
Writeback:           0 kB
AnonPages:      227032 kB
Mapped:          15484 kB
Slab:           525788 kB
PageTables:       4916 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   411588 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71164 MiB
End job on node113 at 2011-04-29 03:12:14