Trace number 3263882

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.46 1201.74

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n28-01-S136611085.shuffled-as.sat03-1627.cnf
MD5SUM17624c574c7f9ee0eff4cc4a13e42e12
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark671.855
Satisfiable
(Un)Satisfiability was proved
Number of variables184
Number of clauses896
Sum of the clauses size3042
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 214
Number of clauses of size 3514
Number of clauses of size 4368
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263882-1304101106/watcher-3263882-1304101106 -o /tmp/evaluation-result-3263882-1304101106/solver-3263882-1304101106 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3263882-1304101106.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.69 4.81 6.42 1/172 27247
/proc/meminfo: memFree=30929664/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=49996 CPUtime=0 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) R 27245 27247 26666 0 -1 4202496 366 1259 0 0 0 0 0 0 23 0 4 0 521707643 51195904 290 33554432000 4194304 4417332 140737389875984 18446744073709551615 239476738138 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 12499 294 167 55 0 10385 0
[pid=27247/tid=27253] ppid=27245 vsize=49996 CPUtime=0 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 32 1259 0 0 0 0 0 0 24 0 5 0 521707644 51195904 294 33554432000 4194304 4417332 140737389875984 18446744073709551615 4275967 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=49996 CPUtime=0 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 0 1259 0 0 0 0 0 0 24 0 5 0 521707644 51195904 294 33554432000 4194304 4417332 140737389875984 18446744073709551615 239476751345 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=49996 CPUtime=0 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 0 1259 0 0 0 0 0 0 24 0 5 0 521707644 51195904 294 33554432000 4194304 4417332 140737389875984 18446744073709551615 239476751345 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=49996 CPUtime=0 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 1 1259 0 0 0 0 0 0 25 0 5 0 521707644 51195904 294 33554432000 4194304 4417332 140737389875984 18446744073709551615 239476387536 0 0 4096 17442 0 0 0 -1 0 0 0 0

[startup+0.0199651 s]
/proc/loadavg: 3.69 4.81 6.42 1/172 27247
/proc/meminfo: memFree=30929664/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=116016 CPUtime=0.05 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 530 1259 0 0 5 0 0 0 23 0 5 0 521707643 118800384 454 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 29004 454 178 55 0 26890 0
[pid=27247/tid=27253] ppid=27245 vsize=116016 CPUtime=0.01 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 50 1259 0 0 1 0 0 0 25 0 5 0 521707644 118800384 454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4272693 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=116016 CPUtime=0.01 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 23 1259 0 0 1 0 0 0 24 0 5 0 521707644 118800384 454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4253280 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=116016 CPUtime=0.01 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 51 1259 0 0 1 0 0 0 25 0 5 0 521707644 118800384 454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4356768 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=116016 CPUtime=0.01 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 69 1259 0 0 1 0 0 0 25 0 5 0 521707644 118800384 454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364475 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 116016

[startup+0.100888 s]
/proc/loadavg: 3.69 4.81 6.42 1/172 27247
/proc/meminfo: memFree=30929664/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=118412 CPUtime=0.37 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 1217 1259 0 0 37 0 0 0 23 0 5 0 521707643 121253888 1044 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 29603 1044 181 55 0 27489 0
[pid=27247/tid=27253] ppid=27245 vsize=118412 CPUtime=0.09 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 179 1259 0 0 9 0 0 0 25 0 5 0 521707644 121253888 1044 33554432000 4194304 4417332 140737389875984 18446744073709551615 4250721 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=118412 CPUtime=0.09 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 121 1259 0 0 9 0 0 0 25 0 5 0 521707644 121253888 1044 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=118412 CPUtime=0.09 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 238 1259 0 0 9 0 0 0 25 0 5 0 521707644 121253888 1044 33554432000 4194304 4417332 140737389875984 18446744073709551615 4357100 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=118412 CPUtime=0.09 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 342 1259 0 0 9 0 0 0 25 0 5 0 521707644 121253888 1044 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 118412

[startup+0.30085 s]
/proc/loadavg: 3.69 4.81 6.42 1/172 27247
/proc/meminfo: memFree=30929664/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=119980 CPUtime=1.17 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 2731 1259 0 0 117 0 0 0 23 0 5 0 521707643 122859520 1454 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 29995 1454 181 55 0 27881 0
[pid=27247/tid=27253] ppid=27245 vsize=119980 CPUtime=0.29 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 601 1259 0 0 29 0 0 0 25 0 5 0 521707644 122859520 1454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=119980 CPUtime=0.29 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 575 1259 0 0 29 0 0 0 25 0 5 0 521707644 122859520 1454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=119980 CPUtime=0.29 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 637 1259 0 0 29 0 0 0 25 0 5 0 521707644 122859520 1454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4255615 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=119980 CPUtime=0.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 581 1259 0 0 29 0 0 0 25 0 5 0 521707644 122859520 1454 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 119980

[startup+0.700774 s]
/proc/loadavg: 3.69 4.81 6.42 1/172 27247
/proc/meminfo: memFree=30929664/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=121976 CPUtime=2.77 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 4849 1259 0 0 276 1 0 0 23 0 5 0 521707643 124903424 1988 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 30494 1988 181 55 0 28380 0
[pid=27247/tid=27253] ppid=27245 vsize=121976 CPUtime=0.69 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 1144 1259 0 0 69 0 0 0 25 0 5 0 521707644 124903424 1988 33554432000 4194304 4417332 140737389875984 18446744073709551615 4236906 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=121976 CPUtime=0.69 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 1128 1259 0 0 69 0 0 0 25 0 5 0 521707644 124903424 1988 33554432000 4194304 4417332 140737389875984 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=121976 CPUtime=0.69 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 1199 1259 0 0 69 0 0 0 25 0 5 0 521707644 124903424 1988 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364614 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=121976 CPUtime=0.69 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 1041 1259 0 0 69 0 0 0 25 0 5 0 521707644 124903424 1988 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 121976

[startup+1.50062 s]
/proc/loadavg: 3.69 4.81 6.42 9/186 27274
/proc/meminfo: memFree=30911992/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=127316 CPUtime=5.96 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 8750 1259 0 0 595 1 0 0 23 0 5 0 521707643 130371584 3326 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 31829 3326 181 55 0 29715 0
[pid=27247/tid=27253] ppid=27245 vsize=127316 CPUtime=1.49 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 2161 1259 0 0 149 0 0 0 25 0 5 0 521707644 130371584 3326 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=127316 CPUtime=1.49 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 1982 1259 0 0 149 0 0 0 25 0 5 0 521707644 130371584 3326 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=127316 CPUtime=1.48 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 2334 1259 0 0 148 0 0 0 25 0 5 0 521707644 130371584 3326 33554432000 4194304 4417332 140737389875984 18446744073709551615 4250882 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=127316 CPUtime=1.48 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 1936 1259 0 0 148 0 0 0 25 0 5 0 521707644 130371584 3326 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.96
Current children cumulated vsize (KiB) 127316

[startup+3.10032 s]
/proc/loadavg: 4.04 4.86 6.43 9/186 27274
/proc/meminfo: memFree=30904428/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=127596 CPUtime=12.36 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 15267 1259 0 0 1234 2 0 0 23 0 5 0 521707643 130658304 3392 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 31899 3392 181 55 0 29785 0
[pid=27247/tid=27253] ppid=27245 vsize=127596 CPUtime=3.09 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 3437 1259 0 0 309 0 0 0 25 0 5 0 521707644 130658304 3392 33554432000 4194304 4417332 140737389875984 18446744073709551615 4355731 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=127596 CPUtime=3.08 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 3377 1259 0 0 308 0 0 0 25 0 5 0 521707644 130658304 3392 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=127596 CPUtime=3.07 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 3992 1259 0 0 307 0 0 0 25 0 5 0 521707644 130658304 3392 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=127596 CPUtime=3.08 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 4124 1259 0 0 308 0 0 0 25 0 5 0 521707644 130658304 3392 33554432000 4194304 4417332 140737389875984 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.36
Current children cumulated vsize (KiB) 127596

[startup+6.3007 s]
/proc/loadavg: 4.04 4.86 6.43 9/186 27275
/proc/meminfo: memFree=30889920/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=137992 CPUtime=25.14 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 28034 1259 0 0 2510 4 0 0 23 0 5 0 521707643 141303808 6004 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 34498 6004 181 55 0 32384 0
[pid=27247/tid=27253] ppid=27245 vsize=137992 CPUtime=6.28 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 6445 1259 0 0 628 0 0 0 25 0 5 0 521707644 141303808 6004 33554432000 4194304 4417332 140737389875984 18446744073709551615 4366042 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=137992 CPUtime=6.28 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 6719 1259 0 0 628 0 0 0 25 0 5 0 521707644 141303808 6004 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=137992 CPUtime=6.25 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 7361 1259 0 0 624 1 0 0 25 0 5 0 521707644 141303808 6004 33554432000 4194304 4417332 140737389875984 18446744073709551615 4365897 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=137992 CPUtime=6.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 7172 1259 0 0 628 1 0 0 25 0 5 0 521707644 141303808 6004 33554432000 4194304 4417332 140737389875984 18446744073709551615 4357131 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.14
Current children cumulated vsize (KiB) 137992

[startup+12.7005 s]
/proc/loadavg: 4.65 4.96 6.44 9/186 27275
/proc/meminfo: memFree=30880108/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=144036 CPUtime=50.71 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 45791 1259 0 0 5064 7 0 0 23 0 5 0 521707643 147492864 7511 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 36009 7511 181 55 0 33895 0
[pid=27247/tid=27253] ppid=27245 vsize=144036 CPUtime=12.69 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 10101 1259 0 0 1268 1 0 0 25 0 5 0 521707644 147492864 7511 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=144036 CPUtime=12.68 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 11683 1259 0 0 1267 1 0 0 25 0 5 0 521707644 147492864 7511 33554432000 4194304 4417332 140737389875984 18446744073709551615 4353386 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=144036 CPUtime=12.62 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 12200 1259 0 0 1260 2 0 0 25 0 5 0 521707644 147492864 7511 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=144036 CPUtime=12.68 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 11470 1259 0 0 1267 1 0 0 25 0 5 0 521707644 147492864 7511 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.71
Current children cumulated vsize (KiB) 144036

[startup+25.501 s]
/proc/loadavg: 5.16 5.06 6.46 9/186 27275
/proc/meminfo: memFree=30855928/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=156576 CPUtime=101.84 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 78438 1259 0 0 10171 13 0 0 23 0 5 0 521707643 160333824 10677 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 39144 10677 181 55 0 37030 0
[pid=27247/tid=27253] ppid=27245 vsize=156576 CPUtime=25.48 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 18455 1259 0 0 2546 2 0 0 25 0 5 0 521707644 160333824 10677 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=156576 CPUtime=25.49 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 20764 1259 0 0 2546 3 0 0 25 0 5 0 521707644 160333824 10677 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=156576 CPUtime=25.35 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 19990 1259 0 0 2532 3 0 0 25 0 5 0 521707644 160333824 10677 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=156576 CPUtime=25.49 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 18892 1259 0 0 2546 3 0 0 25 0 5 0 521707644 160333824 10677 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.84
Current children cumulated vsize (KiB) 156576

[startup+51.1011 s]
/proc/loadavg: 6.13 5.30 6.50 9/186 27276
/proc/meminfo: memFree=30829132/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=169360 CPUtime=204.11 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 123516 1259 0 0 20390 21 0 0 23 0 5 0 521707643 173424640 13902 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 42340 13902 181 55 0 40226 0
[pid=27247/tid=27253] ppid=27245 vsize=169360 CPUtime=51.09 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 32420 1259 0 0 5104 5 0 0 25 0 5 0 521707644 173424640 13902 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364965 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=169360 CPUtime=51.09 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 31994 1259 0 0 5104 5 0 0 25 0 5 0 521707644 173424640 13902 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=169360 CPUtime=50.82 cores=0,2,4,6

################
# More data... #
################

[pid=27247/tid=27254] ppid=27245 vsize=496736 CPUtime=762.32 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 113640 1259 0 0 76213 19 0 0 25 0 5 0 521707644 508657664 95866 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=496736 CPUtime=758.17 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 96625 1259 0 0 75800 17 0 0 25 0 5 0 521707644 508657664 95866 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=496736 CPUtime=762.3 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 205243 1259 0 0 76200 30 0 0 25 0 5 0 521707644 508657664 95866 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3045.14
Current children cumulated vsize (KiB) 496736

[startup+822.3 s]
/proc/loadavg: 8.06 7.84 7.36 9/186 27332
/proc/meminfo: memFree=30124332/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=498740 CPUtime=3284.8 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 568160 1259 0 0 328386 94 0 0 23 0 5 0 521707643 510709760 96371 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 124685 96371 181 55 0 122571 0
[pid=27247/tid=27253] ppid=27245 vsize=498740 CPUtime=822.31 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 129916 1259 0 0 82210 21 0 0 25 0 5 0 521707644 510709760 96371 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=498740 CPUtime=822.31 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 113640 1259 0 0 82212 19 0 0 25 0 5 0 521707644 510709760 96371 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=498740 CPUtime=817.84 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 109397 1259 0 0 81765 19 0 0 25 0 5 0 521707644 510709760 96371 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=498740 CPUtime=822.3 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 214870 1259 0 0 82197 33 0 0 25 0 5 0 521707644 510709760 96371 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3284.8
Current children cumulated vsize (KiB) 498740

[startup+882.301 s]
/proc/loadavg: 8.05 7.88 7.40 9/186 27334
/proc/meminfo: memFree=30060932/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=445732 CPUtime=3524.47 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 619425 1259 0 0 352344 103 0 0 23 0 5 0 521707643 456429568 83171 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 111433 83171 181 55 0 109319 0
[pid=27247/tid=27253] ppid=27245 vsize=445732 CPUtime=882.32 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 142906 1259 0 0 88209 23 0 0 25 0 5 0 521707644 456429568 83171 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=445732 CPUtime=882.32 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 136952 1259 0 0 88208 24 0 0 25 0 5 0 521707644 456429568 83171 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=445732 CPUtime=877.52 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 109397 1259 0 0 87733 19 0 0 25 0 5 0 521707644 456429568 83171 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=445732 CPUtime=882.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 229833 1259 0 0 88194 35 0 0 25 0 5 0 521707644 456429568 83171 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3524.47
Current children cumulated vsize (KiB) 445732

[startup+942.301 s]
/proc/loadavg: 8.06 7.91 7.44 9/186 27335
/proc/meminfo: memFree=29742104/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=444780 CPUtime=3764.16 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 629439 1259 0 0 376311 105 0 0 23 0 5 0 521707643 455454720 82954 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 111195 82954 181 55 0 109081 0
[pid=27247/tid=27253] ppid=27245 vsize=444780 CPUtime=942.32 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 143136 1259 0 0 94209 23 0 0 25 0 5 0 521707644 455454720 82954 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=444780 CPUtime=942.32 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 136952 1259 0 0 94208 24 0 0 25 0 5 0 521707644 455454720 82954 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=444780 CPUtime=937.19 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 109397 1259 0 0 93700 19 0 0 25 0 5 0 521707644 455454720 82954 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=444780 CPUtime=942.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 239617 1259 0 0 94192 37 0 0 25 0 5 0 521707644 455454720 82954 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3764.16
Current children cumulated vsize (KiB) 444780

[startup+1002.3 s]
/proc/loadavg: 8.12 7.96 7.48 9/186 27337
/proc/meminfo: memFree=29618196/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=453096 CPUtime=4003.84 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 636629 1259 0 0 400278 106 0 0 23 0 5 0 521707643 463970304 85035 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 113274 85035 181 55 0 111160 0
[pid=27247/tid=27253] ppid=27245 vsize=453096 CPUtime=1002.32 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 143596 1259 0 0 100209 23 0 0 25 0 5 0 521707644 463970304 85035 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=453096 CPUtime=1002.32 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 136952 1259 0 0 100208 24 0 0 25 0 5 0 521707644 463970304 85035 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=453096 CPUtime=996.87 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 109397 1259 0 0 99668 19 0 0 25 0 5 0 521707644 463970304 85035 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=453096 CPUtime=1002.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 246347 1259 0 0 100191 38 0 0 25 0 5 0 521707644 463970304 85035 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4003.84
Current children cumulated vsize (KiB) 453096

[startup+1062.3 s]
/proc/loadavg: 8.05 7.96 7.51 9/186 27339
/proc/meminfo: memFree=29467504/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=491908 CPUtime=4243.52 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 654779 1259 0 0 424243 109 0 0 23 0 5 0 521707643 503713792 94715 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 122977 94715 181 55 0 120863 0
[pid=27247/tid=27253] ppid=27245 vsize=491908 CPUtime=1062.32 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 152038 1259 0 0 106207 25 0 0 25 0 5 0 521707644 503713792 94715 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=491908 CPUtime=1062.32 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 137582 1259 0 0 106208 24 0 0 25 0 5 0 521707644 503713792 94715 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=491908 CPUtime=1056.54 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 109819 1259 0 0 105635 19 0 0 25 0 5 0 521707644 503713792 94715 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=491908 CPUtime=1062.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 255003 1259 0 0 106190 39 0 0 25 0 5 0 521707644 503713792 94715 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364581 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4243.52
Current children cumulated vsize (KiB) 491908

[startup+1122.3 s]
/proc/loadavg: 8.01 7.97 7.54 9/186 27340
/proc/meminfo: memFree=29829920/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=560148 CPUtime=4483.2 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 677191 1259 0 0 448207 113 0 0 23 0 5 0 521707643 573591552 111781 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 140037 111781 181 55 0 137923 0
[pid=27247/tid=27253] ppid=27245 vsize=560148 CPUtime=1122.33 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 160230 1259 0 0 112207 26 0 0 25 0 5 0 521707644 573591552 111781 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=560148 CPUtime=1122.33 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 146374 1259 0 0 112207 26 0 0 25 0 5 0 521707644 573591552 111781 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=560148 CPUtime=1116.22 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 109819 1259 0 0 111603 19 0 0 25 0 5 0 521707644 573591552 111781 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=560148 CPUtime=1122.29 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 260431 1259 0 0 112189 40 0 0 25 0 5 0 521707644 573591552 111781 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4483.2
Current children cumulated vsize (KiB) 560148

[startup+1182.3 s]
/proc/loadavg: 8.00 7.97 7.56 9/186 27342
/proc/meminfo: memFree=29488532/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=637404 CPUtime=4722.89 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 707538 1259 0 0 472171 118 0 0 23 0 5 0 521707643 652701696 131111 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 159351 131111 181 55 0 157237 0
[pid=27247/tid=27253] ppid=27245 vsize=637404 CPUtime=1182.33 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 160230 1259 0 0 118207 26 0 0 25 0 5 0 521707644 652701696 131111 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=637404 CPUtime=1182.33 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 146374 1259 0 0 118207 26 0 0 25 0 5 0 521707644 652701696 131111 33554432000 4194304 4417332 140737389875984 18446744073709551615 4236891 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=637404 CPUtime=1175.89 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 128984 1259 0 0 117567 22 0 0 25 0 5 0 521707644 652701696 131111 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=637404 CPUtime=1182.3 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 271613 1259 0 0 118188 42 0 0 25 0 5 0 521707644 652701696 131111 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4722.89
Current children cumulated vsize (KiB) 637404



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.00 7.97 7.57 9/186 27342
/proc/meminfo: memFree=29447844/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=640940 CPUtime=4800.4 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 708422 1259 0 0 479922 118 0 0 23 0 5 0 521707643 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 160235 131995 181 55 0 158121 0
[pid=27247/tid=27253] ppid=27245 vsize=640940 CPUtime=1201.74 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 160230 1259 0 0 120148 26 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=640940 CPUtime=1201.74 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 146374 1259 0 0 120148 26 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=640940 CPUtime=1195.19 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 129828 1259 0 0 119497 22 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=640940 CPUtime=1201.7 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 271653 1259 0 0 120128 42 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 640940

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.00 7.97 7.57 9/186 27342
/proc/meminfo: memFree=29447844/32950928 swapFree=67086612/67111528
[pid=27247] ppid=27245 vsize=640940 CPUtime=4800.4 cores=0,2,4,6
/proc/27247/stat : 27247 (plingeling) S 27245 27247 26666 0 -1 4202496 708422 1259 0 0 479922 118 0 0 23 0 5 0 521707643 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 239488497637 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/27247/statm: 160235 131995 181 55 0 158121 0
[pid=27247/tid=27253] ppid=27245 vsize=640940 CPUtime=1201.74 cores=0,2,4,6
/proc/27247/task/27253/stat : 27253 (plingeling) R 27245 27247 26666 0 -1 4202560 160230 1259 0 0 120148 26 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27247/tid=27254] ppid=27245 vsize=640940 CPUtime=1201.74 cores=0,2,4,6
/proc/27247/task/27254/stat : 27254 (plingeling) R 27245 27247 26666 0 -1 4202560 146374 1259 0 0 120148 26 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27247/tid=27255] ppid=27245 vsize=640940 CPUtime=1195.19 cores=0,2,4,6
/proc/27247/task/27255/stat : 27255 (plingeling) R 27245 27247 26666 0 -1 4202560 129828 1259 0 0 119497 22 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27247/tid=27256] ppid=27245 vsize=640940 CPUtime=1201.7 cores=0,2,4,6
/proc/27247/task/27256/stat : 27256 (plingeling) R 27245 27247 26666 0 -1 4202560 271653 1259 0 0 120128 42 0 0 25 0 5 0 521707644 656322560 131995 33554432000 4194304 4417332 140737389875984 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 640940

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.74
CPU time (s): 4800.46
CPU user time (s): 4799.24
CPU system time (s): 1.21381
CPU usage (%): 399.459
Max. virtual memory (cumulated for all children) (KiB): 646556

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.24
system time used= 1.21381
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 709681
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 55
involuntary context switches= 23039

runsolver used 1.9877 second user time and 4.56731 second system time

The end

Launcher Data 

Begin job on node148 at 2011-04-29 20:18:26
IDJOB=3263882
IDBENCH=84030
IDSOLVER=1605
FILE ID=node148/3263882-1304101106
RUNJOBID= node148-1304101106-27235
PBS_JOBID= 13170593
Free space on /tmp= 71436 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n28-01-S136611085.shuffled-as.sat03-1627.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263882-1304101106/watcher-3263882-1304101106 -o /tmp/evaluation-result-3263882-1304101106/solver-3263882-1304101106 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3263882-1304101106.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 17624c574c7f9ee0eff4cc4a13e42e12
RANDOM SEED=426388887

node148.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30930052 kB
Buffers:        694676 kB
Cached:         920932 kB
SwapCached:       3696 kB
Active:         105884 kB
Inactive:      1551416 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30930052 kB
SwapTotal:    67111528 kB
SwapFree:     67086612 kB
Dirty:              84 kB
Writeback:           0 kB
AnonPages:       39832 kB
Mapped:          13476 kB
Slab:           300140 kB
PageTables:       4748 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   187592 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71432 MiB
End job on node148 at 2011-04-29 20:38:30