Trace number 3287459

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.31 1201.75

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k45-pdtpmspalu-tseitin.cnf
MD5SUM8dae13531986e2a9bdb6559418f57130
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1166.09
Satisfiable
(Un)Satisfiability was proved
Number of variables89190
Number of clauses265718
Sum of the clauses size620006
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2177144
Number of clauses of size 388572
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3287459-1304063057/watcher-3287459-1304063057 -o /tmp/evaluation-result-3287459-1304063057/solver-3287459-1304063057 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3287459-1304063057.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.72 7.92 7.62 5/181 21456
/proc/meminfo: memFree=12442916/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=10360 CPUtime=0 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) R 21454 21456 20615 0 -1 4202496 659 1257 0 0 0 0 0 0 25 0 1 0 596536408 10608640 588 33554432000 4194304 4417332 140733512749072 18446744073709551615 216735178496 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21456/statm: 2590 588 147 55 0 476 0

[startup+0.0320851 s]
/proc/loadavg: 7.72 7.92 7.62 5/181 21456
/proc/meminfo: memFree=12442916/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=16688 CPUtime=0.02 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) R 21454 21456 20615 0 -1 4202496 2324 1257 0 0 2 0 0 0 25 0 1 0 596536408 17088512 2195 33554432000 4194304 4417332 140733512749072 18446744073709551615 4322331 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21456/statm: 4172 2195 148 55 0 2058 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 16688

[startup+0.101105 s]
/proc/loadavg: 7.72 7.92 7.62 5/181 21456
/proc/meminfo: memFree=12442916/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=27720 CPUtime=0.09 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) R 21454 21456 20615 0 -1 4202496 5082 1257 0 0 9 0 0 0 25 0 1 0 596536408 28385280 4953 33554432000 4194304 4417332 140733512749072 18446744073709551615 4227863 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21456/statm: 6930 4953 148 55 0 4816 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 27720

[startup+0.301071 s]
/proc/loadavg: 7.72 7.92 7.62 5/181 21456
/proc/meminfo: memFree=12442916/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=63764 CPUtime=0.29 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) R 21454 21456 20615 0 -1 4202496 14108 1257 0 0 28 1 0 0 25 0 1 0 596536408 65294336 13964 33554432000 4194304 4417332 140733512749072 18446744073709551615 216735178496 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21456/statm: 15941 13964 148 55 0 13827 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 63764

[startup+1.11214 s]
/proc/loadavg: 7.72 7.92 7.62 5/181 21456
/proc/meminfo: memFree=12442916/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=187248 CPUtime=3.46 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 36205 1257 0 0 342 4 0 0 25 0 5 0 596536408 191741952 18653 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 46812 18653 173 55 0 44698 0
Current children cumulated CPU time (s) 3.46
Current children cumulated vsize (KiB) 187248

[startup+1.50079 s]
/proc/loadavg: 7.72 7.92 7.62 5/181 21456
/proc/meminfo: memFree=12442916/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=195932 CPUtime=5.01 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 42887 1257 0 0 496 5 0 0 25 0 5 0 596536408 200634368 21457 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 48983 21457 173 55 0 46869 0
Current children cumulated CPU time (s) 5.01
Current children cumulated vsize (KiB) 195932

[startup+3.10051 s]
/proc/loadavg: 7.72 7.92 7.62 9/186 21466
/proc/meminfo: memFree=12356576/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=189280 CPUtime=11.41 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 56374 1257 0 0 1134 7 0 0 25 0 5 0 596536408 193822720 20489 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 47320 20489 179 55 0 45206 0
[pid=21456/tid=21463] ppid=21454 vsize=189280 CPUtime=2.77 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 11135 1257 0 0 276 1 0 0 25 0 5 0 596536440 193822720 20489 33554432000 4194304 4417332 140733512749072 18446744073709551615 4330365 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=189280 CPUtime=2.76 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 15785 1257 0 0 275 1 0 0 25 0 5 0 596536440 193822720 20489 33554432000 4194304 4417332 140733512749072 18446744073709551615 4330853 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=189280 CPUtime=2.75 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 9180 1257 0 0 275 0 0 0 23 0 5 0 596536440 193822720 20489 33554432000 4194304 4417332 140733512749072 18446744073709551615 4377421 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=189280 CPUtime=2.77 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 2058 1257 0 0 277 0 0 0 25 0 5 0 596536440 193822720 20489 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.41
Current children cumulated vsize (KiB) 189280

[startup+6.30095 s]
/proc/loadavg: 7.74 7.92 7.62 9/186 21466
/proc/meminfo: memFree=12317260/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=228544 CPUtime=24.19 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 67237 1257 0 0 2411 8 0 0 25 0 5 0 596536408 234029056 31309 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 57136 31309 179 55 0 55022 0
[pid=21456/tid=21463] ppid=21454 vsize=228544 CPUtime=5.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 14853 1257 0 0 595 2 0 0 25 0 5 0 596536440 234029056 31309 33554432000 4194304 4417332 140733512749072 18446744073709551615 4236017 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=228544 CPUtime=5.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 19858 1257 0 0 595 2 0 0 25 0 5 0 596536440 234029056 31309 33554432000 4194304 4417332 140733512749072 18446744073709551615 4341348 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=228544 CPUtime=5.94 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 12252 1257 0 0 593 1 0 0 25 0 5 0 596536440 234029056 31309 33554432000 4194304 4417332 140733512749072 18446744073709551615 4236593 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=228544 CPUtime=5.97 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 2058 1257 0 0 597 0 0 0 25 0 5 0 596536440 234029056 31309 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.19
Current children cumulated vsize (KiB) 228544

[startup+12.7008 s]
/proc/loadavg: 7.76 7.92 7.62 9/186 21466
/proc/meminfo: memFree=12332816/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=211648 CPUtime=49.77 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 67329 1257 0 0 4969 8 0 0 25 0 5 0 596536408 216727552 27087 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 52912 27087 181 55 0 50798 0
[pid=21456/tid=21463] ppid=21454 vsize=211648 CPUtime=12.38 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 14869 1257 0 0 1236 2 0 0 25 0 5 0 596536440 216727552 27087 33554432000 4194304 4417332 140733512749072 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=211648 CPUtime=12.37 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 19889 1257 0 0 1235 2 0 0 25 0 5 0 596536440 216727552 27087 33554432000 4194304 4417332 140733512749072 18446744073709551615 4366468 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=211648 CPUtime=12.31 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 12297 1257 0 0 1230 1 0 0 25 0 5 0 596536440 216727552 27087 33554432000 4194304 4417332 140733512749072 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=211648 CPUtime=12.37 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 2058 1257 0 0 1237 0 0 0 25 0 5 0 596536440 216727552 27087 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 49.77
Current children cumulated vsize (KiB) 211648

[startup+25.5006 s]
/proc/loadavg: 7.82 7.93 7.63 9/186 21467
/proc/meminfo: memFree=12323512/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=219716 CPUtime=100.9 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 69476 1257 0 0 10081 9 0 0 25 0 5 0 596536408 224989184 29110 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 54929 29110 181 55 0 52815 0
[pid=21456/tid=21463] ppid=21454 vsize=219716 CPUtime=25.17 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 14891 1257 0 0 2515 2 0 0 25 0 5 0 596536440 224989184 29110 33554432000 4194304 4417332 140733512749072 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=219716 CPUtime=25.16 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 19901 1257 0 0 2514 2 0 0 25 0 5 0 596536440 224989184 29110 33554432000 4194304 4417332 140733512749072 18446744073709551615 4365994 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=219716 CPUtime=25.05 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 12362 1257 0 0 2504 1 0 0 25 0 5 0 596536440 224989184 29110 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=219716 CPUtime=25.16 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 4106 1257 0 0 2516 0 0 0 25 0 5 0 596536440 224989184 29110 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 100.9
Current children cumulated vsize (KiB) 219716

[startup+51.1011 s]
/proc/loadavg: 7.93 7.95 7.64 9/186 21467
/proc/meminfo: memFree=12276660/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=276824 CPUtime=203.18 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 85910 1257 0 0 20307 11 0 0 25 0 5 0 596536408 283467776 43387 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 69206 43387 181 55 0 67092 0
[pid=21456/tid=21463] ppid=21454 vsize=276824 CPUtime=50.77 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 26611 1257 0 0 5074 3 0 0 25 0 5 0 596536440 283467776 43387 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=276824 CPUtime=50.76 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 20046 1257 0 0 5074 2 0 0 25 0 5 0 596536440 283467776 43387 33554432000 4194304 4417332 140733512749072 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=276824 CPUtime=50.52 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 13237 1257 0 0 5051 1 0 0 25 0 5 0 596536440 283467776 43387 33554432000 4194304 4417332 140733512749072 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=276824 CPUtime=50.77 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 7800 1257 0 0 5076 1 0 0 25 0 5 0 596536440 283467776 43387 33554432000 4194304 4417332 140733512749072 18446744073709551615 4357108 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.18
Current children cumulated vsize (KiB) 276824

[startup+102.301 s]
/proc/loadavg: 7.97 7.95 7.66 9/186 21469
/proc/meminfo: memFree=12268704/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=270280 CPUtime=407.74 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 149008 1257 0 0 40754 20 0 0 25 0 5 0 596536408 276766720 41751 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 67570 41751 181 55 0 65456 0
[pid=21456/tid=21463] ppid=21454 vsize=270280 CPUtime=101.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 44762 1257 0 0 10191 6 0 0 25 0 5 0 596536440 276766720 41751 33554432000 4194304 4417332 140733512749072 18446744073709551615 4277540 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=270280 CPUtime=101.96 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 27009 1257 0 0 10193 3 0 0 25 0 5 0 596536440 276766720 41751 33554432000 4194304 4417332 140733512749072 18446744073709551615 4277527 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=270280 CPUtime=101.48 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 19420 1257 0 0 10146 2 0 0 25 0 5 0 596536440 276766720 41751 33554432000 4194304 4417332 140733512749072 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=270280 CPUtime=101.97 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 39601 1257 0 0 10192 5 0 0 25 0 5 0 596536440 276766720 41751 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.74
Current children cumulated vsize (KiB) 270280

[startup+162.301 s]
/proc/loadavg: 7.99 7.96 7.67 9/186 21470
/proc/meminfo: memFree=12087136/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=444032 CPUtime=647.46 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 343427 1257 0 0 64700 46 0 0 25 0 5 0 596536408 454688768 85189 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 111008 85189 181 55 0 108894 0
[pid=21456/tid=21463] ppid=21454 vsize=444032 CPUtime=161.98 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 130181 1257 0 0 16179 19 0 0 25 0 5 0 596536440 454688768 85189 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=444032 CPUtime=161.96 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 51385 1257 0 0 16190 6 0 0 25 0 5 0 596536440 454688768 85189 33554432000 4194304 4417332 140733512749072 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=444032 CPUtime=161.2 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 90258 1257 0 0 16109 11 0 0 25 0 5 0 596536440 454688768 85189 33554432000 4194304 4417332 140733512749072 18446744073709551615 4309982 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=444032 CPUtime=161.97 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 53387 1257 0 0 16191 6 0 0 25 0 5 0 596536440 454688768 85189 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 647.46
Current children cumulated vsize (KiB) 444032

[startup+222.3 s]
/proc/loadavg: 7.99 7.96 7.69 9/186 21472
/proc/meminfo: memFree=12170432/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=358840 CPUtime=887.17 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 383297 1257 0 0 88665 52 0 0 25 0 5 0 596536408 367452160 63923 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 89710 63923 181 55 0 87596 0
[pid=21456/tid=21463] ppid=21454 vsize=358840 CPUtime=221.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 136619 1257 0 0 22177 20 0 0 25 0 5 0 596536440 367452160 63923 33554432000 4194304 4417332 140733512749072 18446744073709551615 4354736 0 0 4096 17442 0 0 0 -1 4 0 0 0

################
# More data... #
################

[pid=21456/tid=21464] ppid=21454 vsize=776296 CPUtime=701.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 386452 1257 0 0 70143 54 0 0 25 0 5 0 596536440 794927104 168255 33554432000 4194304 4417332 140733512749072 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=776296 CPUtime=698.68 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 405786 1257 0 0 69808 60 0 0 25 0 5 0 596536440 794927104 168255 33554432000 4194304 4417332 140733512749072 18446744073709551615 4357806 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=776296 CPUtime=701.98 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 89820 1257 0 0 70188 10 0 0 25 0 5 0 596536440 794927104 168255 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2804.94
Current children cumulated vsize (KiB) 776296

[startup+762.301 s]
/proc/loadavg: 8.11 8.07 7.85 9/186 21486
/proc/meminfo: memFree=11805296/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=681852 CPUtime=3044.66 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 1521352 1257 0 0 304253 213 0 0 25 0 5 0 596536408 698216448 144676 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 170463 144676 181 55 0 168349 0
[pid=21456/tid=21463] ppid=21454 vsize=681852 CPUtime=761.96 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 549679 1257 0 0 76119 77 0 0 25 0 5 0 596536440 698216448 144676 33554432000 4194304 4417332 140733512749072 18446744073709551615 4236975 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=681852 CPUtime=761.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 393918 1257 0 0 76142 55 0 0 25 0 5 0 596536440 698216448 144676 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=681852 CPUtime=758.39 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 469719 1257 0 0 75772 67 0 0 25 0 5 0 596536440 698216448 144676 33554432000 4194304 4417332 140733512749072 18446744073709551615 4358652 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=681852 CPUtime=761.99 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 89820 1257 0 0 76188 11 0 0 25 0 5 0 596536440 698216448 144676 33554432000 4194304 4417332 140733512749072 18446744073709551615 4357880 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3044.66
Current children cumulated vsize (KiB) 681852

[startup+822.301 s]
/proc/loadavg: 8.21 8.10 7.87 9/186 21488
/proc/meminfo: memFree=11747020/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=731508 CPUtime=3284.37 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 1658712 1257 0 0 328203 234 0 0 25 0 5 0 596536408 749064192 157058 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 182877 157058 181 55 0 180763 0
[pid=21456/tid=21463] ppid=21454 vsize=731508 CPUtime=821.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 620533 1257 0 0 82109 88 0 0 25 0 5 0 596536440 749064192 157058 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=731508 CPUtime=821.98 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 451712 1257 0 0 82135 63 0 0 25 0 5 0 596536440 749064192 157058 33554432000 4194304 4417332 140733512749072 18446744073709551615 4250873 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=731508 CPUtime=818.21 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 478431 1257 0 0 81753 68 0 0 25 0 5 0 596536440 749064192 157058 33554432000 4194304 4417332 140733512749072 18446744073709551615 4358020 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=731508 CPUtime=821.85 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 89820 1257 0 0 82174 11 0 0 25 0 5 0 596536440 749064192 157058 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364706 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3284.37
Current children cumulated vsize (KiB) 731508

[startup+882.301 s]
/proc/loadavg: 8.13 8.09 7.88 9/186 21489
/proc/meminfo: memFree=11590396/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=895844 CPUtime=3524.07 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 1923626 1257 0 0 352136 271 0 0 25 0 5 0 596536408 917344256 198142 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 223961 198142 181 55 0 221847 0
[pid=21456/tid=21463] ppid=21454 vsize=895844 CPUtime=881.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 680515 1257 0 0 88101 96 0 0 25 0 5 0 596536440 917344256 198142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=895844 CPUtime=881.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 541119 1257 0 0 88121 76 0 0 25 0 5 0 596536440 917344256 198142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=895844 CPUtime=878.22 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 569805 1257 0 0 87740 82 0 0 25 0 5 0 596536440 917344256 198142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=895844 CPUtime=881.57 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 113971 1257 0 0 88143 14 0 0 25 0 5 0 596536440 917344256 198142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3524.07
Current children cumulated vsize (KiB) 895844

[startup+942.307 s]
/proc/loadavg: 7.84 8.04 7.88 5/177 21509
/proc/meminfo: memFree=12002984/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=869304 CPUtime=3763.82 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 2111953 1257 0 0 376083 299 0 0 25 0 5 0 596536408 890167296 191507 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 217326 191507 181 55 0 215212 0
[pid=21456/tid=21463] ppid=21454 vsize=869304 CPUtime=941.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 739050 1257 0 0 94094 103 0 0 25 0 5 0 596536440 890167296 191507 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=869304 CPUtime=941.98 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 587435 1257 0 0 94113 85 0 0 25 0 5 0 596536440 890167296 191507 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=869304 CPUtime=938.22 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 649729 1257 0 0 93730 92 0 0 25 0 5 0 596536440 890167296 191507 33554432000 4194304 4417332 140733512749072 18446744073709551615 4355764 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=869304 CPUtime=941.3 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 117523 1257 0 0 94115 15 0 0 25 0 5 0 596536440 890167296 191507 33554432000 4194304 4417332 140733512749072 18446744073709551615 4264999 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3763.82
Current children cumulated vsize (KiB) 869304

[startup+1002.3 s]
/proc/loadavg: 5.47 7.32 7.64 5/176 21511
/proc/meminfo: memFree=11889304/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=990312 CPUtime=4003.53 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 2450162 1257 0 0 400003 350 0 0 25 0 5 0 596536408 1014079488 221759 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 247578 221759 181 55 0 245464 0
[pid=21456/tid=21463] ppid=21454 vsize=990312 CPUtime=1001.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 859784 1257 0 0 100077 120 0 0 25 0 5 0 596536440 1014079488 221759 33554432000 4194304 4417332 140733512749072 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=990312 CPUtime=1001.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 729814 1257 0 0 100090 107 0 0 25 0 5 0 596536440 1014079488 221759 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=990312 CPUtime=998.22 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 722246 1257 0 0 99719 103 0 0 25 0 5 0 596536440 1014079488 221759 33554432000 4194304 4417332 140733512749072 18446744073709551615 4354419 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=990312 CPUtime=1001.02 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 120102 1257 0 0 100087 15 0 0 25 0 5 0 596536440 1014079488 221759 33554432000 4194304 4417332 140733512749072 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4003.53
Current children cumulated vsize (KiB) 990312

[startup+1062.3 s]
/proc/loadavg: 4.63 6.74 7.42 5/176 21521
/proc/meminfo: memFree=12192584/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=686572 CPUtime=4243.25 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 2666701 1257 0 0 423935 390 0 0 25 0 5 0 596536408 703049728 145824 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 171643 145824 181 55 0 169529 0
[pid=21456/tid=21463] ppid=21454 vsize=686572 CPUtime=1061.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 920222 1257 0 0 106066 131 0 0 25 0 5 0 596536440 703049728 145824 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=686572 CPUtime=1061.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 799494 1257 0 0 106077 120 0 0 25 0 5 0 596536440 703049728 145824 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=686572 CPUtime=1058.21 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 804220 1257 0 0 105702 119 0 0 25 0 5 0 596536440 703049728 145824 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=686572 CPUtime=1060.75 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 124549 1257 0 0 106059 16 0 0 25 0 5 0 596536440 703049728 145824 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4243.25
Current children cumulated vsize (KiB) 686572

[startup+1122.3 s]
/proc/loadavg: 4.34 6.27 7.21 5/176 21523
/proc/meminfo: memFree=11791888/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=1091716 CPUtime=4482.99 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 2899275 1257 0 0 447878 421 0 0 25 0 5 0 596536408 1117917184 247142 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 272929 247142 181 55 0 270815 0
[pid=21456/tid=21463] ppid=21454 vsize=1091716 CPUtime=1121.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 984611 1257 0 0 112057 140 0 0 25 0 5 0 596536440 1117917184 247142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=1091716 CPUtime=1121.97 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 840884 1257 0 0 112072 125 0 0 25 0 5 0 596536440 1117917184 247142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364575 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=1091716 CPUtime=1118.21 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 894592 1257 0 0 111689 132 0 0 25 0 5 0 596536440 1117917184 247142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=1091716 CPUtime=1120.48 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 160972 1257 0 0 112028 20 0 0 25 0 5 0 596536440 1117917184 247142 33554432000 4194304 4417332 140733512749072 18446744073709551615 4251028 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4482.99
Current children cumulated vsize (KiB) 1091716

[startup+1182.3 s]
/proc/loadavg: 4.17 5.87 7.01 5/176 21524
/proc/meminfo: memFree=11926888/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=963648 CPUtime=4722.71 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 3180366 1257 0 0 471803 468 0 0 25 0 5 0 596536408 986775552 215093 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 240912 215093 181 55 0 238798 0
[pid=21456/tid=21463] ppid=21454 vsize=963648 CPUtime=1181.97 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 1067024 1257 0 0 118044 153 0 0 25 0 5 0 596536440 986775552 215093 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=963648 CPUtime=1181.96 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 929396 1257 0 0 118057 139 0 0 25 0 5 0 596536440 986775552 215093 33554432000 4194304 4417332 140733512749072 18446744073709551615 216734875975 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=963648 CPUtime=1178.22 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 977187 1257 0 0 117674 148 0 0 25 0 5 0 596536440 986775552 215093 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=963648 CPUtime=1180.2 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 188543 1257 0 0 117996 24 0 0 25 0 5 0 596536440 986775552 215093 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4722.71
Current children cumulated vsize (KiB) 963648



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 4.12 5.75 6.95 5/176 21525
/proc/meminfo: memFree=11925744/32951124 swapFree=67102140/67111528
[pid=21456] ppid=21454 vsize=953984 CPUtime=4800.23 cores=0,2,4,6
/proc/21456/stat : 21456 (plingeling) S 21454 21456 20615 0 -1 4202496 3267831 1257 0 0 479540 483 0 0 25 0 5 0 596536408 976879616 212677 33554432000 4194304 4417332 140733512749072 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21456/statm: 238496 212677 181 55 0 236382 0
[pid=21456/tid=21463] ppid=21454 vsize=953984 CPUtime=1201.37 cores=0,2,4,6
/proc/21456/task/21463/stat : 21463 (plingeling) R 21454 21456 20615 0 -1 4202560 1085412 1257 0 0 119980 157 0 0 25 0 5 0 596536440 976879616 212677 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21456/tid=21464] ppid=21454 vsize=953984 CPUtime=1201.36 cores=0,2,4,6
/proc/21456/task/21464/stat : 21464 (plingeling) R 21454 21456 20615 0 -1 4202560 972489 1257 0 0 119989 147 0 0 25 0 5 0 596536440 976879616 212677 33554432000 4194304 4417332 140733512749072 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21456/tid=21465] ppid=21454 vsize=953984 CPUtime=1197.63 cores=0,2,4,6
/proc/21456/task/21465/stat : 21465 (plingeling) R 21454 21456 20615 0 -1 4202560 1001554 1257 0 0 119612 151 0 0 25 0 5 0 596536440 976879616 212677 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21456/tid=21466] ppid=21454 vsize=953984 CPUtime=1199.52 cores=0,2,4,6
/proc/21456/task/21466/stat : 21466 (plingeling) R 21454 21456 20615 0 -1 4202560 190160 1257 0 0 119928 24 0 0 25 0 5 0 596536440 976879616 212677 33554432000 4194304 4417332 140733512749072 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.23
Current children cumulated vsize (KiB) 953984

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.75
CPU time (s): 4800.31
CPU user time (s): 4795.43
CPU system time (s): 4.87626
CPU usage (%): 399.444
Max. virtual memory (cumulated for all children) (KiB): 1580356

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.43
system time used= 4.87626
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3269088
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 162
involuntary context switches= 20150

runsolver used 1.73474 second user time and 3.87441 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 09:44:18
IDJOB=3287459
IDBENCH=82467
IDSOLVER=1605
FILE ID=node119/3287459-1304063057
RUNJOBID= node119-1304060531-20633
PBS_JOBID= 13170451
Free space on /tmp= 71800 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtpmspalu-tseitin.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3287459-1304063057/watcher-3287459-1304063057 -o /tmp/evaluation-result-3287459-1304063057/solver-3287459-1304063057 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3287459-1304063057.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8dae13531986e2a9bdb6559418f57130
RANDOM SEED=1340126064

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12443204 kB
Buffers:       2232960 kB
Cached:       17309608 kB
SwapCached:       5504 kB
Active:        8269580 kB
Inactive:     11653892 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12443204 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:            4492 kB
Writeback:           0 kB
AnonPages:      377764 kB
Mapped:          15176 kB
Slab:           520140 kB
PageTables:       5712 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   604112 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71812 MiB
End job on node119 at 2011-04-29 10:04:21