Trace number 3291553

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.18 1205.64

General information on the benchmark

NameSAT03/industrial/kukula/addm_bench/
am_7_7.shuffled-as.sat03-363.cnf
MD5SUMc7f51e1ee7556440ab29e1c4bfe29538
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables4264
Number of clauses14751
Sum of the clauses size40035
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 24214
Number of clauses of size 310535
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3291553-1304101755/watcher-3291553-1304101755 -o /tmp/evaluation-result-3291553-1304101755/solver-3291553-1304101755 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3291553-1304101755.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.77 5.00 6.56 3/172 9373
/proc/meminfo: memFree=19118904/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=9780 CPUtime=0 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) R 9371 9373 8608 0 -1 4202496 497 1257 0 0 0 0 0 0 23 0 1 0 521775120 10014720 425 33554432000 4194304 4417332 140733271705280 18446744073709551615 210646843577 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/9373/statm: 2445 437 147 55 0 331 0

[startup+0.104029 s]
/proc/loadavg: 3.77 5.00 6.56 3/172 9373
/proc/meminfo: memFree=19118904/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=187408 CPUtime=0.31 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 2455 1257 0 0 31 0 0 0 23 0 5 0 521775120 191905792 1929 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 46852 1929 172 55 0 44738 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 187408

[startup+0.201013 s]
/proc/loadavg: 3.77 5.00 6.56 3/172 9373
/proc/meminfo: memFree=19118904/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=187408 CPUtime=0.7 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 2475 1257 0 0 70 0 0 0 23 0 5 0 521775120 191905792 1949 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 46852 1949 180 55 0 44738 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 187408

[startup+0.300996 s]
/proc/loadavg: 3.77 5.00 6.56 3/172 9373
/proc/meminfo: memFree=19118904/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=187752 CPUtime=1.1 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 2631 1257 0 0 110 0 0 0 23 0 5 0 521775120 192258048 2055 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 46938 2055 180 55 0 44824 0
Current children cumulated CPU time (s) 1.1
Current children cumulated vsize (KiB) 187752

[startup+0.700931 s]
/proc/loadavg: 3.77 5.00 6.56 3/172 9373
/proc/meminfo: memFree=19118904/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=189504 CPUtime=2.7 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 4928 1257 0 0 270 0 0 0 23 0 5 0 521775120 194052096 2502 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 47376 2502 180 55 0 45262 0
Current children cumulated CPU time (s) 2.7
Current children cumulated vsize (KiB) 189504

[startup+1.50083 s]
/proc/loadavg: 3.77 5.00 6.56 6/177 9383
/proc/meminfo: memFree=19085004/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=194548 CPUtime=5.78 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 9874 1257 0 0 571 7 0 0 23 0 5 0 521775120 199217152 3766 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 48637 3766 181 55 0 46523 0
[pid=9373/tid=9380] ppid=9371 vsize=194548 CPUtime=1.47 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 1949 1257 0 0 143 4 0 0 25 0 5 0 521775122 199217152 3766 33554432000 4194304 4417332 140733271705280 18446744073709551615 210647204266 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=194548 CPUtime=1.37 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 1943 1257 0 0 136 1 0 0 25 0 5 0 521775122 199217152 3766 33554432000 4194304 4417332 140733271705280 18446744073709551615 4311285 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=194548 CPUtime=1.45 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) D 9371 9373 8608 0 -1 4202560 2182 1257 0 0 144 1 0 0 25 0 5 0 521775122 199217152 3766 33554432000 4194304 4417332 140733271705280 18446744073709551615 210647189594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=194548 CPUtime=1.45 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 1798 1257 0 0 145 0 0 0 25 0 5 0 521775122 199217152 3766 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.78
Current children cumulated vsize (KiB) 194548

[startup+3.10055 s]
/proc/loadavg: 3.77 5.00 6.56 6/177 9383
/proc/meminfo: memFree=19062808/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=196544 CPUtime=10.41 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 14635 1257 0 0 947 94 0 0 23 0 5 0 521775120 201261056 4234 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 49136 4234 181 55 0 47022 0
[pid=9373/tid=9380] ppid=9371 vsize=196544 CPUtime=2.48 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 2695 1257 0 0 204 44 0 0 19 0 5 0 521775122 201261056 4234 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=196544 CPUtime=2.48 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 2876 1257 0 0 240 8 0 0 18 0 5 0 521775122 201261056 4234 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=196544 CPUtime=2.72 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 4384 1257 0 0 256 16 0 0 18 0 5 0 521775122 201261056 4234 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=196544 CPUtime=2.68 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 2678 1257 0 0 244 24 0 0 19 0 5 0 521775122 201261056 4234 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 10.41
Current children cumulated vsize (KiB) 196544

[startup+6.301 s]
/proc/loadavg: 3.95 5.02 6.55 5/176 9384
/proc/meminfo: memFree=19036816/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=200788 CPUtime=20.15 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 23441 1257 0 0 1769 246 0 0 23 0 5 0 521775120 205606912 5300 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 50197 5300 181 55 0 48083 0
[pid=9373/tid=9380] ppid=9371 vsize=200788 CPUtime=4.1 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 3248 1257 0 0 330 80 0 0 25 0 5 0 521775122 205606912 5300 33554432000 4194304 4417332 140733271705280 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=200788 CPUtime=4.84 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 6113 1257 0 0 452 32 0 0 18 0 5 0 521775122 205606912 5300 33554432000 4194304 4417332 140733271705280 18446744073709551615 4353502 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=200788 CPUtime=5.51 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 7312 1257 0 0 476 75 0 0 19 0 5 0 521775122 205606912 5300 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=200788 CPUtime=5.65 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 4766 1257 0 0 508 57 0 0 20 0 5 0 521775122 205606912 5300 33554432000 4194304 4417332 140733271705280 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 20.15
Current children cumulated vsize (KiB) 200788

[startup+12.7009 s]
/proc/loadavg: 4.20 5.05 6.56 6/179 9393
/proc/meminfo: memFree=18579500/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=205324 CPUtime=40.84 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 38641 1257 0 0 3640 444 0 0 23 0 5 0 521775120 210251776 6455 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 51331 6455 181 55 0 49217 0
[pid=9373/tid=9380] ppid=9371 vsize=205324 CPUtime=7.41 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 3750 1257 0 0 615 126 0 0 25 0 5 0 521775122 210251776 6455 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=205324 CPUtime=10.96 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 12395 1257 0 0 1000 96 0 0 18 0 5 0 521775122 210251776 6455 33554432000 4194304 4417332 140733271705280 18446744073709551615 18446744073699065904 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=205324 CPUtime=10.81 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 11872 1257 0 0 969 112 0 0 25 0 5 0 521775122 210251776 6455 33554432000 4194304 4417332 140733271705280 18446744073709551615 4353408 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=205324 CPUtime=11.61 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 8622 1257 0 0 1053 108 0 0 25 0 5 0 521775122 210251776 6455 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40.84
Current children cumulated vsize (KiB) 205324

[startup+25.5007 s]
/proc/loadavg: 5.12 5.21 6.58 11/184 9398
/proc/meminfo: memFree=18116772/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=223392 CPUtime=86.41 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 77289 1257 0 0 7976 665 0 0 23 0 5 0 521775120 228753408 10946 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 55848 10946 181 55 0 53734 0
[pid=9373/tid=9380] ppid=9371 vsize=223392 CPUtime=19.7 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 16349 1257 0 0 1769 201 0 0 25 0 5 0 521775122 228753408 10946 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364534 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=223392 CPUtime=21.59 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 22766 1257 0 0 2009 150 0 0 25 0 5 0 521775122 228753408 10946 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=223392 CPUtime=23.16 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 20849 1257 0 0 2134 182 0 0 25 0 5 0 521775122 228753408 10946 33554432000 4194304 4417332 140733271705280 18446744073709551615 4307267 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=223392 CPUtime=21.91 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 15323 1257 0 0 2061 130 0 0 25 0 5 0 521775122 228753408 10946 33554432000 4194304 4417332 140733271705280 18446744073709551615 4365906 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 86.41
Current children cumulated vsize (KiB) 223392

[startup+51.1004 s]
/proc/loadavg: 6.10 5.43 6.62 10/184 9399
/proc/meminfo: memFree=17902304/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=246756 CPUtime=188.7 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 151524 1257 0 0 18192 678 0 0 23 0 5 0 521775120 252678144 16793 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 61689 16793 181 55 0 59575 0
[pid=9373/tid=9380] ppid=9371 vsize=246756 CPUtime=45.17 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 33536 1257 0 0 4313 204 0 0 25 0 5 0 521775122 252678144 16793 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=246756 CPUtime=47.19 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 44808 1257 0 0 4565 154 0 0 25 0 5 0 521775122 252678144 16793 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=246756 CPUtime=48.76 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 42134 1257 0 0 4691 185 0 0 25 0 5 0 521775122 252678144 16793 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=246756 CPUtime=47.51 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 29044 1257 0 0 4619 132 0 0 25 0 5 0 521775122 252678144 16793 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 188.7
Current children cumulated vsize (KiB) 246756

[startup+102.301 s]
/proc/loadavg: 7.29 5.86 6.70 10/184 9400
/proc/meminfo: memFree=17796456/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=277900 CPUtime=393.28 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 260389 1257 0 0 38631 697 0 0 23 0 5 0 521775120 284569600 24579 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 69475 24579 181 55 0 67361 0
[pid=9373/tid=9380] ppid=9371 vsize=277900 CPUtime=96.14 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 67025 1257 0 0 9404 210 0 0 25 0 5 0 521775122 284569600 24579 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364614 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=277900 CPUtime=98.39 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 75004 1257 0 0 9679 160 0 0 25 0 5 0 521775122 284569600 24579 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=277900 CPUtime=99.97 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 65803 1257 0 0 9808 189 0 0 25 0 5 0 521775122 284569600 24579 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=277900 CPUtime=98.72 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 50555 1257 0 0 9736 136 0 0 25 0 5 0 521775122 284569600 24579 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364811 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 393.28
Current children cumulated vsize (KiB) 277900

[startup+162.301 s]
/proc/loadavg: 7.84 6.28 6.79 10/184 9402
/proc/meminfo: memFree=18000424/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=294936 CPUtime=632.93 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 366357 1257 0 0 62577 716 0 0 23 0 5 0 521775120 302014464 28838 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 73734 28838 181 55 0 71620 0
[pid=9373/tid=9380] ppid=9371 vsize=294936 CPUtime=155.85 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 89710 1257 0 0 15371 214 0 0 25 0 5 0 521775122 302014464 28838 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=294936 CPUtime=158.35 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 102428 1257 0 0 15669 166 0 0 25 0 5 0 521775122 302014464 28838 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=294936 CPUtime=159.97 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 105098 1257 0 0 15801 196 0 0 25 0 5 0 521775122 302014464 28838 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=294936 CPUtime=158.72 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 67119 1257 0 0 15734 138 0 0 25 0 5 0 521775122 302014464 28838 33554432000 4194304 4417332 140733271705280 18446744073709551615 4366004 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 632.93
Current children cumulated vsize (KiB) 294936


################
# More data... #
################

[pid=9373/tid=9381] ppid=9371 vsize=538272 CPUtime=757.92 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 340946 1257 0 0 75575 217 0 0 25 0 5 0 521775122 551190528 89672 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=538272 CPUtime=759.98 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 354360 1257 0 0 75766 232 0 0 25 0 5 0 521775122 551190528 89672 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=538272 CPUtime=758.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 250286 1257 0 0 75710 163 0 0 25 0 5 0 521775122 551190528 89672 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3029.59
Current children cumulated vsize (KiB) 538272

[startup+822.301 s]
/proc/loadavg: 8.19 7.96 7.48 10/184 9419
/proc/meminfo: memFree=17803712/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=504156 CPUtime=3269.21 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1276398 1257 0 0 326051 870 0 0 23 0 5 0 521775120 516255744 81143 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 126039 81143 181 55 0 123925 0
[pid=9373/tid=9380] ppid=9371 vsize=504156 CPUtime=812.58 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 294759 1257 0 0 81008 250 0 0 25 0 5 0 521775122 516255744 81143 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=504156 CPUtime=817.87 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 353048 1257 0 0 81567 220 0 0 25 0 5 0 521775122 516255744 81143 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=504156 CPUtime=819.97 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 362688 1257 0 0 81764 233 0 0 25 0 5 0 521775122 516255744 81143 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=504156 CPUtime=818.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 263901 1257 0 0 81708 165 0 0 25 0 5 0 521775122 516255744 81143 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3269.21
Current children cumulated vsize (KiB) 504156

[startup+882.307 s]
/proc/loadavg: 8.12 7.98 7.52 10/184 9421
/proc/meminfo: memFree=17712316/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=595612 CPUtime=3508.87 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1330820 1257 0 0 350007 880 0 0 23 0 5 0 521775120 609906688 104007 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 148903 104007 181 55 0 146789 0
[pid=9373/tid=9380] ppid=9371 vsize=595612 CPUtime=872.26 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 306097 1257 0 0 86975 251 0 0 25 0 5 0 521775122 609906688 104007 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=595612 CPUtime=877.83 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 367165 1257 0 0 87559 224 0 0 25 0 5 0 521775122 609906688 104007 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=595612 CPUtime=879.99 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 368970 1257 0 0 87764 235 0 0 25 0 5 0 521775122 609906688 104007 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=595612 CPUtime=878.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 286586 1257 0 0 87705 168 0 0 25 0 5 0 521775122 609906688 104007 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3508.87
Current children cumulated vsize (KiB) 595612

[startup+942.301 s]
/proc/loadavg: 8.10 7.99 7.55 10/184 9422
/proc/meminfo: memFree=17642996/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=664916 CPUtime=3748.48 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1384675 1257 0 0 373960 888 0 0 23 0 5 0 521775120 680873984 121333 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 166229 121333 181 55 0 164115 0
[pid=9373/tid=9380] ppid=9371 vsize=664916 CPUtime=931.94 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 323735 1257 0 0 92940 254 0 0 25 0 5 0 521775122 680873984 121333 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=664916 CPUtime=937.78 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 383821 1257 0 0 93550 228 0 0 25 0 5 0 521775122 680873984 121333 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=664916 CPUtime=939.98 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 375352 1257 0 0 93763 235 0 0 25 0 5 0 521775122 680873984 121333 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=664916 CPUtime=938.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 299765 1257 0 0 93703 170 0 0 25 0 5 0 521775122 680873984 121333 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3748.48
Current children cumulated vsize (KiB) 664916

[startup+1002.3 s]
/proc/loadavg: 8.03 7.99 7.57 10/184 9424
/proc/meminfo: memFree=17627100/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=680876 CPUtime=3988.12 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1429637 1257 0 0 397916 896 0 0 23 0 5 0 521775120 697217024 125323 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 170219 125323 181 55 0 168105 0
[pid=9373/tid=9380] ppid=9371 vsize=680876 CPUtime=991.63 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 331858 1257 0 0 98908 255 0 0 25 0 5 0 521775122 697217024 125323 33554432000 4194304 4417332 140733271705280 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=680876 CPUtime=997.73 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 400908 1257 0 0 99541 232 0 0 25 0 5 0 521775122 697217024 125323 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=680876 CPUtime=999.98 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 383648 1257 0 0 99762 236 0 0 25 0 5 0 521775122 697217024 125323 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=680876 CPUtime=998.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 311221 1257 0 0 99702 171 0 0 25 0 5 0 521775122 697217024 125323 33554432000 4194304 4417332 140733271705280 18446744073709551615 4236559 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3988.12
Current children cumulated vsize (KiB) 680876

[startup+1062.3 s]
/proc/loadavg: 8.06 8.01 7.60 10/184 9426
/proc/meminfo: memFree=17698140/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=614216 CPUtime=4227.76 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1490027 1257 0 0 421870 906 0 0 23 0 5 0 521775120 628957184 108690 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 153554 108690 181 55 0 151440 0
[pid=9373/tid=9380] ppid=9371 vsize=614216 CPUtime=1051.31 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 348019 1257 0 0 104873 258 0 0 25 0 5 0 521775122 628957184 108690 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=614216 CPUtime=1057.69 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 422426 1257 0 0 105533 236 0 0 25 0 5 0 521775122 628957184 108690 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=614216 CPUtime=1059.99 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 396709 1257 0 0 105760 239 0 0 25 0 5 0 521775122 628957184 108690 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364652 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=614216 CPUtime=1058.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 320871 1257 0 0 105701 172 0 0 25 0 5 0 521775122 628957184 108690 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4227.76
Current children cumulated vsize (KiB) 614216

[startup+1122.3 s]
/proc/loadavg: 8.07 8.02 7.63 10/184 9427
/proc/meminfo: memFree=17636128/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=675876 CPUtime=4467.39 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1536780 1257 0 0 445824 915 0 0 23 0 5 0 521775120 692097024 124105 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 168969 124105 181 55 0 166855 0
[pid=9373/tid=9380] ppid=9371 vsize=675876 CPUtime=1110.99 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 357465 1257 0 0 110839 260 0 0 25 0 5 0 521775122 692097024 124105 33554432000 4194304 4417332 140733271705280 18446744073709551615 4356864 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=675876 CPUtime=1117.65 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 433200 1257 0 0 111526 239 0 0 25 0 5 0 521775122 692097024 124105 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=675876 CPUtime=1119.98 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 404901 1257 0 0 111758 240 0 0 25 0 5 0 521775122 692097024 124105 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=675876 CPUtime=1118.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 339212 1257 0 0 111698 175 0 0 25 0 5 0 521775122 692097024 124105 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4467.39
Current children cumulated vsize (KiB) 675876

[startup+1182.3 s]
/proc/loadavg: 8.11 8.04 7.66 10/184 9429
/proc/meminfo: memFree=17653356/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=639276 CPUtime=4707.03 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1579538 1257 0 0 469781 922 0 0 23 0 5 0 521775120 654618624 114923 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 159819 114923 181 55 0 157705 0
[pid=9373/tid=9380] ppid=9371 vsize=639276 CPUtime=1170.67 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 372159 1257 0 0 116805 262 0 0 25 0 5 0 521775122 654618624 114923 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=639276 CPUtime=1177.6 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 444322 1257 0 0 117520 240 0 0 25 0 5 0 521775122 654618624 114923 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=639276 CPUtime=1179.98 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 409664 1257 0 0 117757 241 0 0 25 0 5 0 521775122 654618624 114923 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364965 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=639276 CPUtime=1178.73 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 351391 1257 0 0 117696 177 0 0 25 0 5 0 521775122 654618624 114923 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364599 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4707.03
Current children cumulated vsize (KiB) 639276



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.6 s]
/proc/loadavg: 8.22 8.07 7.68 9/184 9430
/proc/meminfo: memFree=17665504/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=650956 CPUtime=4800.11 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1595327 1257 0 0 479086 925 0 0 23 0 5 0 521775120 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 162739 117843 181 55 0 160625 0
[pid=9373/tid=9380] ppid=9371 vsize=650956 CPUtime=1193.85 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 375803 1257 0 0 119122 263 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=650956 CPUtime=1200.88 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 453938 1257 0 0 119846 242 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=650956 CPUtime=1203.29 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 409664 1257 0 0 120088 241 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4366017 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=650956 CPUtime=1202.04 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 353920 1257 0 0 120027 177 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 650956

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.6 s]
/proc/loadavg: 8.22 8.07 7.68 9/184 9430
/proc/meminfo: memFree=17665504/32951132 swapFree=67111400/67111528
[pid=9373] ppid=9371 vsize=650956 CPUtime=4800.11 cores=0,2,4,6
/proc/9373/stat : 9373 (plingeling) S 9371 9373 8608 0 -1 4202496 1595327 1257 0 0 479086 925 0 0 23 0 5 0 521775120 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 210658949093 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/9373/statm: 162739 117843 181 55 0 160625 0
[pid=9373/tid=9380] ppid=9371 vsize=650956 CPUtime=1193.85 cores=0,2,4,6
/proc/9373/task/9380/stat : 9380 (plingeling) R 9371 9373 8608 0 -1 4202560 375803 1257 0 0 119122 263 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9373/tid=9381] ppid=9371 vsize=650956 CPUtime=1200.88 cores=0,2,4,6
/proc/9373/task/9381/stat : 9381 (plingeling) R 9371 9373 8608 0 -1 4202560 453938 1257 0 0 119846 242 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9373/tid=9382] ppid=9371 vsize=650956 CPUtime=1203.29 cores=0,2,4,6
/proc/9373/task/9382/stat : 9382 (plingeling) R 9371 9373 8608 0 -1 4202560 409664 1257 0 0 120088 241 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4366017 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9373/tid=9383] ppid=9371 vsize=650956 CPUtime=1202.04 cores=0,2,4,6
/proc/9373/task/9383/stat : 9383 (plingeling) R 9371 9373 8608 0 -1 4202560 353920 1257 0 0 120027 177 0 0 25 0 5 0 521775122 666578944 117843 33554432000 4194304 4417332 140733271705280 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 650956

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.64
CPU time (s): 4800.18
CPU user time (s): 4790.89
CPU system time (s): 9.28459
CPU usage (%): 398.143
Max. virtual memory (cumulated for all children) (KiB): 738096

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4790.89
system time used= 9.28459
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1596584
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 518
involuntary context switches= 20871

runsolver used 1.75073 second user time and 4.27035 second system time

The end

Launcher Data 

Begin job on node129 at 2011-04-29 20:29:15
IDJOB=3291553
IDBENCH=84237
IDSOLVER=1605
FILE ID=node129/3291553-1304101755
RUNJOBID= node129-1304101755-9359
PBS_JOBID= 13170600
Free space on /tmp= 66372 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT03/industrial/kukula/addm_bench/am_7_7.shuffled-as.sat03-363.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3291553-1304101755/watcher-3291553-1304101755 -o /tmp/evaluation-result-3291553-1304101755/solver-3291553-1304101755 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3291553-1304101755.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c7f51e1ee7556440ab29e1c4bfe29538
RANDOM SEED=612121091

node129.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      19119200 kB
Buffers:       1254428 kB
Cached:       11848972 kB
SwapCached:        128 kB
Active:        2206592 kB
Inactive:     10963192 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      19119200 kB
SwapTotal:    67111528 kB
SwapFree:     67111400 kB
Dirty:             420 kB
Writeback:           0 kB
AnonPages:       66088 kB
Mapped:          14784 kB
Slab:           598716 kB
PageTables:       4720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   192032 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66308 MiB
End job on node129 at 2011-04-29 20:49:23