Trace number 3287137

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.14 1202.43

General information on the benchmark

Nameapplication/jarvisalo/smtqfbv-aigs/
smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
MD5SUMb967fca11e85ef5b614d3452b1c4b2ad
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables257030
Number of clauses769313
Sum of the clauses size1795061
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2512874
Number of clauses of size 3256437
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3287137-1304062793/watcher-3287137-1304062793 -o /tmp/evaluation-result-3287137-1304062793/solver-3287137-1304062793 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3287137-1304062793.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.87 7.89 7.51 5/181 21412
/proc/meminfo: memFree=9940208/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=12184 CPUtime=0 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) R 21410 21412 20615 0 -1 4202496 1124 1252 0 0 0 0 0 0 23 0 1 0 596510061 12476416 1052 33554432000 4194304 4417332 140736911516224 18446744073709551615 216734872557 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/21412/statm: 3046 1052 147 55 0 932 0

[startup+0.0705181 s]
/proc/loadavg: 7.87 7.89 7.51 5/181 21412
/proc/meminfo: memFree=9940208/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=28652 CPUtime=0.06 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) R 21410 21412 20615 0 -1 4202496 5280 1252 0 0 6 0 0 0 23 0 1 0 596510061 29339648 5186 33554432000 4194304 4417332 140736911516224 18446744073709551615 216735236522 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/21412/statm: 7419 5203 148 55 0 5305 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 28652

[startup+0.100509 s]
/proc/loadavg: 7.87 7.89 7.51 5/181 21412
/proc/meminfo: memFree=9940208/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=29676 CPUtime=0.09 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) R 21410 21412 20615 0 -1 4202496 5536 1252 0 0 9 0 0 0 23 0 1 0 596510061 30388224 5442 33554432000 4194304 4417332 140736911516224 18446744073709551615 4209658 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/21412/statm: 7419 5442 148 55 0 5305 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 29676

[startup+0.300474 s]
/proc/loadavg: 7.87 7.89 7.51 5/181 21412
/proc/meminfo: memFree=9940208/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=82436 CPUtime=0.28 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) R 21410 21412 20615 0 -1 4202496 18757 1252 0 0 27 1 0 0 24 0 1 0 596510061 84414464 18632 33554432000 4194304 4417332 140736911516224 18446744073709551615 4259140 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/21412/statm: 20609 18632 148 55 0 18495 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 82436

[startup+0.700406 s]
/proc/loadavg: 7.87 7.89 7.51 5/181 21412
/proc/meminfo: memFree=9940208/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=152444 CPUtime=0.69 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) R 21410 21412 20615 0 -1 4202496 36295 1252 0 0 65 4 0 0 25 0 1 0 596510061 156102656 36134 33554432000 4194304 4417332 140736911516224 18446744073709551615 216734797404 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/21412/statm: 38111 36134 148 55 0 35997 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 152444

[startup+1.50023 s]
/proc/loadavg: 7.87 7.89 7.51 9/186 21422
/proc/meminfo: memFree=9785548/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=282572 CPUtime=3.09 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 62913 1252 0 0 300 9 0 0 25 0 5 0 596510061 289353728 47044 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 70643 47044 171 55 0 68529 0
[pid=21412/tid=21419] ppid=21410 vsize=282572 CPUtime=0.53 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 7046 1252 0 0 52 1 0 0 25 0 5 0 596510158 289353728 47044 33554432000 4194304 4417332 140736911516224 18446744073709551615 4292119 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=282572 CPUtime=0.52 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 7031 1252 0 0 51 1 0 0 25 0 5 0 596510158 289353728 47044 33554432000 4194304 4417332 140736911516224 18446744073709551615 4292014 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=282572 CPUtime=0.52 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 7031 1252 0 0 51 1 0 0 25 0 5 0 596510158 289353728 47044 33554432000 4194304 4417332 140736911516224 18446744073709551615 4290599 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=282572 CPUtime=0.52 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 5498 1252 0 0 52 0 0 0 25 0 5 0 596510158 289353728 47044 33554432000 4194304 4417332 140736911516224 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 282572

[startup+3.10096 s]
/proc/loadavg: 7.87 7.89 7.51 9/186 21422
/proc/meminfo: memFree=9741188/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=294596 CPUtime=9.49 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 90831 1252 0 0 935 14 0 0 25 0 5 0 596510061 301666304 48065 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 73649 48065 175 55 0 71535 0
[pid=21412/tid=21419] ppid=21410 vsize=294596 CPUtime=2.13 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 16891 1252 0 0 211 2 0 0 25 0 5 0 596510158 301666304 48065 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=294596 CPUtime=2.12 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 14229 1252 0 0 210 2 0 0 25 0 5 0 596510158 301666304 48065 33554432000 4194304 4417332 140736911516224 18446744073709551615 4275838 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=294596 CPUtime=2.12 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 17904 1252 0 0 209 3 0 0 25 0 5 0 596510158 301666304 48065 33554432000 4194304 4417332 140736911516224 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=294596 CPUtime=2.12 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 5500 1252 0 0 212 0 0 0 25 0 5 0 596510158 301666304 48065 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.49
Current children cumulated vsize (KiB) 294596

[startup+6.3004 s]
/proc/loadavg: 7.88 7.89 7.51 9/186 21422
/proc/meminfo: memFree=9693548/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=321828 CPUtime=22.25 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 129660 1252 0 0 2205 20 0 0 25 0 5 0 596510061 329551872 59688 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 80457 59688 180 55 0 78343 0
[pid=21412/tid=21419] ppid=21410 vsize=321828 CPUtime=5.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 28083 1252 0 0 528 4 0 0 25 0 5 0 596510158 329551872 59688 33554432000 4194304 4417332 140736911516224 18446744073709551615 4240063 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=321828 CPUtime=5.32 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 25878 1252 0 0 528 4 0 0 25 0 5 0 596510158 329551872 59688 33554432000 4194304 4417332 140736911516224 18446744073709551615 4252216 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=321828 CPUtime=5.29 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 33892 1252 0 0 524 5 0 0 25 0 5 0 596510158 329551872 59688 33554432000 4194304 4417332 140736911516224 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=321828 CPUtime=5.32 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 5500 1252 0 0 532 0 0 0 25 0 5 0 596510158 329551872 59688 33554432000 4194304 4417332 140736911516224 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 22.25
Current children cumulated vsize (KiB) 321828

[startup+12.7003 s]
/proc/loadavg: 7.89 7.89 7.52 9/186 21422
/proc/meminfo: memFree=10062108/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=352228 CPUtime=47.82 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 138798 1252 0 0 4760 22 0 0 25 0 5 0 596510061 360681472 66876 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 88057 66876 180 55 0 85943 0
[pid=21412/tid=21419] ppid=21410 vsize=352228 CPUtime=11.73 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 32182 1252 0 0 1168 5 0 0 25 0 5 0 596510158 360681472 66876 33554432000 4194304 4417332 140736911516224 18446744073709551615 4272705 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=352228 CPUtime=11.72 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 28373 1252 0 0 1167 5 0 0 25 0 5 0 596510158 360681472 66876 33554432000 4194304 4417332 140736911516224 18446744073709551615 4253385 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=352228 CPUtime=11.66 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 36436 1252 0 0 1161 5 0 0 25 0 5 0 596510158 360681472 66876 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=352228 CPUtime=11.72 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 5500 1252 0 0 1172 0 0 0 25 0 5 0 596510158 360681472 66876 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 47.82
Current children cumulated vsize (KiB) 352228

[startup+25.501 s]
/proc/loadavg: 7.91 7.90 7.52 9/186 21422
/proc/meminfo: memFree=10031580/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=351108 CPUtime=98.94 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 163213 1252 0 0 9868 26 0 0 25 0 5 0 596510061 359534592 64254 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 87777 64254 180 55 0 85663 0
[pid=21412/tid=21419] ppid=21410 vsize=351108 CPUtime=24.53 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 40374 1252 0 0 2446 7 0 0 25 0 5 0 596510158 359534592 64254 33554432000 4194304 4417332 140736911516224 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=351108 CPUtime=24.52 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 28373 1252 0 0 2447 5 0 0 25 0 5 0 596510158 359534592 64254 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=351108 CPUtime=24.39 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 52659 1252 0 0 2431 8 0 0 25 0 5 0 596510158 359534592 64254 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=351108 CPUtime=24.52 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 5500 1252 0 0 2452 0 0 0 25 0 5 0 596510158 359534592 64254 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364491 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 98.94
Current children cumulated vsize (KiB) 351108

[startup+51.1006 s]
/proc/loadavg: 8.01 7.92 7.54 9/186 21423
/proc/meminfo: memFree=9741992/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=367416 CPUtime=201.2 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 179412 1252 0 0 20092 28 0 0 25 0 5 0 596510061 376233984 70743 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 91854 70743 181 55 0 89740 0
[pid=21412/tid=21419] ppid=21410 vsize=367416 CPUtime=50.13 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 44569 1252 0 0 5005 8 0 0 25 0 5 0 596510158 376233984 70743 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=367416 CPUtime=50.12 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 28471 1252 0 0 5007 5 0 0 25 0 5 0 596510158 376233984 70743 33554432000 4194304 4417332 140736911516224 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=367416 CPUtime=49.85 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 64565 1252 0 0 4976 9 0 0 25 0 5 0 596510158 376233984 70743 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=367416 CPUtime=50.12 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 5500 1252 0 0 5012 0 0 0 25 0 5 0 596510158 376233984 70743 33554432000 4194304 4417332 140736911516224 18446744073709551615 4356771 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 201.2
Current children cumulated vsize (KiB) 367416

[startup+102.307 s]
/proc/loadavg: 8.08 7.95 7.57 9/186 21424
/proc/meminfo: memFree=10039812/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=389280 CPUtime=405.75 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 224222 1252 0 0 40541 34 0 0 25 0 5 0 596510061 398622720 76859 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 97320 76859 181 55 0 95206 0
[pid=21412/tid=21419] ppid=21410 vsize=389280 CPUtime=101.33 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 44737 1252 0 0 10125 8 0 0 25 0 5 0 596510158 398622720 76859 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=389280 CPUtime=101.32 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 28629 1252 0 0 10127 5 0 0 25 0 5 0 596510158 398622720 76859 33554432000 4194304 4417332 140736911516224 18446744073709551615 4278190 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=389280 CPUtime=100.77 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 104953 1252 0 0 10063 14 0 0 25 0 5 0 596510158 398622720 76859 33554432000 4194304 4417332 140736911516224 18446744073709551615 4253228 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=389280 CPUtime=101.33 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 10132 1 0 0 25 0 5 0 596510158 398622720 76859 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 405.75
Current children cumulated vsize (KiB) 389280

[startup+162.301 s]
/proc/loadavg: 8.09 7.97 7.60 9/186 21426
/proc/meminfo: memFree=9691332/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=407388 CPUtime=645.4 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 234218 1252 0 0 64505 35 0 0 25 0 5 0 596510061 417165312 76806 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 101847 76806 181 55 0 99733 0
[pid=21412/tid=21419] ppid=21410 vsize=407388 CPUtime=161.33 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 45427 1252 0 0 16125 8 0 0 25 0 5 0 596510158 417165312 76806 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=407388 CPUtime=161.31 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 28698 1252 0 0 16126 5 0 0 25 0 5 0 596510158 417165312 76806 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=407388 CPUtime=160.45 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 114190 1252 0 0 16030 15 0 0 25 0 5 0 596510158 417165312 76806 33554432000 4194304 4417332 140736911516224 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=407388 CPUtime=161.33 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 16132 1 0 0 25 0 5 0 596510158 417165312 76806 33554432000 4194304 4417332 140736911516224 18446744073709551615 4366440 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 645.4
Current children cumulated vsize (KiB) 407388


################
# More data... #
################

[pid=21412/tid=21420] ppid=21410 vsize=441468 CPUtime=701.26 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 33557 1252 0 0 70120 6 0 0 25 0 5 0 596510158 452063232 87512 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236713 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=441468 CPUtime=697.51 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 197821 1252 0 0 69725 26 0 0 25 0 5 0 596510158 452063232 87512 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=441468 CPUtime=701.34 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 70133 1 0 0 25 0 5 0 596510158 452063232 87512 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236603 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2802.43
Current children cumulated vsize (KiB) 441468

[startup+762.307 s]
/proc/loadavg: 8.12 8.04 7.78 9/186 21479
/proc/meminfo: memFree=11824700/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=441720 CPUtime=3042.12 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 336122 1252 0 0 304163 49 0 0 25 0 5 0 596510061 452321280 86694 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 110430 86694 181 55 0 108316 0
[pid=21412/tid=21419] ppid=21410 vsize=441720 CPUtime=761.33 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 51255 1252 0 0 76124 9 0 0 25 0 5 0 596510158 452321280 86694 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=441720 CPUtime=761.27 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 33557 1252 0 0 76121 6 0 0 25 0 5 0 596510158 452321280 86694 33554432000 4194304 4417332 140736911516224 18446744073709551615 4365033 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=441720 CPUtime=757.19 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 205407 1252 0 0 75692 27 0 0 25 0 5 0 596510158 452321280 86694 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364887 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=441720 CPUtime=761.35 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 76134 1 0 0 25 0 5 0 596510158 452321280 86694 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3042.12
Current children cumulated vsize (KiB) 441720

[startup+822.301 s]
/proc/loadavg: 8.10 8.04 7.80 9/186 21481
/proc/meminfo: memFree=11934892/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=425532 CPUtime=3281.77 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 341925 1252 0 0 328127 50 0 0 25 0 5 0 596510061 435744768 84803 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 106383 84803 181 55 0 104269 0
[pid=21412/tid=21419] ppid=21410 vsize=425532 CPUtime=821.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 51307 1252 0 0 82123 9 0 0 25 0 5 0 596510158 435744768 84803 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=425532 CPUtime=821.26 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 33611 1252 0 0 82120 6 0 0 25 0 5 0 596510158 435744768 84803 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=425532 CPUtime=816.86 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 211104 1252 0 0 81658 28 0 0 25 0 5 0 596510158 435744768 84803 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=425532 CPUtime=821.34 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 82133 1 0 0 25 0 5 0 596510158 435744768 84803 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3281.77
Current children cumulated vsize (KiB) 425532

[startup+882.301 s]
/proc/loadavg: 8.03 8.03 7.81 9/186 21482
/proc/meminfo: memFree=11598968/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=446104 CPUtime=3521.43 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 353791 1252 0 0 352092 51 0 0 25 0 5 0 596510061 456810496 90444 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 111526 90444 181 55 0 109412 0
[pid=21412/tid=21419] ppid=21410 vsize=446104 CPUtime=881.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 51307 1252 0 0 88123 9 0 0 25 0 5 0 596510158 456810496 90444 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=446104 CPUtime=881.25 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 34710 1252 0 0 88119 6 0 0 25 0 5 0 596510158 456810496 90444 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=446104 CPUtime=876.53 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 221871 1252 0 0 87624 29 0 0 25 0 5 0 596510158 456810496 90444 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364652 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=446104 CPUtime=881.34 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 88133 1 0 0 25 0 5 0 596510158 456810496 90444 33554432000 4194304 4417332 140736911516224 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3521.43
Current children cumulated vsize (KiB) 446104

[startup+942.301 s]
/proc/loadavg: 8.11 8.05 7.83 9/186 21484
/proc/meminfo: memFree=11743056/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=446704 CPUtime=3761.11 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 362366 1252 0 0 376058 53 0 0 25 0 5 0 596510061 457424896 90196 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 111676 90196 181 55 0 109562 0
[pid=21412/tid=21419] ppid=21410 vsize=446704 CPUtime=941.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 51359 1252 0 0 94123 9 0 0 25 0 5 0 596510158 457424896 90196 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=446704 CPUtime=941.24 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 34763 1252 0 0 94118 6 0 0 25 0 5 0 596510158 457424896 90196 33554432000 4194304 4417332 140736911516224 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=446704 CPUtime=936.21 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 230341 1252 0 0 93590 31 0 0 25 0 5 0 596510158 457424896 90196 33554432000 4194304 4417332 140736911516224 18446744073709551615 4365941 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=446704 CPUtime=941.35 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 94134 1 0 0 25 0 5 0 596510158 457424896 90196 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236750 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3761.11
Current children cumulated vsize (KiB) 446704

[startup+1002.3 s]
/proc/loadavg: 8.08 8.06 7.84 9/186 21485
/proc/meminfo: memFree=11942440/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=446104 CPUtime=4000.77 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 370620 1252 0 0 400024 53 0 0 25 0 5 0 596510061 456810496 90728 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 111526 90728 181 55 0 109412 0
[pid=21412/tid=21419] ppid=21410 vsize=446104 CPUtime=1001.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 51359 1252 0 0 100123 9 0 0 25 0 5 0 596510158 456810496 90728 33554432000 4194304 4417332 140736911516224 18446744073709551615 4233447 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=446104 CPUtime=1001.24 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 34763 1252 0 0 100118 6 0 0 25 0 5 0 596510158 456810496 90728 33554432000 4194304 4417332 140736911516224 18446744073709551615 4237189 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=446104 CPUtime=995.88 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 238595 1252 0 0 99556 32 0 0 25 0 5 0 596510158 456810496 90728 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=446104 CPUtime=1001.35 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 100134 1 0 0 25 0 5 0 596510158 456810496 90728 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4000.77
Current children cumulated vsize (KiB) 446104

[startup+1062.3 s]
/proc/loadavg: 8.11 8.07 7.86 9/186 21487
/proc/meminfo: memFree=11718000/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=458540 CPUtime=4240.45 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 381963 1252 0 0 423990 55 0 0 25 0 5 0 596510061 469544960 93501 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 114635 93501 181 55 0 112521 0
[pid=21412/tid=21419] ppid=21410 vsize=458540 CPUtime=1061.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 57600 1252 0 0 106122 10 0 0 25 0 5 0 596510158 469544960 93501 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=458540 CPUtime=1061.23 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 35888 1252 0 0 106117 6 0 0 25 0 5 0 596510158 469544960 93501 33554432000 4194304 4417332 140736911516224 18446744073709551615 4352400 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=458540 CPUtime=1055.56 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 242572 1252 0 0 105524 32 0 0 25 0 5 0 596510158 469544960 93501 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=458540 CPUtime=1061.35 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 106134 1 0 0 25 0 5 0 596510158 469544960 93501 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4240.45
Current children cumulated vsize (KiB) 458540

[startup+1122.3 s]
/proc/loadavg: 8.20 8.10 7.88 9/186 21489
/proc/meminfo: memFree=11441596/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=471312 CPUtime=4480.11 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 394628 1252 0 0 447955 56 0 0 25 0 5 0 596510061 482623488 96471 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 117828 96471 181 55 0 115714 0
[pid=21412/tid=21419] ppid=21410 vsize=471312 CPUtime=1121.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 61739 1252 0 0 112122 10 0 0 25 0 5 0 596510158 482623488 96471 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=471312 CPUtime=1121.22 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 37485 1252 0 0 112116 6 0 0 25 0 5 0 596510158 482623488 96471 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364522 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=471312 CPUtime=1115.23 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 249501 1252 0 0 111490 33 0 0 25 0 5 0 596510158 482623488 96471 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=471312 CPUtime=1121.35 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 112134 1 0 0 25 0 5 0 596510158 482623488 96471 33554432000 4194304 4417332 140736911516224 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4480.11
Current children cumulated vsize (KiB) 471312

[startup+1182.3 s]
/proc/loadavg: 8.14 8.10 7.89 9/186 21490
/proc/meminfo: memFree=11706704/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=465756 CPUtime=4719.79 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 404119 1252 0 0 471921 58 0 0 25 0 5 0 596510061 476934144 94819 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 116439 94819 181 55 0 114325 0
[pid=21412/tid=21419] ppid=21410 vsize=465756 CPUtime=1181.32 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 65940 1252 0 0 118121 11 0 0 25 0 5 0 596510158 476934144 94819 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=465756 CPUtime=1181.22 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 37538 1252 0 0 118116 6 0 0 25 0 5 0 596510158 476934144 94819 33554432000 4194304 4417332 140736911516224 18446744073709551615 4365999 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=465756 CPUtime=1174.91 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 254738 1252 0 0 117457 34 0 0 25 0 5 0 596510158 476934144 94819 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=465756 CPUtime=1181.35 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 118134 1 0 0 25 0 5 0 596510158 476934144 94819 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4719.79
Current children cumulated vsize (KiB) 465756



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.4 s]
/proc/loadavg: 8.18 8.11 7.90 10/186 21491
/proc/meminfo: memFree=11432288/32951124 swapFree=67102140/67111528
[pid=21412] ppid=21410 vsize=469840 CPUtime=4800.09 cores=1,3,5,7
/proc/21412/stat : 21412 (plingeling) S 21410 21412 20615 0 -1 4202496 406410 1252 0 0 479951 58 0 0 25 0 5 0 596510061 481116160 97109 33554432000 4194304 4417332 140736911516224 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/21412/statm: 117460 97109 181 55 0 115346 0
[pid=21412/tid=21419] ppid=21410 vsize=469840 CPUtime=1201.42 cores=1,3,5,7
/proc/21412/task/21419/stat : 21419 (plingeling) R 21410 21412 20615 0 -1 4202560 66961 1252 0 0 120131 11 0 0 25 0 5 0 596510158 481116160 97109 33554432000 4194304 4417332 140736911516224 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21412/tid=21420] ppid=21410 vsize=469840 CPUtime=1201.32 cores=1,3,5,7
/proc/21412/task/21420/stat : 21420 (plingeling) R 21410 21412 20615 0 -1 4202560 37538 1252 0 0 120126 6 0 0 25 0 5 0 596510158 481116160 97109 33554432000 4194304 4417332 140736911516224 18446744073709551615 4357612 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21412/tid=21421] ppid=21410 vsize=469840 CPUtime=1194.9 cores=1,3,5,7
/proc/21412/task/21421/stat : 21421 (plingeling) R 21410 21412 20615 0 -1 4202560 256008 1252 0 0 119456 34 0 0 25 0 5 0 596510158 481116160 97109 33554432000 4194304 4417332 140736911516224 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21412/tid=21422] ppid=21410 vsize=469840 CPUtime=1201.46 cores=1,3,5,7
/proc/21412/task/21422/stat : 21422 (plingeling) R 21410 21412 20615 0 -1 4202560 9596 1252 0 0 120145 1 0 0 25 0 5 0 596510158 481116160 97109 33554432000 4194304 4417332 140736911516224 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 469840

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.43
CPU time (s): 4800.14
CPU user time (s): 4799.53
CPU system time (s): 0.607907
CPU usage (%): 399.204
Max. virtual memory (cumulated for all children) (KiB): 473640

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.53
system time used= 0.607907
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 407662
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 120
involuntary context switches= 38265

runsolver used 1.85072 second user time and 4.71828 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 09:39:54
IDJOB=3287137
IDBENCH=82459
IDSOLVER=1605
FILE ID=node119/3287137-1304062793
RUNJOBID= node119-1304060531-20634
PBS_JOBID= 13170451
Free space on /tmp= 71684 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/jarvisalo/smtqfbv-aigs/smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3287137-1304062793/watcher-3287137-1304062793 -o /tmp/evaluation-result-3287137-1304062793/solver-3287137-1304062793 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3287137-1304062793.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b967fca11e85ef5b614d3452b1c4b2ad
RANDOM SEED=1175407355

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       9940364 kB
Buffers:       2232940 kB
Cached:       17425528 kB
SwapCached:       5504 kB
Active:       10764528 kB
Inactive:     11655480 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       9940364 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           13816 kB
Writeback:           0 kB
AnonPages:     2758592 kB
Mapped:          15148 kB
Slab:           521012 kB
PageTables:       9944 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  2988188 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71796 MiB
End job on node119 at 2011-04-29 09:59:58