Trace number 3282629

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.27 1202.12

General information on the benchmark

NameAPPLICATIONS/crypto/
md5gen/gus-md5-11.cnf
MD5SUM49867705b76fc90285dd6fc48de2613c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables69561
Number of clauses226787
Sum of the clauses size587727
Maximum clause length3
Minimum clause length1
Number of clauses of size 1129
Number of clauses of size 292376
Number of clauses of size 3134282
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3282629-1304042916/watcher-3282629-1304042916 -o /tmp/evaluation-result-3282629-1304042916/solver-3282629-1304042916 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3282629-1304042916.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.72 8.26 8.12 2/177 2943
/proc/meminfo: memFree=31847872/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=11900 CPUtime=0 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) R 2941 2943 1629 0 -1 4202496 752 1260 0 0 0 0 0 0 21 0 1 0 496707206 12185600 682 33554432000 4194304 4417332 140736840096032 18446744073709551615 270338077789 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/2943/statm: 2975 693 143 55 0 861 0

[startup+0.0322739 s]
/proc/loadavg: 6.72 8.26 8.12 2/177 2943
/proc/meminfo: memFree=31847872/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=23732 CPUtime=0.02 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) R 2941 2943 1629 0 -1 4202496 4010 1260 0 0 2 0 0 0 21 0 1 0 496707206 24301568 3940 33554432000 4194304 4417332 140736840096032 18446744073709551615 4264900 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/2943/statm: 5933 3940 148 55 0 3819 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 23732

[startup+0.100294 s]
/proc/loadavg: 6.72 8.26 8.12 2/177 2943
/proc/meminfo: memFree=31847872/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=36788 CPUtime=0.08 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) R 2941 2943 1629 0 -1 4202496 7297 1260 0 0 8 0 0 0 21 0 1 0 496707206 37670912 7220 33554432000 4194304 4417332 140736840096032 18446744073709551615 4264940 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/2943/statm: 9197 7220 148 55 0 7083 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 36788

[startup+0.300219 s]
/proc/loadavg: 6.72 8.26 8.12 2/177 2943
/proc/meminfo: memFree=31847872/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=80196 CPUtime=0.29 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) R 2941 2943 1629 0 -1 4202496 18149 1260 0 0 28 1 0 0 22 0 1 0 496707206 82120704 18072 33554432000 4194304 4417332 140736840096032 18446744073709551615 4231537 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/2943/statm: 20049 18072 148 55 0 17935 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 80196

[startup+1.15101 s]
/proc/loadavg: 6.72 8.26 8.12 2/177 2943
/proc/meminfo: memFree=31847872/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=182984 CPUtime=3.54 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 31091 1260 0 0 352 2 0 0 22 0 5 0 496707206 187375616 19103 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 45746 19103 171 55 0 43632 0
Current children cumulated CPU time (s) 3.54
Current children cumulated vsize (KiB) 182984

[startup+1.50095 s]
/proc/loadavg: 6.72 8.26 8.12 2/177 2943
/proc/meminfo: memFree=31847872/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=182444 CPUtime=4.94 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 33800 1260 0 0 492 2 0 0 22 0 5 0 496707206 186822656 18967 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 45611 18967 171 55 0 43497 0
Current children cumulated CPU time (s) 4.94
Current children cumulated vsize (KiB) 182444

[startup+3.10063 s]
/proc/loadavg: 6.82 8.26 8.12 9/186 2959
/proc/meminfo: memFree=31744192/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=190432 CPUtime=11.33 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 44372 1260 0 0 1129 4 0 0 22 0 5 0 496707206 195002368 20913 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 47608 20913 179 55 0 45494 0
[pid=2943/tid=2955] ppid=2941 vsize=190432 CPUtime=2.74 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 5431 1260 0 0 274 0 0 0 25 0 5 0 496707240 195002368 20913 33554432000 4194304 4417332 140736840096032 18446744073709551615 4320401 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=190432 CPUtime=2.75 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 9114 1260 0 0 274 1 0 0 25 0 5 0 496707240 195002368 20913 33554432000 4194304 4417332 140736840096032 18446744073709551615 4233560 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=190432 CPUtime=2.75 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 9944 1260 0 0 274 1 0 0 25 0 5 0 496707240 195002368 20913 33554432000 4194304 4417332 140736840096032 18446744073709551615 4248516 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=190432 CPUtime=2.74 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 1722 1260 0 0 274 0 0 0 23 0 5 0 496707240 195002368 20913 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236544 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.33
Current children cumulated vsize (KiB) 190432

[startup+6.30099 s]
/proc/loadavg: 6.82 8.26 8.12 9/186 2959
/proc/meminfo: memFree=31730180/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=208092 CPUtime=24.12 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 48800 1260 0 0 2407 5 0 0 22 0 5 0 496707206 213086208 24350 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 52023 24350 179 55 0 49909 0
[pid=2943/tid=2955] ppid=2941 vsize=208092 CPUtime=5.95 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 7805 1260 0 0 594 1 0 0 25 0 5 0 496707240 213086208 24350 33554432000 4194304 4417332 140736840096032 18446744073709551615 4252204 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=208092 CPUtime=5.94 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 11168 1260 0 0 593 1 0 0 25 0 5 0 496707240 213086208 24350 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236267 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=208092 CPUtime=5.95 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 9944 1260 0 0 594 1 0 0 25 0 5 0 496707240 213086208 24350 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=208092 CPUtime=5.92 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 1722 1260 0 0 592 0 0 0 25 0 5 0 496707240 213086208 24350 33554432000 4194304 4417332 140736840096032 18446744073709551615 4237332 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.12
Current children cumulated vsize (KiB) 208092

[startup+12.7007 s]
/proc/loadavg: 7.00 8.25 8.11 9/186 2959
/proc/meminfo: memFree=31726452/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=211160 CPUtime=49.7 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 49641 1260 0 0 4965 5 0 0 22 0 5 0 496707206 216227840 25117 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 52790 25117 179 55 0 50676 0
[pid=2943/tid=2955] ppid=2941 vsize=211160 CPUtime=12.36 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 7805 1260 0 0 1235 1 0 0 25 0 5 0 496707240 216227840 25117 33554432000 4194304 4417332 140736840096032 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=211160 CPUtime=12.35 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 11168 1260 0 0 1234 1 0 0 25 0 5 0 496707240 216227840 25117 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236672 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=211160 CPUtime=12.35 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 9944 1260 0 0 1234 1 0 0 25 0 5 0 496707240 216227840 25117 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236559 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=211160 CPUtime=12.29 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 2563 1260 0 0 1229 0 0 0 25 0 5 0 496707240 216227840 25117 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.7
Current children cumulated vsize (KiB) 211160

[startup+25.5012 s]
/proc/loadavg: 7.23 8.26 8.12 9/186 2959
/proc/meminfo: memFree=31724344/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=215240 CPUtime=100.82 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 50661 1260 0 0 10077 5 0 0 22 0 5 0 496707206 220405760 26137 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 53810 26137 179 55 0 51696 0
[pid=2943/tid=2955] ppid=2941 vsize=215240 CPUtime=25.15 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 8361 1260 0 0 2514 1 0 0 25 0 5 0 496707240 220405760 26137 33554432000 4194304 4417332 140736840096032 18446744073709551615 4354063 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=215240 CPUtime=25.14 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 11168 1260 0 0 2513 1 0 0 25 0 5 0 496707240 220405760 26137 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=215240 CPUtime=25.15 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 9944 1260 0 0 2514 1 0 0 25 0 5 0 496707240 220405760 26137 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=215240 CPUtime=25.02 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 3027 1260 0 0 2502 0 0 0 25 0 5 0 496707240 220405760 26137 33554432000 4194304 4417332 140736840096032 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.82
Current children cumulated vsize (KiB) 215240

[startup+51.1011 s]
/proc/loadavg: 7.49 8.24 8.11 9/186 2960
/proc/meminfo: memFree=31699328/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=240432 CPUtime=203.08 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 58921 1260 0 0 20302 6 0 0 22 0 5 0 496707206 246202368 32435 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 60108 32435 179 55 0 57994 0
[pid=2943/tid=2955] ppid=2941 vsize=240432 CPUtime=50.75 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 11712 1260 0 0 5074 1 0 0 25 0 5 0 496707240 246202368 32435 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=240432 CPUtime=50.74 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 13712 1260 0 0 5073 1 0 0 25 0 5 0 496707240 246202368 32435 33554432000 4194304 4417332 140736840096032 18446744073709551615 4352392 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=240432 CPUtime=50.75 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 11287 1260 0 0 5074 1 0 0 25 0 5 0 496707240 246202368 32435 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250961 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=240432 CPUtime=50.48 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 4049 1260 0 0 5048 0 0 0 25 0 5 0 496707240 246202368 32435 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.08
Current children cumulated vsize (KiB) 240432

[startup+102.301 s]
/proc/loadavg: 7.94 8.24 8.12 9/186 2997
/proc/meminfo: memFree=31695372/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=242740 CPUtime=407.61 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 62633 1260 0 0 40754 7 0 0 22 0 5 0 496707206 248565760 33013 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 60685 33013 180 55 0 58571 0
[pid=2943/tid=2955] ppid=2941 vsize=242740 CPUtime=101.95 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 11712 1260 0 0 10193 2 0 0 25 0 5 0 496707240 248565760 33013 33554432000 4194304 4417332 140736840096032 18446744073709551615 4252192 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=242740 CPUtime=101.94 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 17004 1260 0 0 10193 1 0 0 25 0 5 0 496707240 248565760 33013 33554432000 4194304 4417332 140736840096032 18446744073709551615 4251063 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=242740 CPUtime=101.94 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 11287 1260 0 0 10193 1 0 0 25 0 5 0 496707240 248565760 33013 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236895 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=242740 CPUtime=101.4 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 4469 1260 0 0 10140 0 0 0 25 0 5 0 496707240 248565760 33013 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.61
Current children cumulated vsize (KiB) 242740

[startup+162.301 s]
/proc/loadavg: 8.08 8.22 8.12 9/186 3000
/proc/meminfo: memFree=31691004/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=243372 CPUtime=647.31 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 77129 1260 0 0 64722 9 0 0 22 0 5 0 496707206 249212928 32387 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 60843 32387 180 55 0 58729 0
[pid=2943/tid=2955] ppid=2941 vsize=243372 CPUtime=161.95 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 14743 1260 0 0 16193 2 0 0 25 0 5 0 496707240 249212928 32387 33554432000 4194304 4417332 140736840096032 18446744073709551615 4365869 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=243372 CPUtime=161.95 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 23530 1260 0 0 16193 2 0 0 25 0 5 0 496707240 249212928 32387 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=243372 CPUtime=161.95 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 16226 1260 0 0 16194 1 0 0 25 0 5 0 496707240 249212928 32387 33554432000 4194304 4417332 140736840096032 18446744073709551615 4353472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=243372 CPUtime=161.08 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 4469 1260 0 0 16108 0 0 0 25 0 5 0 496707240 249212928 32387 33554432000 4194304 4417332 140736840096032 18446744073709551615 4357874 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.31
Current children cumulated vsize (KiB) 243372

[startup+222.301 s]
/proc/loadavg: 8.12 8.21 8.12 9/186 3002
/proc/meminfo: memFree=31692692/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=241932 CPUtime=886.98 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 88016 1260 0 0 88686 12 0 0 22 0 5 0 496707206 247738368 32027 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 60483 32027 180 55 0 58369 0
[pid=2943/tid=2955] ppid=2941 vsize=241932 CPUtime=221.96 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 17701 1260 0 0 22193 3 0 0 25 0 5 0 496707240 247738368 32027 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 2 0 0 0

################
# More data... #
################

[pid=2943/tid=2956] ppid=2941 vsize=297984 CPUtime=701.97 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 87358 1260 0 0 70183 14 0 0 25 0 5 0 496707240 305135616 46526 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=297984 CPUtime=701.96 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 50906 1260 0 0 70189 7 0 0 25 0 5 0 496707240 305135616 46526 33554432000 4194304 4417332 140736840096032 18446744073709551615 4352479 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=297984 CPUtime=698.17 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 38176 1260 0 0 69811 6 0 0 25 0 5 0 496707240 305135616 46526 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2804.46
Current children cumulated vsize (KiB) 297984

[startup+762.301 s]
/proc/loadavg: 8.15 8.12 8.09 9/186 3016
/proc/meminfo: memFree=31612264/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=305136 CPUtime=3044.14 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 281933 1260 0 0 304369 45 0 0 22 0 5 0 496707206 312459264 48314 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 76284 48314 180 55 0 74170 0
[pid=2943/tid=2955] ppid=2941 vsize=305136 CPUtime=761.98 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 70955 1260 0 0 76187 11 0 0 25 0 5 0 496707240 312459264 48314 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=305136 CPUtime=761.97 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 94025 1260 0 0 76182 15 0 0 25 0 5 0 496707240 312459264 48314 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250972 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=305136 CPUtime=761.96 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 57539 1260 0 0 76188 8 0 0 25 0 5 0 496707240 312459264 48314 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=305136 CPUtime=757.85 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 41253 1260 0 0 75778 7 0 0 25 0 5 0 496707240 312459264 48314 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3044.14
Current children cumulated vsize (KiB) 305136

[startup+822.301 s]
/proc/loadavg: 8.12 8.11 8.09 9/186 3018
/proc/meminfo: memFree=31626188/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=293664 CPUtime=3283.84 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 296390 1260 0 0 328336 48 0 0 22 0 5 0 496707206 300711936 45446 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 73416 45446 180 55 0 71302 0
[pid=2943/tid=2955] ppid=2941 vsize=293664 CPUtime=821.99 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 73857 1260 0 0 82187 12 0 0 25 0 5 0 496707240 300711936 45446 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=293664 CPUtime=821.99 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 99713 1260 0 0 82182 17 0 0 25 0 5 0 496707240 300711936 45446 33554432000 4194304 4417332 140736840096032 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=293664 CPUtime=821.96 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 62927 1260 0 0 82187 9 0 0 25 0 5 0 496707240 300711936 45446 33554432000 4194304 4417332 140736840096032 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=293664 CPUtime=817.53 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 41732 1260 0 0 81746 7 0 0 25 0 5 0 496707240 300711936 45446 33554432000 4194304 4417332 140736840096032 18446744073709551615 4237214 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3283.84
Current children cumulated vsize (KiB) 293664

[startup+882.3 s]
/proc/loadavg: 8.11 8.10 8.09 9/186 3019
/proc/meminfo: memFree=31611468/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=308656 CPUtime=3523.5 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 314523 1260 0 0 352300 50 0 0 22 0 5 0 496707206 316063744 49194 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 77164 49194 180 55 0 75050 0
[pid=2943/tid=2955] ppid=2941 vsize=308656 CPUtime=881.99 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 80718 1260 0 0 88186 13 0 0 25 0 5 0 496707240 316063744 49194 33554432000 4194304 4417332 140736840096032 18446744073709551615 4354498 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=308656 CPUtime=881.97 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 104539 1260 0 0 88180 17 0 0 25 0 5 0 496707240 316063744 49194 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=308656 CPUtime=881.96 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 68662 1260 0 0 88186 10 0 0 25 0 5 0 496707240 316063744 49194 33554432000 4194304 4417332 140736840096032 18446744073709551615 4357851 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=308656 CPUtime=877.21 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 42443 1260 0 0 87714 7 0 0 25 0 5 0 496707240 316063744 49194 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3523.5
Current children cumulated vsize (KiB) 308656

[startup+942.306 s]
/proc/loadavg: 8.09 8.10 8.09 9/186 3021
/proc/meminfo: memFree=31610640/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=305340 CPUtime=3763.22 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 334278 1260 0 0 376268 54 0 0 22 0 5 0 496707206 312668160 48365 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 76335 48365 180 55 0 74221 0
[pid=2943/tid=2955] ppid=2941 vsize=305340 CPUtime=942 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 85612 1260 0 0 94186 14 0 0 25 0 5 0 496707240 312668160 48365 33554432000 4194304 4417332 140736840096032 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=305340 CPUtime=941.98 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 111846 1260 0 0 94180 18 0 0 25 0 5 0 496707240 312668160 48365 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=305340 CPUtime=941.97 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 75868 1260 0 0 94186 11 0 0 25 0 5 0 496707240 312668160 48365 33554432000 4194304 4417332 140736840096032 18446744073709551615 4353734 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=305340 CPUtime=936.89 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 42791 1260 0 0 93682 7 0 0 25 0 5 0 496707240 312668160 48365 33554432000 4194304 4417332 140736840096032 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3763.22
Current children cumulated vsize (KiB) 305340

[startup+1002.3 s]
/proc/loadavg: 8.25 8.12 8.10 10/187 3025
/proc/meminfo: memFree=31595716/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=299908 CPUtime=4002.29 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 353590 1260 0 0 400171 58 0 0 22 0 5 0 496707206 307105792 47007 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 74977 47007 180 55 0 72863 0
[pid=2943/tid=2955] ppid=2941 vsize=299908 CPUtime=1001.98 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 91627 1260 0 0 100183 15 0 0 25 0 5 0 496707240 307105792 47007 33554432000 4194304 4417332 140736840096032 18446744073709551615 4365035 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=299908 CPUtime=1001.84 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 116317 1260 0 0 100164 20 0 0 25 0 5 0 496707240 307105792 47007 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=299908 CPUtime=1001.54 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 81832 1260 0 0 100142 12 0 0 25 0 5 0 496707240 307105792 47007 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=299908 CPUtime=996.56 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 45653 1260 0 0 99648 8 0 0 25 0 5 0 496707240 307105792 47007 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4002.29
Current children cumulated vsize (KiB) 299908

[startup+1062.3 s]
/proc/loadavg: 8.17 8.13 8.10 9/186 3027
/proc/meminfo: memFree=31557576/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=353360 CPUtime=4241.75 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 376895 1260 0 0 424113 62 0 0 22 0 5 0 496707206 361840640 60370 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 88340 60370 180 55 0 86226 0
[pid=2943/tid=2955] ppid=2941 vsize=353360 CPUtime=1061.97 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 100843 1260 0 0 106181 16 0 0 25 0 5 0 496707240 361840640 60370 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=353360 CPUtime=1061.85 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 120560 1260 0 0 106164 21 0 0 25 0 5 0 496707240 361840640 60370 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=353360 CPUtime=1061.31 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 84507 1260 0 0 106119 12 0 0 25 0 5 0 496707240 361840640 60370 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=353360 CPUtime=1056.24 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 52824 1260 0 0 105614 10 0 0 25 0 5 0 496707240 361840640 60370 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4241.75
Current children cumulated vsize (KiB) 353360

[startup+1122.3 s]
/proc/loadavg: 8.15 8.13 8.10 9/186 3029
/proc/meminfo: memFree=31567772/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=346388 CPUtime=4481.43 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 395000 1260 0 0 448079 64 0 0 22 0 5 0 496707206 354701312 58627 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 86597 58627 180 55 0 84483 0
[pid=2943/tid=2955] ppid=2941 vsize=346388 CPUtime=1121.99 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 109379 1260 0 0 112181 18 0 0 25 0 5 0 496707240 354701312 58627 33554432000 4194304 4417332 140736840096032 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=346388 CPUtime=1121.85 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 123576 1260 0 0 112164 21 0 0 25 0 5 0 496707240 354701312 58627 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=346388 CPUtime=1121.32 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 87224 1260 0 0 112119 13 0 0 25 0 5 0 496707240 354701312 58627 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=346388 CPUtime=1115.91 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 56660 1260 0 0 111581 10 0 0 25 0 5 0 496707240 354701312 58627 33554432000 4194304 4417332 140736840096032 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4481.43
Current children cumulated vsize (KiB) 346388

[startup+1182.3 s]
/proc/loadavg: 8.14 8.13 8.10 9/186 3030
/proc/meminfo: memFree=31574908/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=335216 CPUtime=4721.11 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 414038 1260 0 0 472044 67 0 0 22 0 5 0 496707206 343261184 55834 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 83804 55834 180 55 0 81690 0
[pid=2943/tid=2955] ppid=2941 vsize=335216 CPUtime=1181.98 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 111722 1260 0 0 118180 18 0 0 25 0 5 0 496707240 343261184 55834 33554432000 4194304 4417332 140736840096032 18446744073709551615 4251010 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=335216 CPUtime=1181.85 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 130108 1260 0 0 118163 22 0 0 25 0 5 0 496707240 343261184 55834 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=335216 CPUtime=1181.32 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 94508 1260 0 0 118119 13 0 0 25 0 5 0 496707240 343261184 55834 33554432000 4194304 4417332 140736840096032 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=335216 CPUtime=1175.59 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 59539 1260 0 0 117548 11 0 0 25 0 5 0 496707240 343261184 55834 33554432000 4194304 4417332 140736840096032 18446744073709551615 4353511 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4721.11
Current children cumulated vsize (KiB) 335216



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.1 s]
/proc/loadavg: 8.25 8.16 8.10 9/186 3031
/proc/meminfo: memFree=31611568/32951124 swapFree=67111356/67111528
[pid=2943] ppid=2941 vsize=297916 CPUtime=4800.22 cores=0,2,4,6
/proc/2943/stat : 2943 (plingeling) S 2941 2943 1629 0 -1 4202496 419133 1260 0 0 479954 68 0 0 22 0 5 0 496707206 305065984 46509 33554432000 4194304 4417332 140736840096032 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/2943/statm: 74479 46509 180 55 0 72365 0
[pid=2943/tid=2955] ppid=2941 vsize=297916 CPUtime=1201.79 cores=0,2,4,6
/proc/2943/task/2955/stat : 2955 (plingeling) R 2941 2943 1629 0 -1 4202560 112280 1260 0 0 120161 18 0 0 25 0 5 0 496707240 305065984 46509 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250961 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=2943/tid=2956] ppid=2941 vsize=297916 CPUtime=1201.66 cores=0,2,4,6
/proc/2943/task/2956/stat : 2956 (plingeling) R 2941 2943 1629 0 -1 4202560 133237 1260 0 0 120143 23 0 0 25 0 5 0 496707240 305065984 46509 33554432000 4194304 4417332 140736840096032 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=2943/tid=2958] ppid=2941 vsize=297916 CPUtime=1201.12 cores=0,2,4,6
/proc/2943/task/2958/stat : 2958 (plingeling) R 2941 2943 1629 0 -1 4202560 95916 1260 0 0 120099 13 0 0 25 0 5 0 496707240 305065984 46509 33554432000 4194304 4417332 140736840096032 18446744073709551615 4355764 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=2943/tid=2959] ppid=2941 vsize=297916 CPUtime=1195.29 cores=0,2,4,6
/proc/2943/task/2959/stat : 2959 (plingeling) R 2941 2943 1629 0 -1 4202560 59539 1260 0 0 119518 11 0 0 25 0 5 0 496707240 305065984 46509 33554432000 4194304 4417332 140736840096032 18446744073709551615 4365875 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 297916

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.12
CPU time (s): 4800.27
CPU user time (s): 4799.57
CPU system time (s): 0.697893
CPU usage (%): 399.316
Max. virtual memory (cumulated for all children) (KiB): 365912

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.57
system time used= 0.697893
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 420393
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 72
involuntary context switches= 23299

runsolver used 1.9717 second user time and 4.55331 second system time

The end

Launcher Data 

Begin job on node143 at 2011-04-29 04:08:36
IDJOB=3282629
IDBENCH=70830
IDSOLVER=1605
FILE ID=node143/3282629-1304042916
RUNJOBID= node143-1304042916-2915
PBS_JOBID= 13170363
Free space on /tmp= 72920 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT09/APPLICATIONS/crypto/md5gen/gus-md5-11.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3282629-1304042916/watcher-3282629-1304042916 -o /tmp/evaluation-result-3282629-1304042916/solver-3282629-1304042916 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3282629-1304042916.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 49867705b76fc90285dd6fc48de2613c
RANDOM SEED=162539580

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31848152 kB
Buffers:        166964 kB
Cached:         581000 kB
SwapCached:        168 kB
Active:         248740 kB
Inactive:       581924 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31848152 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            7416 kB
Writeback:           0 kB
AnonPages:       82180 kB
Mapped:          15084 kB
Slab:           207716 kB
PageTables:       5152 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   225504 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72920 MiB
End job on node143 at 2011-04-29 04:28:40