Trace number 3266434

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.38 1201.93

General information on the benchmark

NameSAT02/handmade/simon/
satex-challenges/c6288.shuffled.cnf
MD5SUM75bd8319132b47c2d2e6f0889735a248
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.865867
Satisfiable
(Un)Satisfiability was proved
Number of variables5008
Number of clauses14813
Sum of the clauses size34485
Maximum clause length33
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 29982
Number of clauses of size 34829
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3266434-1304108798/watcher-3266434-1304108798 -o /tmp/evaluation-result-3266434-1304108798/solver-3266434-1304108798 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3266434-1304108798.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.17 8.10 8.01 5/181 6434
/proc/meminfo: memFree=30192736/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=10420 CPUtime=0 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) R 6432 6434 5950 0 -1 4202496 699 1250 0 0 0 0 0 0 25 0 1 0 601110516 10670080 628 33554432000 4194304 4417332 140736668900656 18446744073709551615 260905393920 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/6434/statm: 2605 628 147 55 0 491 0

[startup+0.0172709 s]
/proc/loadavg: 8.17 8.10 8.01 5/181 6434
/proc/meminfo: memFree=30192736/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=12488 CPUtime=0.01 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) R 6432 6434 5950 0 -1 4202496 1235 1250 0 0 1 0 0 0 25 0 1 0 601110516 12787712 1146 33554432000 4194304 4417332 140736668900656 18446744073709551615 4262069 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/6434/statm: 3122 1146 148 55 0 1008 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 12488

[startup+0.131251 s]
/proc/loadavg: 8.17 8.10 8.01 5/181 6434
/proc/meminfo: memFree=30192736/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=120000 CPUtime=0.44 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 2129 1250 0 0 44 0 0 0 25 0 5 0 601110516 122880000 1432 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 30000 1432 175 55 0 27886 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 120000

[startup+0.30022 s]
/proc/loadavg: 8.17 8.10 8.01 5/181 6434
/proc/meminfo: memFree=30192736/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=120552 CPUtime=1.12 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 2299 1250 0 0 112 0 0 0 25 0 5 0 601110516 123445248 1602 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 30138 1602 181 55 0 28024 0
Current children cumulated CPU time (s) 1.12
Current children cumulated vsize (KiB) 120552

[startup+0.700147 s]
/proc/loadavg: 8.17 8.10 8.01 5/181 6434
/proc/meminfo: memFree=30192736/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=124192 CPUtime=2.72 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 3408 1250 0 0 272 0 0 0 25 0 5 0 601110516 127172608 2498 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 31048 2498 181 55 0 28934 0
Current children cumulated CPU time (s) 2.72
Current children cumulated vsize (KiB) 124192

[startup+1.501 s]
/proc/loadavg: 8.24 8.12 8.02 9/186 6444
/proc/meminfo: memFree=30180800/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=124696 CPUtime=5.91 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 5027 1250 0 0 591 0 0 0 25 0 5 0 601110516 127688704 2624 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 31174 2624 181 55 0 29060 0
[pid=6434/tid=6441] ppid=6432 vsize=124696 CPUtime=1.47 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 930 1250 0 0 147 0 0 0 25 0 5 0 601110518 127688704 2624 33554432000 4194304 4417332 140736668900656 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=124696 CPUtime=1.47 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 611 1250 0 0 147 0 0 0 25 0 5 0 601110518 127688704 2624 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=124696 CPUtime=1.47 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 1072 1250 0 0 147 0 0 0 25 0 5 0 601110518 127688704 2624 33554432000 4194304 4417332 140736668900656 18446744073709551615 4309920 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=124696 CPUtime=1.47 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 911 1250 0 0 147 0 0 0 25 0 5 0 601110518 127688704 2624 33554432000 4194304 4417332 140736668900656 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.91
Current children cumulated vsize (KiB) 124696

[startup+3.10071 s]
/proc/loadavg: 8.24 8.12 8.02 9/186 6444
/proc/meminfo: memFree=30171692/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=129336 CPUtime=12.31 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 9863 1250 0 0 1230 1 0 0 25 0 5 0 601110516 132440064 3784 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 32334 3784 181 55 0 30220 0
[pid=6434/tid=6441] ppid=6432 vsize=129336 CPUtime=3.07 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 2288 1250 0 0 307 0 0 0 25 0 5 0 601110518 132440064 3784 33554432000 4194304 4417332 140736668900656 18446744073709551615 4356888 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=129336 CPUtime=3.07 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 1570 1250 0 0 307 0 0 0 25 0 5 0 601110518 132440064 3784 33554432000 4194304 4417332 140736668900656 18446744073709551615 4354600 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=129336 CPUtime=3.06 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 2088 1250 0 0 306 0 0 0 25 0 5 0 601110518 132440064 3784 33554432000 4194304 4417332 140736668900656 18446744073709551615 4358027 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=129336 CPUtime=3.07 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 2414 1250 0 0 307 0 0 0 25 0 5 0 601110518 132440064 3784 33554432000 4194304 4417332 140736668900656 18446744073709551615 4353502 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.31
Current children cumulated vsize (KiB) 129336

[startup+6.30112 s]
/proc/loadavg: 8.30 8.13 8.02 9/186 6444
/proc/meminfo: memFree=30161524/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=136664 CPUtime=25.09 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 17502 1250 0 0 2507 2 0 0 25 0 5 0 601110516 139943936 5616 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 34166 5616 181 55 0 32052 0
[pid=6434/tid=6441] ppid=6432 vsize=136664 CPUtime=6.27 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 3729 1250 0 0 627 0 0 0 25 0 5 0 601110518 139943936 5616 33554432000 4194304 4417332 140736668900656 18446744073709551615 4232695 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=136664 CPUtime=6.26 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 3973 1250 0 0 626 0 0 0 25 0 5 0 601110518 139943936 5616 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=136664 CPUtime=6.24 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 3734 1250 0 0 624 0 0 0 25 0 5 0 601110518 139943936 5616 33554432000 4194304 4417332 140736668900656 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=136664 CPUtime=6.27 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 4563 1250 0 0 627 0 0 0 25 0 5 0 601110518 139943936 5616 33554432000 4194304 4417332 140736668900656 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 25.09
Current children cumulated vsize (KiB) 136664

[startup+12.7009 s]
/proc/loadavg: 8.27 8.13 8.02 9/186 6445
/proc/meminfo: memFree=30156684/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=137204 CPUtime=50.65 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 31855 1250 0 0 5060 5 0 0 25 0 5 0 601110516 140496896 5779 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 34301 5779 181 55 0 32187 0
[pid=6434/tid=6441] ppid=6432 vsize=137204 CPUtime=12.67 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 7402 1250 0 0 1266 1 0 0 25 0 5 0 601110518 140496896 5779 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=137204 CPUtime=12.67 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 8661 1250 0 0 1266 1 0 0 25 0 5 0 601110518 140496896 5779 33554432000 4194304 4417332 140736668900656 18446744073709551615 4291616 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=137204 CPUtime=12.6 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 7377 1250 0 0 1259 1 0 0 25 0 5 0 601110518 140496896 5779 33554432000 4194304 4417332 140736668900656 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=137204 CPUtime=12.66 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 6912 1250 0 0 1266 0 0 0 25 0 5 0 601110518 140496896 5779 33554432000 4194304 4417332 140736668900656 18446744073709551615 4357440 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 50.65
Current children cumulated vsize (KiB) 137204

[startup+25.5006 s]
/proc/loadavg: 8.21 8.12 8.02 9/186 6445
/proc/meminfo: memFree=30150480/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=145196 CPUtime=101.78 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 54161 1250 0 0 10169 9 0 0 25 0 5 0 601110516 148680704 7781 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 36299 7781 181 55 0 34185 0
[pid=6434/tid=6441] ppid=6432 vsize=145196 CPUtime=25.47 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 14766 1250 0 0 2544 3 0 0 25 0 5 0 601110518 148680704 7781 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=145196 CPUtime=25.47 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 13705 1250 0 0 2545 2 0 0 25 0 5 0 601110518 148680704 7781 33554432000 4194304 4417332 140736668900656 18446744073709551615 4366324 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=145196 CPUtime=25.33 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 13017 1250 0 0 2531 2 0 0 25 0 5 0 601110518 148680704 7781 33554432000 4194304 4417332 140736668900656 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=145196 CPUtime=25.47 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 11170 1250 0 0 2546 1 0 0 25 0 5 0 601110518 148680704 7781 33554432000 4194304 4417332 140736668900656 18446744073709551615 4365849 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.78
Current children cumulated vsize (KiB) 145196

[startup+51.1009 s]
/proc/loadavg: 8.14 8.11 8.02 9/186 6446
/proc/meminfo: memFree=30150328/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=157928 CPUtime=204.05 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 93408 1250 0 0 20388 17 0 0 25 0 5 0 601110516 161718272 10932 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 39482 10932 181 55 0 37368 0
[pid=6434/tid=6441] ppid=6432 vsize=157928 CPUtime=51.07 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 25039 1250 0 0 5102 5 0 0 25 0 5 0 601110518 161718272 10932 33554432000 4194304 4417332 140736668900656 18446744073709551615 4353565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=157928 CPUtime=51.06 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 22708 1250 0 0 5103 3 0 0 25 0 5 0 601110518 161718272 10932 33554432000 4194304 4417332 140736668900656 18446744073709551615 4353598 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=157928 CPUtime=50.79 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 25745 1250 0 0 5075 4 0 0 25 0 5 0 601110518 161718272 10932 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=157928 CPUtime=51.07 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 18413 1250 0 0 5105 2 0 0 25 0 5 0 601110518 161718272 10932 33554432000 4194304 4417332 140736668900656 18446744073709551615 4365037 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 204.05
Current children cumulated vsize (KiB) 157928

[startup+102.307 s]
/proc/loadavg: 8.13 8.11 8.02 9/186 6447
/proc/meminfo: memFree=30129192/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=167704 CPUtime=408.59 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 161049 1250 0 0 40831 28 0 0 25 0 5 0 601110516 171728896 13389 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 41926 13389 181 55 0 39812 0
[pid=6434/tid=6441] ppid=6432 vsize=167704 CPUtime=102.27 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 41027 1250 0 0 10220 7 0 0 25 0 5 0 601110518 171728896 13389 33554432000 4194304 4417332 140736668900656 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=167704 CPUtime=102.27 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 37659 1250 0 0 10222 5 0 0 25 0 5 0 601110518 171728896 13389 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=167704 CPUtime=101.72 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 48418 1250 0 0 10163 9 0 0 25 0 5 0 601110518 171728896 13389 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=167704 CPUtime=102.28 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 32442 1250 0 0 10223 5 0 0 25 0 5 0 601110518 171728896 13389 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 408.59
Current children cumulated vsize (KiB) 167704

[startup+162.301 s]
/proc/loadavg: 8.11 8.10 8.02 9/186 6449
/proc/meminfo: memFree=30115264/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=178996 CPUtime=648.25 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 228221 1250 0 0 64786 39 0 0 25 0 5 0 601110516 183291904 16226 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 44749 16226 181 55 0 42635 0
[pid=6434/tid=6441] ppid=6432 vsize=178996 CPUtime=162.27 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 58954 1250 0 0 16217 10 0 0 25 0 5 0 601110518 183291904 16226 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=178996 CPUtime=162.27 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 52536 1250 0 0 16219 8 0 0 25 0 5 0 601110518 183291904 16226 33554432000 4194304 4417332 140736668900656 18446744073709551615 4354428 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=178996 CPUtime=161.39 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 68782 1250 0 0 16126 13 0 0 25 0 5 0 601110518 183291904 16226 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=178996 CPUtime=162.28 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 46446 1250 0 0 16221 7 0 0 25 0 5 0 601110518 183291904 16226 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 648.25
Current children cumulated vsize (KiB) 178996


################
# More data... #
################

[pid=6434/tid=6442] ppid=6432 vsize=288628 CPUtime=762.29 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 167630 1250 0 0 76202 27 0 0 25 0 5 0 601110518 295555072 43613 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=288628 CPUtime=758.14 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 201314 1250 0 0 75782 32 0 0 25 0 5 0 601110518 295555072 43613 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=288628 CPUtime=762.29 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 166264 1250 0 0 76205 24 0 0 25 0 5 0 601110518 295555072 43613 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364504 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3044.25
Current children cumulated vsize (KiB) 288628

[startup+822.3 s]
/proc/loadavg: 8.16 8.15 8.04 9/186 6469
/proc/meminfo: memFree=29979508/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=286632 CPUtime=3283.9 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 757178 1250 0 0 328269 121 0 0 25 0 5 0 601110516 293511168 43114 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 71658 43114 181 55 0 69544 0
[pid=6434/tid=6441] ppid=6432 vsize=286632 CPUtime=821.46 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 191018 1250 0 0 82115 31 0 0 25 0 5 0 601110518 293511168 43114 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=286632 CPUtime=822.29 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 173800 1250 0 0 82200 29 0 0 25 0 5 0 601110518 293511168 43114 33554432000 4194304 4417332 140736668900656 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=286632 CPUtime=817.81 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 213118 1250 0 0 81748 33 0 0 25 0 5 0 601110518 293511168 43114 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=286632 CPUtime=822.29 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 177739 1250 0 0 82203 26 0 0 25 0 5 0 601110518 293511168 43114 33554432000 4194304 4417332 140736668900656 18446744073709551615 4357444 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3283.9
Current children cumulated vsize (KiB) 286632

[startup+882.3 s]
/proc/loadavg: 8.17 8.15 8.05 9/186 6471
/proc/meminfo: memFree=29956276/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=310152 CPUtime=3523.58 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 788089 1250 0 0 352231 127 0 0 25 0 5 0 601110516 317595648 48994 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 77538 48994 181 55 0 75424 0
[pid=6434/tid=6441] ppid=6432 vsize=310152 CPUtime=881.46 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 196487 1250 0 0 88114 32 0 0 25 0 5 0 601110518 317595648 48994 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=310152 CPUtime=882.29 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 180047 1250 0 0 88199 30 0 0 25 0 5 0 601110518 317595648 48994 33554432000 4194304 4417332 140736668900656 18446744073709551615 4366324 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=310152 CPUtime=877.48 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 224424 1250 0 0 87713 35 0 0 25 0 5 0 601110518 317595648 48994 33554432000 4194304 4417332 140736668900656 18446744073709551615 4354509 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=310152 CPUtime=882.3 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 185628 1250 0 0 88202 28 0 0 25 0 5 0 601110518 317595648 48994 33554432000 4194304 4417332 140736668900656 18446744073709551615 4354248 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3523.58
Current children cumulated vsize (KiB) 310152

[startup+942.3 s]
/proc/loadavg: 6.48 7.74 7.92 5/176 6490
/proc/meminfo: memFree=30033356/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=337228 CPUtime=3763.27 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 815348 1250 0 0 376195 132 0 0 25 0 5 0 601110516 345321472 55763 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 84307 55763 181 55 0 82193 0
[pid=6434/tid=6441] ppid=6432 vsize=337228 CPUtime=941.46 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 201701 1250 0 0 94113 33 0 0 25 0 5 0 601110518 345321472 55763 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=337228 CPUtime=942.3 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 183486 1250 0 0 94199 31 0 0 25 0 5 0 601110518 345321472 55763 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=337228 CPUtime=937.17 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 235808 1250 0 0 93680 37 0 0 25 0 5 0 601110518 345321472 55763 33554432000 4194304 4417332 140736668900656 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=337228 CPUtime=942.3 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 192850 1250 0 0 94201 29 0 0 25 0 5 0 601110518 345321472 55763 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3763.27
Current children cumulated vsize (KiB) 337228

[startup+1002.3 s]
/proc/loadavg: 4.91 7.05 7.67 5/176 6492
/proc/meminfo: memFree=29997188/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=373260 CPUtime=4002.96 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 858795 1250 0 0 400156 140 0 0 25 0 5 0 601110516 382218240 64771 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 93315 64771 181 55 0 91201 0
[pid=6434/tid=6441] ppid=6432 vsize=373260 CPUtime=1001.47 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 213186 1250 0 0 100111 36 0 0 25 0 5 0 601110518 382218240 64771 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=373260 CPUtime=1002.3 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 193696 1250 0 0 100197 33 0 0 25 0 5 0 601110518 382218240 64771 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=373260 CPUtime=996.85 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 249541 1250 0 0 99646 39 0 0 25 0 5 0 601110518 382218240 64771 33554432000 4194304 4417332 140736668900656 18446744073709551615 4357187 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=373260 CPUtime=1002.3 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 200869 1250 0 0 100199 31 0 0 25 0 5 0 601110518 382218240 64771 33554432000 4194304 4417332 140736668900656 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4002.96
Current children cumulated vsize (KiB) 373260

[startup+1062.3 s]
/proc/loadavg: 4.33 6.50 7.44 5/176 6494
/proc/meminfo: memFree=30014636/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=355892 CPUtime=4242.65 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 892787 1250 0 0 424118 147 0 0 25 0 5 0 601110516 364433408 60429 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 88973 60429 181 55 0 86859 0
[pid=6434/tid=6441] ppid=6432 vsize=355892 CPUtime=1061.46 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 221764 1250 0 0 106109 37 0 0 25 0 5 0 601110518 364433408 60429 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=355892 CPUtime=1062.3 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 200730 1250 0 0 106196 34 0 0 25 0 5 0 601110518 364433408 60429 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=355892 CPUtime=1056.53 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 260377 1250 0 0 105612 41 0 0 25 0 5 0 601110518 364433408 60429 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=355892 CPUtime=1062.3 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 208413 1250 0 0 106198 32 0 0 25 0 5 0 601110518 364433408 60429 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4242.65
Current children cumulated vsize (KiB) 355892

[startup+1122.3 s]
/proc/loadavg: 4.18 6.05 7.22 5/176 6495
/proc/meminfo: memFree=29995716/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=374820 CPUtime=4482.33 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 909760 1250 0 0 448084 149 0 0 25 0 5 0 601110516 383815680 65161 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 93705 65161 181 55 0 91591 0
[pid=6434/tid=6441] ppid=6432 vsize=374820 CPUtime=1121.47 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 225371 1250 0 0 112109 38 0 0 25 0 5 0 601110518 383815680 65161 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=374820 CPUtime=1122.3 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 200730 1250 0 0 112196 34 0 0 25 0 5 0 601110518 383815680 65161 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=374820 CPUtime=1116.21 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 267242 1250 0 0 111579 42 0 0 25 0 5 0 601110518 383815680 65161 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=374820 CPUtime=1122.31 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 214914 1250 0 0 112197 34 0 0 25 0 5 0 601110518 383815680 65161 33554432000 4194304 4417332 140736668900656 18446744073709551615 4366445 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4482.33
Current children cumulated vsize (KiB) 374820

[startup+1182.3 s]
/proc/loadavg: 4.12 5.69 7.02 5/176 6497
/proc/meminfo: memFree=29949160/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=426200 CPUtime=4722.02 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 939936 1250 0 0 472048 154 0 0 25 0 5 0 601110516 436428800 78006 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 106550 78006 181 55 0 104436 0
[pid=6434/tid=6441] ppid=6432 vsize=426200 CPUtime=1181.47 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 226967 1250 0 0 118109 38 0 0 25 0 5 0 601110518 436428800 78006 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=426200 CPUtime=1182.3 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 211409 1250 0 0 118195 35 0 0 25 0 5 0 601110518 436428800 78006 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=426200 CPUtime=1175.89 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 276312 1250 0 0 117546 43 0 0 25 0 5 0 601110518 436428800 78006 33554432000 4194304 4417332 140736668900656 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=426200 CPUtime=1182.31 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 223745 1250 0 0 118195 36 0 0 25 0 5 0 601110518 436428800 78006 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4722.02
Current children cumulated vsize (KiB) 426200



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 4.08 5.58 6.96 5/176 6497
/proc/meminfo: memFree=29957324/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=412440 CPUtime=4800.33 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 965035 1250 0 0 479875 158 0 0 25 0 5 0 601110516 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 103110 74566 181 55 0 100996 0
[pid=6434/tid=6441] ppid=6432 vsize=412440 CPUtime=1201.08 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 233800 1250 0 0 120068 40 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=412440 CPUtime=1201.91 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 220166 1250 0 0 120154 37 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=412440 CPUtime=1195.38 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 280724 1250 0 0 119495 43 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=412440 CPUtime=1201.91 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 228842 1250 0 0 120155 36 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 412440

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 4.08 5.58 6.96 5/176 6497
/proc/meminfo: memFree=29957324/32950928 swapFree=67054472/67111528
[pid=6434] ppid=6432 vsize=412440 CPUtime=4800.33 cores=1,3,5,7
/proc/6434/stat : 6434 (plingeling) S 6432 6434 5950 0 -1 4202496 965035 1250 0 0 479875 158 0 0 25 0 5 0 601110516 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/6434/statm: 103110 74566 181 55 0 100996 0
[pid=6434/tid=6441] ppid=6432 vsize=412440 CPUtime=1201.08 cores=1,3,5,7
/proc/6434/task/6441/stat : 6441 (plingeling) R 6432 6434 5950 0 -1 4202560 233800 1250 0 0 120068 40 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=6434/tid=6442] ppid=6432 vsize=412440 CPUtime=1201.91 cores=1,3,5,7
/proc/6434/task/6442/stat : 6442 (plingeling) R 6432 6434 5950 0 -1 4202560 220166 1250 0 0 120154 37 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6434/tid=6443] ppid=6432 vsize=412440 CPUtime=1195.38 cores=1,3,5,7
/proc/6434/task/6443/stat : 6443 (plingeling) R 6432 6434 5950 0 -1 4202560 280724 1250 0 0 119495 43 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=6434/tid=6444] ppid=6432 vsize=412440 CPUtime=1201.91 cores=1,3,5,7
/proc/6434/task/6444/stat : 6444 (plingeling) R 6432 6434 5950 0 -1 4202560 228842 1250 0 0 120155 36 0 0 25 0 5 0 601110518 422338560 74566 33554432000 4194304 4417332 140736668900656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 412440

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.93
CPU time (s): 4800.38
CPU user time (s): 4798.77
CPU system time (s): 1.60576
CPU usage (%): 399.39
Max. virtual memory (cumulated for all children) (KiB): 428640

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.77
system time used= 1.60575
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 966285
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 109
involuntary context switches= 36345

runsolver used 1.89271 second user time and 4.6353 second system time

The end

Launcher Data 

Begin job on node116 at 2011-04-29 22:26:38
IDJOB=3266434
IDBENCH=87734
IDSOLVER=1605
FILE ID=node116/3266434-1304108798
RUNJOBID= node116-1304107594-6265
PBS_JOBID= 13170643
Free space on /tmp= 72516 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT02/handmade/simon/satex-challenges/c6288.shuffled.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3266434-1304108798/watcher-3266434-1304108798 -o /tmp/evaluation-result-3266434-1304108798/solver-3266434-1304108798 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3266434-1304108798.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 75bd8319132b47c2d2e6f0889735a248
RANDOM SEED=1630014554

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30192892 kB
Buffers:        976348 kB
Cached:        1361092 kB
SwapCached:       9656 kB
Active:         116316 kB
Inactive:      2304160 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30192892 kB
SwapTotal:    67111528 kB
SwapFree:     67054472 kB
Dirty:              52 kB
Writeback:           0 kB
AnonPages:       81348 kB
Mapped:          13380 kB
Slab:           274508 kB
PageTables:       4516 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   314240 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72516 MiB
End job on node116 at 2011-04-29 22:46:42