Trace number 3371183

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.36 1203.93

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n220-s24359701-sat.cnf
MD5SUMceff918324677440f95867755c4e7133
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark5.76412
Satisfiable
(Un)Satisfiability was proved
Number of variables220
Number of clauses528
Sum of the clauses size1320
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2440
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 588
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371183-1305424081/watcher-3371183-1305424081 -o /tmp/evaluation-result-3371183-1305424081/solver-3371183-1305424081 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371183-1305424081.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.43 4.59 6.23 6/370 7620
/proc/meminfo: memFree=18208880/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=49836 CPUtime=0 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 338 1252 0 0 0 0 0 0 22 0 5 0 732638776 51032064 265 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 12459 265 172 55 0 10345 0
[pid=7620/tid=7626] ppid=7618 vsize=115708 CPUtime=0 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 38 1252 0 0 0 0 0 0 24 0 5 0 732638776 118484992 343 33554432000 4194304 4417452 140736526938096 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=115708 CPUtime=0 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 6 1252 0 0 0 0 0 0 24 0 5 0 732638776 118484992 343 33554432000 4194304 4417452 140736526938096 18446744073709551615 224285030894 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=115708 CPUtime=0 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 18 1252 0 0 0 0 0 0 24 0 5 0 732638776 118484992 345 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365150 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=115708 CPUtime=0 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 73 1252 0 0 0 0 0 0 24 0 5 0 732638776 118484992 359 33554432000 4194304 4417452 140736526938096 18446744073709551615 224284619723 0 0 4096 17442 0 0 0 -1 5 0 0 0

[startup+0.054041 s]
/proc/loadavg: 3.43 4.59 6.23 6/370 7620
/proc/meminfo: memFree=18208880/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=117264 CPUtime=0.18 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 860 1252 0 0 18 0 0 0 22 0 5 0 732638776 120078336 756 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 29316 756 180 55 0 27202 0
[pid=7620/tid=7626] ppid=7618 vsize=117264 CPUtime=0.04 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 102 1252 0 0 4 0 0 0 24 0 5 0 732638776 120078336 756 33554432000 4194304 4417452 140736526938096 18446744073709551615 4366102 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=117264 CPUtime=0.03 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 52 1252 0 0 3 0 0 0 24 0 5 0 732638776 120078336 756 33554432000 4194304 4417452 140736526938096 18446744073709551615 4310328 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=117264 CPUtime=0.04 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 108 1252 0 0 4 0 0 0 24 0 5 0 732638776 120078336 756 33554432000 4194304 4417452 140736526938096 18446744073709551615 4358106 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=117264 CPUtime=0.04 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 301 1252 0 0 4 0 0 0 25 0 5 0 732638776 120078336 756 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 117264

[startup+0.101033 s]
/proc/loadavg: 3.43 4.59 6.23 6/370 7620
/proc/meminfo: memFree=18208880/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=118140 CPUtime=0.37 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1306 1252 0 0 37 0 0 0 22 0 5 0 732638776 120975360 976 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 29535 976 180 55 0 27421 0
[pid=7620/tid=7626] ppid=7618 vsize=118140 CPUtime=0.09 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 175 1252 0 0 9 0 0 0 24 0 5 0 732638776 120975360 976 33554432000 4194304 4417452 140736526938096 18446744073709551615 4310251 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=118140 CPUtime=0.08 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 162 1252 0 0 8 0 0 0 24 0 5 0 732638776 120975360 976 33554432000 4194304 4417452 140736526938096 18446744073709551615 4250755 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=118140 CPUtime=0.09 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 173 1252 0 0 9 0 0 0 25 0 5 0 732638776 120975360 976 33554432000 4194304 4417452 140736526938096 18446744073709551615 4311296 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=118140 CPUtime=0.09 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 499 1252 0 0 9 0 0 0 25 0 5 0 732638776 120975360 976 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 118140

[startup+0.301001 s]
/proc/loadavg: 3.43 4.59 6.23 6/370 7620
/proc/meminfo: memFree=18208880/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=120632 CPUtime=1.17 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 3196 1252 0 0 117 0 0 0 22 0 5 0 732638776 123527168 1610 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 30158 1610 180 55 0 28044 0
[pid=7620/tid=7626] ppid=7618 vsize=120632 CPUtime=0.29 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 468 1252 0 0 29 0 0 0 24 0 5 0 732638776 123527168 1610 33554432000 4194304 4417452 140736526938096 18446744073709551615 4306366 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=120632 CPUtime=0.28 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 525 1252 0 0 28 0 0 0 25 0 5 0 732638776 123527168 1610 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=120632 CPUtime=0.29 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 680 1252 0 0 29 0 0 0 25 0 5 0 732638776 123527168 1610 33554432000 4194304 4417452 140736526938096 18446744073709551615 4250795 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=120632 CPUtime=0.29 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 1226 1252 0 0 29 0 0 0 25 0 5 0 732638776 123527168 1610 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 120632

[startup+0.700935 s]
/proc/loadavg: 3.43 4.59 6.23 6/370 7620
/proc/meminfo: memFree=18208880/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=122824 CPUtime=2.77 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 6773 1252 0 0 276 1 0 0 22 0 5 0 732638776 125771776 2161 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 30706 2161 180 55 0 28592 0
[pid=7620/tid=7626] ppid=7618 vsize=122824 CPUtime=0.69 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 1551 1252 0 0 69 0 0 0 25 0 5 0 732638776 125771776 2161 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364500 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=122824 CPUtime=0.68 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 1405 1252 0 0 68 0 0 0 25 0 5 0 732638776 125771776 2161 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365103 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=122824 CPUtime=0.69 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 1491 1252 0 0 69 0 0 0 25 0 5 0 732638776 125771776 2161 33554432000 4194304 4417452 140736526938096 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=122824 CPUtime=0.69 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 2029 1252 0 0 69 0 0 0 25 0 5 0 732638776 125771776 2161 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 122824

[startup+1.5008 s]
/proc/loadavg: 3.43 4.59 6.23 10/375 7630
/proc/meminfo: memFree=18188156/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=126072 CPUtime=5.96 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 12273 1252 0 0 594 2 0 0 22 0 5 0 732638776 129097728 2940 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 31518 2940 180 55 0 29404 0
[pid=7620/tid=7626] ppid=7618 vsize=126072 CPUtime=1.49 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 3058 1252 0 0 149 0 0 0 25 0 5 0 732638776 129097728 2940 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=126072 CPUtime=1.46 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 2595 1252 0 0 146 0 0 0 25 0 5 0 732638776 129097728 2940 33554432000 4194304 4417452 140736526938096 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=126072 CPUtime=1.49 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 2694 1252 0 0 149 0 0 0 25 0 5 0 732638776 129097728 2940 33554432000 4194304 4417452 140736526938096 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=126072 CPUtime=1.48 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 3629 1252 0 0 148 0 0 0 25 0 5 0 732638776 129097728 2940 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.96
Current children cumulated vsize (KiB) 126072

[startup+3.10054 s]
/proc/loadavg: 3.80 4.65 6.24 10/375 7630
/proc/meminfo: memFree=18180344/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=130996 CPUtime=12.34 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 22583 1252 0 0 1231 3 0 0 22 0 5 0 732638776 134139904 4183 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 32749 4183 180 55 0 30635 0
[pid=7620/tid=7626] ppid=7618 vsize=130996 CPUtime=3.09 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 6030 1252 0 0 308 1 0 0 25 0 5 0 732638776 134139904 4183 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=130996 CPUtime=3.05 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 4917 1252 0 0 305 0 0 0 25 0 5 0 732638776 134139904 4183 33554432000 4194304 4417452 140736526938096 18446744073709551615 4366412 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=130996 CPUtime=3.08 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 5012 1252 0 0 308 0 0 0 25 0 5 0 732638776 134139904 4183 33554432000 4194304 4417452 140736526938096 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=130996 CPUtime=3.08 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 6327 1252 0 0 308 0 0 0 25 0 5 0 732638776 134139904 4183 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.34
Current children cumulated vsize (KiB) 130996

[startup+6.301 s]
/proc/loadavg: 3.80 4.65 6.24 10/375 7630
/proc/meminfo: memFree=18166584/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=137368 CPUtime=25.1 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 35458 1252 0 0 2505 5 0 0 22 0 5 0 732638776 140664832 5778 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 34342 5778 180 55 0 32228 0
[pid=7620/tid=7626] ppid=7618 vsize=137368 CPUtime=6.29 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 9321 1252 0 0 628 1 0 0 25 0 5 0 732638776 140664832 5778 33554432000 4194304 4417452 140736526938096 18446744073709551615 4310613 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=137368 CPUtime=6.22 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 7532 1252 0 0 621 1 0 0 25 0 5 0 732638776 140664832 5778 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=137368 CPUtime=6.29 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 8371 1252 0 0 628 1 0 0 25 0 5 0 732638776 140664832 5778 33554432000 4194304 4417452 140736526938096 18446744073709551615 4259910 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=137368 CPUtime=6.29 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 9937 1252 0 0 628 1 0 0 25 0 5 0 732638776 140664832 5778 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 137368

[startup+12.7009 s]
/proc/loadavg: 4.44 4.76 6.26 10/375 7632
/proc/meminfo: memFree=18148992/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=146736 CPUtime=50.63 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 61446 1252 0 0 5053 10 0 0 22 0 5 0 732638776 150257664 8136 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 36684 8136 180 55 0 34570 0
[pid=7620/tid=7626] ppid=7618 vsize=146736 CPUtime=12.69 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 15560 1252 0 0 1267 2 0 0 25 0 5 0 732638776 150257664 8136 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=146736 CPUtime=12.54 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 13852 1252 0 0 1252 2 0 0 25 0 5 0 732638776 150257664 8136 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=146736 CPUtime=12.69 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 15550 1252 0 0 1267 2 0 0 25 0 5 0 732638776 150257664 8136 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=146736 CPUtime=12.69 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 16187 1252 0 0 1267 2 0 0 25 0 5 0 732638776 150257664 8136 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.63
Current children cumulated vsize (KiB) 146736

[startup+25.5008 s]
/proc/loadavg: 5.15 4.90 6.29 10/375 7632
/proc/meminfo: memFree=18130652/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=145360 CPUtime=101.67 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 100744 1252 0 0 10150 17 0 0 22 0 5 0 732638776 148848640 7824 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 36340 7824 180 55 0 34226 0
[pid=7620/tid=7626] ppid=7618 vsize=145360 CPUtime=25.49 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 28819 1252 0 0 2544 5 0 0 25 0 5 0 732638776 148848640 7824 33554432000 4194304 4417452 140736526938096 18446744073709551615 4236761 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=145360 CPUtime=25.18 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 22202 1252 0 0 2515 3 0 0 25 0 5 0 732638776 148848640 7824 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=145360 CPUtime=25.49 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 24323 1252 0 0 2545 4 0 0 25 0 5 0 732638776 148848640 7824 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=145360 CPUtime=25.49 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 25103 1252 0 0 2545 4 0 0 25 0 5 0 732638776 148848640 7824 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.67
Current children cumulated vsize (KiB) 145360

[startup+51.1005 s]
/proc/loadavg: 6.12 5.15 6.33 10/375 7632
/proc/meminfo: memFree=18111424/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=157748 CPUtime=203.77 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 166918 1252 0 0 20350 27 0 0 22 0 5 0 732638776 161533952 10921 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 39437 10921 180 55 0 37323 0
[pid=7620/tid=7626] ppid=7618 vsize=157748 CPUtime=51.08 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 44824 1252 0 0 5101 7 0 0 25 0 5 0 732638776 161533952 10921 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=157748 CPUtime=50.48 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 39925 1252 0 0 5042 6 0 0 25 0 5 0 732638776 161533952 10921 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=157748 CPUtime=51.09 cores=1,3,5,7

################
# More data... #
################

[pid=7620/tid=7627] ppid=7618 vsize=235236 CPUtime=753.29 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 221595 1252 0 0 75292 37 0 0 25 0 5 0 732638776 240881664 30261 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=235236 CPUtime=762.31 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 222927 1252 0 0 76199 32 0 0 25 0 5 0 732638776 240881664 30261 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=235236 CPUtime=762.31 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 225180 1252 0 0 76196 35 0 0 25 0 5 0 732638776 240881664 30261 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3040.25
Current children cumulated vsize (KiB) 235236

[startup+822.301 s]
/proc/loadavg: 8.10 7.86 7.31 10/375 7661
/proc/meminfo: memFree=17896444/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=222200 CPUtime=3279.48 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 929613 1252 0 0 327801 147 0 0 22 0 5 0 732638776 227532800 27034 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 55550 27034 180 55 0 53436 0
[pid=7620/tid=7626] ppid=7618 vsize=222200 CPUtime=822.28 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 228998 1252 0 0 82192 36 0 0 25 0 5 0 732638776 227532800 27034 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=222200 CPUtime=812.58 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 233082 1252 0 0 81219 39 0 0 25 0 5 0 732638776 227532800 27034 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364607 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=222200 CPUtime=822.29 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 235293 1252 0 0 82195 34 0 0 25 0 5 0 732638776 227532800 27034 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=222200 CPUtime=822.3 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 231943 1252 0 0 82194 36 0 0 25 0 5 0 732638776 227532800 27034 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3279.48
Current children cumulated vsize (KiB) 222200

[startup+882.301 s]
/proc/loadavg: 8.56 8.02 7.40 11/380 8222
/proc/meminfo: memFree=17940924/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=235588 CPUtime=3517.77 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 970748 1252 0 0 351623 154 0 0 22 0 5 0 732638776 241242112 30349 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 58897 30349 180 55 0 56783 0
[pid=7620/tid=7626] ppid=7618 vsize=235588 CPUtime=882.12 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 237045 1252 0 0 88174 38 0 0 25 0 5 0 732638776 241242112 30349 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=235588 CPUtime=871.29 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 242488 1252 0 0 87088 41 0 0 25 0 5 0 732638776 241242112 30349 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=235588 CPUtime=882.24 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 244906 1252 0 0 88188 36 0 0 25 0 5 0 732638776 241242112 30349 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=235588 CPUtime=882.09 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 246012 1252 0 0 88171 38 0 0 25 0 5 0 732638776 241242112 30349 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3517.77
Current children cumulated vsize (KiB) 235588

[startup+942.301 s]
/proc/loadavg: 8.93 8.21 7.50 11/380 8427
/proc/meminfo: memFree=17874780/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=242284 CPUtime=3757.03 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1011987 1252 0 0 375542 161 0 0 22 0 5 0 732638776 248098816 32055 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 60571 32055 180 55 0 58457 0
[pid=7620/tid=7626] ppid=7618 vsize=242284 CPUtime=942.12 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 249085 1252 0 0 94173 39 0 0 25 0 5 0 732638776 248098816 32055 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=242284 CPUtime=930.59 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 250024 1252 0 0 93017 42 0 0 25 0 5 0 732638776 248098816 32055 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=242284 CPUtime=942.24 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 253645 1252 0 0 94187 37 0 0 25 0 5 0 732638776 248098816 32055 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=242284 CPUtime=942.06 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 258936 1252 0 0 94165 41 0 0 25 0 5 0 732638776 248098816 32055 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3757.03
Current children cumulated vsize (KiB) 242284

[startup+1002.3 s]
/proc/loadavg: 9.14 8.42 7.62 11/380 8624
/proc/meminfo: memFree=17859608/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=253692 CPUtime=3996.29 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1054898 1252 0 0 399462 167 0 0 22 0 5 0 732638776 259780608 34907 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 63423 34907 180 55 0 61309 0
[pid=7620/tid=7626] ppid=7618 vsize=253692 CPUtime=1002.13 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 260826 1252 0 0 100171 42 0 0 25 0 5 0 732638776 259780608 34907 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=253692 CPUtime=989.87 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 259223 1252 0 0 98943 44 0 0 25 0 5 0 732638776 259780608 34907 33554432000 4194304 4417452 140736526938096 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=253692 CPUtime=1002.24 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 263742 1252 0 0 100185 39 0 0 25 0 5 0 732638776 259780608 34907 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=253692 CPUtime=1002.04 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 270810 1252 0 0 100162 42 0 0 25 0 5 0 732638776 259780608 34907 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3996.29
Current children cumulated vsize (KiB) 253692

[startup+1062.3 s]
/proc/loadavg: 9.10 8.53 7.70 11/380 8858
/proc/meminfo: memFree=17847600/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=237264 CPUtime=4235.56 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1093794 1252 0 0 423383 173 0 0 22 0 5 0 732638776 242958336 30800 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 59316 30800 180 55 0 57202 0
[pid=7620/tid=7626] ppid=7618 vsize=237264 CPUtime=1062.12 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 269498 1252 0 0 106169 43 0 0 25 0 5 0 732638776 242958336 30800 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=237264 CPUtime=1049.16 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 264345 1252 0 0 104872 44 0 0 25 0 5 0 732638776 242958336 30800 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=237264 CPUtime=1062.24 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 273815 1252 0 0 106184 40 0 0 25 0 5 0 732638776 242958336 30800 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=237264 CPUtime=1062.01 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 285839 1252 0 0 106157 44 0 0 25 0 5 0 732638776 242958336 30800 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4235.56
Current children cumulated vsize (KiB) 237264

[startup+1122.3 s]
/proc/loadavg: 9.08 8.64 7.79 11/380 9061
/proc/meminfo: memFree=17842436/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=237576 CPUtime=4474.84 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1138411 1252 0 0 447304 180 0 0 22 0 5 0 732638776 243277824 30878 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 59394 30878 180 55 0 57280 0
[pid=7620/tid=7626] ppid=7618 vsize=237576 CPUtime=1122.13 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 277130 1252 0 0 112169 44 0 0 25 0 5 0 732638776 243277824 30878 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=237576 CPUtime=1108.46 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 273252 1252 0 0 110800 46 0 0 25 0 5 0 732638776 243277824 30878 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=237576 CPUtime=1122.24 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 280810 1252 0 0 112182 42 0 0 25 0 5 0 732638776 243277824 30878 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=237576 CPUtime=1121.99 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 306922 1252 0 0 112152 47 0 0 25 0 5 0 732638776 243277824 30878 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4474.84
Current children cumulated vsize (KiB) 237576

[startup+1182.3 s]
/proc/loadavg: 9.07 8.71 7.86 11/380 9269
/proc/meminfo: memFree=17816612/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=251960 CPUtime=4714.14 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1178776 1252 0 0 471227 187 0 0 22 0 5 0 732638776 258007040 34474 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 62990 34474 180 55 0 60876 0
[pid=7620/tid=7626] ppid=7618 vsize=251960 CPUtime=1182.12 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 291549 1252 0 0 118166 46 0 0 25 0 5 0 732638776 258007040 34474 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=251960 CPUtime=1167.76 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 280684 1252 0 0 116729 47 0 0 25 0 5 0 732638776 258007040 34474 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=251960 CPUtime=1182.24 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 287625 1252 0 0 118182 42 0 0 25 0 5 0 732638776 258007040 34474 33554432000 4194304 4417452 140736526938096 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=251960 CPUtime=1181.99 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 318621 1252 0 0 118149 50 0 0 25 0 5 0 732638776 258007040 34474 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4714.14
Current children cumulated vsize (KiB) 251960



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.9 s]
/proc/loadavg: 9.12 8.75 7.89 5/375 9348
/proc/meminfo: memFree=18002520/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=258028 CPUtime=4800.3 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1189269 1252 0 0 479841 189 0 0 22 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 64507 35991 180 55 0 62393 0
[pid=7620/tid=7626] ppid=7618 vsize=258028 CPUtime=1203.73 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 291549 1252 0 0 120327 46 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4251086 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=258028 CPUtime=1189.09 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 286412 1252 0 0 118862 47 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=258028 CPUtime=1203.85 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 289965 1252 0 0 120342 43 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=258028 CPUtime=1203.6 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 321046 1252 0 0 120310 50 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 258028

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.9 s]
/proc/loadavg: 9.12 8.75 7.89 5/375 9348
/proc/meminfo: memFree=18002520/32951124 swapFree=45193364/67111528
[pid=7620] ppid=7618 vsize=258028 CPUtime=4800.3 cores=1,3,5,7
/proc/7620/stat : 7620 (plingeling) S 7618 7620 6932 0 -1 4202496 1189269 1252 0 0 479841 189 0 0 22 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/7620/statm: 64507 35991 180 55 0 62393 0
[pid=7620/tid=7626] ppid=7618 vsize=258028 CPUtime=1203.73 cores=1,3,5,7
/proc/7620/task/7626/stat : 7626 (plingeling) R 7618 7620 6932 0 -1 4202560 291549 1252 0 0 120327 46 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4251086 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7620/tid=7627] ppid=7618 vsize=258028 CPUtime=1189.09 cores=1,3,5,7
/proc/7620/task/7627/stat : 7627 (plingeling) R 7618 7620 6932 0 -1 4202560 286412 1252 0 0 118862 47 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7620/tid=7628] ppid=7618 vsize=258028 CPUtime=1203.85 cores=1,3,5,7
/proc/7620/task/7628/stat : 7628 (plingeling) R 7618 7620 6932 0 -1 4202560 289965 1252 0 0 120342 43 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7620/tid=7629] ppid=7618 vsize=258028 CPUtime=1203.6 cores=1,3,5,7
/proc/7620/task/7629/stat : 7629 (plingeling) R 7618 7620 6932 0 -1 4202560 321046 1252 0 0 120310 50 0 0 25 0 5 0 732638776 264220672 35991 33554432000 4194304 4417452 140736526938096 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 258028

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.93
CPU time (s): 4800.36
CPU user time (s): 4798.46
CPU system time (s): 1.89971
CPU usage (%): 398.725
Max. virtual memory (cumulated for all children) (KiB): 275832

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.46
system time used= 1.89971
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1190521
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 106
involuntary context switches= 38803

runsolver used 4.01139 second user time and 10.2444 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-15 03:48:01
IDJOB=3371183
IDBENCH=83146
IDSOLVER=1848
FILE ID=node118/3371183-1305424081
RUNJOBID= node118-1305424081-7592
PBS_JOBID= 13324586
Free space on /tmp= 73052 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n220-s24359701-sat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371183-1305424081/watcher-3371183-1305424081 -o /tmp/evaluation-result-3371183-1305424081/solver-3371183-1305424081 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371183-1305424081.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ceff918324677440f95867755c4e7133
RANDOM SEED=1277674830

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18208912 kB
Buffers:        281708 kB
Cached:        4460868 kB
SwapCached:      22084 kB
Active:       11818004 kB
Inactive:      2620336 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18208912 kB
SwapTotal:    67111528 kB
SwapFree:     45193364 kB
Dirty:             508 kB
Writeback:           0 kB
AnonPages:     9687008 kB
Mapped:          15044 kB
Slab:           148936 kB
PageTables:      90056 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 88692408 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73052 MiB
End job on node118 at 2011-05-15 04:08:07