Trace number 1779885

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (16 threads)? (MO) 1716.25 116.297

General information on the benchmark

NameAPPLICATIONS/bioinfo/
q_query_3_l40_lambda.cnf
MD5SUM660e95de0ccce382abc30cb1546100bc
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark18.37
Satisfiable
(Un)Satisfiability was proved
Number of variables28488
Number of clauses140544
Sum of the clauses size404432
Maximum clause length9
Minimum clause length1
Number of clauses of size 13800
Number of clauses of size 235941
Number of clauses of size 381078
Number of clauses of size 413589
Number of clauses of size 56016
Number of clauses of size over 5120

Solver Data 

0.20/0.26	c Command: HOME/ttsth-5-0 -num-threads 16 HOME/instance-1779885-1244956959.cnf 
0.20/0.26	c Using SATTIMEOUT of 10000
0.20/0.26	c Using SATRAM of 2047
0.20/0.26	c Size of subset node = 16
0.20/0.26	c Variables = 28488
0.20/0.26	c Clauses = 140544
0.20/0.26	c Literals = 404432
0.20/0.26	c Reduced size:
0.20/0.26	c Variables = 28488
0.20/0.26	c Clauses = 125907
0.20/0.26	c Literals = 367661
0.20/0.26	c a2
0.40/0.48	c a1
7.00/7.01	c Independent sub-problems = 1
7.00/7.01	c Sub-problem with 23780 variables and 125907 clauses
7.00/7.01	c using permutation method 4
7.00/7.01	c count = 1
7.00/7.01	c Ordering finished 6.960 seconds
7.00/7.01	c mapped 7.000 seconds
7.00/7.01	c permuted 7.000 seconds
7.00/7.01	c about to sort
7.00/7.04	c sorted
8.70/8.71	c clauses now 125907
8.70/8.71	c init finished 7.490 seconds
8.70/8.71	c tree built 8.690 seconds
8.70/8.71	c 
8.70/8.71	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
8.70/8.71	c ---------------------------------|----------------------------------|------------------------------------------------------
8.70/8.71	c ................................ | ..........                       |                        8.7           0.0            2
8.70/8.71	c ................................ | ..........                     1 |                        8.7           0.1            3
8.70/8.71	c ****************................ | ..........                     2 |                        8.7           0.2            3
8.70/8.71	c ****************................ | ..........                     3 |                        8.7           0.3            3
8.70/8.71	c ****************................ | ..........                     4 |                        8.7           0.5            3
8.70/8.71	c ****************................ | ..........                     5 |                        8.7           0.6            3
8.70/8.71	c ****************................ | ..........                     6 |                        8.7           0.7            3
10.11/8.88	c ****************................ | ..........                  1000 | 2                     10.1          99.0          899
245.08/23.80	c ****************................ | ..........                370000 | 18999                200.6        1844.6     10713542
449.57/36.79	c ****************................ | ..........                591000 | 31982                400.1        1477.1     26024981
653.40/49.55	c ****************................ | ..........                872000 | 50520                600.4        1452.5     41341544
855.61/62.21	c ****************................ | ..........               1056000 | 63314                800.7        1318.8     54407602
1058.07/74.96	c ****************................ | *.........               1146000 | 69450               1003.2        1142.4     65882710
1259.19/87.52	c ****************................ | *.........               1219000 | 74642               1203.2        1013.2     75628668
1460.42/100.17	c ****************................ | *.........               1285000 | 79370               1405.0         914.6     84699073
1661.43/112.79	c ****************................ | *.........               1338000 | 83132               1606.2         833.0     92029641

Verifier Data 

sh: BIN/SAT-verifier: No such file or directory

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1779885-1244956959/watcher-1779885-1244956959 -o /tmp/evaluation-result-1779885-1244956959/solver-1779885-1244956959 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 16 HOME/instance-1779885-1244956959.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 15.53 14.87 14.56 2/184 25981
/proc/meminfo: memFree=19041176/36427776 swapFree=4312/7584
[pid=25981] ppid=25979 vsize=1548 CPUtime=0
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 275 0 0 0 0 0 0 0 20 0 1 0 655002279 1585152 231 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134732614 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 387 231 75 177 0 209 0

[startup+0.088901 s]
/proc/loadavg: 15.53 14.87 14.56 2/184 25981
/proc/meminfo: memFree=19041176/36427776 swapFree=4312/7584
[pid=25981] ppid=25979 vsize=6288 CPUtime=0.08
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 1464 0 0 0 8 0 0 0 20 0 1 0 655002279 6438912 1420 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134732614 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 1572 1421 75 177 0 1394 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 6288

[startup+0.100888 s]
/proc/loadavg: 15.53 14.87 14.56 2/184 25981
/proc/meminfo: memFree=19041176/36427776 swapFree=4312/7584
[pid=25981] ppid=25979 vsize=7128 CPUtime=0.1
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 1632 0 0 0 10 0 0 0 20 0 1 0 655002279 7299072 1588 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134732614 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 1782 1589 75 177 0 1604 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 7128

[startup+0.300953 s]
/proc/loadavg: 15.53 14.87 14.56 2/184 25981
/proc/meminfo: memFree=19041176/36427776 swapFree=4312/7584
[pid=25981] ppid=25979 vsize=19668 CPUtime=0.3
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 4639 0 0 0 26 4 0 0 20 0 1 0 655002279 20140032 4595 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134554919 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 4917 4595 76 177 0 4739 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 19668

[startup+0.700952 s]
/proc/loadavg: 15.53 14.87 14.56 2/184 25981
/proc/meminfo: memFree=19041176/36427776 swapFree=4312/7584
[pid=25981] ppid=25979 vsize=22128 CPUtime=0.7
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 5761 0 0 0 66 4 0 0 20 0 1 0 655002279 22659072 5334 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134555279 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 5532 5334 77 177 0 5354 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 22128

[startup+1.50101 s]
/proc/loadavg: 15.53 14.87 14.56 2/185 25982
/proc/meminfo: memFree=19017508/36427776 swapFree=4452/30880
[pid=25981] ppid=25979 vsize=24184 CPUtime=1.5
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 6241 0 0 0 146 4 0 0 20 0 1 0 655002279 24764416 5814 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134599426 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 6046 5814 77 177 0 5868 0
Current children cumulated CPU time (s) 1.5
Current children cumulated vsize (KiB) 24184

[startup+3.10118 s]
/proc/loadavg: 15.53 14.87 14.56 2/185 25982
/proc/meminfo: memFree=19017160/36427776 swapFree=4360/32148
[pid=25981] ppid=25979 vsize=24316 CPUtime=3.1
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 6303 0 0 0 306 4 0 0 20 0 1 0 655002279 24899584 5876 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134600707 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 6079 5876 77 177 0 5901 0
Current children cumulated CPU time (s) 3.1
Current children cumulated vsize (KiB) 24316

[startup+6.30542 s]
/proc/loadavg: 14.37 14.63 14.49 2/185 25982
/proc/meminfo: memFree=19016912/36427776 swapFree=4620/32508
[pid=25981] ppid=25979 vsize=24316 CPUtime=6.3
/proc/25981/stat : 25981 (ttsth-5-0) R 25979 25981 15497 0 -1 4202496 6309 0 0 0 626 4 0 0 20 0 1 0 655002279 24899584 5882 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134600718 0 0 1 0 0 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 6079 5882 77 177 0 5901 0
Current children cumulated CPU time (s) 6.3
Current children cumulated vsize (KiB) 24316

[startup+12.7046 s]
/proc/loadavg: 13.30 14.41 14.42 17/201 25998
/proc/meminfo: memFree=18705532/36427776 swapFree=4692/346168
[pid=25981] ppid=25979 vsize=470004 CPUtime=70.4
/proc/25981/stat : 25981 (ttsth-5-0) S 25979 25981 15497 0 -1 4202496 87262 0 0 0 3387 3653 0 0 20 0 17 0 655002279 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134629544 0 0 1 0 18446744073709551615 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 117501 86589 95 177 0 117323 0
[pid=25981/tid=25983] ppid=25979 vsize=470004 CPUtime=3.92
/proc/25981/task/25983/stat : 25983 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 2595 0 0 0 360 32 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546026 0 0 1 0 0 0 0 -1 13 0 0 0 0 0
[pid=25981/tid=25984] ppid=25979 vsize=470004 CPUtime=3.9
/proc/25981/task/25984/stat : 25984 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 640 0 0 0 160 230 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 14 0 0 0 0 0
[pid=25981/tid=25985] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25985/stat : 25985 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 442 0 0 0 140 244 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 0 0 0 -1 15 0 0 0 0 0
[pid=25981/tid=25986] ppid=25979 vsize=470004 CPUtime=3.86
/proc/25981/task/25986/stat : 25986 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 1059 0 0 0 164 222 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134589526 0 0 1 0 0 0 0 -1 0 0 0 0 0 0
[pid=25981/tid=25987] ppid=25979 vsize=470004 CPUtime=3.86
/proc/25981/task/25987/stat : 25987 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 742 0 0 0 160 226 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 1 0 0 0 0 0
[pid=25981/tid=25988] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25988/stat : 25988 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 498 0 0 0 152 232 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546026 0 0 1 0 0 0 0 -1 2 0 0 0 0 0
[pid=25981/tid=25989] ppid=25979 vsize=470004 CPUtime=3.82
/proc/25981/task/25989/stat : 25989 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 734 0 0 0 154 228 0 0 20 0 17 0 655003150 481284096 86589 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 0 0 0 -1 3 0 0 0 0 0
[pid=25981/tid=25990] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25990/stat : 25990 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 982 0 0 0 140 244 0 0 20 0 17 0 655003150 481284096 86590 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 0 0 0 -1 4 0 0 0 0 0
[pid=25981/tid=25991] ppid=25979 vsize=470004 CPUtime=3.82
/proc/25981/task/25991/stat : 25991 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 916 0 0 0 140 242 0 0 20 0 17 0 655003150 481284096 86590 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546034 0 0 1 0 0 0 0 -1 5 0 0 0 0 0
[pid=25981/tid=25992] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25992/stat : 25992 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 723 0 0 0 156 228 0 0 20 0 17 0 655003150 481284096 86590 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 6 0 0 0 0 0
[pid=25981/tid=25993] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25993/stat : 25993 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 612 0 0 0 122 262 0 0 20 0 17 0 655003150 481284096 86590 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 7 0 0 0 0 0
[pid=25981/tid=25994] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25994/stat : 25994 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 542 0 0 0 144 240 0 0 20 0 17 0 655003150 481284096 86590 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 8 0 0 0 0 0
[pid=25981/tid=25995] ppid=25979 vsize=470004 CPUtime=3.84
/proc/25981/task/25995/stat : 25995 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 596 0 0 0 126 258 0 0 20 0 17 0 655003151 481284096 86591 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 9 0 0 0 0 0
[pid=25981/tid=25996] ppid=25979 vsize=470004 CPUtime=3.86
/proc/25981/task/25996/stat : 25996 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 648 0 0 0 138 248 0 0 20 0 17 0 655003151 481284096 86591 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546034 0 0 1 0 0 0 0 -1 11 0 0 0 0 0
[pid=25981/tid=25997] ppid=25979 vsize=470004 CPUtime=3.78
/proc/25981/task/25997/stat : 25997 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 543 0 0 0 128 250 0 0 20 0 17 0 655003151 481284096 86591 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134519962 0 0 1 0 0 0 0 -1 10 0 0 0 0 0
[pid=25981/tid=25998] ppid=25979 vsize=470004 CPUtime=3.88
/proc/25981/task/25998/stat : 25998 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 611 0 0 0 160 228 0 0 20 0 17 0 655003151 481284096 86591 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134591254 0 0 1 0 0 0 0 -1 12 0 0 0 0 0
Current children cumulated CPU time (s) 70.4
Current children cumulated vsize (KiB) 470004

[startup+25.5031 s]
/proc/loadavg: 13.90 14.49 14.44 17/201 25998
/proc/meminfo: memFree=18511580/36427776 swapFree=4692/541032
[pid=25981] ppid=25979 vsize=676044 CPUtime=271.85
/proc/25981/stat : 25981 (ttsth-5-0) S 25979 25981 15497 0 -1 4202496 135656 0 0 0 19470 7715 0 0 20 0 17 0 655002279 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134629544 0 0 1 0 18446744073709551615 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 169011 134983 95 177 0 168833 0
[pid=25981/tid=25983] ppid=25979 vsize=676044 CPUtime=16.7
/proc/25981/task/25983/stat : 25983 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 9593 0 0 0 1630 40 0 0 20 0 17 0 655003150 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562454 0 0 1 0 0 0 0 -1 13 0 0 0 0 0
[pid=25981/tid=25984] ppid=25979 vsize=676044 CPUtime=16.54
/proc/25981/task/25984/stat : 25984 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 2987 0 0 0 1172 482 0 0 20 0 17 0 655003150 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 14 0 0 0 0 0
[pid=25981/tid=25985] ppid=25979 vsize=676044 CPUtime=16.44
/proc/25981/task/25985/stat : 25985 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3058 0 0 0 1102 542 0 0 20 0 17 0 655003150 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134591433 0 0 1 0 0 0 0 -1 15 0 0 0 0 0
[pid=25981/tid=25986] ppid=25979 vsize=676044 CPUtime=16.38
/proc/25981/task/25986/stat : 25986 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3861 0 0 0 1132 506 0 0 20 0 17 0 655003150 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 0 0 0 0 0 0
[pid=25981/tid=25987] ppid=25979 vsize=676044 CPUtime=16.42
/proc/25981/task/25987/stat : 25987 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3290 0 0 0 1138 504 0 0 20 0 17 0 655003150 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546034 0 0 1 0 0 0 0 -1 1 0 0 0 0 0
[pid=25981/tid=25988] ppid=25979 vsize=676044 CPUtime=16.4
/proc/25981/task/25988/stat : 25988 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 2871 0 0 0 1128 512 0 0 20 0 17 0 655003150 692269056 134983 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134680308 0 0 1 0 0 0 0 -1 2 0 0 0 0 0
[pid=25981/tid=25989] ppid=25979 vsize=676044 CPUtime=16.42
/proc/25981/task/25989/stat : 25989 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3986 0 0 0 1126 516 0 0 20 0 17 0 655003150 692269056 134984 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 3 0 0 0 0 0
[pid=25981/tid=25990] ppid=25979 vsize=676044 CPUtime=16.44
/proc/25981/task/25990/stat : 25990 (ttsth-5-0) S 25979 25981 15497 0 -1 4202560 4275 0 0 0 1146 498 0 0 20 0 17 0 655003150 692269056 134985 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 18446744071564503368 0 0 -1 4 0 0 0 0 0
[pid=25981/tid=25991] ppid=25979 vsize=676044 CPUtime=16.42
/proc/25981/task/25991/stat : 25991 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 4326 0 0 0 1130 512 0 0 20 0 17 0 655003150 692269056 134985 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 5 0 0 0 0 0
[pid=25981/tid=25992] ppid=25979 vsize=676044 CPUtime=16.44
/proc/25981/task/25992/stat : 25992 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3271 0 0 0 1164 480 0 0 20 0 17 0 655003150 692269056 134985 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562278 0 0 1 0 0 0 0 -1 6 0 0 0 0 0
[pid=25981/tid=25993] ppid=25979 vsize=676044 CPUtime=16.42
/proc/25981/task/25993/stat : 25993 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3320 0 0 0 1136 506 0 0 20 0 17 0 655003150 692269056 134985 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546026 0 0 1 0 0 0 0 -1 7 0 0 0 0 0
[pid=25981/tid=25994] ppid=25979 vsize=676044 CPUtime=16.46
/proc/25981/task/25994/stat : 25994 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3031 0 0 0 1120 526 0 0 20 0 17 0 655003150 692269056 134986 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546026 0 0 1 0 0 0 0 -1 8 0 0 0 0 0
[pid=25981/tid=25995] ppid=25979 vsize=676044 CPUtime=16.46
/proc/25981/task/25995/stat : 25995 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3378 0 0 0 1132 514 0 0 20 0 17 0 655003151 692269056 134986 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 9 0 0 0 0 0
[pid=25981/tid=25996] ppid=25979 vsize=676044 CPUtime=16.46
/proc/25981/task/25996/stat : 25996 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3939 0 0 0 1132 514 0 0 20 0 17 0 655003151 692269056 134986 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 11 0 0 0 0 0
[pid=25981/tid=25997] ppid=25979 vsize=676044 CPUtime=16.14
/proc/25981/task/25997/stat : 25997 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 2810 0 0 0 1096 518 0 0 20 0 17 0 655003151 692269056 134986 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134591438 0 0 1 0 0 0 0 -1 10 0 0 0 0 0
[pid=25981/tid=25998] ppid=25979 vsize=676044 CPUtime=16.48
/proc/25981/task/25998/stat : 25998 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 3282 0 0 0 1138 510 0 0 20 0 17 0 655003151 692269056 134986 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 12 0 0 0 0 0
Current children cumulated CPU time (s) 271.85
Current children cumulated vsize (KiB) 676044

[startup+51.1064 s]
/proc/loadavg: 14.69 14.62 14.49 17/201 25998
/proc/meminfo: memFree=18000616/36427776 swapFree=4692/1051116
[pid=25981] ppid=25979 vsize=1221988 CPUtime=678.76
/proc/25981/stat : 25981 (ttsth-5-0) S 25979 25981 15497 0 -1 4202496 263887 0 0 0 59107 8769 0 0 20 0 17 0 655002279 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134629544 0 0 1 0 18446744073709551615 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 305497 263214 95 177 0 305319 0
[pid=25981/tid=25983] ppid=25979 vsize=1221988 CPUtime=42.29
/proc/25981/task/25983/stat : 25983 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 17262 0 0 0 4179 50 0 0 20 0 17 0 655003150 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562391 0 0 1 0 0 0 0 -1 13 0 0 0 0 0
[pid=25981/tid=25984] ppid=25979 vsize=1221988 CPUtime=41.99
/proc/25981/task/25984/stat : 25984 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 12937 0 0 0 3637 562 0 0 20 0 17 0 655003150 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 0 0 0 -1 14 0 0 0 0 0
[pid=25981/tid=25985] ppid=25979 vsize=1221988 CPUtime=41.9
/proc/25981/task/25985/stat : 25985 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 11716 0 0 0 3582 608 0 0 20 0 17 0 655003150 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 15 0 0 0 0 0
[pid=25981/tid=25986] ppid=25979 vsize=1221988 CPUtime=41.75
/proc/25981/task/25986/stat : 25986 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 11799 0 0 0 3587 588 0 0 20 0 17 0 655003150 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 0 0 0 0 0 0
[pid=25981/tid=25987] ppid=25979 vsize=1221988 CPUtime=41.84
/proc/25981/task/25987/stat : 25987 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 9951 0 0 0 3604 580 0 0 20 0 17 0 655003150 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 1 0 0 0 0 0
[pid=25981/tid=25988] ppid=25979 vsize=1221988 CPUtime=41.84
/proc/25981/task/25988/stat : 25988 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 10314 0 0 0 3590 594 0 0 20 0 17 0 655003150 1251315712 263214 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 2 0 0 0 0 0
[pid=25981/tid=25989] ppid=25979 vsize=1221988 CPUtime=41.87
/proc/25981/task/25989/stat : 25989 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 13181 0 0 0 3615 572 0 0 20 0 17 0 655003150 1251315712 263215 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 3 0 0 0 0 0
[pid=25981/tid=25990] ppid=25979 vsize=1221988 CPUtime=41.88
/proc/25981/task/25990/stat : 25990 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 14700 0 0 0 3628 560 0 0 20 0 17 0 655003150 1251315712 263215 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 4 0 0 0 0 0
[pid=25981/tid=25991] ppid=25979 vsize=1221988 CPUtime=41.88
/proc/25981/task/25991/stat : 25991 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 14528 0 0 0 3598 590 0 0 20 0 17 0 655003150 1251315712 263215 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 0 0 0 -1 5 0 0 0 0 0
[pid=25981/tid=25992] ppid=25979 vsize=1221988 CPUtime=42.06
/proc/25981/task/25992/stat : 25992 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 5474 0 0 0 3716 490 0 0 20 0 17 0 655003150 1251315712 263215 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562420 0 0 1 0 0 0 0 -1 6 0 0 0 0 0
[pid=25981/tid=25993] ppid=25979 vsize=1221988 CPUtime=41.9
/proc/25981/task/25993/stat : 25993 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 14619 0 0 0 3630 560 0 0 20 0 17 0 655003150 1251315712 263216 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636382 0 0 1 0 0 0 0 -1 7 0 0 0 0 0
[pid=25981/tid=25994] ppid=25979 vsize=1221988 CPUtime=41.91
/proc/25981/task/25994/stat : 25994 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 10788 0 0 0 3577 614 0 0 20 0 17 0 655003150 1251315712 263216 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 8 0 0 0 0 0
[pid=25981/tid=25995] ppid=25979 vsize=1221988 CPUtime=41.97
/proc/25981/task/25995/stat : 25995 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 12665 0 0 0 3627 570 0 0 20 0 17 0 655003151 1251315712 263216 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546026 0 0 1 0 0 0 0 -1 9 0 0 0 0 0
[pid=25981/tid=25996] ppid=25979 vsize=1221988 CPUtime=41.88
/proc/25981/task/25996/stat : 25996 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 9831 0 0 0 3592 596 0 0 20 0 17 0 655003151 1251315712 263217 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 11 0 0 0 0 0
[pid=25981/tid=25997] ppid=25979 vsize=1221988 CPUtime=41.09
/proc/25981/task/25997/stat : 25997 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 10002 0 0 0 3523 586 0 0 20 0 17 0 655003151 1251315712 263217 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 10 0 0 0 0 0
[pid=25981/tid=25998] ppid=25979 vsize=1221988 CPUtime=41.9
/proc/25981/task/25998/stat : 25998 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 9739 0 0 0 3580 610 0 0 20 0 17 0 655003151 1251315712 263217 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134593191 0 0 1 0 0 0 0 -1 12 0 0 0 0 0
Current children cumulated CPU time (s) 678.76
Current children cumulated vsize (KiB) 1221988

[startup+102.318 s]
/proc/loadavg: 15.43 14.83 14.57 17/201 25998
/proc/meminfo: memFree=17294660/36427776 swapFree=4692/1755092
[pid=25981] ppid=25979 vsize=1964188 CPUtime=1495.64
/proc/25981/stat : 25981 (ttsth-5-0) S 25979 25981 15497 0 -1 4202496 438123 0 0 0 140114 9450 0 0 20 0 17 0 655002279 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134629544 0 0 1 0 18446744073709551615 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 491047 437450 95 177 0 490869 0
[pid=25981/tid=25983] ppid=25979 vsize=1964188 CPUtime=93.48
/proc/25981/task/25983/stat : 25983 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 44958 0 0 0 9254 94 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562360 0 0 1 0 0 0 0 -1 13 0 0 0 0 0
[pid=25981/tid=25984] ppid=25979 vsize=1964188 CPUtime=93.11
/proc/25981/task/25984/stat : 25984 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 20459 0 0 0 8703 608 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134680308 0 0 1 0 0 0 0 -1 14 0 0 0 0 0
[pid=25981/tid=25985] ppid=25979 vsize=1964188 CPUtime=92.99
/proc/25981/task/25985/stat : 25985 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 27258 0 0 0 8633 666 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 15 0 0 0 0 0
[pid=25981/tid=25986] ppid=25979 vsize=1964188 CPUtime=92.83
/proc/25981/task/25986/stat : 25986 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 20267 0 0 0 8643 640 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562351 0 0 1 0 0 0 0 -1 0 0 0 0 0 0
[pid=25981/tid=25987] ppid=25979 vsize=1964188 CPUtime=92.96
/proc/25981/task/25987/stat : 25987 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 21065 0 0 0 8672 624 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 1 0 0 0 0 0
[pid=25981/tid=25988] ppid=25979 vsize=1964188 CPUtime=92.99
/proc/25981/task/25988/stat : 25988 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 17607 0 0 0 8677 622 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562351 0 0 1 0 0 0 0 -1 2 0 0 0 0 0
[pid=25981/tid=25989] ppid=25979 vsize=1964188 CPUtime=92.95
/proc/25981/task/25989/stat : 25989 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 20635 0 0 0 8677 618 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134550989 0 0 1 0 0 0 0 -1 3 0 0 0 0 0
[pid=25981/tid=25990] ppid=25979 vsize=1964188 CPUtime=93.01
/proc/25981/task/25990/stat : 25990 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 22731 0 0 0 8703 598 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 4 0 0 0 0 0
[pid=25981/tid=25991] ppid=25979 vsize=1964188 CPUtime=92.99
/proc/25981/task/25991/stat : 25991 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 31894 0 0 0 8661 638 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 5 0 0 0 0 0
[pid=25981/tid=25992] ppid=25979 vsize=1964188 CPUtime=93.16
/proc/25981/task/25992/stat : 25992 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 8953 0 0 0 8814 502 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562380 0 0 1 0 0 0 0 -1 6 0 0 0 0 0
[pid=25981/tid=25993] ppid=25979 vsize=1964188 CPUtime=93.03
/proc/25981/task/25993/stat : 25993 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 27493 0 0 0 8711 592 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 7 0 0 0 0 0
[pid=25981/tid=25994] ppid=25979 vsize=1964188 CPUtime=93.04
/proc/25981/task/25994/stat : 25994 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 19062 0 0 0 8638 666 0 0 20 0 17 0 655003150 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 8 0 0 0 0 0
[pid=25981/tid=25995] ppid=25979 vsize=1964188 CPUtime=93.08
/proc/25981/task/25995/stat : 25995 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 24757 0 0 0 8688 620 0 0 20 0 17 0 655003151 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 9 0 0 0 0 0
[pid=25981/tid=25996] ppid=25979 vsize=1964188 CPUtime=93.01
/proc/25981/task/25996/stat : 25996 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 16063 0 0 0 8663 638 0 0 20 0 17 0 655003151 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 11 0 0 0 0 0
[pid=25981/tid=25997] ppid=25979 vsize=1964188 CPUtime=91.17
/proc/25981/task/25997/stat : 25997 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 18054 0 0 0 8479 638 0 0 20 0 17 0 655003151 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134591181 0 0 1 0 0 0 0 -1 10 0 0 0 0 0
[pid=25981/tid=25998] ppid=25979 vsize=1964188 CPUtime=93.03
/proc/25981/task/25998/stat : 25998 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 22486 0 0 0 8653 650 0 0 20 0 17 0 655003151 2011328512 437450 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 12 0 0 0 0 0
Current children cumulated CPU time (s) 1495.64
Current children cumulated vsize (KiB) 1964188



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+116.102 s]
/proc/loadavg: 15.56 14.89 14.59 17/201 25998
/proc/meminfo: memFree=17168128/36427776 swapFree=4692/1881432
[pid=25981] ppid=25979 vsize=2096468 CPUtime=1715.9
/proc/25981/stat : 25981 (ttsth-5-0) S 25979 25981 15497 0 -1 4202496 469011 0 0 0 162023 9567 0 0 20 0 17 0 655002279 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134629544 0 0 1 0 18446744073709551615 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 524117 468338 95 177 0 523939 0
[pid=25981/tid=25983] ppid=25979 vsize=2096468 CPUtime=107.27
/proc/25981/task/25983/stat : 25983 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 50987 0 0 0 10621 106 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 13 0 0 0 0 0
[pid=25981/tid=25984] ppid=25979 vsize=2096468 CPUtime=106.9
/proc/25981/task/25984/stat : 25984 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 22395 0 0 0 10074 616 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562380 0 0 1 0 0 0 0 -1 14 0 0 0 0 0
[pid=25981/tid=25985] ppid=25979 vsize=2096468 CPUtime=106.75
/proc/25981/task/25985/stat : 25985 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 28963 0 0 0 10001 674 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134546034 0 0 1 0 0 0 0 -1 15 0 0 0 0 0
[pid=25981/tid=25986] ppid=25979 vsize=2096468 CPUtime=106.6
/proc/25981/task/25986/stat : 25986 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 21706 0 0 0 10012 648 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562344 0 0 1 0 0 0 0 -1 0 0 0 0 0 0
[pid=25981/tid=25987] ppid=25979 vsize=2096468 CPUtime=106.74
/proc/25981/task/25987/stat : 25987 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 22684 0 0 0 10042 632 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562341 0 0 1 0 0 0 0 -1 1 0 0 0 0 0
[pid=25981/tid=25988] ppid=25979 vsize=2096468 CPUtime=106.77
/proc/25981/task/25988/stat : 25988 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 18673 0 0 0 10053 624 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562380 0 0 1 0 0 0 0 -1 2 0 0 0 0 0
[pid=25981/tid=25989] ppid=25979 vsize=2096468 CPUtime=106.75
/proc/25981/task/25989/stat : 25989 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 21992 0 0 0 10049 626 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 3 0 0 0 0 0
[pid=25981/tid=25990] ppid=25979 vsize=2096468 CPUtime=106.77
/proc/25981/task/25990/stat : 25990 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 23953 0 0 0 10073 604 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134590777 0 0 1 0 0 0 0 -1 4 0 0 0 0 0
[pid=25981/tid=25991] ppid=25979 vsize=2096468 CPUtime=106.78
/proc/25981/task/25991/stat : 25991 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 34730 0 0 0 10026 652 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 5 0 0 0 0 0
[pid=25981/tid=25992] ppid=25979 vsize=2096468 CPUtime=106.94
/proc/25981/task/25992/stat : 25992 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 9571 0 0 0 10190 504 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 6 0 0 0 0 0
[pid=25981/tid=25993] ppid=25979 vsize=2096468 CPUtime=106.83
/proc/25981/task/25993/stat : 25993 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 28452 0 0 0 10087 596 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 7 0 0 0 0 0
[pid=25981/tid=25994] ppid=25979 vsize=2096468 CPUtime=106.82
/proc/25981/task/25994/stat : 25994 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 20091 0 0 0 10014 668 0 0 20 0 17 0 655003150 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134589646 0 0 1 0 0 0 0 -1 8 0 0 0 0 0
[pid=25981/tid=25995] ppid=25979 vsize=2096468 CPUtime=106.86
/proc/25981/task/25995/stat : 25995 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 28190 0 0 0 10056 630 0 0 20 0 17 0 655003151 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 9 0 0 0 0 0
[pid=25981/tid=25996] ppid=25979 vsize=2096468 CPUtime=106.79
/proc/25981/task/25996/stat : 25996 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 17290 0 0 0 10033 646 0 0 20 0 17 0 655003151 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134636641 0 0 1 0 0 0 0 -1 11 0 0 0 0 0
[pid=25981/tid=25997] ppid=25979 vsize=2096468 CPUtime=104.68
/proc/25981/task/25997/stat : 25997 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 20174 0 0 0 9820 648 0 0 20 0 17 0 655003151 2146783232 468338 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134590285 0 0 1 0 0 0 0 -1 10 0 0 0 0 0
[pid=25981/tid=25998] ppid=25979 vsize=2096468 CPUtime=106.8
/proc/25981/task/25998/stat : 25998 (ttsth-5-0) R 25979 25981 15497 0 -1 4202560 24779 0 0 0 10024 656 0 0 20 0 17 0 655003151 2146783232 468339 18446744073709551615 134512640 135237372 4287514560 18446744073709551615 134562338 0 0 1 0 0 0 0 -1 12 0 0 0 0 0
Current children cumulated CPU time (s) 1715.9
Current children cumulated vsize (KiB) 2096468

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+116.227 s]
/proc/loadavg: 15.56 14.89 14.59 17/201 25998
/proc/meminfo: memFree=17168128/36427776 swapFree=4692/1881432
[pid=25981] ppid=25979 vsize=0 CPUtime=1716.16
/proc/25981/stat : 25981 (ttsth-5-0) Z 25979 25981 15497 0 -1 4203532 469024 0 0 0 162037 9579 0 0 20 0 2 0 655002279 0 0 18446744073709551615 0 0 0 0 0 0 0 1 0 18446744073709551615 0 0 17 12 0 0 0 0 0
/proc/25981/statm: 0 0 0 0 0 0 0
[pid=25981/tid=25998] ppid=25979 vsize=0 CPUtime=106.92
/proc/25981/task/25998/stat : 25998 (ttsth-5-0) R 25979 25981 15497 0 -1 4203588 24779 0 0 0 10024 668 0 0 20 0 2 0 655003151 0 0 18446744073709551615 0 0 0 0 0 0 0 1 0 0 0 0 -1 12 0 0 0 0 0
Current children cumulated CPU time (s) 1716.16
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 116.297
CPU time (s): 1716.25
CPU user time (s): 1620.39
CPU system time (s): 95.862
CPU usage (%): 1475.74
Max. virtual memory (cumulated for all children) (KiB): 2096468

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1620.39
system time used= 95.862
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 469024
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 87433
involuntary context switches= 3901

runsolver used 0.532033 second user time and 1.6841 second system time

The end

Launcher Data 

Begin job on xenhost3-49 at 2009-06-14 07:22:39
IDJOB=1779885
IDBENCH=70938
IDSOLVER=634
FILE ID=xenhost3-49/1779885-1244956959
PBS_JOBID=
Free space on /tmp= 7021 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (16 threads)
BENCH NAME= SAT09/APPLICATIONS/bioinfo/q_query_3_l40_lambda.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1779885-1244956959/watcher-1779885-1244956959 -o /tmp/evaluation-result-1779885-1244956959/solver-1779885-1244956959 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 16 HOME/instance-1779885-1244956959.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 16

MD5SUM BENCH= 660e95de0ccce382abc30cb1546100bc
RANDOM SEED= 1596870469

Linux xenhost3-49 2.6.27.5 #1 SMP Mon Mar 30 10:34:22 CEST 2009 x86_64 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 2
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 8
initial apicid	: 8
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 4
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 6
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 24
initial apicid	: 24
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 2
cpu cores	: 1
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 2
siblings	: 1
core id		: 2
cpu cores	: 1
apicid		: 10
initial apicid	: 10
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 4
siblings	: 1
core id		: 2
cpu cores	: 1
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 6
siblings	: 1
core id		: 2
cpu cores	: 1
apicid		: 26
initial apicid	: 26
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 1
cpu cores	: 1
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 2
siblings	: 1
core id		: 1
cpu cores	: 1
apicid		: 9
initial apicid	: 9
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 4
siblings	: 1
core id		: 1
cpu cores	: 1
apicid		: 17
initial apicid	: 17
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 6
siblings	: 1
core id		: 1
cpu cores	: 1
apicid		: 25
initial apicid	: 25
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 0
siblings	: 1
core id		: 3
cpu cores	: 1
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 2
siblings	: 1
core id		: 3
cpu cores	: 1
apicid		: 11
initial apicid	: 11
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 4
siblings	: 1
core id		: 3
cpu cores	: 1
apicid		: 19
initial apicid	: 19
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU           X7350  @ 2.93GHz
stepping	: 11
cpu MHz		: 2925.864
cache size	: 4096 KB
physical id	: 6
siblings	: 1
core id		: 3
cpu cores	: 1
apicid		: 27
initial apicid	: 27
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 5855.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:     36427776 kB
MemFree:      19041176 kB
Buffers:        484460 kB
Cached:       14119472 kB
SwapCached:          0 kB
Active:        4129828 kB
Inactive:     10481700 kB
SwapTotal:    25398692 kB
SwapFree:     25398692 kB
Dirty:            3336 kB
Writeback:           0 kB
AnonPages:        7584 kB
Mapped:           3648 kB
Slab:           721680 kB
SReclaimable:   694896 kB
SUnreclaim:      26784 kB
PageTables:          0 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
WritebackTmp:        0 kB
CommitLimit:  43612580 kB
Committed_AS:    29616 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    275248 kB
VmallocChunk: 34359462839 kB
DirectMap4k:  99216660 kB
DirectMap2M:         0 kB

Free space on /tmp at the end= 7021 MiB
End job on xenhost3-49 at 2009-06-14 07:24:37