Trace number 1777003

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (4 threads)? (TO) 3856.56 1000.1

General information on the benchmark

Namemizh-md5-48-5.cnf
MD5SUM8ab50418497c0326ed1ff5d73a4e24a3
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1504.44
Satisfiable
(Un)Satisfiability was proved
Number of variables66892
Number of clauses279256
Sum of the clauses size832648
Maximum clause length4
Minimum clause length1
Number of clauses of size 1201
Number of clauses of size 2120496
Number of clauses of size 342781
Number of clauses of size 4115778
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

1.81/1.82	c Command: HOME/ttsth-5-0 -num-threads 4 HOME/instance-1777003-1243676550.cnf 
1.81/1.82	c Using SATTIMEOUT of 10000
1.81/1.82	c Using SATRAM of 2047
1.81/1.82	c Size of subset node = 16
1.81/1.82	c Variables = 66892
1.81/1.82	c Clauses = 279256
1.81/1.82	c Literals = 832648
1.81/1.82	c Reduced size:
1.81/1.82	c Variables = 66892
1.81/1.82	c Clauses = 238959
1.81/1.82	c Literals = 689357
1.81/1.82	c a2
2.30/2.38	c a1
44.57/44.65	c Independent sub-problems = 1
44.57/44.65	c Sub-problem with 61584 variables and 238959 clauses
44.57/44.65	c using permutation method 4
44.57/44.65	c count = 1
44.57/44.65	c Ordering finished 44.490 seconds
44.57/44.65	c mapped 44.600 seconds
44.57/44.65	c permuted 44.600 seconds
44.57/44.65	c about to sort
44.67/44.73	c sorted
45.97/46.08	c clauses now 238959
45.97/46.08	c init finished 45.080 seconds
45.97/46.08	c tree built 46.030 seconds
45.97/46.08	c 
45.97/46.08	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
45.97/46.08	c ---------------------------------|----------------------------------|------------------------------------------------------
45.97/46.08	c ................................ | ..........                       |                       46.0           0.0             
45.97/46.08	c ................................ | ..........                     1 |                       46.0           0.0             
45.97/46.08	c ................................ | ..........                     2 |                       46.0           0.0             
45.97/46.08	c ................................ | ..........                     3 |                       46.0           0.1             
45.97/46.08	c ................................ | ..........                     4 |                       46.0           0.1             
45.97/46.08	c ................................ | ..........                     5 |                       46.0           0.1             
45.97/46.08	c ................................ | ..........                  1000 |                       46.0          21.7            6
199.79/84.67	c ................................ | ..........               4864000 | 30304                200.0       24316.4      4577253
399.86/134.77	c ................................ | ..........              10174000 | 60676                400.0       25435.0      9944900
599.63/184.81	c ................................ | ..........              15996000 | 96658                600.0       26658.7     13010618
800.17/234.98	c ................................ | ..........              21257000 | 131301               800.5       26556.3     16226690
1000.11/285.08	c ................................ | *.........              25962000 | 159842              1000.4       25950.8     19133770
1199.74/335.08	c ................................ | *.........              30684000 | 190831              1200.0       25569.6     21615764
1399.89/385.18	c ................................ | *.........              33861000 | 251814              1400.1       24184.2     24598971
1599.93/435.25	c ................................ | *.........              36718000 | 316160              1600.1       22947.5     27354807
1799.85/485.26	c ................................ | *.........              39637000 | 379610              1800.0       22019.9     30287231
2000.01/535.38	c ................................ | **........              42533000 | 447265              2000.2       21263.8     33496354

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1777003-1243676550/watcher-1777003-1243676550 -o /tmp/evaluation-result-1777003-1243676550/solver-1777003-1243676550 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 4 HOME/instance-1777003-1243676550.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 0.92 1.03 1.33 3/99 5329
/proc/meminfo: memFree=3065448/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1716 CPUtime=0.02
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 309 0 0 0 2 0 0 0 25 0 1 0 1132319448 1757184 265 4294967295 134512640 135237372 3213945952 3213944476 134639534 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 429 266 75 177 0 251 0

[startup+0.093741 s]
/proc/loadavg: 0.92 1.03 1.33 3/99 5329
/proc/meminfo: memFree=3065448/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=5036 CPUtime=0.1
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 1138 0 0 0 10 0 0 0 25 0 1 0 1132319448 5156864 1094 4294967295 134512640 135237372 3213945952 3213944748 134639534 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 1259 1095 75 177 0 1081 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 5036

[startup+0.103728 s]
/proc/loadavg: 0.92 1.03 1.33 3/99 5329
/proc/meminfo: memFree=3065448/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=5432 CPUtime=0.11
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 1245 0 0 0 11 0 0 0 25 0 1 0 1132319448 5562368 1201 4294967295 134512640 135237372 3213945952 3213945400 134732614 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 1358 1201 75 177 0 1180 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 5432

[startup+0.303708 s]
/proc/loadavg: 0.92 1.03 1.33 3/99 5329
/proc/meminfo: memFree=3065448/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=13932 CPUtime=0.31
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 3365 0 0 0 31 0 0 0 25 0 1 0 1132319448 14266368 3321 4294967295 134512640 135237372 3213945952 3213944476 134639534 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 3483 3322 75 177 0 3305 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 13932

[startup+0.703672 s]
/proc/loadavg: 0.92 1.03 1.33 3/99 5329
/proc/meminfo: memFree=3065448/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=21860 CPUtime=0.71
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 5205 0 0 0 71 0 0 0 25 0 1 0 1132319448 22384640 5161 4294967295 134512640 135237372 3213945952 3213945644 134534043 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 5465 5161 75 177 0 5287 0
Current children cumulated CPU time (s) 0.71
Current children cumulated vsize (KiB) 21860

[startup+1.5036 s]
/proc/loadavg: 0.92 1.03 1.33 2/100 5330
/proc/meminfo: memFree=3045120/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=21860 CPUtime=1.51
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 5205 0 0 0 151 0 0 0 25 0 1 0 1132319448 22384640 5161 4294967295 134512640 135237372 3213945952 3213945644 134534088 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 5465 5161 75 177 0 5287 0
Current children cumulated CPU time (s) 1.51
Current children cumulated vsize (KiB) 21860

[startup+3.10344 s]
/proc/loadavg: 0.92 1.03 1.33 2/100 5330
/proc/meminfo: memFree=3023808/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=47220 CPUtime=3.1
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 12331 0 0 0 307 3 0 0 25 0 1 0 1132319448 48353280 11459 4294967295 134512640 135237372 3213945952 3213945312 134600975 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 11805 11459 77 177 0 11627 0
Current children cumulated CPU time (s) 3.1
Current children cumulated vsize (KiB) 47220

[startup+6.30313 s]
/proc/loadavg: 0.93 1.03 1.33 2/100 5331
/proc/meminfo: memFree=3019508/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=47904 CPUtime=6.3
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 12499 0 0 0 627 3 0 0 25 0 1 0 1132319448 49053696 11627 4294967295 134512640 135237372 3213945952 3213945312 134600960 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 11976 11627 77 177 0 11798 0
Current children cumulated CPU time (s) 6.3
Current children cumulated vsize (KiB) 47904

[startup+12.7025 s]
/proc/loadavg: 0.93 1.03 1.32 3/100 5331
/proc/meminfo: memFree=3018152/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=49340 CPUtime=12.7
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 12821 0 0 0 1267 3 0 0 25 0 1 0 1132319448 50524160 11949 4294967295 134512640 135237372 3213945952 3213945312 134600975 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 12335 11949 77 177 0 12157 0
Current children cumulated CPU time (s) 12.7
Current children cumulated vsize (KiB) 49340

[startup+25.5013 s]
/proc/loadavg: 0.95 1.03 1.32 2/100 5514
/proc/meminfo: memFree=3017424/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=49604 CPUtime=25.49
/proc/5329/stat : 5329 (ttsth-5-0) R 5327 5329 20984 0 -1 4202496 12877 0 0 0 2544 5 0 0 25 0 1 0 1132319448 50794496 12005 4294967295 134512640 135237372 3213945952 3213945312 134600718 0 0 0 0 0 0 0 17 2 0 0 0
/proc/5329/statm: 12401 12005 77 177 0 12223 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 49604

[startup+51.1189 s]
/proc/loadavg: 1.20 1.07 1.33 5/104 5522
/proc/meminfo: memFree=2776916/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=317708 CPUtime=66.06
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 74710 0 0 0 6577 29 0 0 25 0 5 0 1132319448 325332992 73007 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 79427 73007 95 177 0 79249 0
[pid=5329/tid=5519] ppid=5327 vsize=317708 CPUtime=5
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 1095 0 0 0 499 1 0 0 25 0 5 0 1132324057 325332992 73007 4294967295 134512640 135237372 3213945952 3071307488 134751534 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=317708 CPUtime=4.99
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 2420 0 0 0 499 0 0 0 25 0 5 0 1132324057 325332992 73007 4294967295 134512640 135237372 3213945952 3062295636 134680212 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=317708 CPUtime=5.01
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 2255 0 0 0 499 2 0 0 25 0 5 0 1132324057 325332992 73007 4294967295 134512640 135237372 3213945952 3054079708 134549950 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=317708 CPUtime=5.03
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 1979 0 0 0 502 1 0 0 25 0 5 0 1132324057 325332992 73007 4294967295 134512640 135237372 3213945952 3045754336 134590285 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 66.06
Current children cumulated vsize (KiB) 317708

[startup+102.314 s]
/proc/loadavg: 2.88 1.56 1.48 5/104 5708
/proc/meminfo: memFree=2681556/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=416024 CPUtime=270.57
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 97943 0 0 0 26980 77 0 0 25 0 5 0 1132319448 426008576 96240 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 104006 96240 95 177 0 103828 0
[pid=5329/tid=5519] ppid=5327 vsize=416024 CPUtime=56.16
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 1599 0 0 0 5608 8 0 0 25 0 5 0 1132324057 426008576 96240 4294967295 134512640 135237372 3213945952 3072731392 134593167 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=416024 CPUtime=56.01
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 9847 0 0 0 5583 18 0 0 25 0 5 0 1132324057 426008576 96240 4294967295 134512640 135237372 3213945952 3062827536 134680285 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=416024 CPUtime=56.19
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 11377 0 0 0 5605 14 0 0 25 0 5 0 1132324057 426008576 96240 4294967295 134512640 135237372 3213945952 3054570912 134588671 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=416024 CPUtime=56.18
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 8159 0 0 0 5606 12 0 0 25 0 5 0 1132324057 426008576 96240 4294967295 134512640 135237372 3213945952 3045862464 134680329 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 270.57
Current children cumulated vsize (KiB) 416024

[startup+162.308 s]
/proc/loadavg: 3.63 1.99 1.62 5/104 5896
/proc/meminfo: memFree=2603444/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=497628 CPUtime=510.08
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 117407 0 0 0 50875 133 0 0 25 0 5 0 1132319448 509571072 115704 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 124407 115704 95 177 0 124229 0
[pid=5329/tid=5519] ppid=5327 vsize=497628 CPUtime=115.94
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 1684 0 0 0 11574 20 0 0 25 0 5 0 1132324057 509571072 115704 4294967295 134512640 135237372 3213945952 3071891376 134551475 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=497628 CPUtime=115.96
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 16962 0 0 0 11574 22 0 0 25 0 5 0 1132324057 509571072 115704 4294967295 134512640 135237372 3213945952 3062461664 134562336 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=497628 CPUtime=116.01
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 18470 0 0 0 11569 32 0 0 25 0 5 0 1132324057 509571072 115704 4294967295 134512640 135237372 3213945952 3054071584 134562344 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=497628 CPUtime=116.14
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 13330 0 0 0 11580 34 0 0 25 0 5 0 1132324057 509571072 115704 4294967295 134512640 135237372 3213945952 3045613360 134590129 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 510.08
Current children cumulated vsize (KiB) 497628

[startup+222.313 s]
/proc/loadavg: 3.95 2.38 1.78 5/104 6084
/proc/meminfo: memFree=2542112/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=562376 CPUtime=749.79
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 132699 0 0 0 74797 182 0 0 25 0 5 0 1132319448 575873024 130996 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 140594 130996 95 177 0 140416 0
[pid=5329/tid=5519] ppid=5327 vsize=562376 CPUtime=175.87
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 1963 0 0 0 17562 25 0 0 25 0 5 0 1132324057 575873024 130996 4294967295 134512640 135237372 3213945952 3070674784 134588649 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=562376 CPUtime=175.95
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 21672 0 0 0 17565 30 0 0 25 0 5 0 1132324057 575873024 130996 4294967295 134512640 135237372 3213945952 3062470640 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=562376 CPUtime=175.84
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 23953 0 0 0 17540 44 0 0 25 0 5 0 1132324057 575873024 130996 4294967295 134512640 135237372 3213945952 3054007616 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=562376 CPUtime=176.1
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 18150 0 0 0 17552 58 0 0 25 0 5 0 1132324057 575873024 130996 4294967295 134512640 135237372 3213945952 3045676432 134589170 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 749.79
Current children cumulated vsize (KiB) 562376

[startup+282.307 s]
/proc/loadavg: 4.18 2.73 1.94 5/104 6272
/proc/meminfo: memFree=2483980/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=624976 CPUtime=989.33
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 147373 0 0 0 98712 221 0 0 25 0 5 0 1132319448 639975424 145670 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 156244 145670 95 177 0 156066 0
[pid=5329/tid=5519] ppid=5327 vsize=624976 CPUtime=235.73
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 2774 0 0 0 23539 34 0 0 25 0 5 0 1132324057 639975424 145670 4294967295 134512640 135237372 3213945952 3071035332 134680212 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=624976 CPUtime=235.75
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 25720 0 0 0 23536 39 0 0 25 0 5 0 1132324057 639975424 145670 4294967295 134512640 135237372 3213945952 3062545056 134561428 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=624976 CPUtime=235.79
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 29468 0 0 0 23522 57 0 0 25 0 5 0 1132324057 639975424 145670 4294967295 134512640 135237372 3213945952 3054130092 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=624976 CPUtime=236.03
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 22450 0 0 0 23537 66 0 0 25 0 5 0 1132324057 639975424 145670 4294967295 134512640 135237372 3213945952 3045759788 134549950 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 989.33
Current children cumulated vsize (KiB) 624976

[startup+342.311 s]
/proc/loadavg: 4.29 3.02 2.08 6/104 6467
/proc/meminfo: memFree=2434624/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=676572 CPUtime=1228.94
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 159624 0 0 0 122628 266 0 0 25 0 5 0 1132319448 692809728 157921 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 169143 157921 95 177 0 168965 0
[pid=5329/tid=5519] ppid=5327 vsize=676572 CPUtime=295.55
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 3787 0 0 0 29517 38 0 0 25 0 5 0 1132324057 692809728 157921 4294967295 134512640 135237372 3213945952 3070820368 134749348 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=676572 CPUtime=295.67
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 28521 0 0 0 29506 61 0 0 25 0 5 0 1132324057 692809728 157921 4294967295 134512640 135237372 3213945952 3062459904 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=676572 CPUtime=295.79
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 34338 0 0 0 29512 67 0 0 25 0 5 0 1132324057 692809728 157921 4294967295 134512640 135237372 3213945952 3054076752 134562278 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=676572 CPUtime=295.9
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 26017 0 0 0 29515 75 0 0 25 0 5 0 1132324057 692809728 157921 4294967295 134512640 135237372 3213945952 3045546320 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1228.94

################
# More data... #
################

[pid=5329/tid=5520] ppid=5327 vsize=790276 CPUtime=415.3
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 34223 0 0 0 41442 88 0 0 25 0 5 0 1132324057 809242624 185135 4294967295 134512640 135237372 3213945952 3062470880 134749305 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=790276 CPUtime=415.68
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 43872 0 0 0 41475 93 0 0 25 0 5 0 1132324057 809242624 185135 4294967295 134512640 135237372 3213945952 3054072640 134749351 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=790276 CPUtime=415.74
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 31710 0 0 0 41478 96 0 0 25 0 5 0 1132324057 809242624 185135 4294967295 134512640 135237372 3213945952 3045792096 134680285 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1708.27
Current children cumulated vsize (KiB) 790276

[startup+522.304 s]
/proc/loadavg: 4.09 3.51 2.43 5/104 7029
/proc/meminfo: memFree=2267780/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=853408 CPUtime=1947.97
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 201462 0 0 0 194411 386 0 0 25 0 5 0 1132319448 873889792 199759 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 213352 199759 95 177 0 213174 0
[pid=5329/tid=5519] ppid=5327 vsize=853408 CPUtime=475.49
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 12953 0 0 0 47505 44 0 0 25 0 5 0 1132324057 873889792 199759 4294967295 134512640 135237372 3213945952 3070901456 134749353 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=853408 CPUtime=475.09
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 36537 0 0 0 47401 108 0 0 25 0 5 0 1132324057 873889792 199759 4294967295 134512640 135237372 3213945952 3062815776 134552734 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=853408 CPUtime=475.68
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 48421 0 0 0 47468 100 0 0 25 0 5 0 1132324057 873889792 199759 4294967295 134512640 135237372 3213945952 3054356784 134591065 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=853408 CPUtime=475.68
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 36590 0 0 0 47459 109 0 0 25 0 5 0 1132324057 873889792 199759 4294967295 134512640 135237372 3213945952 3045702444 134519936 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1947.97
Current children cumulated vsize (KiB) 853408

[startup+582.308 s]
/proc/loadavg: 4.06 3.61 2.53 5/104 7217
/proc/meminfo: memFree=2198860/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=925856 CPUtime=2187.69
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 218568 0 0 0 218345 424 0 0 25 0 5 0 1132319448 948076544 216865 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 231464 216865 95 177 0 231286 0
[pid=5329/tid=5519] ppid=5327 vsize=925856 CPUtime=535.49
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 15472 0 0 0 53504 45 0 0 25 0 5 0 1132324057 948076544 216865 4294967295 134512640 135237372 3213945952 3071289664 134591168 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=925856 CPUtime=534.92
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 39966 0 0 0 53369 123 0 0 25 0 5 0 1132324057 948076544 216865 4294967295 134512640 135237372 3213945952 3062454736 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=925856 CPUtime=535.66
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 54750 0 0 0 53453 113 0 0 25 0 5 0 1132324057 948076544 216865 4294967295 134512640 135237372 3213945952 3053929680 134680229 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=925856 CPUtime=535.59
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 41419 0 0 0 53441 118 0 0 25 0 5 0 1132324057 948076544 216865 4294967295 134512640 135237372 3213945952 3045692380 134636641 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2187.69
Current children cumulated vsize (KiB) 925856

[startup+642.313 s]
/proc/loadavg: 4.27 3.76 2.65 5/104 7406
/proc/meminfo: memFree=2130652/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=996084 CPUtime=2427.42
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 235322 0 0 0 242276 466 0 0 25 0 5 0 1132319448 1019990016 233619 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 249021 233619 95 177 0 248843 0
[pid=5329/tid=5519] ppid=5327 vsize=996084 CPUtime=595.5
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 18217 0 0 0 59501 49 0 0 25 0 5 0 1132324057 1019990016 233619 4294967295 134512640 135237372 3213945952 3071049696 134749337 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=996084 CPUtime=594.8
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 44941 0 0 0 59342 138 0 0 25 0 5 0 1132324057 1019990016 233619 4294967295 134512640 135237372 3213945952 3062477804 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=996084 CPUtime=595.54
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 59529 0 0 0 59428 126 0 0 25 0 5 0 1132324057 1019990016 233619 4294967295 134512640 135237372 3213945952 3054121920 134590501 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=996084 CPUtime=595.55
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 45674 0 0 0 59427 128 0 0 25 0 5 0 1132324057 1019990016 233619 4294967295 134512640 135237372 3213945952 3045696556 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2427.42
Current children cumulated vsize (KiB) 996084

[startup+702.317 s]
/proc/loadavg: 4.22 3.83 2.74 6/104 7594
/proc/meminfo: memFree=2074360/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1057784 CPUtime=2667.17
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 249620 0 0 0 266207 510 0 0 25 0 5 0 1132319448 1083170816 247917 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 264446 247917 95 177 0 264268 0
[pid=5329/tid=5519] ppid=5327 vsize=1057784 CPUtime=655.48
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 20836 0 0 0 65498 50 0 0 25 0 5 0 1132324057 1083170816 247917 4294967295 134512640 135237372 3213945952 3070909872 134680326 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=1057784 CPUtime=654.63
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 48271 0 0 0 65306 157 0 0 25 0 5 0 1132324057 1083170816 247917 4294967295 134512640 135237372 3213945952 3062571008 134590241 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=1057784 CPUtime=655.53
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 63869 0 0 0 65414 139 0 0 25 0 5 0 1132324057 1083170816 247917 4294967295 134512640 135237372 3213945952 3054122128 134680311 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=1057784 CPUtime=655.5
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 49683 0 0 0 65411 139 0 0 25 0 5 0 1132324057 1083170816 247917 4294967295 134512640 135237372 3213945952 3045789040 134590129 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2667.17
Current children cumulated vsize (KiB) 1057784

[startup+762.311 s]
/proc/loadavg: 4.20 3.89 2.83 5/104 7784
/proc/meminfo: memFree=2015272/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1118744 CPUtime=2906.85
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 264130 0 0 0 290139 546 0 0 25 0 5 0 1132319448 1145593856 262427 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 279686 262427 95 177 0 279508 0
[pid=5329/tid=5519] ppid=5327 vsize=1118744 CPUtime=715.45
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 23810 0 0 0 71492 53 0 0 25 0 5 0 1132324057 1145593856 262427 4294967295 134512640 135237372 3213945952 3071074672 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=1118744 CPUtime=714.52
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 52463 0 0 0 71285 167 0 0 25 0 5 0 1132324057 1145593856 262427 4294967295 134512640 135237372 3213945952 3062459968 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=1118744 CPUtime=715.5
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 68200 0 0 0 71393 157 0 0 25 0 5 0 1132324057 1145593856 262427 4294967295 134512640 135237372 3213945952 3054649084 134680502 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=1118744 CPUtime=715.35
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 52696 0 0 0 71391 144 0 0 25 0 5 0 1132324057 1145593856 262427 4294967295 134512640 135237372 3213945952 3046073644 134680502 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2906.85
Current children cumulated vsize (KiB) 1118744

[startup+822.305 s]
/proc/loadavg: 4.12 3.92 2.90 6/104 7972
/proc/meminfo: memFree=1956640/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1181508 CPUtime=3146.59
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 279039 0 0 0 314070 589 0 0 25 0 5 0 1132319448 1209864192 277336 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 295377 277336 95 177 0 295199 0
[pid=5329/tid=5519] ppid=5327 vsize=1181508 CPUtime=775.39
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 26618 0 0 0 77482 57 0 0 25 0 5 0 1132324057 1209864192 277336 4294967295 134512640 135237372 3213945952 3070507568 134552734 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=1181508 CPUtime=774.4
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 56750 0 0 0 77256 184 0 0 25 0 5 0 1132324057 1209864192 277336 4294967295 134512640 135237372 3213945952 3062454480 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=1181508 CPUtime=775.42
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 72510 0 0 0 77374 168 0 0 25 0 5 0 1132324057 1209864192 277336 4294967295 134512640 135237372 3213945952 3054074448 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=1181508 CPUtime=775.35
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 56200 0 0 0 77380 155 0 0 25 0 5 0 1132324057 1209864192 277336 4294967295 134512640 135237372 3213945952 3045868160 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3146.59
Current children cumulated vsize (KiB) 1181508

[startup+882.32 s]
/proc/loadavg: 4.20 3.98 2.98 6/104 8160
/proc/meminfo: memFree=1908796/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1231048 CPUtime=3386.3
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 290809 0 0 0 338005 625 0 0 25 0 5 0 1132319448 1260593152 289106 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 307762 289106 95 177 0 307584 0
[pid=5329/tid=5519] ppid=5327 vsize=1231048 CPUtime=835.35
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 28877 0 0 0 83475 60 0 0 25 0 5 0 1132324057 1260593152 289106 4294967295 134512640 135237372 3213945952 3070975776 134680273 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=1231048 CPUtime=834.24
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 59435 0 0 0 83227 197 0 0 25 0 5 0 1132324057 1260593152 289106 4294967295 134512640 135237372 3213945952 3062738732 134636641 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=1231048 CPUtime=835.42
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 75844 0 0 0 83365 177 0 0 25 0 5 0 1132324057 1260593152 289106 4294967295 134512640 135237372 3213945952 3054134048 134749353 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=1231048 CPUtime=835.26
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 59692 0 0 0 83360 166 0 0 25 0 5 0 1132324057 1260593152 289106 4294967295 134512640 135237372 3213945952 3045861868 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3386.3
Current children cumulated vsize (KiB) 1231048

[startup+942.314 s]
/proc/loadavg: 4.18 4.01 3.05 5/104 8349
/proc/meminfo: memFree=1846680/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1298520 CPUtime=3625.99
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 306524 0 0 0 361926 673 0 0 25 0 5 0 1132319448 1329684480 304821 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 324630 304821 95 177 0 324452 0
[pid=5329/tid=5519] ppid=5327 vsize=1298520 CPUtime=895.33
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 31568 0 0 0 89472 61 0 0 25 0 5 0 1132324057 1329684480 304821 4294967295 134512640 135237372 3213945952 3070388304 134562399 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=1298520 CPUtime=894.24
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 63572 0 0 0 89215 209 0 0 25 0 5 0 1132324057 1329684480 304821 4294967295 134512640 135237372 3213945952 3062574572 134636641 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=1298520 CPUtime=895.34
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 79897 0 0 0 89346 188 0 0 25 0 5 0 1132324057 1329684480 304821 4294967295 134512640 135237372 3213945952 3054169276 134749296 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=1298520 CPUtime=895.05
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 64526 0 0 0 89315 190 0 0 25 0 5 0 1132324057 1329684480 304821 4294967295 134512640 135237372 3213945952 3046143964 134636382 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3625.99
Current children cumulated vsize (KiB) 1298520



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1000.01 s]
/proc/loadavg: 4.22 4.05 3.13 7/104 8534
/proc/meminfo: memFree=1788144/3368532 swapFree=1953580/1959888
[pid=5329] ppid=5327 vsize=1359100 CPUtime=3856.43
/proc/5329/stat : 5329 (ttsth-5-0) S 5327 5329 20984 0 -1 4202496 320873 0 0 0 384932 711 0 0 25 0 5 0 1132319448 1391718400 319170 4294967295 134512640 135237372 3213945952 3213945548 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/5329/statm: 339775 319170 95 177 0 339597 0
[pid=5329/tid=5519] ppid=5327 vsize=1359100 CPUtime=953
/proc/5329/task/5519/stat : 5519 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 34041 0 0 0 95235 65 0 0 25 0 5 0 1132324057 1391718400 319170 4294967295 134512640 135237372 3213945952 3070932184 134519962 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=5329/tid=5520] ppid=5327 vsize=1359100 CPUtime=951.91
/proc/5329/task/5520/stat : 5520 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 67158 0 0 0 94972 219 0 0 25 0 5 0 1132324057 1391718400 319170 4294967295 134512640 135237372 3213945952 3062470316 134636641 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=5329/tid=5521] ppid=5327 vsize=1359100 CPUtime=953
/proc/5329/task/5521/stat : 5521 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 83912 0 0 0 95104 196 0 0 25 0 5 0 1132324057 1391718400 319170 4294967295 134512640 135237372 3213945952 3054091644 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=5329/tid=5522] ppid=5327 vsize=1359100 CPUtime=952.49
/proc/5329/task/5522/stat : 5522 (ttsth-5-0) R 5327 5329 20984 0 -1 4202560 68801 0 0 0 95043 206 0 0 25 0 5 0 1132324057 1391718400 319170 4294967295 134512640 135237372 3213945952 3045633340 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3856.43
Current children cumulated vsize (KiB) 1359100

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1000.1
CPU time (s): 3856.56
CPU user time (s): 3849.4
CPU system time (s): 7.16
CPU usage (%): 385.618
Max. virtual memory (cumulated for all children) (KiB): 1359100

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3849.4
system time used= 7.16
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 320877
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 45556
involuntary context switches= 70188

runsolver used 0.01 second user time and 0 second system time

The end

Launcher Data 

Begin job on  at 2009-05-30 11:42:30
IDJOB=1777003
IDBENCH=69651
IDSOLVER=632
FILE ID=/1777003-1243676550
PBS_JOBID=
Free space on /tmp= 15880 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (4 threads)
BENCH NAME= SAT_RACE08/cnf/mizh-md5-48-5.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1777003-1243676550/watcher-1777003-1243676550 -o /tmp/evaluation-result-1777003-1243676550/solver-1777003-1243676550 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 4 HOME/instance-1777003-1243676550.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 4

MD5SUM BENCH= 8ab50418497c0326ed1ff5d73a4e24a3
RANDOM SEED= 458776530

Linux (none) 2.6.21.5 #6 SMP Wed Jul 4 17:16:38 CEST 2007 i686 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3992.42
clflush size	: 64

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.11
clflush size	: 64

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.16
clflush size	: 64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.15
clflush size	: 64


/proc/meminfo:
MemTotal:      3368532 kB
MemFree:       3065596 kB
Buffers:         77556 kB
Cached:         124804 kB
SwapCached:       2424 kB
Active:         137604 kB
Inactive:        75436 kB
HighTotal:     2489216 kB
HighFree:      2351452 kB
LowTotal:       879316 kB
LowFree:        714144 kB
SwapTotal:     1959888 kB
SwapFree:      1953580 kB
Dirty:            6244 kB
Writeback:           0 kB
AnonPages:        9740 kB
Mapped:           4992 kB
Slab:            80488 kB
SReclaimable:    71568 kB
SUnreclaim:       8920 kB
PageTables:        540 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3644152 kB
Committed_AS:    93564 kB
VmallocTotal:   114680 kB
VmallocUsed:      1444 kB
VmallocChunk:   113072 kB

Free space on /tmp at the end= 15880 MiB
End job on  at 2009-05-30 11:59:12