Trace number 1780978

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (4 threads)? (TO) 1971 1000.23

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
rpoc_xits_12_UNKNOWN.cnf
MD5SUMfbf37caab2e9eafe910c795f49053484
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1898
Number of clauses128735
Sum of the clauses size383430
Maximum clause length13
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 23005
Number of clauses of size 3125707
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 523

Solver Data 

0.11/0.27	c Command: HOME/ttsth-5-0 -num-threads 4 HOME/instance-1780978-1243684659.cnf 
0.11/0.27	c Using SATTIMEOUT of 10000
0.11/0.27	c Using SATRAM of 2047
0.11/0.27	c Size of subset node = 16
0.11/0.27	c Variables = 1898
0.11/0.27	c Clauses = 128735
0.11/0.27	c Literals = 383430
0.11/0.27	c Reduced size:
0.11/0.27	c Variables = 1898
0.11/0.27	c Clauses = 128735
0.11/0.27	c Literals = 383430
0.11/0.27	c a2
0.51/0.61	c a1
1.15/1.74	c Independent sub-problems = 1
1.15/1.74	c Sub-problem with 1898 variables and 128735 clauses
1.15/1.74	c using permutation method 4
1.15/1.74	c count = 1
1.15/1.74	c Ordering finished 1.130 seconds
1.15/1.74	c mapped 1.200 seconds
1.15/1.74	c permuted 1.200 seconds
1.15/1.74	c about to sort
1.25/1.88	c sorted
1.95/3.33	c clauses now 128735
1.95/3.33	c init finished 1.270 seconds
1.95/3.33	c tree built 1.990 seconds
1.95/3.33	c 
1.95/3.33	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
1.95/3.33	c ---------------------------------|----------------------------------|------------------------------------------------------
1.95/3.33	c ................................ | ..........                       |                        2.0           0.0             
1.95/3.33	c ................................ | ..........                     1 |                        2.0           0.5             
1.95/3.33	c ................................ | ..........                     2 |                        2.0           1.0             
1.95/3.33	c ................................ | ..........                     3 |                        2.0           1.5             
1.95/3.33	c ................................ | ..........                     4 |                        2.0           2.0             
1.95/3.33	c ................................ | ..........                     5 |                        2.0           2.5             
1.95/3.33	c ................................ | ..........                  1000 | 6                      2.0         500.0          405
200.24/106.82	c ................................ | ..........               1242000 | 104704               200.3        6201.0     20334040
400.56/208.64	c ................................ | ..........               1786000 | 142576               400.7        4457.3     34028913
600.09/308.08	c ................................ | ..........               2129000 | 155324               600.1        3547.8     44779901
800.22/409.22	c ................................ | ..........               2462000 | 166660               800.3        3076.2     54588828
1000.04/509.15	c ................................ | *.........               2769000 | 179382              1000.2        2768.5     63211803
1200.45/611.95	c ................................ | *.........               3038000 | 193093              1200.6        2530.4     72312956
1400.37/713.56	c ................................ | *.........               3313000 | 206950              1400.5        2365.5     80399267
1600.28/815.30	c ................................ | *.........               3611000 | 222211              1600.3        2256.4     87686183
1800.28/914.54	c ................................ | *.........               3918000 | 239641              1800.4        2176.1     94873225

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1780978-1243684659/watcher-1780978-1243684659 -o /tmp/evaluation-result-1780978-1243684659/solver-1780978-1243684659 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 4 HOME/instance-1780978-1243684659.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.20 7.27 7/113 22634
/proc/meminfo: memFree=2609028/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1140 CPUtime=0.01
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 181 0 0 0 0 1 0 0 18 0 1 0 1122603220 1167360 137 4294967295 134512640 135237372 3213851712 3213851160 134732614 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 285 137 75 177 0 107 0

[startup+0.030003 s]
/proc/loadavg: 8.02 8.20 7.27 7/113 22634
/proc/meminfo: memFree=2609028/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=2196 CPUtime=0.03
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 455 0 0 0 1 2 0 0 18 0 1 0 1122603220 2248704 411 4294967295 134512640 135237372 3213851712 3213850508 134639534 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 549 411 75 177 0 371 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 2196

[startup+0.109996 s]
/proc/loadavg: 8.02 8.20 7.27 7/113 22634
/proc/meminfo: memFree=2609028/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=5728 CPUtime=0.11
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 1312 0 0 0 8 3 0 0 18 0 1 0 1122603220 5865472 1268 4294967295 134512640 135237372 3213851712 3213850236 134639534 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 1432 1269 75 177 0 1254 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 5728

[startup+0.309978 s]
/proc/loadavg: 8.02 8.20 7.27 7/113 22634
/proc/meminfo: memFree=2609028/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=17624 CPUtime=0.31
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 4205 0 0 0 27 4 0 0 18 0 1 0 1122603220 18046976 4160 4294967295 134512640 135237372 3213851712 3213849936 134749339 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 4406 4160 76 177 0 4228 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 17624

[startup+0.709945 s]
/proc/loadavg: 8.02 8.20 7.27 7/113 22634
/proc/meminfo: memFree=2609028/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=17584 CPUtime=0.71
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 4525 0 0 0 67 4 0 0 18 0 1 0 1122603220 18006016 4228 4294967295 134512640 135237372 3213851712 3213850840 134730914 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 4396 4228 77 177 0 4218 0
Current children cumulated CPU time (s) 0.71
Current children cumulated vsize (KiB) 17584

[startup+1.50988 s]
/proc/loadavg: 8.02 8.20 7.27 6/114 22635
/proc/meminfo: memFree=2592168/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=19168 CPUtime=1.14
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 4955 0 0 0 110 4 0 0 19 0 1 0 1122603220 19628032 4658 4294967295 134512640 135237372 3213851712 3213849396 134852327 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 4792 4658 83 177 0 4614 0
Current children cumulated CPU time (s) 1.14
Current children cumulated vsize (KiB) 19168

[startup+3.10976 s]
/proc/loadavg: 8.02 8.20 7.27 6/114 22635
/proc/meminfo: memFree=2588024/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=30356 CPUtime=1.94
/proc/22634/stat : 22634 (ttsth-5-0) R 22632 22634 14986 0 -1 4202496 7999 0 0 0 189 5 0 0 21 0 1 0 1122603220 31084544 7450 4294967295 134512640 135237372 3213851712 3213768696 134732614 0 0 0 0 0 0 0 17 2 0 0 0
/proc/22634/statm: 7589 7450 88 177 0 7411 0
Current children cumulated CPU time (s) 1.94
Current children cumulated vsize (KiB) 30356
heavy processes:
  %CPU=84 pid=12341 uid=2256 cmd=./multiboost --fileformat arff --traintest /users/tao/kegl/Research/Samples/Isolet/isoletTrain.arff /users/tao/kegl/Research/Sam
  %CPU=84 pid=12739 uid=2256 cmd=./multiboost --fileformat arff --traintest /users/tao/kegl/Research/Samples/Isolet/isoletTrain.arff /users/tao/kegl/Research/Sam
  %CPU=84 pid=13124 uid=2256 cmd=./multiboost --fileformat arff --traintest /users/tao/kegl/Research/Samples/LetterRecognition/letterTrain.arff /users/tao/kegl/R
  %CPU=86 pid=13512 uid=2256 cmd=./multiboost --fileformat arff --traintest /users/tao/kegl/Research/Samples/LetterRecognition/letterTrain.arff /users/tao/kegl/R

[startup+6.30951 s]
/proc/loadavg: 8.02 8.19 7.27 9/118 22640
/proc/meminfo: memFree=2569216/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=78216 CPUtime=6.76
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 11581 0 0 0 670 6 0 0 21 0 5 0 1122603220 80093184 11032 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 19554 11032 95 177 0 19376 0
[pid=22634/tid=22637] ppid=22632 vsize=78216 CPUtime=1.33
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 472 0 0 0 133 0 0 0 25 0 5 0 1122603546 80093184 11032 4294967295 134512640 135237372 3213851712 3084829552 134680467 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=78216 CPUtime=1.12
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 944 0 0 0 111 1 0 0 25 0 5 0 1122603546 80093184 11032 4294967295 134512640 135237372 3213851712 3076486960 134749351 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=78216 CPUtime=1.22
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 863 0 0 0 122 0 0 0 24 0 5 0 1122603546 80093184 11032 4294967295 134512640 135237372 3213851712 3068061808 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=78216 CPUtime=1.09
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 1139 0 0 0 109 0 0 0 22 0 5 0 1122603561 80093184 11032 4294967295 134512640 135237372 3213851712 3059663696 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 6.76
Current children cumulated vsize (KiB) 78216

[startup+12.709 s]
/proc/loadavg: 8.01 8.19 7.27 8/118 22641
/proc/meminfo: memFree=2543856/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=103660 CPUtime=18.5
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 17756 0 0 0 1841 9 0 0 21 0 5 0 1122603220 106147840 17207 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 25915 17207 95 177 0 25737 0
[pid=22634/tid=22637] ppid=22632 vsize=103660 CPUtime=4.33
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 1021 0 0 0 433 0 0 0 25 0 5 0 1122603546 106147840 17207 4294967295 134512640 135237372 3213851712 3084839984 134749335 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=103660 CPUtime=4.02
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 3300 0 0 0 400 2 0 0 24 0 5 0 1122603546 106147840 17207 4294967295 134512640 135237372 3213851712 3076464496 134749353 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=103660 CPUtime=4.18
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 2200 0 0 0 417 1 0 0 25 0 5 0 1122603546 106147840 17207 4294967295 134512640 135237372 3213851712 3068092384 134546026 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=103660 CPUtime=3.97
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 3072 0 0 0 396 1 0 0 25 0 5 0 1122603561 106147840 17207 4294967295 134512640 135237372 3213851712 3059663564 134636382 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 18.5
Current children cumulated vsize (KiB) 103660

[startup+25.508 s]
/proc/loadavg: 8.08 8.20 7.29 10/118 22641
/proc/meminfo: memFree=2483392/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=164192 CPUtime=42.4
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 32478 0 0 0 4223 17 0 0 21 0 5 0 1122603220 168132608 31929 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 41048 31930 95 177 0 40870 0
[pid=22634/tid=22637] ppid=22632 vsize=164192 CPUtime=10.39
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 3306 0 0 0 1039 0 0 0 25 0 5 0 1122603546 168132608 31930 4294967295 134512640 135237372 3213851712 3084818960 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=164192 CPUtime=9.91
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 6978 0 0 0 988 3 0 0 25 0 5 0 1122603546 168132608 31930 4294967295 134512640 135237372 3213851712 3076440096 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=164192 CPUtime=10.21
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 5618 0 0 0 1017 4 0 0 25 0 5 0 1122603546 168132608 31930 4294967295 134512640 135237372 3213851712 3068075088 134549905 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=164192 CPUtime=9.89
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 8414 0 0 0 984 5 0 0 25 0 5 0 1122603561 168132608 31930 4294967295 134512640 135237372 3213851712 3059679872 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 42.4
Current children cumulated vsize (KiB) 164192

[startup+51.106 s]
/proc/loadavg: 8.01 8.18 7.31 9/118 22825
/proc/meminfo: memFree=2400500/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=250824 CPUtime=91.93
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 53117 0 0 0 9164 29 0 0 21 0 5 0 1122603220 256843776 52568 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 62706 52568 95 177 0 62528 0
[pid=22634/tid=22637] ppid=22632 vsize=250824 CPUtime=22.89
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 7122 0 0 0 2288 1 0 0 25 0 5 0 1122603546 256843776 52568 4294967295 134512640 135237372 3213851712 3084845312 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=250824 CPUtime=23.11
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 12304 0 0 0 2302 9 0 0 25 0 5 0 1122603546 256843776 52568 4294967295 134512640 135237372 3213851712 3076453632 134590245 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=250824 CPUtime=21.81
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 10084 0 0 0 2174 7 0 0 25 0 5 0 1122603546 256843776 52568 4294967295 134512640 135237372 3213851712 3068051504 134562401 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=250824 CPUtime=22.12
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 15444 0 0 0 2205 7 0 0 25 0 5 0 1122603561 256843776 52568 4294967295 134512640 135237372 3213851712 3059663568 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 91.93
Current children cumulated vsize (KiB) 250824

[startup+102.322 s]
/proc/loadavg: 8.41 8.26 7.38 10/118 23017
/proc/meminfo: memFree=2265572/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=388664 CPUtime=191.56
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 86028 0 0 0 19113 43 0 0 21 0 5 0 1122603220 397991936 85479 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 97166 85479 95 177 0 96988 0
[pid=22634/tid=22637] ppid=22632 vsize=388664 CPUtime=47.93
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 12721 0 0 0 4790 3 0 0 25 0 5 0 1122603546 397991936 85479 4294967295 134512640 135237372 3213851712 3084838288 134680254 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=388664 CPUtime=48.41
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 20605 0 0 0 4828 13 0 0 25 0 5 0 1122603546 397991936 85479 4294967295 134512640 135237372 3213851712 3076447280 134565869 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=388664 CPUtime=46.31
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 16348 0 0 0 4620 11 0 0 25 0 5 0 1122603546 397991936 85479 4294967295 134512640 135237372 3213851712 3068068048 134591139 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=388664 CPUtime=46.91
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 28191 0 0 0 4680 11 0 0 25 0 5 0 1122603561 397991936 85479 4294967295 134512640 135237372 3213851712 3059682512 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 191.56
Current children cumulated vsize (KiB) 388664

[startup+162.317 s]
/proc/loadavg: 8.52 8.30 7.45 10/118 23214
/proc/meminfo: memFree=2130984/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=525652 CPUtime=310.75
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 118935 0 0 0 31007 68 0 0 21 0 5 0 1122603220 538267648 118386 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 131413 118386 95 177 0 131235 0
[pid=22634/tid=22637] ppid=22632 vsize=525652 CPUtime=77.28
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 19097 0 0 0 7721 7 0 0 25 0 5 0 1122603546 538267648 118386 4294967295 134512640 135237372 3213851712 3084874400 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=525652 CPUtime=77.91
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 29057 0 0 0 7766 25 0 0 25 0 5 0 1122603546 538267648 118386 4294967295 134512640 135237372 3213851712 3076448016 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=525652 CPUtime=77.11
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 25249 0 0 0 7696 15 0 0 25 0 5 0 1122603546 538267648 118386 4294967295 134512640 135237372 3213851712 3068032368 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=525652 CPUtime=76.45
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 37369 0 0 0 7629 16 0 0 25 0 5 0 1122603561 538267648 118386 4294967295 134512640 135237372 3213851712 3059691104 134680282 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 310.75
Current children cumulated vsize (KiB) 525652

[startup+222.313 s]
/proc/loadavg: 8.62 8.35 7.51 9/118 23401
/proc/meminfo: memFree=2008412/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=650960 CPUtime=429.5
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 149074 0 0 0 42862 88 0 0 21 0 5 0 1122603220 666583040 148525 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 162740 148525 95 177 0 162562 0
[pid=22634/tid=22637] ppid=22632 vsize=650960 CPUtime=106.77
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 22481 0 0 0 10669 8 0 0 25 0 5 0 1122603546 666583040 148525 4294967295 134512640 135237372 3213851712 3084837424 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=650960 CPUtime=106.54
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 37384 0 0 0 10620 34 0 0 25 0 5 0 1122603546 666583040 148525 4294967295 134512640 135237372 3213851712 3076419936 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=650960 CPUtime=106.71
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 34630 0 0 0 10653 18 0 0 25 0 5 0 1122603546 666583040 148525 4294967295 134512640 135237372 3213851712 3068042832 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=650960 CPUtime=107.48
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 46416 0 0 0 10725 23 0 0 25 0 5 0 1122603561 666583040 148525 4294967295 134512640 135237372 3213851712 3059706620 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 429.5
Current children cumulated vsize (KiB) 650960

[startup+282.308 s]
/proc/loadavg: 8.54 8.36 7.57 9/118 23588
/proc/meminfo: memFree=1910544/3368532 swapFree=1953000/1959888

################
# More data... #
################

/proc/meminfo: memFree=1558016/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1121860 CPUtime=1025.9
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 260704 0 0 0 102428 162 0 0 21 0 5 0 1122603220 1148784640 260155 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 280465 260155 95 177 0 280287 0
[pid=22634/tid=22637] ppid=22632 vsize=1121860 CPUtime=255.23
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 35824 0 0 0 25511 12 0 0 25 0 5 0 1122603546 1148784640 260155 4294967295 134512640 135237372 3213851712 3084837104 134562430 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1121860 CPUtime=257.51
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 63590 0 0 0 25698 53 0 0 25 0 5 0 1122603546 1148784640 260155 4294967295 134512640 135237372 3213851712 3076431296 134562344 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1121860 CPUtime=255.81
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 60355 0 0 0 25553 28 0 0 25 0 5 0 1122603546 1148784640 260155 4294967295 134512640 135237372 3213851712 3068060832 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1121860 CPUtime=255.35
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 92772 0 0 0 25471 64 0 0 25 0 5 0 1122603561 1148784640 260155 4294967295 134512640 135237372 3213851712 3059708880 134680245 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1025.9
Current children cumulated vsize (KiB) 1121860

[startup+582.315 s]
/proc/loadavg: 8.43 8.39 7.79 11/119 24417
/proc/meminfo: memFree=1471972/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1209836 CPUtime=1143.8
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 281801 0 0 0 114199 181 0 0 21 0 5 0 1122603220 1238872064 281252 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 302459 281252 95 177 0 302281 0
[pid=22634/tid=22637] ppid=22632 vsize=1209836 CPUtime=284.73
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 38198 0 0 0 28458 15 0 0 25 0 5 0 1122603546 1238872064 281252 4294967295 134512640 135237372 3213851712 3084818672 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1209836 CPUtime=287.11
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 70112 0 0 0 28657 54 0 0 25 0 5 0 1122603546 1238872064 281252 4294967295 134512640 135237372 3213851712 3076446624 134749351 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1209836 CPUtime=285.21
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 65419 0 0 0 28493 28 0 0 25 0 5 0 1122603546 1238872064 281252 4294967295 134512640 135237372 3213851712 3068049460 134588805 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1209836 CPUtime=284.76
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 99909 0 0 0 28397 79 0 0 25 0 5 0 1122603561 1238872064 281252 4294967295 134512640 135237372 3213851712 3059669360 134591426 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1143.8
Current children cumulated vsize (KiB) 1209836

[startup+642.31 s]
/proc/loadavg: 8.64 8.45 7.84 9/118 24723
/proc/meminfo: memFree=1390632/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1294352 CPUtime=1260.2
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 301736 0 0 0 125831 189 0 0 21 0 5 0 1122603220 1325416448 301187 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 323588 301187 95 177 0 323410 0
[pid=22634/tid=22637] ppid=22632 vsize=1294352 CPUtime=313.67
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 40470 0 0 0 31348 19 0 0 25 0 5 0 1122603546 1325416448 301187 4294967295 134512640 135237372 3213851712 3084839456 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1294352 CPUtime=315.81
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 76634 0 0 0 31525 56 0 0 25 0 5 0 1122603546 1325416448 301187 4294967295 134512640 135237372 3213851712 3076446672 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1294352 CPUtime=314.87
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 69265 0 0 0 31458 29 0 0 25 0 5 0 1122603546 1325416448 301187 4294967295 134512640 135237372 3213851712 3068060272 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1294352 CPUtime=313.85
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 107204 0 0 0 31305 80 0 0 25 0 5 0 1122603561 1325416448 301187 4294967295 134512640 135237372 3213851712 3059703536 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1260.2
Current children cumulated vsize (KiB) 1294352

[startup+702.306 s]
/proc/loadavg: 8.79 8.51 7.90 10/118 24911
/proc/meminfo: memFree=1315224/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1372680 CPUtime=1378.15
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 320404 0 0 0 137612 203 0 0 21 0 5 0 1122603220 1405624320 319855 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 343170 319855 95 177 0 342992 0
[pid=22634/tid=22637] ppid=22632 vsize=1372680 CPUtime=343.27
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 43040 0 0 0 34307 20 0 0 25 0 5 0 1122603546 1405624320 319855 4294967295 134512640 135237372 3213851712 3084839984 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1372680 CPUtime=344.91
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 82356 0 0 0 34429 62 0 0 25 0 5 0 1122603546 1405624320 319855 4294967295 134512640 135237372 3213851712 3076411152 134590146 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1372680 CPUtime=344.92
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 72341 0 0 0 34461 31 0 0 25 0 5 0 1122603546 1405624320 319855 4294967295 134512640 135237372 3213851712 3068052192 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1372680 CPUtime=343.05
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 114504 0 0 0 34220 85 0 0 25 0 5 0 1122603561 1405624320 319855 4294967295 134512640 135237372 3213851712 3059664128 134562334 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1378.15
Current children cumulated vsize (KiB) 1372680

[startup+762.311 s]
/proc/loadavg: 8.71 8.52 7.94 10/119 25060
/proc/meminfo: memFree=1246524/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1443312 CPUtime=1495.74
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 337260 0 0 0 149356 218 0 0 21 0 5 0 1122603220 1477951488 336711 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 360828 336711 95 177 0 360650 0
[pid=22634/tid=22637] ppid=22632 vsize=1443312 CPUtime=372.27
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 45146 0 0 0 37207 20 0 0 25 0 5 0 1122603546 1477951488 336711 4294967295 134512640 135237372 3213851712 3084810416 134562341 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1443312 CPUtime=374.71
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 87994 0 0 0 37406 65 0 0 25 0 5 0 1122603546 1477951488 336711 4294967295 134512640 135237372 3213851712 3076430928 134562380 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1443312 CPUtime=374.61
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 75356 0 0 0 37429 32 0 0 25 0 5 0 1122603546 1477951488 336711 4294967295 134512640 135237372 3213851712 3068051408 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1443312 CPUtime=372.15
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 120601 0 0 0 37119 96 0 0 25 0 5 0 1122603561 1477951488 336711 4294967295 134512640 135237372 3213851712 3059662672 134562328 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1495.74
Current children cumulated vsize (KiB) 1443312

[startup+822.316 s]
/proc/loadavg: 8.65 8.53 7.98 9/118 25291
/proc/meminfo: memFree=1179040/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1513940 CPUtime=1615.71
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 354123 0 0 0 161348 223 0 0 21 0 5 0 1122603220 1550274560 353574 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 378485 353574 95 177 0 378307 0
[pid=22634/tid=22637] ppid=22632 vsize=1513940 CPUtime=401.83
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 47987 0 0 0 40163 20 0 0 25 0 5 0 1122603546 1550274560 353574 4294967295 134512640 135237372 3213851712 3084839632 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1513940 CPUtime=403.71
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 93389 0 0 0 40303 68 0 0 25 0 5 0 1122603546 1550274560 353574 4294967295 134512640 135237372 3213851712 3076420656 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1513940 CPUtime=406.51
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 79639 0 0 0 40618 33 0 0 25 0 5 0 1122603546 1550274560 353574 4294967295 134512640 135237372 3213851712 3068062064 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1513940 CPUtime=401.66
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 124945 0 0 0 40069 97 0 0 25 0 5 0 1122603561 1550274560 353574 4294967295 134512640 135237372 3213851712 3059671488 134562459 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1615.71
Current children cumulated vsize (KiB) 1513940

[startup+882.312 s]
/proc/loadavg: 8.53 8.52 8.01 9/118 25478
/proc/meminfo: memFree=1108124/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1585728 CPUtime=1736.16
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 371391 0 0 0 173379 237 0 0 21 0 5 0 1122603220 1623785472 370842 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 396432 370842 95 177 0 396254 0
[pid=22634/tid=22637] ppid=22632 vsize=1585728 CPUtime=431.33
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 50683 0 0 0 43112 21 0 0 25 0 5 0 1122603546 1623785472 370842 4294967295 134512640 135237372 3213851712 3084833744 134562407 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1585728 CPUtime=434.77
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 99365 0 0 0 43405 72 0 0 25 0 5 0 1122603546 1623785472 370842 4294967295 134512640 135237372 3213851712 3076441008 134562380 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1585728 CPUtime=436.41
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 83383 0 0 0 43607 34 0 0 25 0 5 0 1122603546 1623785472 370842 4294967295 134512640 135237372 3213851712 3068049808 134562309 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1585728 CPUtime=431.65
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 129797 0 0 0 43060 105 0 0 25 0 5 0 1122603561 1623785472 370842 4294967295 134512640 135237372 3213851712 3059683872 134562351 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1736.16
Current children cumulated vsize (KiB) 1585728

[startup+942.307 s]
/proc/loadavg: 8.50 8.50 8.03 11/118 25666
/proc/meminfo: memFree=1040840/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1656432 CPUtime=1855.56
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 388109 0 0 0 185308 248 0 0 21 0 5 0 1122603220 1696186368 387560 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 414108 387560 95 177 0 413930 0
[pid=22634/tid=22637] ppid=22632 vsize=1656432 CPUtime=460.81
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 53602 0 0 0 46059 22 0 0 25 0 5 0 1122603546 1696186368 387560 4294967295 134512640 135237372 3213851712 3084837824 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1656432 CPUtime=464.85
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 104053 0 0 0 46410 75 0 0 25 0 5 0 1122603546 1696186368 387560 4294967295 134512640 135237372 3213851712 3076433824 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1656432 CPUtime=466.31
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 87716 0 0 0 46594 37 0 0 25 0 5 0 1122603546 1696186368 387560 4294967295 134512640 135237372 3213851712 3068050976 134562355 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1656432 CPUtime=461.59
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 134575 0 0 0 46050 109 0 0 25 0 5 0 1122603561 1696186368 387560 4294967295 134512640 135237372 3213851712 3059678848 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1855.56
Current children cumulated vsize (KiB) 1656432



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1000.01 s]
/proc/loadavg: 8.58 8.50 8.05 12/119 25723
/proc/meminfo: memFree=970940/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=1729196 CPUtime=1970.85
/proc/22634/stat : 22634 (ttsth-5-0) S 22632 22634 14986 0 -1 4202496 405181 0 0 0 196826 259 0 0 21 0 5 0 1122603220 1770696704 404632 4294967295 134512640 135237372 3213851712 3213851308 134629544 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 432299 404632 95 177 0 432121 0
[pid=22634/tid=22637] ppid=22632 vsize=1729196 CPUtime=489.85
/proc/22634/task/22637/stat : 22637 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 55924 0 0 0 48960 25 0 0 25 0 5 0 1122603546 1770696704 404632 4294967295 134512640 135237372 3213851712 3084839536 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=22634/tid=22638] ppid=22632 vsize=1729196 CPUtime=493.51
/proc/22634/task/22638/stat : 22638 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 109519 0 0 0 49274 77 0 0 25 0 5 0 1122603546 1770696704 404632 4294967295 134512640 135237372 3213851712 3076441648 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22639] ppid=22632 vsize=1729196 CPUtime=495.16
/proc/22634/task/22639/stat : 22639 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 92333 0 0 0 49476 40 0 0 25 0 5 0 1122603546 1770696704 404632 4294967295 134512640 135237372 3213851712 3068033920 134591116 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=1729196 CPUtime=490.33
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4202560 139242 0 0 0 48921 112 0 0 25 0 5 0 1122603561 1770696704 404632 4294967295 134512640 135237372 3213851712 3059709728 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1970.85
Current children cumulated vsize (KiB) 1729196

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1000.21 s]
/proc/loadavg: 8.58 8.50 8.05 12/119 25723
/proc/meminfo: memFree=970940/3368532 swapFree=1953000/1959888
[pid=22634] ppid=22632 vsize=0 CPUtime=1970.99
/proc/22634/stat : 22634 (ttsth-5-0) Z 22632 22634 14986 0 -1 4203524 405192 0 0 0 196832 267 0 0 18 0 2 0 1122603220 0 0 4294967295 0 0 0 0 0 0 0 0 0 4294967295 0 0 17 2 0 0 0
/proc/22634/statm: 0 0 0 0 0 0 0
[pid=22634/tid=22640] ppid=22632 vsize=0 CPUtime=490.43
/proc/22634/task/22640/stat : 22640 (ttsth-5-0) R 22632 22634 14986 0 -1 4203588 139242 0 0 0 48923 120 0 0 25 0 2 0 1122603561 0 0 4294967295 0 0 0 0 0 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1970.99
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1000.23
CPU time (s): 1971
CPU user time (s): 1968.32
CPU system time (s): 2.68
CPU usage (%): 197.055
Max. virtual memory (cumulated for all children) (KiB): 1729196

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1968.32
system time used= 2.68
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 405192
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 625
involuntary context switches= 53447

runsolver used 0 second user time and 0 second system time

The end

Launcher Data 

Begin job on tipi45 at 2009-05-30 13:57:39
IDJOB=1780978
IDBENCH=71084
IDSOLVER=632
FILE ID=tipi45/1780978-1243684659
PBS_JOBID=
Free space on /tmp= 15500 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (4 threads)
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/rpoc_xits_12_UNKNOWN.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1780978-1243684659/watcher-1780978-1243684659 -o /tmp/evaluation-result-1780978-1243684659/solver-1780978-1243684659 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 4 HOME/instance-1780978-1243684659.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 4

MD5SUM BENCH= fbf37caab2e9eafe910c795f49053484
RANDOM SEED= 205734483

Linux tipi45 2.6.21.5 #6 SMP Wed Jul 4 17:16:38 CEST 2007 i686 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.081
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3992.41
clflush size	: 64

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.081
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.10
clflush size	: 64

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.081
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.16
clflush size	: 64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.081
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.14
clflush size	: 64


/proc/meminfo:
MemTotal:      3368532 kB
MemFree:       2609028 kB
Buffers:         34988 kB
Cached:         165696 kB
SwapCached:       1728 kB
Active:         592188 kB
Inactive:        88580 kB
HighTotal:     2489216 kB
HighFree:      1857932 kB
LowTotal:       879316 kB
LowFree:        751096 kB
SwapTotal:     1959888 kB
SwapFree:      1953000 kB
Dirty:            4008 kB
Writeback:           0 kB
AnonPages:      479524 kB
Mapped:           6600 kB
Slab:            65648 kB
SReclaimable:    54712 kB
SUnreclaim:      10936 kB
PageTables:       1124 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3644152 kB
Committed_AS:   567796 kB
VmallocTotal:   114680 kB
VmallocUsed:     17900 kB
VmallocChunk:    96244 kB

Free space on /tmp at the end= 15500 MiB
End job on tipi45 at 2009-05-30 14:14:22