Trace number 1780873

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (4 threads)? (MO) 1723.57 872.051

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
rbcl_xits_14_SAT.cnf
MD5SUM0678a0ef5b4345961915559db9bf689b
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark4.23
Satisfiable
(Un)Satisfiability was proved
Number of variables2220
Number of clauses148488
Sum of the clauses size442317
Maximum clause length15
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 23423
Number of clauses of size 3145042
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 523

Solver Data 

0.21/0.24	c Command: HOME/ttsth-5-0 -num-threads 4 HOME/instance-1780873-1243684116.cnf 
0.21/0.24	c Using SATTIMEOUT of 10000
0.21/0.24	c Using SATRAM of 2047
0.21/0.24	c Size of subset node = 16
0.21/0.24	c Variables = 2220
0.21/0.24	c Clauses = 148488
0.21/0.24	c Literals = 442317
0.21/0.24	c Reduced size:
0.21/0.24	c Variables = 2220
0.21/0.24	c Clauses = 148488
0.21/0.24	c Literals = 442317
0.21/0.24	c a2
0.61/0.63	c a1
1.31/1.38	c Independent sub-problems = 1
1.31/1.38	c Sub-problem with 2220 variables and 148488 clauses
1.31/1.38	c using permutation method 4
1.31/1.38	c count = 1
1.31/1.38	c Ordering finished 1.320 seconds
1.31/1.38	c mapped 1.390 seconds
1.31/1.38	c permuted 1.390 seconds
1.31/1.38	c about to sort
1.41/1.44	c sorted
2.21/2.29	c clauses now 148488
2.21/2.29	c init finished 1.460 seconds
2.21/2.29	c tree built 2.290 seconds
2.21/2.29	c 
2.21/2.29	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
2.21/2.29	c ---------------------------------|----------------------------------|------------------------------------------------------
2.21/2.29	c ................................ | ..........                       |                        2.3           0.0             
2.21/2.29	c ................................ | ..........                     1 |                        2.3           0.4             
2.21/2.29	c ................................ | ..........                     2 |                        2.3           0.9             
2.21/2.29	c ................................ | ..........                     3 |                        2.3           1.3             
2.21/2.29	c ................................ | ..........                     4 |                        2.3           1.7             
2.21/2.29	c ................................ | ..........                     5 |                        2.3           2.2             
2.21/2.30	c ................................ | ..........                  1000 |                        2.3         434.8          128
199.88/103.75	c ................................ | ..........                953000 | 51088                200.0        4764.8     23435483
400.08/205.50	c ................................ | ..........               1547000 | 100800               400.3        3864.5     41159947
599.92/306.38	c ................................ | ..........               2003000 | 144359               600.1        3337.8     54842880
800.19/407.18	c ................................ | ..........               2413000 | 183547               800.3        3015.0     67091321
1000.02/507.46	c ................................ | *.........               2769000 | 216260              1000.1        2768.6     78195382
1200.16/608.45	c ................................ | *.........               3114000 | 247850              1200.2        2594.5     93712110
1400.16/709.05	c ................................ | *.........               3427000 | 277201              1400.2        2447.5    106212019
1600.09/809.90	c ................................ | *.........               3747000 | 308869              1600.2        2341.5    116541516

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1780873-1243684116/watcher-1780873-1243684116 -o /tmp/evaluation-result-1780873-1243684116/solver-1780873-1243684116 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 4 HOME/instance-1780873-1243684116.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 6.93 6.75 5.83 5/111 27143
/proc/meminfo: memFree=2951944/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=3024 CPUtime=0.05
/proc/27143/stat : 27143 (ttsth-5-0) R 27141 27143 20065 0 -1 4202496 648 0 0 0 5 0 0 0 25 0 1 0 1133075726 3096576 603 4294967295 134512640 135237372 3219013936 3219013492 134738859 0 0 0 0 0 0 0 17 0 0 0 0
/proc/27143/statm: 756 603 75 177 0 578 0

[startup+0.148935 s]
/proc/loadavg: 6.93 6.75 5.83 5/111 27143
/proc/meminfo: memFree=2951944/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=7100 CPUtime=0.15
/proc/27143/stat : 27143 (ttsth-5-0) R 27141 27143 20065 0 -1 4202496 1626 0 0 0 14 1 0 0 25 0 1 0 1133075726 7270400 1581 4294967295 134512640 135237372 3219013936 3219013384 134732614 0 0 0 0 0 0 0 17 0 0 0 0
/proc/27143/statm: 1775 1581 75 177 0 1597 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 7100

[startup+0.208932 s]
/proc/loadavg: 6.93 6.75 5.83 5/111 27143
/proc/meminfo: memFree=2951944/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=9612 CPUtime=0.21
/proc/27143/stat : 27143 (ttsth-5-0) R 27141 27143 20065 0 -1 4202496 2219 0 0 0 19 2 0 0 25 0 1 0 1133075726 9842688 2174 4294967295 134512640 135237372 3219013936 3219012804 134884286 0 0 0 0 0 0 0 17 0 0 0 0
/proc/27143/statm: 2403 2174 75 177 0 2225 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 9612

[startup+0.308924 s]
/proc/loadavg: 6.93 6.75 5.83 5/111 27143
/proc/meminfo: memFree=2951944/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=20432 CPUtime=0.31
/proc/27143/stat : 27143 (ttsth-5-0) R 27141 27143 20065 0 -1 4202496 4843 0 0 0 27 4 0 0 25 0 1 0 1133075726 20922368 4797 4294967295 134512640 135237372 3219013936 3219012256 134749353 0 0 0 0 0 0 0 17 0 0 0 0
/proc/27143/statm: 5108 4797 76 177 0 4930 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 20432

[startup+0.708895 s]
/proc/loadavg: 6.93 6.75 5.83 5/111 27143
/proc/meminfo: memFree=2951944/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=20204 CPUtime=0.71
/proc/27143/stat : 27143 (ttsth-5-0) R 27141 27143 20065 0 -1 4202496 5140 0 0 0 67 4 0 0 25 0 1 0 1133075726 20688896 4802 4294967295 134512640 135237372 3219013936 3219013264 134555272 0 0 0 0 0 0 0 17 0 0 0 0
/proc/27143/statm: 5051 4802 77 177 0 4873 0
Current children cumulated CPU time (s) 0.71
Current children cumulated vsize (KiB) 20204

[startup+1.50883 s]
/proc/loadavg: 6.93 6.75 5.83 5/112 27144
/proc/meminfo: memFree=2932496/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=23108 CPUtime=1.51
/proc/27143/stat : 27143 (ttsth-5-0) R 27141 27143 20065 0 -1 4202496 6105 0 0 0 146 5 0 0 25 0 1 0 1133075726 23662592 5621 4294967295 134512640 135237372 3219013936 3219009048 134727552 0 0 0 0 0 0 0 17 0 0 0 0
/proc/27143/statm: 5777 5621 88 177 0 5599 0
Current children cumulated CPU time (s) 1.51
Current children cumulated vsize (KiB) 23108

[startup+3.1087 s]
/proc/loadavg: 6.93 6.75 5.83 5/112 27144
/proc/meminfo: memFree=2920952/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=75124 CPUtime=3.36
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 10692 0 0 0 330 6 0 0 25 0 5 0 1133075726 76926976 9964 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 18781 9964 95 177 0 18603 0
Current children cumulated CPU time (s) 3.36
Current children cumulated vsize (KiB) 75124

[startup+6.30843 s]
/proc/loadavg: 7.02 6.77 5.84 9/116 27148
/proc/meminfo: memFree=2900268/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=94448 CPUtime=9.17
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 15592 0 0 0 910 7 0 0 25 0 5 0 1133075726 96714752 14864 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 23612 14865 95 177 0 23434 0
[pid=27143/tid=27145] ppid=27141 vsize=94448 CPUtime=1.73
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 1000 0 0 0 173 0 0 0 25 0 5 0 1133075956 96714752 14865 4294967295 134512640 135237372 3219013936 3083426656 134591102 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=94448 CPUtime=1.79
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 1386 0 0 0 179 0 0 0 25 0 5 0 1133075956 96714752 14865 4294967295 134512640 135237372 3219013936 3074993408 134587151 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=94448 CPUtime=1.71
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 1904 0 0 0 170 1 0 0 25 0 5 0 1133075956 96714752 14865 4294967295 134512640 135237372 3219013936 3066639120 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=94448 CPUtime=1.65
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 1851 0 0 0 165 0 0 0 25 0 5 0 1133075961 96714752 14865 4294967295 134512640 135237372 3219013936 3058199376 134565869 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 9.17
Current children cumulated vsize (KiB) 94448

[startup+12.7079 s]
/proc/loadavg: 7.01 6.77 5.85 8/116 27149
/proc/meminfo: memFree=2865008/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=127364 CPUtime=21.91
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 23656 0 0 0 2183 8 0 0 25 0 5 0 1133075726 130420736 22928 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 31841 22928 95 177 0 31663 0
[pid=27143/tid=27145] ppid=27141 vsize=127364 CPUtime=5
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 2533 0 0 0 500 0 0 0 25 0 5 0 1133075956 130420736 22928 4294967295 134512640 135237372 3219013936 3083399888 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=127364 CPUtime=4.92
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 2948 0 0 0 491 1 0 0 25 0 5 0 1133075956 130420736 22928 4294967295 134512640 135237372 3219013936 3075006352 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=127364 CPUtime=4.91
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 4354 0 0 0 490 1 0 0 25 0 5 0 1133075956 130420736 22928 4294967295 134512640 135237372 3219013936 3066600944 134749337 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=127364 CPUtime=4.8
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 4369 0 0 0 480 0 0 0 25 0 5 0 1133075961 130420736 22928 4294967295 134512640 135237372 3219013936 3058195360 134680474 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 21.91
Current children cumulated vsize (KiB) 127364

[startup+25.5068 s]
/proc/loadavg: 7.01 6.78 5.87 8/116 27334
/proc/meminfo: memFree=2805920/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=186888 CPUtime=46.16
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 37989 0 0 0 4602 14 0 0 25 0 5 0 1133075726 191373312 37261 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 46722 37261 95 177 0 46544 0
[pid=27143/tid=27145] ppid=27141 vsize=186888 CPUtime=11.28
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 5092 0 0 0 1126 2 0 0 25 0 5 0 1133075956 191373312 37261 4294967295 134512640 135237372 3219013936 3083427632 134680285 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=186888 CPUtime=10.86
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 5553 0 0 0 1084 2 0 0 25 0 5 0 1133075956 191373312 37261 4294967295 134512640 135237372 3219013936 3074997260 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=186888 CPUtime=10.81
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 8648 0 0 0 1078 3 0 0 25 0 5 0 1133075956 191373312 37261 4294967295 134512640 135237372 3219013936 3066598176 134591184 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=186888 CPUtime=10.93
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 9244 0 0 0 1092 1 0 0 25 0 5 0 1133075961 191373312 37261 4294967295 134512640 135237372 3219013936 3058208096 134749353 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 46.16
Current children cumulated vsize (KiB) 186888

[startup+51.1147 s]
/proc/loadavg: 7.07 6.82 5.90 10/116 27336
/proc/meminfo: memFree=2712012/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=286764 CPUtime=95.95
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 61840 0 0 0 9561 34 0 0 25 0 5 0 1133075726 293646336 61112 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 71691 61113 95 177 0 71513 0
[pid=27143/tid=27145] ppid=27141 vsize=286764 CPUtime=24.19
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 9697 0 0 0 2409 10 0 0 25 0 5 0 1133075956 293646336 61113 4294967295 134512640 135237372 3219013936 3083430976 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=286764 CPUtime=23.18
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 11893 0 0 0 2314 4 0 0 25 0 5 0 1133075956 293646336 61113 4294967295 134512640 135237372 3219013936 3075003636 134680212 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=286764 CPUtime=23.04
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 14470 0 0 0 2293 11 0 0 25 0 5 0 1133075956 293646336 61113 4294967295 134512640 135237372 3219013936 3066605952 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=286764 CPUtime=23.25
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 16329 0 0 0 2322 3 0 0 25 0 5 0 1133075961 293646336 61113 4294967295 134512640 135237372 3219013936 3058189216 134562430 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 95.95
Current children cumulated vsize (KiB) 286764

[startup+102.31 s]
/proc/loadavg: 7.36 6.93 5.99 9/116 27524
/proc/meminfo: memFree=2553316/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=449828 CPUtime=197.11
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 100970 0 0 0 19648 63 0 0 25 0 5 0 1133075726 460623872 100242 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 112457 100242 95 177 0 112279 0
[pid=27143/tid=27145] ppid=27141 vsize=449828 CPUtime=49.73
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 15627 0 0 0 4954 19 0 0 25 0 5 0 1133075956 460623872 100242 4294967295 134512640 135237372 3219013936 3083420688 134580209 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=449828 CPUtime=48.61
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 20088 0 0 0 4851 10 0 0 25 0 5 0 1133075956 460623872 100242 4294967295 134512640 135237372 3219013936 3075011056 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=449828 CPUtime=48.06
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 26291 0 0 0 4791 15 0 0 25 0 5 0 1133075956 460623872 100242 4294967295 134512640 135237372 3219013936 3066638880 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=449828 CPUtime=48.42
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 29512 0 0 0 4829 13 0 0 25 0 5 0 1133075961 460623872 100242 4294967295 134512640 135237372 3219013936 3058208240 134561682 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 197.11
Current children cumulated vsize (KiB) 449828

[startup+162.316 s]
/proc/loadavg: 7.29 6.99 6.06 9/116 27710
/proc/meminfo: memFree=2381284/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=633964 CPUtime=315.36
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 144351 0 0 0 31450 86 0 0 25 0 5 0 1133075726 649179136 143623 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 158491 143623 95 177 0 158313 0
[pid=27143/tid=27145] ppid=27141 vsize=633964 CPUtime=79.33
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 22576 0 0 0 7907 26 0 0 25 0 5 0 1133075956 649179136 143623 4294967295 134512640 135237372 3219013936 3083414720 134680285 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=633964 CPUtime=78.32
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 28835 0 0 0 7815 17 0 0 25 0 5 0 1133075956 649179136 143623 4294967295 134512640 135237372 3219013936 3075002224 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=633964 CPUtime=77.41
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 40774 0 0 0 7720 21 0 0 25 0 5 0 1133075956 649179136 143623 4294967295 134512640 135237372 3219013936 3066612000 134562407 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=633964 CPUtime=78.01
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 42714 0 0 0 7785 16 0 0 25 0 5 0 1133075961 649179136 143623 4294967295 134512640 135237372 3219013936 3058178048 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 315.36
Current children cumulated vsize (KiB) 633964

[startup+222.321 s]
/proc/loadavg: 7.44 7.08 6.15 8/116 27898
/proc/meminfo: memFree=2232768/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=786372 CPUtime=433.16
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 180924 0 0 0 43214 102 0 0 25 0 5 0 1133075726 805244928 180196 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 196593 180196 95 177 0 196415 0
[pid=27143/tid=27145] ppid=27141 vsize=786372 CPUtime=108.6
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 32334 0 0 0 10831 29 0 0 25 0 5 0 1133075956 805244928 180196 4294967295 134512640 135237372 3219013936 3083425520 134680326 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=786372 CPUtime=108.02
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 37038 0 0 0 10779 23 0 0 25 0 5 0 1133075956 805244928 180196 4294967295 134512640 135237372 3219013936 3075006444 134562272 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=786372 CPUtime=106.76
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 48841 0 0 0 10651 25 0 0 25 0 5 0 1133075956 805244928 180196 4294967295 134512640 135237372 3219013936 3066628240 134561437 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=786372 CPUtime=107.5
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 53259 0 0 0 10731 19 0 0 25 0 5 0 1133075961 805244928 180196 4294967295 134512640 135237372 3219013936 3058213648 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 433.16
Current children cumulated vsize (KiB) 786372

[startup+282.316 s]
/proc/loadavg: 7.29 7.09 6.21 8/116 28086
/proc/meminfo: memFree=2103288/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=924932 CPUtime=552.49
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 213688 0 0 0 55131 118 0 0 25 0 5 0 1133075726 947130368 212960 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 231233 212960 95 177 0 231055 0
[pid=27143/tid=27145] ppid=27141 vsize=924932 CPUtime=138.53
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 39814 0 0 0 13822 31 0 0 25 0 5 0 1133075956 947130368 212960 4294967295 134512640 135237372 3219013936 3083414140 134680208 0 0 0 0 0 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=1873428/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1167660 CPUtime=790.67
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 270927 0 0 0 78911 156 0 0 25 0 5 0 1133075726 1195683840 270199 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 291915 270199 95 177 0 291737 0
[pid=27143/tid=27145] ppid=27141 vsize=1167660 CPUtime=198.23
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 54749 0 0 0 19782 41 0 0 25 0 5 0 1133075956 1195683840 270199 4294967295 134512640 135237372 3219013936 3083419264 134562380 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1167660 CPUtime=197.52
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 57021 0 0 0 19720 32 0 0 25 0 5 0 1133075956 1195683840 270199 4294967295 134512640 135237372 3219013936 3075006992 134562331 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1167660 CPUtime=195.88
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 73490 0 0 0 19551 37 0 0 25 0 5 0 1133075956 1195683840 270199 4294967295 134512640 135237372 3219013936 3066613584 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1167660 CPUtime=196.75
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 76215 0 0 0 19635 40 0 0 25 0 5 0 1133075961 1195683840 270199 4294967295 134512640 135237372 3219013936 3058187760 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 790.67
Current children cumulated vsize (KiB) 1167660

[startup+462.311 s]
/proc/loadavg: 7.35 7.17 6.39 8/116 28655
/proc/meminfo: memFree=1762204/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1283740 CPUtime=910.24
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 298486 0 0 0 90855 169 0 0 25 0 5 0 1133075726 1314549760 297758 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 320935 297758 95 177 0 320757 0
[pid=27143/tid=27145] ppid=27141 vsize=1283740 CPUtime=228
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 60985 0 0 0 22759 41 0 0 25 0 5 0 1133075956 1314549760 297758 4294967295 134512640 135237372 3219013936 3083407456 134562344 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1283740 CPUtime=227.43
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 62561 0 0 0 22706 37 0 0 25 0 5 0 1133075956 1314549760 297758 4294967295 134512640 135237372 3219013936 3075007616 134562459 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1283740 CPUtime=225.81
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 83243 0 0 0 22539 42 0 0 25 0 5 0 1133075956 1314549760 297758 4294967295 134512640 135237372 3219013936 3066597856 134561510 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1283740 CPUtime=226.72
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 82245 0 0 0 22629 43 0 0 25 0 5 0 1133075961 1314549760 297758 4294967295 134512640 135237372 3219013936 3058227056 134562391 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 910.24
Current children cumulated vsize (KiB) 1283740

[startup+522.306 s]
/proc/loadavg: 7.42 7.21 6.45 8/116 28851
/proc/meminfo: memFree=1654736/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1397740 CPUtime=1029.82
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 325761 0 0 0 102799 183 0 0 25 0 5 0 1133075726 1431285760 325033 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 349435 325033 95 177 0 349257 0
[pid=27143/tid=27145] ppid=27141 vsize=1397740 CPUtime=258.03
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 67544 0 0 0 25761 42 0 0 25 0 5 0 1133075956 1431285760 325033 4294967295 134512640 135237372 3219013936 3083431520 134564225 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1397740 CPUtime=257.38
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 66326 0 0 0 25694 44 0 0 25 0 5 0 1133075956 1431285760 325033 4294967295 134512640 135237372 3219013936 3075010240 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1397740 CPUtime=255.67
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 90911 0 0 0 25522 45 0 0 25 0 5 0 1133075956 1431285760 325033 4294967295 134512640 135237372 3219013936 3066640464 134562351 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1397740 CPUtime=256.45
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 91528 0 0 0 25599 46 0 0 25 0 5 0 1133075961 1431285760 325033 4294967295 134512640 135237372 3219013936 3058219888 134749353 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1029.82
Current children cumulated vsize (KiB) 1397740

[startup+582.311 s]
/proc/loadavg: 7.20 7.19 6.49 8/116 29039
/proc/meminfo: memFree=1504768/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1553932 CPUtime=1148.76
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 362962 0 0 0 114680 196 0 0 25 0 5 0 1133075726 1591226368 362234 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 388483 362234 95 177 0 388305 0
[pid=27143/tid=27145] ppid=27141 vsize=1553932 CPUtime=287.83
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 73743 0 0 0 28740 43 0 0 25 0 5 0 1133075956 1591226368 362234 4294967295 134512640 135237372 3219013936 3083429392 134591529 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1553932 CPUtime=287.02
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 71111 0 0 0 28657 45 0 0 25 0 5 0 1133075956 1591226368 362234 4294967295 134512640 135237372 3219013936 3075008832 134562360 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1553932 CPUtime=285.71
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 97743 0 0 0 28523 48 0 0 25 0 5 0 1133075956 1591226368 362234 4294967295 134512640 135237372 3219013936 3066594320 134562309 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1553932 CPUtime=285.91
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 110913 0 0 0 28537 54 0 0 25 0 5 0 1133075961 1591226368 362234 4294967295 134512640 135237372 3219013936 3058202800 134680322 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1148.76
Current children cumulated vsize (KiB) 1553932

[startup+642.316 s]
/proc/loadavg: 7.18 7.18 6.53 8/116 29228
/proc/meminfo: memFree=1376208/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1690580 CPUtime=1267.91
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 395356 0 0 0 126576 215 0 0 25 0 5 0 1133075726 1731153920 394628 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 422645 394628 95 177 0 422467 0
[pid=27143/tid=27145] ppid=27141 vsize=1690580 CPUtime=317.71
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 79692 0 0 0 31724 47 0 0 25 0 5 0 1133075956 1731153920 394628 4294967295 134512640 135237372 3219013936 3083425312 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1690580 CPUtime=316.71
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 75464 0 0 0 31624 47 0 0 25 0 5 0 1133075956 1731153920 394628 4294967295 134512640 135237372 3219013936 3074996032 134562363 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1690580 CPUtime=315.61
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 103782 0 0 0 31508 53 0 0 25 0 5 0 1133075956 1731153920 394628 4294967295 134512640 135237372 3219013936 3066595696 134562351 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1690580 CPUtime=315.59
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 126966 0 0 0 31497 62 0 0 25 0 5 0 1133075961 1731153920 394628 4294967295 134512640 135237372 3219013936 3058218320 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1267.91
Current children cumulated vsize (KiB) 1690580

[startup+702.311 s]
/proc/loadavg: 7.40 7.25 6.59 8/116 29418
/proc/meminfo: memFree=1262696/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1809216 CPUtime=1386.78
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 423300 0 0 0 138448 230 0 0 25 0 5 0 1133075726 1852637184 422572 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 452304 422572 95 177 0 452126 0
[pid=27143/tid=27145] ppid=27141 vsize=1809216 CPUtime=347.5
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 83947 0 0 0 34698 52 0 0 25 0 5 0 1133075956 1852637184 422572 4294967295 134512640 135237372 3219013936 3083424768 134549905 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1809216 CPUtime=346.49
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 78951 0 0 0 34602 47 0 0 25 0 5 0 1133075956 1852637184 422572 4294967295 134512640 135237372 3219013936 3075004512 134562437 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1809216 CPUtime=345.15
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 109848 0 0 0 34461 54 0 0 25 0 5 0 1133075956 1852637184 422572 4294967295 134512640 135237372 3219013936 3066631168 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1809216 CPUtime=345.35
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 141102 0 0 0 34464 71 0 0 25 0 5 0 1133075961 1852637184 422572 4294967295 134512640 135237372 3219013936 3058208432 134562370 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1386.78
Current children cumulated vsize (KiB) 1809216

[startup+762.316 s]
/proc/loadavg: 7.40 7.26 6.63 9/116 29606
/proc/meminfo: memFree=1157056/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=1920060 CPUtime=1505.62
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 449427 0 0 0 150320 242 0 0 25 0 5 0 1133075726 1966141440 448699 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 480015 448699 95 177 0 479837 0
[pid=27143/tid=27145] ppid=27141 vsize=1920060 CPUtime=377.13
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 89192 0 0 0 37661 52 0 0 25 0 5 0 1133075956 1966141440 448699 4294967295 134512640 135237372 3219013936 3083401328 134562463 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=1920060 CPUtime=376.14
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 82597 0 0 0 37567 47 0 0 25 0 5 0 1133075956 1966141440 448699 4294967295 134512640 135237372 3219013936 3075012784 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=1920060 CPUtime=374.91
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 115830 0 0 0 37430 61 0 0 25 0 5 0 1133075956 1966141440 448699 4294967295 134512640 135237372 3219013936 3066607872 134680367 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=1920060 CPUtime=375.15
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 152356 0 0 0 37439 76 0 0 25 0 5 0 1133075961 1966141440 448699 4294967295 134512640 135237372 3219013936 3058203376 134562459 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1505.62
Current children cumulated vsize (KiB) 1920060

[startup+822.311 s]
/proc/loadavg: 7.49 7.31 6.68 8/116 29794
/proc/meminfo: memFree=1068300/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=2016240 CPUtime=1624.63
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 471953 0 0 0 162199 264 0 0 25 0 5 0 1133075726 2064629760 471225 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 504060 471225 95 177 0 503882 0
[pid=27143/tid=27145] ppid=27141 vsize=2016240 CPUtime=406.98
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 94205 0 0 0 40639 59 0 0 25 0 5 0 1133075956 2064629760 471225 4294967295 134512640 135237372 3219013936 3083406892 134680502 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=2016240 CPUtime=405.82
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 85797 0 0 0 40526 56 0 0 25 0 5 0 1133075956 2064629760 471225 4294967295 134512640 135237372 3219013936 3075008640 134562407 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=2016240 CPUtime=404.71
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 120366 0 0 0 40409 62 0 0 25 0 5 0 1133075956 2064629760 471225 4294967295 134512640 135237372 3219013936 3066627168 134562331 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=2016240 CPUtime=404.83
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 162133 0 0 0 40402 81 0 0 25 0 5 0 1133075961 2064629760 471225 4294967295 134512640 135237372 3219013936 3058192424 134732535 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1624.63
Current children cumulated vsize (KiB) 2016240



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+871.907 s]
/proc/loadavg: 7.42 7.32 6.72 8/116 29984
/proc/meminfo: memFree=991128/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=2096816 CPUtime=1723.45
/proc/27143/stat : 27143 (ttsth-5-0) S 27141 27143 20065 0 -1 4202496 490972 0 0 0 172065 280 0 0 25 0 5 0 1133075726 2147139584 490244 4294967295 134512640 135237372 3219013936 3219013532 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 524204 490244 95 177 0 524026 0
[pid=27143/tid=27145] ppid=27141 vsize=2096816 CPUtime=431.63
/proc/27143/task/27145/stat : 27145 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 98510 0 0 0 43096 67 0 0 25 0 5 0 1133075956 2147139584 490244 4294967295 134512640 135237372 3219013936 3083412824 134551012 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27146] ppid=27141 vsize=2096816 CPUtime=430.52
/proc/27143/task/27146/stat : 27146 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 88739 0 0 0 42995 57 0 0 25 0 5 0 1133075956 2147139584 490244 4294967295 134512640 135237372 3219013936 3075015024 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=27143/tid=27147] ppid=27141 vsize=2096816 CPUtime=429.46
/proc/27143/task/27147/stat : 27147 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 124175 0 0 0 42883 63 0 0 25 0 5 0 1133075956 2147139584 490244 4294967295 134512640 135237372 3219013936 3066630240 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=2096816 CPUtime=429.55
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4202560 170096 0 0 0 42868 87 0 0 25 0 5 0 1133075961 2147139584 490244 4294967295 134512640 135237372 3219013936 3058233776 134562360 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1723.45
Current children cumulated vsize (KiB) 2096816

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+872.027 s]
/proc/loadavg: 7.42 7.32 6.72 8/116 29984
/proc/meminfo: memFree=991128/3368532 swapFree=1953788/1959888
[pid=27143] ppid=27141 vsize=0 CPUtime=1723.55
/proc/27143/stat : 27143 (ttsth-5-0) Z 27141 27143 20065 0 -1 4203524 490973 0 0 0 172067 288 0 0 18 0 2 0 1133075726 0 0 4294967295 0 0 0 0 0 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/27143/statm: 0 0 0 0 0 0 0
[pid=27143/tid=27148] ppid=27141 vsize=0 CPUtime=429.63
/proc/27143/task/27148/stat : 27148 (ttsth-5-0) R 27141 27143 20065 0 -1 4203588 170096 0 0 0 42868 95 0 0 25 0 2 0 1133075961 0 0 4294967295 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1723.55
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 872.051
CPU time (s): 1723.57
CPU user time (s): 1720.67
CPU system time (s): 2.9
CPU usage (%): 197.645
Max. virtual memory (cumulated for all children) (KiB): 2096816

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1720.67
system time used= 2.9
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 490973
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 635
involuntary context switches= 69839

runsolver used 0 second user time and 0 second system time

The end

Launcher Data 

Begin job on  at 2009-05-30 13:48:36
IDJOB=1780873
IDBENCH=71075
IDSOLVER=632
FILE ID=/1780873-1243684116
PBS_JOBID=
Free space on /tmp= 6474 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (4 threads)
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/rbcl_xits_14_SAT.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1780873-1243684116/watcher-1780873-1243684116 -o /tmp/evaluation-result-1780873-1243684116/solver-1780873-1243684116 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 4 HOME/instance-1780873-1243684116.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 4

MD5SUM BENCH= 0678a0ef5b4345961915559db9bf689b
RANDOM SEED= 1607287588

Linux (none) 2.6.21.5 #6 SMP Wed Jul 4 17:16:38 CEST 2007 i686 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3992.41
clflush size	: 64

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.12
clflush size	: 64

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.14
clflush size	: 64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.14
clflush size	: 64


/proc/meminfo:
MemTotal:      3368532 kB
MemFree:       2953868 kB
Buffers:        149124 kB
Cached:         143500 kB
SwapCached:       2256 kB
Active:         189760 kB
Inactive:       119696 kB
HighTotal:     2489216 kB
HighFree:      2328032 kB
LowTotal:       879316 kB
LowFree:        625836 kB
SwapTotal:     1959888 kB
SwapFree:      1953788 kB
Dirty:            3140 kB
Writeback:           0 kB
AnonPages:       16068 kB
Mapped:           5280 kB
Slab:            96036 kB
SReclaimable:    82764 kB
SUnreclaim:      13272 kB
PageTables:        704 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3644152 kB
Committed_AS:   112684 kB
VmallocTotal:   114680 kB
VmallocUsed:      1444 kB
VmallocChunk:   113072 kB

Free space on /tmp at the end= 6474 MiB
End job on  at 2009-05-30 14:03:10