Trace number 3263894

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.29 1231.82

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n28-01-S136611085.shuffled-as.sat03-1627.cnf
MD5SUM17624c574c7f9ee0eff4cc4a13e42e12
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark671.855
Satisfiable
(Un)Satisfiability was proved
Number of variables184
Number of clauses896
Sum of the clauses size3042
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 214
Number of clauses of size 3514
Number of clauses of size 4368
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3263894-1304098485.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 184
0.00/0.00	c Clauses = 896
0.00/0.00	c Literals = 3042
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 184
0.00/0.00	c Clauses = 896
0.00/0.00	c Literals = 3042
0.00/0.00	c a2
0.00/0.00	c a1
0.20/0.23	c Independent sub-problems = 1
0.20/0.23	c Sub-problem with 184 variables and 896 clauses
0.20/0.23	c using permutation method 4
0.20/0.23	c count = 3
0.20/0.23	c Ordering finished 0.230 seconds
0.20/0.23	c mapped 0.230 seconds
0.20/0.23	c permuted 0.230 seconds
0.20/0.23	c about to sort
0.20/0.23	c sorted
0.20/0.23	c clauses now 896
0.20/0.23	c init finished 0.230 seconds
0.20/0.23	c tree built 0.230 seconds
0.20/0.23	c 
0.20/0.23	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.20/0.23	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.20/0.23	c ................................ | ..........                    |                                  0.2         0.0             
0.20/0.23	c ................................ | ..........                  1 |                                  0.2         4.3             
0.20/0.23	c ................................ | ..........                  2 |                                  0.2         8.7             
0.20/0.23	c ................................ | ..........                  3 |                                  0.2        13.0             
0.20/0.23	c ................................ | ..........                  4 |                                  0.2        17.4             
0.20/0.23	c ................................ | ..........                  5 |                                  0.2        21.7             
0.20/0.26	c ................................ | ..........               1000 | 19           803                 0.2      4166.7        20259
95.65/24.49	c ................................ | ..........            1550000 | 116526       986168             96.0     16142.5     19379895
191.79/48.85	c ................................ | ..........            2684000 | 230025       1703040           192.0     13979.2     28332187
287.85/73.35	c ................................ | ..........            3907000 | 333323       2470905           288.1     13563.1     36062903
383.78/97.77	c ................................ | ..........            5082000 | 432287       3198520           384.1     13231.6     38263221
479.76/122.36	c ................................ | *.........            6001000 | 539629       3845576           480.0     12501.8     44956134
576.03/146.90	c ................................ | *.........            7094000 | 625578       4513185           576.0     12315.1     46074488
671.85/171.34	c ................................ | *.........            7982000 | 725112       5114341           672.0     11877.4     50272452
768.23/196.27	c ................................ | *.........            8990000 | 809256       5715173           768.5     11697.7     57449502
864.03/221.01	c ................................ | *.........           10073000 | 897010       6321612           864.0     11657.9     60063383
959.67/245.68	c ................................ | **........           10968000 | 978845       6887958           960.0     11424.6     60489879
1056.06/270.40	c ................................ | **........           11736000 | 1068470      7401509          1056.1     11112.8     62496160
1151.98/295.01	c ................................ | **........           12677000 | 1148836      7952921          1152.0     11003.9     64916907
1247.77/319.68	c ................................ | **........           13461000 | 1246285      8484531          1248.1     10785.1     68920041
1343.94/344.43	c ................................ | **........           14273000 | 1323967      8990785          1344.1     10619.2     69612331
1440.02/368.90	c ................................ | ***.......           15103000 | 1407869      9531559          1440.1     10487.8     70316815
1535.96/393.55	c ................................ | ***.......           15909000 | 1490351      10045447         1536.2     10356.3     76371966
1631.95/418.12	c ................................ | ***.......           16539000 | 1577092      10510475         1632.0     10133.9     77155930
1727.96/442.82	c ................................ | ***.......           17342000 | 1651794      10994194         1728.0     10035.6     77155930
1823.69/467.69	c ................................ | ***.......           18020000 | 1739843      11456541         1824.1      9879.0     79296037
1919.80/492.15	c ................................ | ****......           18652000 | 1821511      11891547         1920.0      9714.6     79416534
2015.89/516.75	c ................................ | ****......           19406000 | 1894283      12361034         2016.1      9625.5     79416534
2111.81/541.48	c ................................ | ****......           20079000 | 1969409      12793177         2112.2      9506.4     79416534
2208.06/566.00	c ................................ | ****......           20751000 | 2045751      13250626         2208.1      9397.8     79561216
2303.87/590.74	c ................................ | ****......           21430000 | 2124701      13721341         2304.0      9301.1     85314226
2399.73/615.68	c ................................ | *****.....           22243000 | 2200945      14190036         2400.1      9267.7     91671414
2496.10/640.46	c ................................ | *****.....           22990000 | 2270924      14660791         2496.4      9209.4     91671414
2591.96/665.05	c ................................ | *****.....           23537000 | 2333909      15089569         2592.2      9080.1     91671414
2687.84/689.77	c ................................ | *****.....           24153000 | 2393500      15509790         2688.1      8985.1     92945520
2783.89/714.43	c ................................ | *****.....           24770000 | 2450660      15938038         2784.0      8897.2     92991112
2879.73/739.09	c ................................ | ******....           25459000 | 2510773      16396650         2880.1      8839.6     94056759
2975.71/763.68	c ................................ | ******....           26125000 | 2564959      16829880         2976.1      8778.4     95553761
3071.95/788.33	c ................................ | ******....           26691000 | 2612209      17234555         3072.1      8688.2     95895600
3168.01/812.83	c ................................ | ******....           27153000 | 2671224      17645206         3168.1      8570.7     98934735
3263.83/837.86	c ................................ | ******....           27787000 | 2730502      18040996         3264.1      8512.9    105772929
3359.65/862.39	c ................................ | *******...           28370000 | 2788351      18411317         3360.0      8443.5    105772929
3456.05/886.90	c ................................ | *******...           28871000 | 2849402      18794719         3456.1      8353.7    105772929
3551.96/911.54	c ................................ | *******...           29302000 | 2910841      19172823         3552.1      8249.1    105772929
3647.97/935.93	c ................................ | *******...           29900000 | 2966520      19596831         3648.1      8196.0    105772929
3743.92/960.93	c ................................ | *******...           30476000 | 3023799      20015131         3744.0      8139.9    105772929
3840.05/985.62	c ................................ | ********..           31018000 | 3077417      20381869         3840.1      8077.3    106239319
3936.03/1010.32	c ................................ | ********..           31546000 | 3133119      20767302         3936.1      8014.5    106818503
4031.96/1034.82	c ................................ | ********..           32001000 | 3202288      21169209         4032.1      7936.6    109665548
4127.83/1059.57	c ................................ | ********..           32481000 | 3257058      21532922         4128.1      7868.2    110494821
4223.99/1084.01	c ................................ | ********..           32944000 | 3310222      21888665         4224.1      7799.1    112828496
4319.85/1108.66	c ................................ | *********.           33392000 | 3363706      22221899         4320.1      7729.5    115041720
4416.04/1133.02	c ................................ | *********.           33848000 | 3417322      22643104         4416.1      7664.7    117130560
4511.87/1157.45	c ................................ | *********.           34263000 | 3467133      23026410         4512.1      7593.6    118446182
4607.87/1182.46	c ................................ | *********.           34766000 | 3514417      23433378         4608.1      7544.5    118446182
4703.99/1207.01	c ................................ | *********.           35199000 | 3558874      23835637         4704.1      7482.7    118446182

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263894-1304098485/watcher-3263894-1304098485 -o /tmp/evaluation-result-3263894-1304098485/solver-3263894-1304098485 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3263894-1304098485.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.87 8.01 7.92 6/179 13066
/proc/meminfo: memFree=26557880/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=8972 CPUtime=0 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) R 13064 13066 12928 0 -1 4202496 312 0 0 0 0 0 0 0 25 0 1 0 521447187 9187328 254 33554432000 4194304 4338100 140735794161312 18446744073709551615 4295949 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13066/statm: 2243 254 151 36 0 148 0

[startup+0.015094 s]
/proc/loadavg: 7.87 8.01 7.92 6/179 13066
/proc/meminfo: memFree=26557880/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=8972 CPUtime=0.01 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) R 13064 13066 12928 0 -1 4202496 313 0 0 0 1 0 0 0 25 0 1 0 521447187 9187328 255 33554432000 4194304 4338100 140735794161312 18446744073709551615 4295793 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13066/statm: 2243 255 152 36 0 148 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 8972

[startup+0.100121 s]
/proc/loadavg: 7.87 8.01 7.92 6/179 13066
/proc/meminfo: memFree=26557880/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=8972 CPUtime=0.09 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) R 13064 13066 12928 0 -1 4202496 313 0 0 0 9 0 0 0 25 0 1 0 521447187 9187328 255 33554432000 4194304 4338100 140735794161312 18446744073709551615 4296000 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13066/statm: 2243 255 152 36 0 148 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 8972

[startup+0.301048 s]
/proc/loadavg: 7.87 8.01 7.92 6/179 13066
/proc/meminfo: memFree=26557880/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=196044 CPUtime=0.48 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 2666 0 0 0 48 0 0 0 25 0 5 0 521447187 200749056 2608 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 49011 2609 173 36 0 46916 0
Current children cumulated CPU time (s) 0.48
Current children cumulated vsize (KiB) 196044

[startup+0.700981 s]
/proc/loadavg: 7.87 8.01 7.92 6/179 13066
/proc/meminfo: memFree=26557880/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=237224 CPUtime=2.07 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 13234 0 0 0 206 1 0 0 25 0 5 0 521447187 242917376 13176 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 59306 13176 173 36 0 57211 0
Current children cumulated CPU time (s) 2.07
Current children cumulated vsize (KiB) 237224

[startup+1.50084 s]
/proc/loadavg: 7.87 8.01 7.92 9/184 13071
/proc/meminfo: memFree=26411304/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=297172 CPUtime=5.25 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 28564 0 0 0 522 3 0 0 25 0 5 0 521447187 304304128 28506 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 74293 28506 173 36 0 72198 0
[pid=13066/tid=13068] ppid=13064 vsize=297172 CPUtime=1.24 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 7775 0 0 0 124 0 0 0 25 0 5 0 521447210 304304128 28506 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254827 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=297172 CPUtime=1.25 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 7067 0 0 0 125 0 0 0 19 0 5 0 521447210 304304128 28506 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=297172 CPUtime=1.25 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 7114 0 0 0 123 2 0 0 19 0 5 0 521447210 304304128 28507 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=297172 CPUtime=1.25 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 5733 0 0 0 125 0 0 0 25 0 5 0 521447210 304304128 28507 33554432000 4194304 4338100 140735794161312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.25
Current children cumulated vsize (KiB) 297172

[startup+3.10057 s]
/proc/loadavg: 7.87 8.01 7.92 9/184 13071
/proc/meminfo: memFree=26265356/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=363248 CPUtime=11.46 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 44531 0 0 0 1131 15 0 0 25 0 5 0 521447187 371965952 44473 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 90812 44473 175 36 0 88717 0
[pid=13066/tid=13068] ppid=13064 vsize=363248 CPUtime=2.8 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 13416 0 0 0 277 3 0 0 18 0 5 0 521447210 371965952 44473 33554432000 4194304 4338100 140735794161312 18446744073709551615 242970246699 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=363248 CPUtime=2.74 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 10381 0 0 0 270 4 0 0 19 0 5 0 521447210 371965952 44473 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982362830 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=363248 CPUtime=2.82 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 10842 0 0 0 279 3 0 0 24 0 5 0 521447210 371965952 44473 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=363248 CPUtime=2.83 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 9016 0 0 0 280 3 0 0 19 0 5 0 521447210 371965952 44475 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982364315 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.46
Current children cumulated vsize (KiB) 363248

[startup+6.30102 s]
/proc/loadavg: 7.88 8.01 7.92 9/184 13071
/proc/meminfo: memFree=25988720/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=463192 CPUtime=24.14 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 68506 0 0 0 2394 20 0 0 25 0 5 0 521447187 474308608 68448 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 115798 68448 175 36 0 113703 0
[pid=13066/tid=13068] ppid=13064 vsize=463192 CPUtime=5.96 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 20958 0 0 0 591 5 0 0 25 0 5 0 521447210 474308608 68449 33554432000 4194304 4338100 140735794161312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=463192 CPUtime=5.91 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 14816 0 0 0 586 5 0 0 25 0 5 0 521447210 474308608 68449 33554432000 4194304 4338100 140735794161312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=463192 CPUtime=6.01 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 17827 0 0 0 596 5 0 0 25 0 5 0 521447210 474308608 68449 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945357 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=463192 CPUtime=6 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 14030 0 0 0 596 4 0 0 25 0 5 0 521447210 474308608 68449 33554432000 4194304 4338100 140735794161312 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.14
Current children cumulated vsize (KiB) 463192

[startup+12.7009 s]
/proc/loadavg: 7.97 8.02 7.93 9/184 13071
/proc/meminfo: memFree=25443276/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=677504 CPUtime=49.36 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 123160 0 0 0 4898 38 0 0 25 0 5 0 521447187 693764096 123102 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 169376 123102 175 36 0 167281 0
[pid=13066/tid=13068] ppid=13064 vsize=677504 CPUtime=12.23 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 38806 0 0 0 1211 12 0 0 25 0 5 0 521447210 693764096 123102 33554432000 4194304 4338100 140735794161312 18446744073709551615 4256331 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=677504 CPUtime=12.21 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 25440 0 0 0 1213 8 0 0 21 0 5 0 521447210 693764096 123102 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=677504 CPUtime=12.33 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 32442 0 0 0 1224 9 0 0 25 0 5 0 521447210 693764096 123102 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254424 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=677504 CPUtime=12.31 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 25596 0 0 0 1225 6 0 0 25 0 5 0 521447210 693764096 123102 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.36
Current children cumulated vsize (KiB) 677504

[startup+25.5007 s]
/proc/loadavg: 7.98 8.02 7.93 9/184 13072
/proc/meminfo: memFree=25006300/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=881984 CPUtime=99.99 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 175063 0 0 0 9947 52 0 0 25 0 5 0 521447187 903151616 175005 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 220496 175005 175 36 0 218401 0
[pid=13066/tid=13068] ppid=13064 vsize=881984 CPUtime=24.86 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 63159 0 0 0 2468 18 0 0 25 0 5 0 521447210 903151616 175005 33554432000 4194304 4338100 140735794161312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=881984 CPUtime=24.91 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 30401 0 0 0 2480 11 0 0 25 0 5 0 521447210 903151616 175005 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=881984 CPUtime=24.96 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 46565 0 0 0 2483 13 0 0 25 0 5 0 521447210 903151616 175005 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=881984 CPUtime=24.99 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 34062 0 0 0 2490 9 0 0 25 0 5 0 521447210 903151616 175005 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 99.99
Current children cumulated vsize (KiB) 881984

[startup+51.1003 s]
/proc/loadavg: 8.06 8.03 7.93 9/184 13073
/proc/meminfo: memFree=24429444/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=1228176 CPUtime=200.82 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 262119 0 0 0 20002 80 0 0 25 0 5 0 521447187 1257652224 262061 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 307044 262061 175 36 0 304949 0
[pid=13066/tid=13068] ppid=13064 vsize=1228176 CPUtime=49.89 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 106422 0 0 0 4959 30 0 0 25 0 5 0 521447210 1257652224 262061 33554432000 4194304 4338100 140735794161312 18446744073709551615 4257000 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=1228176 CPUtime=50.05 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 31240 0 0 0 4990 15 0 0 25 0 5 0 521447210 1257652224 262061 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=1228176 CPUtime=50.33 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 76405 0 0 0 5011 22 0 0 25 0 5 0 521447210 1257652224 262061 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=1228176 CPUtime=50.29 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 47176 0 0 0 5017 12 0 0 25 0 5 0 521447210 1257652224 262061 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 200.82
Current children cumulated vsize (KiB) 1228176

[startup+102.306 s]
/proc/loadavg: 8.03 8.03 7.93 9/184 13074
/proc/meminfo: memFree=23214240/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=1653000 CPUtime=401.75 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 366908 0 0 0 40053 122 0 0 25 0 5 0 521447187 1692672000 366850 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 413250 366850 175 36 0 411155 0
[pid=13066/tid=13068] ppid=13064 vsize=1653000 CPUtime=99.76 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 152669 0 0 0 9929 47 0 0 25 0 5 0 521447210 1692672000 366850 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=1653000 CPUtime=100.29 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 42915 0 0 0 10008 21 0 0 25 0 5 0 521447210 1692672000 366850 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254400 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=1653000 CPUtime=100.69 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 110312 0 0 0 10036 33 0 0 25 0 5 0 521447210 1692672000 366850 33554432000 4194304 4338100 140735794161312 18446744073709551615 4257956 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=1653000 CPUtime=100.72 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 60136 0 0 0 10054 18 0 0 25 0 5 0 521447210 1692672000 366850 33554432000 4194304 4338100 140735794161312 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 401.75
Current children cumulated vsize (KiB) 1653000

[startup+162.301 s]
/proc/loadavg: 7.89 7.99 7.92 9/184 13075
/proc/meminfo: memFree=22785920/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=1957088 CPUtime=636.37 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 457586 0 0 0 63475 162 0 0 25 0 5 0 521447187 2004058112 457528 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 489272 457529 175 36 0 487177 0
[pid=13066/tid=13068] ppid=13064 vsize=1957088 CPUtime=158.05 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 211224 0 0 0 15735 70 0 0 25 0 5 0 521447210 2004058112 457529 33554432000 4194304 4338100 140735794161312 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=1957088 CPUtime=158.92 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 51270 0 0 0 15866 26 0 0 25 0 5 0 521447210 2004058112 457529 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=1957088 CPUtime=159.5 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 128704 0 0 0 15908 42 0 0 25 0 5 0 521447210 2004058112 457529 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945104 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=1957088 CPUtime=159.64 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 65513 0 0 0 15941 23 0 0 25 0 5 0 521447210 2004058112 457529 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 636.37
Current children cumulated vsize (KiB) 1957088


################
# More data... #
################

/proc/meminfo: memFree=19271880/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=4165104 CPUtime=2970.54 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 974493 0 0 0 296619 435 0 0 25 0 5 0 521447187 4265066496 974435 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1041276 974435 175 36 0 1039181 0
[pid=13066/tid=13068] ppid=13064 vsize=4165104 CPUtime=736.99 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 534875 0 0 0 73502 197 0 0 25 0 5 0 521447210 4265066496 974435 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=4165104 CPUtime=742.47 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 119897 0 0 0 74177 70 0 0 25 0 5 0 521447210 4265066496 974435 33554432000 4194304 4338100 140735794161312 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=4165104 CPUtime=745.37 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 226423 0 0 0 74436 101 0 0 25 0 5 0 521447210 4265066496 974435 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945114 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=4165104 CPUtime=745.44 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 92422 0 0 0 74478 66 0 0 25 0 5 0 521447210 4265066496 974435 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945104 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2970.54
Current children cumulated vsize (KiB) 4165104

[startup+822.3 s]
/proc/loadavg: 7.84 7.94 7.91 9/184 13093
/proc/meminfo: memFree=18933476/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=4352104 CPUtime=3205.08 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1025117 0 0 0 320049 459 0 0 25 0 5 0 521447187 4456554496 1025059 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1088026 1025059 175 36 0 1085931 0
[pid=13066/tid=13068] ppid=13064 vsize=4352104 CPUtime=795.11 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 562717 0 0 0 79304 207 0 0 25 0 5 0 521447210 4456554496 1025059 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945525 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=4352104 CPUtime=801.4 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 121050 0 0 0 80068 72 0 0 25 0 5 0 521447210 4456554496 1025059 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982364315 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=4352104 CPUtime=803.98 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 228332 0 0 0 80292 106 0 0 25 0 5 0 521447210 4456554496 1025059 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=4352104 CPUtime=804.31 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 112142 0 0 0 80360 71 0 0 25 0 5 0 521447210 4456554496 1025059 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3205.08
Current children cumulated vsize (KiB) 4352104

[startup+882.301 s]
/proc/loadavg: 7.94 7.94 7.91 9/184 13095
/proc/meminfo: memFree=18624204/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=4828352 CPUtime=3438.06 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1077218 0 0 0 343317 489 0 0 25 0 5 0 521447187 4944232448 1077160 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1207088 1077160 175 36 0 1204993 0
[pid=13066/tid=13068] ppid=13064 vsize=4828352 CPUtime=852.76 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 587063 0 0 0 85059 217 0 0 25 0 5 0 521447210 4944232448 1077160 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945318 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=4828352 CPUtime=859.96 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 130581 0 0 0 85913 83 0 0 19 0 5 0 521447210 4944232448 1077160 33554432000 4194304 4338100 140735794161312 18446744073709551615 4241488 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=4828352 CPUtime=862.35 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 238922 0 0 0 86123 112 0 0 25 0 5 0 521447210 4944232448 1077160 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=4828352 CPUtime=862.72 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 119776 0 0 0 86197 75 0 0 25 0 5 0 521447210 4944232448 1077160 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945192 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3438.06
Current children cumulated vsize (KiB) 4828352

[startup+942.301 s]
/proc/loadavg: 7.90 7.93 7.90 9/184 13096
/proc/meminfo: memFree=18360204/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=4836928 CPUtime=3673.07 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1087870 0 0 0 366800 507 0 0 25 0 5 0 521447187 4953014272 1087812 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1209232 1087812 175 36 0 1207137 0
[pid=13066/tid=13068] ppid=13064 vsize=4836928 CPUtime=911.31 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 588675 0 0 0 90910 221 0 0 25 0 5 0 521447210 4953014272 1087812 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=4836928 CPUtime=918.72 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 136046 0 0 0 91785 87 0 0 25 0 5 0 521447210 4953014272 1087812 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=4836928 CPUtime=920.92 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 240763 0 0 0 91974 118 0 0 25 0 5 0 521447210 4953014272 1087812 33554432000 4194304 4338100 140735794161312 18446744073709551615 4257956 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=4836928 CPUtime=921.86 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 121510 0 0 0 92106 80 0 0 25 0 5 0 521447210 4953014272 1087812 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3673.07
Current children cumulated vsize (KiB) 4836928

[startup+1002.3 s]
/proc/loadavg: 7.70 7.89 7.89 7/184 13098
/proc/meminfo: memFree=18064316/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=4879144 CPUtime=3904.62 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1107184 0 0 0 389937 525 0 0 25 0 5 0 521447187 4996243456 1107126 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1219786 1107127 175 36 0 1217691 0
[pid=13066/tid=13068] ppid=13064 vsize=4879144 CPUtime=969.07 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 598437 0 0 0 96679 228 0 0 25 0 5 0 521447210 4996243456 1107127 33554432000 4194304 4338100 140735794161312 18446744073709551615 4235847 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=4879144 CPUtime=976.74 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 142160 0 0 0 97584 90 0 0 19 0 5 0 521447210 4996243456 1107127 33554432000 4194304 4338100 140735794161312 18446744073709551615 4240533 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=4879144 CPUtime=979.2 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 242608 0 0 0 97799 121 0 0 25 0 5 0 521447210 4996243456 1107127 33554432000 4194304 4338100 140735794161312 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=4879144 CPUtime=979.36 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 123104 0 0 0 97851 85 0 0 22 0 5 0 521447210 4996243456 1107127 33554432000 4194304 4338100 140735794161312 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3904.62
Current children cumulated vsize (KiB) 4879144

[startup+1062.3 s]
/proc/loadavg: 7.99 7.93 7.91 9/184 13099
/proc/meminfo: memFree=17745356/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=5002820 CPUtime=4138.52 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1148711 0 0 0 413305 547 0 0 25 0 5 0 521447187 5122887680 1148653 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1250705 1148653 175 36 0 1248610 0
[pid=13066/tid=13068] ppid=13064 vsize=5002820 CPUtime=1027.02 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 620798 0 0 0 102464 238 0 0 25 0 5 0 521447210 5122887680 1148653 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=5002820 CPUtime=1035.05 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 149869 0 0 0 103412 93 0 0 25 0 5 0 521447210 5122887680 1148653 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=5002820 CPUtime=1037.98 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 252584 0 0 0 103672 126 0 0 25 0 5 0 521447210 5122887680 1148653 33554432000 4194304 4338100 140735794161312 18446744073709551615 242969945534 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=5002820 CPUtime=1038.2 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 124584 0 0 0 103732 88 0 0 25 0 5 0 521447210 5122887680 1148653 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4138.52
Current children cumulated vsize (KiB) 5002820

[startup+1122.3 s]
/proc/loadavg: 7.98 7.94 7.91 9/184 13101
/proc/meminfo: memFree=17469812/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=5176412 CPUtime=4373.7 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1200903 0 0 0 436798 572 0 0 25 0 5 0 521447187 5300645888 1200845 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1294103 1200845 175 36 0 1292008 0
[pid=13066/tid=13068] ppid=13064 vsize=5176412 CPUtime=1085.85 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 648620 0 0 0 108334 251 0 0 25 0 5 0 521447210 5300645888 1200845 33554432000 4194304 4338100 140735794161312 18446744073709551615 242970246576 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=5176412 CPUtime=1093.79 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 156037 0 0 0 109283 96 0 0 25 0 5 0 521447210 5300645888 1200845 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=5176412 CPUtime=1096.58 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 269278 0 0 0 109526 132 0 0 25 0 5 0 521447210 5300645888 1200845 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=5176412 CPUtime=1097.21 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 126092 0 0 0 109630 91 0 0 25 0 5 0 521447210 5300645888 1200845 33554432000 4194304 4338100 140735794161312 18446744073709551615 4257010 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4373.7
Current children cumulated vsize (KiB) 5176412

[startup+1182.3 s]
/proc/loadavg: 8.17 8.00 7.93 9/184 13103
/proc/meminfo: memFree=17279076/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=5397260 CPUtime=4607.46 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1232947 0 0 0 460152 594 0 0 25 0 5 0 521447187 5526794240 1232889 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1349315 1232889 175 36 0 1347220 0
[pid=13066/tid=13068] ppid=13064 vsize=5397260 CPUtime=1144.01 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 650087 0 0 0 114146 255 0 0 25 0 5 0 521447210 5526794240 1232889 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=5397260 CPUtime=1152.4 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 161959 0 0 0 115140 100 0 0 25 0 5 0 521447210 5526794240 1232889 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=5397260 CPUtime=1155.17 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 292346 0 0 0 115375 142 0 0 25 0 5 0 521447210 5526794240 1232889 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=5397260 CPUtime=1155.61 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 127679 0 0 0 115466 95 0 0 25 0 5 0 521447210 5526794240 1232889 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4607.46
Current children cumulated vsize (KiB) 5397260



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1231.6 s]
/proc/loadavg: 8.01 7.97 7.91 9/184 13136
/proc/meminfo: memFree=21519240/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=5422940 CPUtime=4800.05 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) S 13064 13066 12928 0 -1 4202496 1242170 0 0 0 479396 609 0 0 25 0 5 0 521447187 5553090560 1242112 33554432000 4194304 4338100 140735794161312 18446744073709551615 242982352869 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 1355735 1242112 175 36 0 1353640 0
[pid=13066/tid=13068] ppid=13064 vsize=5422940 CPUtime=1191.85 cores=0,2,4,6
/proc/13066/task/13068/stat : 13068 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 651200 0 0 0 118927 258 0 0 25 0 5 0 521447210 5553090560 1242112 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=13066/tid=13069] ppid=13064 vsize=5422940 CPUtime=1200.8 cores=0,2,4,6
/proc/13066/task/13069/stat : 13069 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 162909 0 0 0 119976 104 0 0 25 0 5 0 521447210 5553090560 1242112 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=5422940 CPUtime=1203.16 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 298195 0 0 0 120170 146 0 0 25 0 5 0 521447210 5553090560 1242112 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=13066/tid=13071] ppid=13064 vsize=5422940 CPUtime=1203.96 cores=0,2,4,6
/proc/13066/task/13071/stat : 13071 (ttsth-5-4) R 13064 13066 12928 0 -1 4202560 128990 0 0 0 120297 99 0 0 25 0 5 0 521447210 5553090560 1242112 33554432000 4194304 4338100 140735794161312 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.05
Current children cumulated vsize (KiB) 5422940

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1231.8 s]
/proc/loadavg: 8.01 7.97 7.91 9/184 13136
/proc/meminfo: memFree=21519240/32951132 swapFree=67111364/67111528
[pid=13066] ppid=13064 vsize=0 CPUtime=4800.27 cores=0,2,4,6
/proc/13066/stat : 13066 (ttsth-5-4) Z 13064 13066 12928 0 -1 4203548 1242174 0 0 0 479399 628 0 0 18 0 2 0 521447187 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/13066/statm: 0 0 0 0 0 0 0
[pid=13066/tid=13070] ppid=13064 vsize=0 CPUtime=1203.36 cores=0,2,4,6
/proc/13066/task/13070/stat : 13070 (ttsth-5-4) R 13064 13066 12928 0 -1 4203588 298199 0 0 0 120171 165 0 0 25 0 2 0 521447210 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1231.82
CPU time (s): 4800.29
CPU user time (s): 4793.99
CPU system time (s): 6.29604
CPU usage (%): 389.692
Max. virtual memory (cumulated for all children) (KiB): 5422940

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4793.99
system time used= 6.29604
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1242174
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 79918
involuntary context switches= 19261

runsolver used 1.9747 second user time and 4.24735 second system time

The end

Launcher Data 

Begin job on node128 at 2011-04-29 19:34:45
IDJOB=3263894
IDBENCH=84030
IDSOLVER=1654
FILE ID=node128/3263894-1304098485
RUNJOBID= node128-1304097280-12946
PBS_JOBID= 13170611
Free space on /tmp= 66976 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n28-01-S136611085.shuffled-as.sat03-1627.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263894-1304098485/watcher-3263894-1304098485 -o /tmp/evaluation-result-3263894-1304098485/solver-3263894-1304098485 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3263894-1304098485.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 17624c574c7f9ee0eff4cc4a13e42e12
RANDOM SEED=1713142050

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      26558268 kB
Buffers:       1281988 kB
Cached:        4391472 kB
SwapCached:        160 kB
Active:         406224 kB
Inactive:      5400964 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      26558268 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:             256 kB
Writeback:          12 kB
AnonPages:      132820 kB
Mapped:          15504 kB
Slab:           523952 kB
PageTables:       4528 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   321236 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66972 MiB
End job on node128 at 2011-04-29 19:55:19