Trace number 3265610

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.5 1202.2

General information on the benchmark

NameSAT05/crafted/sorge05/QG7a/
gensys-brn002.shuffled-as.sat05-3935.cnf
MD5SUMeda2d1e7d2975da4e019beddc323767b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.366944
Satisfiable
(Un)Satisfiability was proved
Number of variables2401
Number of clauses15897
Sum of the clauses size111804
Maximum clause length13
Minimum clause length1
Number of clauses of size 135
Number of clauses of size 2385
Number of clauses of size 32205
Number of clauses of size 4294
Number of clauses of size 5441
Number of clauses of size over 512537

Solver Data 

0.00/0.03	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265610-1304102391.cnf 
0.00/0.03	c Using SATTIMEOUT of 4800
0.00/0.03	c Using SATRAM of 15500
0.00/0.03	c Size of subset node = 32
0.00/0.03	c Variables = 2401
0.00/0.03	c Clauses = 15897
0.00/0.03	c Literals = 111804
0.00/0.03	c Reduced size:
0.00/0.03	c Variables = 2401
0.00/0.03	c Clauses = 15393
0.00/0.03	c Literals = 108532
0.00/0.03	c a2
0.09/0.15	c a1
0.29/0.30	c Independent sub-problems = 1
0.29/0.30	c Sub-problem with 2345 variables and 15393 clauses
0.29/0.30	c using permutation method 4
0.29/0.30	c count = 1
0.29/0.30	c Ordering finished 0.290 seconds
0.29/0.30	c mapped 0.290 seconds
0.29/0.30	c permuted 0.290 seconds
0.29/0.30	c about to sort
0.29/0.31	c sorted
0.29/0.39	c clauses now 15393
0.29/0.39	c init finished 0.330 seconds
0.29/0.39	c tree built 0.390 seconds
0.29/0.39	c 
0.29/0.39	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.39	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.39	c ................................ | ..........                    |                                  0.4         0.0             
0.29/0.39	c ................................ | ..........                  1 |                                  0.4         2.6             
0.29/0.39	c ................................ | ..........                  2 |                                  0.4         5.1             
0.29/0.39	c ................................ | ..........                  3 |                                  0.4         7.7             
0.29/0.39	c ................................ | ..........                  4 |                                  0.4        10.3             
0.29/0.39	c ................................ | ..........                  5 |                                  0.4        12.8             
0.40/0.40	c ................................ | ..........               1000 |              825                 0.4      2500.0         1701
95.90/24.33	c ................................ | ..........            3495000 | 225992       1673210            96.0     36406.2      3994245
191.74/48.36	c ................................ | ..........            5605000 | 381255       2924471           192.0     29191.2      6203849
288.01/72.42	c ................................ | ..........            6898000 | 517969       3939930           288.1     23943.9      7420817
383.87/96.45	c ................................ | ..........            7964000 | 645479       4810012           384.1     20735.8      8579799
479.74/120.46	c ................................ | *.........            8976000 | 767998       5653576           480.0     18700.0      9586122
576.01/144.55	c ................................ | *.........            9938000 | 886135       6442817           576.2     17246.9     10461774
671.87/168.54	c ................................ | *.........           10945000 | 1004844      7115143           672.0     16286.5     11717852
767.76/192.59	c ................................ | *.........           11932000 | 1108038      7805406           768.1     15534.4     13505687
864.01/216.61	c ................................ | *.........           12981000 | 1207935      8516970           864.1     15023.1     14309117
959.88/240.66	c ................................ | **........           13597000 | 1320126      9234863           960.1     14161.6     15771049
1055.74/264.69	c ................................ | **........           14185000 | 1428875      9870337          1056.1     13431.5     16184036
1152.00/288.77	c ................................ | **........           14686000 | 1528065      10434470         1152.3     12745.3     16883336
1247.85/312.76	c ................................ | **........           15274000 | 1607284      10899786         1248.1     12237.9     18446674
1343.75/336.79	c ................................ | **........           15885000 | 1686725      11281061         1344.1     11818.4     20047370
1440.01/360.82	c ................................ | ***.......           16502000 | 1774071      11677397         1440.1     11459.1     21199590
1535.87/384.84	c ................................ | ***.......           17087000 | 1846384      12052557         1536.0     11124.2     22294260
1632.14/408.91	c ................................ | ***.......           17995000 | 1934832      12421196         1632.2     11025.0     23092681
1728.00/432.91	c ................................ | ***.......           18589000 | 2008710      12829260         1728.0     10757.2     23683968
1823.86/456.93	c ................................ | ***.......           19094000 | 2077251      13221495         1824.0     10468.1     24112324
1919.72/480.99	c ................................ | ****......           19550000 | 2141785      13535431         1920.1     10181.9     24827706
2016.01/505.01	c ................................ | ****......           20205000 | 2213431      13861802         2016.1     10022.0     25527080
2111.86/529.06	c ................................ | ****......           20697000 | 2280041      14190361         2112.1      9799.2     26234147
2207.70/553.09	c ................................ | ****......           21241000 | 2346139      14498839         2208.1      9619.6     27319495
2303.94/577.15	c ................................ | ****......           21885000 | 2416800      14772723         2304.1      9498.1     28073401
2399.80/601.15	c ................................ | *****.....           22420000 | 2486703      15145978         2400.0      9341.7     28572659
2495.65/625.18	c ................................ | *****.....           22901000 | 2570303      15488355         2496.0      9175.1     28817327
2591.92/649.22	c ................................ | *****.....           23365000 | 2644694      15790559         2592.0      9014.2     28858452
2687.81/673.27	c ................................ | *****.....           23798000 | 2718363      16076376         2688.1      8853.1     29336520
2784.05/697.32	c ................................ | *****.....           24265000 | 2784299      16395372         2784.2      8715.4     29565241
2880.31/721.44	c ................................ | ******....           24628000 | 2848720      16774851         2880.5      8549.9     30501373
2976.16/745.43	c ................................ | ******....           25452000 | 2933153      17159902         2976.3      8551.6     31377271
3072.01/769.40	c ................................ | ******....           26036000 | 2993872      17500938         3072.0      8475.2     32550482
3167.86/793.46	c ................................ | ******....           26599000 | 3061157      17840746         3168.1      8395.9     33427248
3264.14/817.56	c ................................ | ******....           26983000 | 3120804      18120797         3264.4      8265.8     34050949
3360.02/841.55	c ................................ | *******...           27346000 | 3172638      18353065         3360.2      8138.1     34889845
3455.88/865.57	c ................................ | *******...           27896000 | 3226884      18563405         3456.2      8071.3     35200928
3551.75/889.56	c ................................ | *******...           28296000 | 3282102      18760286         3552.0      7966.2     35491079
3648.02/913.64	c ................................ | *******...           28700000 | 3340722      18965734         3648.2      7867.0     36232891
3744.27/937.74	c ................................ | *******...           28975000 | 3387057      19144751         3744.5      7738.1     36662799
3840.12/961.70	c ................................ | ********..           29296000 | 3435216      19341159         3840.2      7628.8     37169264
3936.02/985.77	c ................................ | ********..           29634000 | 3483718      19531803         3936.3      7528.4     37734206
4031.87/1009.77	c ................................ | ********..           29959000 | 3527697      19714294         4032.2      7430.0     38016661
4128.11/1033.85	c ................................ | ********..           30203000 | 3568724      19899517         4128.3      7316.0     38525567
4224.37/1057.90	c ................................ | ********..           30476000 | 3610717      20070453         4224.4      7214.3     39009341
4320.23/1081.91	c ................................ | *********.           30758000 | 3659589      20256461         4320.3      7119.4     39414529
4415.69/1105.89	c ................................ | *********.           31027000 | 3705031      20455474         4416.1      7025.9     40116205
4511.95/1129.95	c ................................ | *********.           31344000 | 3754472      20658223         4512.2      6946.5     40861622
4608.25/1154.00	c ................................ | *********.           31604000 | 3799329      20846921         4608.2      6858.1     41740668
4703.71/1177.98	c ................................ | *********.           31821000 | 3841055      21025803         4704.0      6764.6     42441371

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265610-1304102391/watcher-3265610-1304102391 -o /tmp/evaluation-result-3265610-1304102391/solver-3265610-1304102391 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265610-1304102391.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.35 6.79 7.61 2/172 20401
/proc/meminfo: memFree=17906860/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=9104 CPUtime=0 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) R 20399 20401 19929 0 -1 4202496 331 0 0 0 0 0 0 0 21 0 1 0 521841715 9322496 273 33554432000 4194304 4338100 140735332321520 18446744073709551615 216634515200 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/20401/statm: 2276 273 147 36 0 181 0

[startup+0.105413 s]
/proc/loadavg: 4.35 6.79 7.61 2/172 20401
/proc/meminfo: memFree=17906860/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=24632 CPUtime=0.09 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) R 20399 20401 19929 0 -1 4202496 3831 0 0 0 9 0 0 0 22 0 1 0 521841715 25223168 3772 33554432000 4194304 4338100 140735332321520 18446744073709551615 4206990 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/20401/statm: 6158 3772 152 36 0 4063 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24632

[startup+0.2004 s]
/proc/loadavg: 4.35 6.79 7.61 2/172 20401
/proc/meminfo: memFree=17906860/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=24672 CPUtime=0.19 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) R 20399 20401 19929 0 -1 4202496 3893 0 0 0 19 0 0 0 22 0 1 0 521841715 25264128 3802 33554432000 4194304 4338100 140735332321520 18446744073709551615 4296793 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/20401/statm: 6168 3802 154 36 0 4073 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 24672

[startup+0.300381 s]
/proc/loadavg: 4.35 6.79 7.61 2/172 20401
/proc/meminfo: memFree=17906860/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=24672 CPUtime=0.29 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) R 20399 20401 19929 0 -1 4202496 3894 0 0 0 29 0 0 0 22 0 1 0 521841715 25264128 3803 33554432000 4194304 4338100 140735332321520 18446744073709551615 4297822 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/20401/statm: 6168 3803 155 36 0 4073 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 24672

[startup+0.700276 s]
/proc/loadavg: 4.35 6.79 7.61 2/172 20401
/proc/meminfo: memFree=17906860/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=239012 CPUtime=1.6 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 14141 0 0 0 158 2 0 0 22 0 5 0 521841715 244748288 12603 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 59753 12603 175 36 0 57658 0
Current children cumulated CPU time (s) 1.6
Current children cumulated vsize (KiB) 239012

[startup+1.50114 s]
/proc/loadavg: 4.35 6.79 7.61 10/186 20416
/proc/meminfo: memFree=17788276/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=240332 CPUtime=4.8 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 16753 0 0 0 477 3 0 0 22 0 5 0 521841715 246099968 15215 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 60083 15215 175 36 0 57988 0
[pid=20401/tid=20409] ppid=20399 vsize=240332 CPUtime=1.1 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 914 0 0 0 110 0 0 0 19 0 5 0 521841754 246099968 15215 33554432000 4194304 4338100 140735332321520 18446744073709551615 4241252 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=240332 CPUtime=1.1 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 931 0 0 0 110 0 0 0 19 0 5 0 521841754 246099968 15216 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=240332 CPUtime=1.09 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 1027 0 0 0 109 0 0 0 25 0 5 0 521841754 246099968 15216 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910381 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=240332 CPUtime=1.1 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 1062 0 0 0 110 0 0 0 19 0 5 0 521841754 246099968 15216 33554432000 4194304 4338100 140735332321520 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4.8
Current children cumulated vsize (KiB) 240332

[startup+3.10086 s]
/proc/loadavg: 4.64 6.81 7.61 9/186 20416
/proc/meminfo: memFree=17764724/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=257424 CPUtime=11.19 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 20831 0 0 0 1116 3 0 0 22 0 5 0 521841715 263602176 19293 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 64356 19293 175 36 0 62261 0
[pid=20401/tid=20409] ppid=20399 vsize=257424 CPUtime=2.7 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 1811 0 0 0 270 0 0 0 23 0 5 0 521841754 263602176 19293 33554432000 4194304 4338100 140735332321520 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=257424 CPUtime=2.7 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 1999 0 0 0 270 0 0 0 23 0 5 0 521841754 263602176 19293 33554432000 4194304 4338100 140735332321520 18446744073709551615 4256352 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=257424 CPUtime=2.68 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 1856 0 0 0 268 0 0 0 25 0 5 0 521841754 263602176 19293 33554432000 4194304 4338100 140735332321520 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=257424 CPUtime=2.7 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 2345 0 0 0 270 0 0 0 23 0 5 0 521841754 263602176 19293 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.19
Current children cumulated vsize (KiB) 257424

[startup+6.30029 s]
/proc/loadavg: 4.64 6.81 7.61 9/186 20416
/proc/meminfo: memFree=17704460/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=281712 CPUtime=23.97 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 27914 0 0 0 2391 6 0 0 22 0 5 0 521841715 288473088 26376 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 70428 26376 175 36 0 68333 0
[pid=20401/tid=20409] ppid=20399 vsize=281712 CPUtime=5.89 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 3873 0 0 0 588 1 0 0 25 0 5 0 521841754 288473088 26377 33554432000 4194304 4338100 140735332321520 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=281712 CPUtime=5.9 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 3573 0 0 0 590 0 0 0 25 0 5 0 521841754 288473088 26377 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910718 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=281712 CPUtime=5.87 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 3632 0 0 0 587 0 0 0 25 0 5 0 521841754 288473088 26377 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633911034 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=281712 CPUtime=5.88 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 4017 0 0 0 587 1 0 0 25 0 5 0 521841754 288473088 26377 33554432000 4194304 4338100 140735332321520 18446744073709551615 4258005 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 23.97
Current children cumulated vsize (KiB) 281712

[startup+12.7012 s]
/proc/loadavg: 5.16 6.85 7.62 9/186 20417
/proc/meminfo: memFree=17602528/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=317344 CPUtime=49.54 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 38244 0 0 0 4942 12 0 0 22 0 5 0 521841715 324960256 36706 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 79336 36706 176 36 0 77241 0
[pid=20401/tid=20409] ppid=20399 vsize=317344 CPUtime=12.3 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 7607 0 0 0 1225 5 0 0 25 0 5 0 521841754 324960256 36706 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=317344 CPUtime=12.3 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 5129 0 0 0 1229 1 0 0 25 0 5 0 521841754 324960256 36706 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=317344 CPUtime=12.28 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 6817 0 0 0 1227 1 0 0 25 0 5 0 521841754 324960256 36706 33554432000 4194304 4338100 140735332321520 18446744073709551615 4256091 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=317344 CPUtime=12.24 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 5871 0 0 0 1223 1 0 0 25 0 5 0 521841754 324960256 36706 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910319 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.54
Current children cumulated vsize (KiB) 317344

[startup+25.5009 s]
/proc/loadavg: 5.59 6.89 7.62 9/186 20417
/proc/meminfo: memFree=17439312/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=376972 CPUtime=100.67 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 54987 0 0 0 10046 21 0 0 22 0 5 0 521841715 386019328 53449 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 94243 53449 176 36 0 92148 0
[pid=20401/tid=20409] ppid=20399 vsize=376972 CPUtime=25.1 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 12968 0 0 0 2502 8 0 0 25 0 5 0 521841754 386019328 53449 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=376972 CPUtime=25.1 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 9216 0 0 0 2507 3 0 0 25 0 5 0 521841754 386019328 53449 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646329499 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=376972 CPUtime=25.08 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 10982 0 0 0 2504 4 0 0 25 0 5 0 521841754 386019328 53449 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=376972 CPUtime=24.98 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 9001 0 0 0 2495 3 0 0 25 0 5 0 521841754 386019328 53449 33554432000 4194304 4338100 140735332321520 18446744073709551615 4287100 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.67
Current children cumulated vsize (KiB) 376972

[startup+51.1004 s]
/proc/loadavg: 6.41 6.98 7.63 9/186 20418
/proc/meminfo: memFree=17180516/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=464884 CPUtime=202.92 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 81322 0 0 0 20255 37 0 0 22 0 5 0 521841715 476041216 79784 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 116221 79784 176 36 0 114126 0
[pid=20401/tid=20409] ppid=20399 vsize=464884 CPUtime=50.69 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 21126 0 0 0 5055 14 0 0 25 0 5 0 521841754 476041216 79784 33554432000 4194304 4338100 140735332321520 18446744073709551615 216634211760 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=464884 CPUtime=50.69 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 16410 0 0 0 5063 6 0 0 25 0 5 0 521841754 476041216 79784 33554432000 4194304 4338100 140735332321520 18446744073709551615 4257956 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=464884 CPUtime=50.57 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 16455 0 0 0 5050 7 0 0 25 0 5 0 521841754 476041216 79784 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=464884 CPUtime=50.55 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 14511 0 0 0 5049 6 0 0 25 0 5 0 521841754 476041216 79784 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.92
Current children cumulated vsize (KiB) 464884

[startup+102.3 s]
/proc/loadavg: 7.42 7.16 7.65 9/186 20419
/proc/meminfo: memFree=16785408/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=557696 CPUtime=407.46 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 111496 0 0 0 40686 60 0 0 22 0 5 0 521841715 571080704 109958 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 139424 109958 176 36 0 137329 0
[pid=20401/tid=20409] ppid=20399 vsize=557696 CPUtime=101.89 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 29623 0 0 0 10168 21 0 0 25 0 5 0 521841754 571080704 109959 33554432000 4194304 4338100 140735332321520 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=557696 CPUtime=101.9 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 27525 0 0 0 10176 14 0 0 25 0 5 0 521841754 571080704 109959 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254840 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=557696 CPUtime=101.49 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 20438 0 0 0 10138 11 0 0 25 0 5 0 521841754 571080704 109959 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=557696 CPUtime=101.76 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 21091 0 0 0 10166 10 0 0 25 0 5 0 521841754 571080704 109959 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.46
Current children cumulated vsize (KiB) 557696

[startup+162.301 s]
/proc/loadavg: 7.79 7.31 7.67 9/186 20421
/proc/meminfo: memFree=16412272/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=644424 CPUtime=647.11 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 141238 0 0 0 64623 88 0 0 22 0 5 0 521841715 659890176 139700 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 161106 139700 176 36 0 159011 0
[pid=20401/tid=20409] ppid=20399 vsize=644424 CPUtime=161.89 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 39157 0 0 0 16158 31 0 0 25 0 5 0 521841754 659890176 139700 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910319 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=644424 CPUtime=161.89 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 36111 0 0 0 16166 23 0 0 25 0 5 0 521841754 659890176 139700 33554432000 4194304 4338100 140735332321520 18446744073709551615 4256087 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=644424 CPUtime=161.15 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 27401 0 0 0 16099 16 0 0 25 0 5 0 521841754 659890176 139700 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=644424 CPUtime=161.75 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 25749 0 0 0 16161 14 0 0 25 0 5 0 521841754 659890176 139700 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 647.11
Current children cumulated vsize (KiB) 644424


################
# More data... #
################

[pid=20401/tid=20410] ppid=20399 vsize=1449276 CPUtime=701.07 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 84201 0 0 0 70049 58 0 0 25 0 5 0 521841754 1484058624 340649 33554432000 4194304 4338100 140735332321520 18446744073709551615 4287287 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1449276 CPUtime=700.61 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 46066 0 0 0 70024 37 0 0 25 0 5 0 521841754 1484058624 340649 33554432000 4194304 4338100 140735332321520 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1449276 CPUtime=701.39 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 118557 0 0 0 70075 64 0 0 25 0 5 0 521841754 1484058624 340649 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2804.02
Current children cumulated vsize (KiB) 1449276

[startup+762.3 s]
/proc/loadavg: 8.17 7.97 7.85 9/186 20437
/proc/meminfo: memFree=14193500/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1538260 CPUtime=3043.67 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 367461 0 0 0 304103 264 0 0 22 0 5 0 521841715 1575178240 365923 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 384565 365923 176 36 0 382470 0
[pid=20401/tid=20409] ppid=20399 vsize=1538260 CPUtime=760.46 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 85764 0 0 0 75959 87 0 0 25 0 5 0 521841754 1575178240 365923 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1538260 CPUtime=760.8 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 93928 0 0 0 76017 63 0 0 25 0 5 0 521841754 1575178240 365923 33554432000 4194304 4338100 140735332321520 18446744073709551615 4239500 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1538260 CPUtime=760.61 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 47338 0 0 0 76021 40 0 0 25 0 5 0 521841754 1575178240 365923 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1538260 CPUtime=761.38 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 127611 0 0 0 76068 70 0 0 25 0 5 0 521841754 1575178240 365923 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3043.67
Current children cumulated vsize (KiB) 1538260

[startup+822.301 s]
/proc/loadavg: 8.16 7.99 7.86 9/186 20438
/proc/meminfo: memFree=14038224/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1665644 CPUtime=3283.31 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 386082 0 0 0 328053 278 0 0 22 0 5 0 521841715 1705619456 384544 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 416411 384544 176 36 0 414316 0
[pid=20401/tid=20409] ppid=20399 vsize=1665644 CPUtime=820.45 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 89849 0 0 0 81954 91 0 0 25 0 5 0 521841754 1705619456 384544 33554432000 4194304 4338100 140735332321520 18446744073709551615 4286315 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1665644 CPUtime=820.46 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 98994 0 0 0 81979 67 0 0 25 0 5 0 521841754 1705619456 384544 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1665644 CPUtime=820.61 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 48343 0 0 0 82019 42 0 0 25 0 5 0 521841754 1705619456 384544 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1665644 CPUtime=821.37 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 136076 0 0 0 82062 75 0 0 25 0 5 0 521841754 1705619456 384544 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3283.31
Current children cumulated vsize (KiB) 1665644

[startup+882.306 s]
/proc/loadavg: 8.18 8.02 7.87 9/186 20440
/proc/meminfo: memFree=13875892/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1775804 CPUtime=3523.01 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 399114 0 0 0 352012 289 0 0 22 0 5 0 521841715 1818423296 397576 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 443951 397576 176 36 0 441856 0
[pid=20401/tid=20409] ppid=20399 vsize=1775804 CPUtime=880.13 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 92091 0 0 0 87918 95 0 0 25 0 5 0 521841754 1818423296 397576 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1775804 CPUtime=880.46 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 107386 0 0 0 87976 70 0 0 25 0 5 0 521841754 1818423296 397576 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1775804 CPUtime=880.6 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 49091 0 0 0 88017 43 0 0 25 0 5 0 521841754 1818423296 397576 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1775804 CPUtime=881.37 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 137726 0 0 0 88061 76 0 0 25 0 5 0 521841754 1818423296 397576 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3523.01
Current children cumulated vsize (KiB) 1775804

[startup+942.301 s]
/proc/loadavg: 8.24 8.06 7.89 9/186 20441
/proc/meminfo: memFree=13721496/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1817168 CPUtime=3762.64 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 412043 0 0 0 375964 300 0 0 22 0 5 0 521841715 1860780032 410505 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 454292 410505 176 36 0 452197 0
[pid=20401/tid=20409] ppid=20399 vsize=1817168 CPUtime=939.81 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 95711 0 0 0 93881 100 0 0 25 0 5 0 521841754 1860780032 410505 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1817168 CPUtime=940.45 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 112329 0 0 0 93972 73 0 0 25 0 5 0 521841754 1860780032 410505 33554432000 4194304 4338100 140735332321520 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1817168 CPUtime=940.59 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 49620 0 0 0 94014 45 0 0 25 0 5 0 521841754 1860780032 410505 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1817168 CPUtime=941.37 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 141563 0 0 0 94058 79 0 0 25 0 5 0 521841754 1860780032 410505 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3762.64
Current children cumulated vsize (KiB) 1817168

[startup+1002.3 s]
/proc/loadavg: 8.14 8.06 7.91 9/186 20443
/proc/meminfo: memFree=13614584/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1861528 CPUtime=4002.3 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 424297 0 0 0 399919 311 0 0 22 0 5 0 521841715 1906204672 422759 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 465382 422759 176 36 0 463287 0
[pid=20401/tid=20409] ppid=20399 vsize=1861528 CPUtime=999.48 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 99070 0 0 0 99845 103 0 0 25 0 5 0 521841754 1906204672 422759 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1861528 CPUtime=1000.45 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 117901 0 0 0 99968 77 0 0 25 0 5 0 521841754 1906204672 422759 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1861528 CPUtime=1000.59 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 50023 0 0 0 100013 46 0 0 25 0 5 0 521841754 1906204672 422759 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1861528 CPUtime=1001.37 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 144483 0 0 0 100055 82 0 0 25 0 5 0 521841754 1906204672 422759 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4002.3
Current children cumulated vsize (KiB) 1861528

[startup+1062.3 s]
/proc/loadavg: 8.11 8.06 7.91 9/186 20445
/proc/meminfo: memFree=13524656/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1895844 CPUtime=4241.95 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 434936 0 0 0 423875 320 0 0 22 0 5 0 521841715 1941344256 433398 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 473961 433398 176 36 0 471866 0
[pid=20401/tid=20409] ppid=20399 vsize=1895844 CPUtime=1059.15 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 101487 0 0 0 105809 106 0 0 25 0 5 0 521841754 1941344256 433398 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1895844 CPUtime=1060.42 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 123234 0 0 0 105962 80 0 0 25 0 5 0 521841754 1941344256 433398 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1895844 CPUtime=1060.58 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 50688 0 0 0 106011 47 0 0 25 0 5 0 521841754 1941344256 433398 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1895844 CPUtime=1061.37 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 146707 0 0 0 106054 83 0 0 25 0 5 0 521841754 1941344256 433398 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4241.95
Current children cumulated vsize (KiB) 1895844

[startup+1122.3 s]
/proc/loadavg: 8.14 8.08 7.93 9/186 20446
/proc/meminfo: memFree=13385124/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=1946592 CPUtime=4481.59 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 449341 0 0 0 447829 330 0 0 22 0 5 0 521841715 1993310208 447803 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 486648 447803 176 36 0 484553 0
[pid=20401/tid=20409] ppid=20399 vsize=1946592 CPUtime=1118.82 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 105441 0 0 0 111772 110 0 0 25 0 5 0 521841754 1993310208 447803 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910352 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=1946592 CPUtime=1120.41 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 127673 0 0 0 111957 84 0 0 25 0 5 0 521841754 1993310208 447803 33554432000 4194304 4338100 140735332321520 18446744073709551615 4289086 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=1946592 CPUtime=1120.58 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 51369 0 0 0 112011 47 0 0 25 0 5 0 521841754 1993310208 447803 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=1946592 CPUtime=1121.36 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 152038 0 0 0 112051 85 0 0 25 0 5 0 521841754 1993310208 447803 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254294 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4481.59
Current children cumulated vsize (KiB) 1946592

[startup+1182.3 s]
/proc/loadavg: 8.14 8.09 7.94 9/186 20448
/proc/meminfo: memFree=13269284/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=2023088 CPUtime=4721.27 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 468653 0 0 0 471783 344 0 0 22 0 5 0 521841715 2071642112 467115 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 505772 467115 176 36 0 503677 0
[pid=20401/tid=20409] ppid=20399 vsize=2023088 CPUtime=1178.5 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 108498 0 0 0 117736 114 0 0 25 0 5 0 521841754 2071642112 467115 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910288 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=2023088 CPUtime=1180.4 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 135934 0 0 0 117952 88 0 0 25 0 5 0 521841754 2071642112 467115 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=2023088 CPUtime=1180.58 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 53605 0 0 0 118009 49 0 0 25 0 5 0 521841754 2071642112 467115 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=2023088 CPUtime=1181.36 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 157796 0 0 0 118047 89 0 0 25 0 5 0 521841754 2071642112 467115 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4721.27
Current children cumulated vsize (KiB) 2023088



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.1 s]
/proc/loadavg: 8.10 8.08 7.94 9/186 20448
/proc/meminfo: memFree=13245336/32951124 swapFree=67111528/67111528
[pid=20401] ppid=20399 vsize=2029264 CPUtime=4800.38 cores=1,3,5,7
/proc/20401/stat : 20401 (ttsth-5-4) S 20399 20401 19929 0 -1 4202496 470808 0 0 0 479692 346 0 0 22 0 5 0 521841715 2077966336 469270 33554432000 4194304 4338100 140735332321520 18446744073709551615 216646318053 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/20401/statm: 507316 469270 176 36 0 505221 0
[pid=20401/tid=20409] ppid=20399 vsize=2029264 CPUtime=1198.2 cores=1,3,5,7
/proc/20401/task/20409/stat : 20409 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 108686 0 0 0 119705 115 0 0 25 0 5 0 521841754 2077966336 469270 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910319 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=20401/tid=20410] ppid=20399 vsize=2029264 CPUtime=1200.21 cores=1,3,5,7
/proc/20401/task/20410/stat : 20410 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 136854 0 0 0 119932 89 0 0 25 0 5 0 521841754 2077966336 469270 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=20401/tid=20411] ppid=20399 vsize=2029264 CPUtime=1200.38 cores=1,3,5,7
/proc/20401/task/20411/stat : 20411 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 54507 0 0 0 119989 49 0 0 25 0 5 0 521841754 2077966336 469270 33554432000 4194304 4338100 140735332321520 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=20401/tid=20412] ppid=20399 vsize=2029264 CPUtime=1201.17 cores=1,3,5,7
/proc/20401/task/20412/stat : 20412 (ttsth-5-4) R 20399 20401 19929 0 -1 4202560 157941 0 0 0 120028 89 0 0 25 0 5 0 521841754 2077966336 469270 33554432000 4194304 4338100 140735332321520 18446744073709551615 216633910373 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.38
Current children cumulated vsize (KiB) 2029264

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.2
CPU time (s): 4800.5
CPU user time (s): 4796.95
CPU system time (s): 3.54946
CPU usage (%): 399.31
Max. virtual memory (cumulated for all children) (KiB): 2029264

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.95
system time used= 3.54946
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 470808
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1882
involuntary context switches= 31824

runsolver used 1.9497 second user time and 4.6023 second system time

The end

Launcher Data 

Begin job on node103 at 2011-04-29 20:39:51
IDJOB=3265610
IDBENCH=85786
IDSOLVER=1654
FILE ID=node103/3265610-1304102391
RUNJOBID= node103-1304102391-20388
PBS_JOBID= 13170621
Free space on /tmp= 71516 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/sorge05/QG7a/gensys-brn002.shuffled-as.sat05-3935.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265610-1304102391/watcher-3265610-1304102391 -o /tmp/evaluation-result-3265610-1304102391/solver-3265610-1304102391 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265610-1304102391.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= eda2d1e7d2975da4e019beddc323767b
RANDOM SEED=1540586023

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17904412 kB
Buffers:       2148988 kB
Cached:       12248132 kB
SwapCached:          0 kB
Active:        5588000 kB
Inactive:      8875716 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17904412 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1256 kB
Writeback:           0 kB
AnonPages:       65308 kB
Mapped:          14444 kB
Slab:           519844 kB
PageTables:       4232 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   189484 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71516 MiB
End job on node103 at 2011-04-29 20:59:55