Trace number 3249768

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.38 1215.83

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-21x39.cnf
MD5SUMe7679ca49bb432c9b90305d907365c4e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables3276
Number of clauses628173
Sum of the clauses size2502864
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 24914
Number of clauses of size 30
Number of clauses of size 4623259
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.69/0.71	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249768-1304065877.cnf 
0.69/0.71	c Using SATTIMEOUT of 4800
0.69/0.71	c Using SATRAM of 15500
0.69/0.71	c Size of subset node = 32
0.69/0.71	c Variables = 3276
0.69/0.71	c Clauses = 628173
0.69/0.71	c Literals = 2502864
0.69/0.71	c Reduced size:
0.69/0.71	c Variables = 3276
0.69/0.71	c Clauses = 628173
0.69/0.71	c Literals = 2502864
0.69/0.71	c a2
1.89/1.97	c a1
6.50/6.56	c Independent sub-problems = 1
6.50/6.56	c Sub-problem with 3276 variables and 628173 clauses
6.50/6.56	c using permutation method 4
6.50/6.56	c count = 1
6.50/6.56	c Ordering finished 6.430 seconds
6.50/6.56	c mapped 6.550 seconds
6.50/6.56	c permuted 6.550 seconds
6.50/6.56	c about to sort
6.60/6.64	c sorted
7.99/8.00	c clauses now 628173
7.99/8.00	c init finished 6.710 seconds
7.99/8.00	c tree built 8.000 seconds
7.99/8.00	c 
7.99/8.00	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
7.99/8.00	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
7.99/8.00	c ................................ | ..........                    |                                  8.0         0.0             
7.99/8.00	c ................................ | ..........                  1 |                                  8.0         0.1             
7.99/8.00	c ................................ | ..........                  2 |                                  8.0         0.2             
7.99/8.00	c ................................ | ..........                  3 |                                  8.0         0.4             
7.99/8.00	c ................................ | ..........                  4 |                                  8.0         0.5             
7.99/8.00	c ................................ | ..........                  5 |                                  8.0         0.6             
7.99/8.02	c ................................ | ..........               1000 | 4            294                 8.0       124.4          163
95.96/30.22	c ................................ | ..........            1038000 | 97158        535668             96.0     10808.0      3538228
191.91/54.42	c ................................ | ..........            2029000 | 196506       1109701           192.0     10567.7      7239873
287.74/78.68	c ................................ | ..........            2946000 | 289186       1610248           288.1     10227.0      9991410
383.96/102.93	c ................................ | ..........            3818000 | 383189       2098795           384.1      9940.1     12844041
479.76/127.18	c ................................ | *.........            4642000 | 468615       2551795           480.1      9668.8     16257839
575.70/151.37	c ................................ | *.........            5419000 | 553266       2991420           576.0      9408.0     18724584
671.99/175.61	c ................................ | *.........            6150000 | 630351       3396828           672.1      9151.0     20870312
767.92/199.83	c ................................ | *.........            6835000 | 702027       3778282           768.0      8899.3     22833182
863.83/224.07	c ................................ | *.........            7583000 | 778835       4203404           864.1      8775.3     24546798
959.77/248.29	c ................................ | **........            8291000 | 852952       4598736           960.1      8635.3     26686458
1055.69/272.47	c ................................ | **........            8991000 | 921583       4996403          1056.0      8514.2     28330802
1151.94/296.72	c ................................ | **........            9728000 | 996777       5395732          1152.0      8444.4     30237178
1247.70/320.99	c ................................ | **........           10572000 | 1079335      5912071          1248.1      8470.6     31649296
1343.77/345.25	c ................................ | **........           11379000 | 1156613      6386229          1344.0      8466.5     32991925
1439.99/369.51	c ................................ | ***.......           12176000 | 1238856      6878710          1440.0      8455.4     34183202
1535.98/393.74	c ................................ | ***.......           12862000 | 1304929      7325321          1536.1      8373.0     34660255
1632.01/417.92	c ................................ | ***.......           13600000 | 1376299      7749358          1632.1      8332.8     35156579
1727.64/442.09	c ................................ | ***.......           14243000 | 1438504      8129827          1728.0      8242.5     36223849
1823.69/466.30	c ................................ | ***.......           14888000 | 1503067      8499692          1824.1      8161.9     37612675
1920.09/490.50	c ................................ | ****......           15527000 | 1568611      8865624          1920.1      8086.6     39599807
2015.85/514.74	c ................................ | ****......           16243000 | 1639514      9280896          2016.0      8057.0     41125056
2112.06/539.01	c ................................ | ****......           16957000 | 1707587      9694328          2112.1      8028.5     42398348
2207.73/563.17	c ................................ | ****......           17579000 | 1772825      10056347         2208.0      7961.5     43270580
2303.71/587.39	c ................................ | ****......           18166000 | 1833986      10384823         2304.0      7884.4     43588958
2399.72/611.57	c ................................ | *****.....           18812000 | 1904711      10727444         2400.0      7838.3     45593841
2496.04/635.81	c ................................ | *****.....           19446000 | 1978165      10993244         2496.1      7790.7     47518013
2592.08/660.01	c ................................ | *****.....           19964000 | 2038511      11229582         2592.1      7701.8     49465199
2688.16/684.22	c ................................ | *****.....           20467000 | 2097085      11474300         2688.2      7613.6     51522664
2783.80/708.39	c ................................ | *****.....           20937000 | 2152680      11671688         2784.1      7520.1     53403184
2879.91/732.56	c ................................ | ******....           21454000 | 2211920      11923015         2880.2      7448.9     55175539
2975.95/756.72	c ................................ | ******....           21942000 | 2269005      12129940         2976.0      7372.9     56746189
3072.07/780.91	c ................................ | ******....           22345000 | 2314495      12334413         3072.1      7273.5     57989970
3167.76/805.08	c ................................ | ******....           22829000 | 2368515      12573190         3168.1      7205.9     60273854
3263.81/829.28	c ................................ | ******....           23315000 | 2420499      12820028         3264.2      7142.7     61474220
3359.97/853.43	c ................................ | *******...           23645000 | 2460633      12939696         3360.1      7037.0     63056214
3455.75/877.56	c ................................ | *******...           23976000 | 2497629      13082896         3456.0      6937.5     64003630
3551.97/901.71	c ................................ | *******...           24311000 | 2534899      13232726         3552.0      6844.2     66177132
3648.19/925.91	c ................................ | *******...           24665000 | 2569764      13391244         3648.3      6760.8     67629569
3744.05/950.01	c ................................ | *******...           24917000 | 2598725      13497466         3744.1      6655.0     67889540
3839.92/974.13	c ................................ | ********..           25234000 | 2635249      13655758         3840.0      6571.3     68644490
3936.09/998.33	c ................................ | ********..           25563000 | 2676244      13822130         3936.2      6494.3     70312938
4031.90/1022.47	c ................................ | ********..           25887000 | 2712828      13983163         4032.2      6420.1     71339547
4127.80/1046.57	c ................................ | ********..           26166000 | 2744850      14115992         4128.1      6338.5     71873546
4223.94/1070.73	c ................................ | ********..           26520000 | 2784924      14280981         4224.1      6278.3     72544410
4319.71/1094.87	c ................................ | *********.           26855000 | 2821819      14446810         4320.0      6216.4     73994030
4415.97/1119.01	c ................................ | *********.           27131000 | 2852554      14588197         4416.0      6143.8     75048541
4512.23/1143.20	c ................................ | *********.           27456000 | 2887542      14740366         4512.3      6084.8     76690566
4608.08/1167.33	c ................................ | *********.           27856000 | 2934632      14955940         4608.2      6044.9     78321344
4703.85/1191.48	c ................................ | *********.           28175000 | 2977195      15072798         4704.2      5989.4     80655448
4799.87/1215.69	c CPU limit (4800) reached
4799.87/1215.69	c Time = 4800.23 seconds
4799.87/1215.69	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249768-1304065877/watcher-3249768-1304065877 -o /tmp/evaluation-result-3249768-1304065877/solver-3249768-1304065877 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249768-1304065877.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.84 8.04 7.68 5/181 11739
/proc/meminfo: memFree=15541316/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=9408 CPUtime=0 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 413 0 0 0 0 0 0 0 22 0 1 0 518188193 9633792 355 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446606021 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/11739/statm: 2352 356 147 36 0 257 0

[startup+0.0530931 s]
/proc/loadavg: 7.84 8.04 7.68 5/181 11739
/proc/meminfo: memFree=15541316/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=15256 CPUtime=0.04 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 1864 0 0 0 4 0 0 0 22 0 1 0 518188193 15622144 1806 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446606021 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/11739/statm: 3814 1807 148 36 0 1719 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 15256

[startup+0.101079 s]
/proc/loadavg: 7.84 8.04 7.68 5/181 11739
/proc/meminfo: memFree=15541316/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=21004 CPUtime=0.09 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 3252 0 0 0 9 0 0 0 22 0 1 0 518188193 21508096 3194 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446606021 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/11739/statm: 5251 3195 148 36 0 3156 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 21004

[startup+0.301044 s]
/proc/loadavg: 7.84 8.04 7.68 5/181 11739
/proc/meminfo: memFree=15541316/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=44132 CPUtime=0.29 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 9079 0 0 0 28 1 0 0 23 0 1 0 518188193 45191168 9021 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446606021 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/11739/statm: 11033 9022 148 36 0 8938 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 44132

[startup+0.700983 s]
/proc/loadavg: 7.84 8.04 7.68 5/181 11739
/proc/meminfo: memFree=15541316/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=82952 CPUtime=0.69 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 18667 0 0 0 66 3 0 0 25 0 1 0 518188193 84942848 18609 33554432000 4194304 4338100 140736630853696 18446744073709551615 4221551 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/11739/statm: 20738 18609 150 36 0 18643 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 82952

[startup+1.50082 s]
/proc/loadavg: 7.84 8.04 7.68 6/182 11740
/proc/meminfo: memFree=15363004/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=190100 CPUtime=1.49 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 40104 0 0 0 144 5 0 0 25 0 1 0 518188193 194662400 39148 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340200 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/11739/statm: 47525 39148 1019 36 0 45430 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 190100

[startup+3.10052 s]
/proc/loadavg: 7.84 8.04 7.68 6/182 11740
/proc/meminfo: memFree=15342432/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=120000 CPUtime=3.09 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 47599 0 0 0 302 7 0 0 25 0 1 0 518188193 122880000 28034 33554432000 4194304 4338100 140736630853696 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/11739/statm: 30033 28035 299 36 0 27938 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 120000

[startup+6.30093 s]
/proc/loadavg: 7.61 7.99 7.67 6/182 11740
/proc/meminfo: memFree=15301024/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=161184 CPUtime=6.29 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) R 11737 11739 9766 0 -1 4202496 58017 0 0 0 621 8 0 0 25 0 1 0 518188193 165052416 38309 33554432000 4194304 4338100 140736630853696 18446744073709551615 4295806 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/11739/statm: 40296 38309 157 36 0 38201 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 161184

[startup+12.7007 s]
/proc/loadavg: 7.64 7.99 7.67 9/186 11745
/proc/meminfo: memFree=15158660/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=457796 CPUtime=26.6 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 73999 0 0 0 2608 52 0 0 25 0 5 0 518188193 468783104 53063 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 114449 53063 176 36 0 112354 0
[pid=11739/tid=11742] ppid=11737 vsize=457796 CPUtime=4.64 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 1219 0 0 0 451 13 0 0 25 0 5 0 518188993 468783104 53063 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340521 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=457796 CPUtime=4.63 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 1150 0 0 0 452 11 0 0 21 0 5 0 518188993 468783104 53063 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340577 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=457796 CPUtime=4.62 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 1217 0 0 0 450 12 0 0 18 0 5 0 518188993 468783104 53063 33554432000 4194304 4338100 140736630853696 18446744073709551615 4258042 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=457796 CPUtime=4.68 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 2252 0 0 0 462 6 0 0 19 0 5 0 518188993 468783104 53063 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340148 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 26.6
Current children cumulated vsize (KiB) 457796

[startup+25.5003 s]
/proc/loadavg: 7.79 8.00 7.68 9/186 11745
/proc/meminfo: memFree=14985588/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=461400 CPUtime=77.31 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 90630 0 0 0 7538 193 0 0 25 0 5 0 518188193 472473600 69694 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 115350 69694 176 36 0 113255 0
[pid=11739/tid=11742] ppid=11737 vsize=461400 CPUtime=17.34 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 4817 0 0 0 1677 57 0 0 25 0 5 0 518188993 472473600 69694 33554432000 4194304 4338100 140736630853696 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=461400 CPUtime=17.31 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 5365 0 0 0 1676 55 0 0 23 0 5 0 518188993 472473600 69694 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340243 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=461400 CPUtime=17.18 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 4232 0 0 0 1673 45 0 0 21 0 5 0 518188993 472473600 69694 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340326 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=461400 CPUtime=17.46 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 8055 0 0 0 1721 25 0 0 19 0 5 0 518188993 472473600 69694 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254840 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 77.31
Current children cumulated vsize (KiB) 461400

[startup+51.1005 s]
/proc/loadavg: 7.86 8.00 7.69 9/186 11746
/proc/meminfo: memFree=14706488/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=614512 CPUtime=178.84 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 125334 0 0 0 17427 457 0 0 25 0 5 0 518188193 629260288 104398 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 153628 104398 176 36 0 151533 0
[pid=11739/tid=11742] ppid=11737 vsize=614512 CPUtime=42.71 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 12397 0 0 0 4133 138 0 0 18 0 5 0 518188993 629260288 104398 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=614512 CPUtime=42.68 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 13524 0 0 0 4135 133 0 0 25 0 5 0 518188993 629260288 104398 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458757926 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=614512 CPUtime=42.75 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 13086 0 0 0 4192 83 0 0 19 0 5 0 518188993 629260288 104398 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=614512 CPUtime=42.67 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 18166 0 0 0 4174 93 0 0 18 0 5 0 518188993 629260288 104398 33554432000 4194304 4338100 140736630853696 18446744073709551615 4257010 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 178.84
Current children cumulated vsize (KiB) 614512

[startup+102.306 s]
/proc/loadavg: 8.03 8.03 7.71 9/186 11747
/proc/meminfo: memFree=14268268/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=729620 CPUtime=381.6 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 175689 0 0 0 37110 1050 0 0 25 0 5 0 518188193 747130880 154753 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 182405 154753 176 36 0 180310 0
[pid=11739/tid=11742] ppid=11737 vsize=729620 CPUtime=93.27 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 22506 0 0 0 9030 297 0 0 25 0 5 0 518188993 747130880 154753 33554432000 4194304 4338100 140736630853696 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=729620 CPUtime=93.27 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 25586 0 0 0 9025 302 0 0 18 0 5 0 518188993 747130880 154753 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340193 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=729620 CPUtime=93.79 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 25096 0 0 0 9195 184 0 0 18 0 5 0 518188993 747130880 154753 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=729620 CPUtime=93.23 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 34340 0 0 0 9066 257 0 0 20 0 5 0 518188993 747130880 154754 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 381.6
Current children cumulated vsize (KiB) 729620

[startup+162.301 s]
/proc/loadavg: 7.99 8.02 7.73 9/186 11749
/proc/meminfo: memFree=13858312/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=987312 CPUtime=619.26 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 233663 0 0 0 60232 1694 0 0 25 0 5 0 518188193 1011007488 212727 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 246828 212727 176 36 0 244733 0
[pid=11739/tid=11742] ppid=11737 vsize=987312 CPUtime=153.21 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 33293 0 0 0 14923 398 0 0 25 0 5 0 518188993 1011007488 212727 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446612482 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=987312 CPUtime=152.33 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 40571 0 0 0 14763 470 0 0 20 0 5 0 518188993 1011007488 212727 33554432000 4194304 4338100 140736630853696 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=987312 CPUtime=153.09 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 37015 0 0 0 14936 373 0 0 18 0 5 0 518188993 1011007488 212727 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=987312 CPUtime=152.61 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 54623 0 0 0 14818 443 0 0 20 0 5 0 518188993 1011007488 212727 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 619.26
Current children cumulated vsize (KiB) 987312

[startup+222.301 s]
/proc/loadavg: 7.96 8.01 7.74 6/186 11750
/proc/meminfo: memFree=13402108/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=1221816 CPUtime=857.08 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 274194 0 0 0 83386 2322 0 0 25 0 5 0 518188193 1251139584 253258 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 305454 253258 176 36 0 303359 0
[pid=11739/tid=11742] ppid=11737 vsize=1221816 CPUtime=212.56 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 40981 0 0 0 20709 547 0 0 20 0 5 0 518188993 1251139584 253258 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=1221816 CPUtime=212.05 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 51311 0 0 0 20617 588 0 0 25 0 5 0 518188993 1251139584 253258 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=1221816 CPUtime=212.39 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 46730 0 0 0 20689 550 0 0 18 0 5 0 518188993 1251139584 253258 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=1221816 CPUtime=212.04 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 67011 0 0 0 20578 626 0 0 18 0 5 0 518188993 1251139584 253258 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 857.08
Current children cumulated vsize (KiB) 1221816

[startup+282.3 s]
/proc/loadavg: 8.06 8.03 7.76 9/186 11752
/proc/meminfo: memFree=13155956/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=1393324 CPUtime=1094.9 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 313945 0 0 0 106531 2959 0 0 25 0 5 0 518188193 1426763776 293009 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 348331 293009 176 36 0 346236 0
[pid=11739/tid=11742] ppid=11737 vsize=1393324 CPUtime=271.82 cores=0,2,4,6

################
# More data... #
################

/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 139070 0 0 0 66652 1997 0 0 21 0 5 0 518188993 2290642944 503098 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2759.98
Current children cumulated vsize (KiB) 2236956

[startup+762.307 s]
/proc/loadavg: 8.02 8.01 7.84 9/186 11765
/proc/meminfo: memFree=11640792/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2332128 CPUtime=2998.22 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 556543 0 0 0 292097 7725 0 0 25 0 5 0 518188193 2388099072 535607 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 583032 535607 176 36 0 580937 0
[pid=11739/tid=11742] ppid=11737 vsize=2332128 CPUtime=746.95 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 108731 0 0 0 72752 1943 0 0 25 0 5 0 518188993 2388099072 535607 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2332128 CPUtime=748.52 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 119161 0 0 0 73073 1779 0 0 18 0 5 0 518188993 2388099072 535607 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2332128 CPUtime=748.76 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 116873 0 0 0 73022 1854 0 0 25 0 5 0 518188993 2388099072 535607 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2332128 CPUtime=745.95 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 143617 0 0 0 72457 2138 0 0 18 0 5 0 518188993 2388099072 535607 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2998.22
Current children cumulated vsize (KiB) 2332128

[startup+822.3 s]
/proc/loadavg: 8.06 8.02 7.85 9/186 11766
/proc/meminfo: memFree=11456908/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2437000 CPUtime=3236.39 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 589589 0 0 0 315390 8249 0 0 25 0 5 0 518188193 2495488000 568653 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 609250 568653 176 36 0 607155 0
[pid=11739/tid=11742] ppid=11737 vsize=2437000 CPUtime=806.35 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 117455 0 0 0 78545 2090 0 0 20 0 5 0 518188993 2495488000 568653 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2437000 CPUtime=808.22 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 128068 0 0 0 78935 1887 0 0 18 0 5 0 518188993 2495488000 568653 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2437000 CPUtime=808.3 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 122062 0 0 0 78856 1974 0 0 24 0 5 0 518188993 2495488000 568653 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2437000 CPUtime=805.48 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 153843 0 0 0 78261 2287 0 0 18 0 5 0 518188993 2495488000 568653 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3236.39
Current children cumulated vsize (KiB) 2437000

[startup+882.301 s]
/proc/loadavg: 8.02 8.01 7.86 9/186 11768
/proc/meminfo: memFree=11306624/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2597376 CPUtime=3474.83 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 617312 0 0 0 338800 8683 0 0 25 0 5 0 518188193 2659713024 596376 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 649344 596376 176 36 0 647249 0
[pid=11739/tid=11742] ppid=11737 vsize=2597376 CPUtime=865.91 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 123025 0 0 0 84383 2208 0 0 19 0 5 0 518188993 2659713024 596376 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340135 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2597376 CPUtime=867.93 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 135971 0 0 0 84801 1992 0 0 21 0 5 0 518188993 2659713024 596376 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340205 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2597376 CPUtime=867.77 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 128268 0 0 0 84685 2092 0 0 21 0 5 0 518188993 2659713024 596376 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340502 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2597376 CPUtime=865.19 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 161887 0 0 0 84138 2381 0 0 19 0 5 0 518188993 2659713024 596376 33554432000 4194304 4338100 140736630853696 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3474.83
Current children cumulated vsize (KiB) 2597376

[startup+942.301 s]
/proc/loadavg: 8.01 8.01 7.87 9/186 11769
/proc/meminfo: memFree=11145288/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2699204 CPUtime=3713.44 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 646263 0 0 0 362266 9078 0 0 25 0 5 0 518188193 2763984896 625327 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 674801 625327 176 36 0 672706 0
[pid=11739/tid=11742] ppid=11737 vsize=2699204 CPUtime=925.47 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 128989 0 0 0 90224 2323 0 0 18 0 5 0 518188993 2763984896 625327 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2699204 CPUtime=927.88 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 142474 0 0 0 90740 2048 0 0 18 0 5 0 518188993 2763984896 625327 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2699204 CPUtime=927.21 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 135107 0 0 0 90516 2205 0 0 18 0 5 0 518188993 2763984896 625327 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2699204 CPUtime=924.83 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 171532 0 0 0 89993 2490 0 0 20 0 5 0 518188993 2763984896 625327 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3713.44
Current children cumulated vsize (KiB) 2699204

[startup+1002.3 s]
/proc/loadavg: 8.08 8.02 7.88 9/186 11771
/proc/meminfo: memFree=10965720/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2840044 CPUtime=3951.99 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 667362 0 0 0 385730 9469 0 0 25 0 5 0 518188193 2908205056 646426 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 710011 646426 176 36 0 707916 0
[pid=11739/tid=11742] ppid=11737 vsize=2840044 CPUtime=985.05 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 136250 0 0 0 96067 2438 0 0 21 0 5 0 518188993 2908205056 646426 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2840044 CPUtime=987.46 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 146610 0 0 0 96598 2148 0 0 18 0 5 0 518188993 2908205056 646426 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2840044 CPUtime=986.74 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 141270 0 0 0 96359 2315 0 0 18 0 5 0 518188993 2908205056 646426 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2840044 CPUtime=984.71 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 175071 0 0 0 95914 2557 0 0 18 0 5 0 518188993 2908205056 646426 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3951.99
Current children cumulated vsize (KiB) 2840044

[startup+1062.3 s]
/proc/loadavg: 7.84 7.98 7.87 9/186 11773
/proc/meminfo: memFree=10858964/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2849812 CPUtime=4190.56 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 681006 0 0 0 409213 9843 0 0 25 0 5 0 518188193 2918207488 660070 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 712453 660070 176 36 0 710358 0
[pid=11739/tid=11742] ppid=11737 vsize=2849812 CPUtime=1044.68 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 139058 0 0 0 101921 2547 0 0 18 0 5 0 518188993 2918207488 660071 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340521 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2849812 CPUtime=1047.1 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 152877 0 0 0 102454 2256 0 0 18 0 5 0 518188993 2918207488 660071 33554432000 4194304 4338100 140736630853696 18446744073709551615 4234400 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2849812 CPUtime=1046.07 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 141793 0 0 0 102190 2417 0 0 18 0 5 0 518188993 2918207488 660071 33554432000 4194304 4338100 140736630853696 18446744073709551615 4286323 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2849812 CPUtime=1044.68 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 179118 0 0 0 101855 2613 0 0 18 0 5 0 518188993 2918207488 660071 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4190.56
Current children cumulated vsize (KiB) 2849812

[startup+1122.3 s]
/proc/loadavg: 7.98 7.99 7.88 9/186 11774
/proc/meminfo: memFree=10722816/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=2872604 CPUtime=4429.07 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 705392 0 0 0 432657 10250 0 0 25 0 5 0 518188193 2941546496 684456 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 718151 684457 176 36 0 716056 0
[pid=11739/tid=11742] ppid=11737 vsize=2872604 CPUtime=1104.29 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 144714 0 0 0 107763 2666 0 0 18 0 5 0 518188993 2941546496 684457 33554432000 4194304 4338100 140736630853696 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=2872604 CPUtime=1106.61 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 158779 0 0 0 108293 2368 0 0 18 0 5 0 518188993 2941546496 684458 33554432000 4194304 4338100 140736630853696 18446744073709551615 4234400 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=2872604 CPUtime=1105.51 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 148073 0 0 0 108019 2532 0 0 19 0 5 0 518188993 2941546496 684458 33554432000 4194304 4338100 140736630853696 18446744073709551615 244446340321 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=2872604 CPUtime=1104.62 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 185667 0 0 0 107789 2673 0 0 25 0 5 0 518188993 2941546496 684458 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4429.07
Current children cumulated vsize (KiB) 2872604

[startup+1182.3 s]
/proc/loadavg: 8.07 8.01 7.89 9/186 11776
/proc/meminfo: memFree=10557396/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=3105028 CPUtime=4667.72 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) S 11737 11739 9766 0 -1 4202496 740765 0 0 0 456097 10675 0 0 25 0 5 0 518188193 3179548672 719829 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 776257 719829 176 36 0 774162 0
[pid=11739/tid=11742] ppid=11737 vsize=3105028 CPUtime=1163.96 cores=0,2,4,6
/proc/11739/task/11742/stat : 11742 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 153893 0 0 0 113610 2786 0 0 18 0 5 0 518188993 3179548672 719829 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=11739/tid=11743] ppid=11737 vsize=3105028 CPUtime=1166.27 cores=0,2,4,6
/proc/11739/task/11743/stat : 11743 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 168778 0 0 0 114135 2492 0 0 18 0 5 0 518188993 3179548672 719829 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=11739/tid=11744] ppid=11737 vsize=3105028 CPUtime=1164.87 cores=0,2,4,6
/proc/11739/task/11744/stat : 11744 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 157646 0 0 0 113839 2648 0 0 19 0 5 0 518188993 3179548672 719829 33554432000 4194304 4338100 140736630853696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=3105028 CPUtime=1164.58 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4202560 192287 0 0 0 113720 2738 0 0 21 0 5 0 518188993 3179548672 719829 33554432000 4194304 4338100 140736630853696 18446744073709551615 244458771125 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4667.72
Current children cumulated vsize (KiB) 3105028



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1215.7 s]
/proc/loadavg: 7.48 7.89 7.86 2/173 11795
/proc/meminfo: memFree=12764156/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=0 CPUtime=4800.24 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) Z 11737 11739 9766 0 -1 4203548 764169 0 0 0 469127 10897 0 0 18 0 2 0 518188193 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 0 0 0 0 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=0 CPUtime=1197.95 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4203588 195051 0 0 0 117019 2776 0 0 18 0 2 0 518188993 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1215.81 s]
/proc/loadavg: 7.48 7.89 7.86 2/173 11795
/proc/meminfo: memFree=12764156/32951124 swapFree=67111528/67111528
[pid=11739] ppid=11737 vsize=0 CPUtime=4800.35 cores=0,2,4,6
/proc/11739/stat : 11739 (ttsth-5-4) Z 11737 11739 9766 0 -1 4203548 764169 0 0 0 469127 10908 0 0 18 0 2 0 518188193 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/11739/statm: 0 0 0 0 0 0 0
[pid=11739/tid=11745] ppid=11737 vsize=0 CPUtime=1198.06 cores=0,2,4,6
/proc/11739/task/11745/stat : 11745 (ttsth-5-4) R 11737 11739 9766 0 -1 4203588 195051 0 0 0 117019 2787 0 0 19 0 2 0 518188993 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1215.83
CPU time (s): 4800.38
CPU user time (s): 4691.28
CPU system time (s): 109.095
CPU usage (%): 394.824
Max. virtual memory (cumulated for all children) (KiB): 3131076

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4691.28
system time used= 109.095
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 764169
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 961474
involuntary context switches= 18306

runsolver used 1.79573 second user time and 4.02939 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-29 10:31:18
IDJOB=3249768
IDBENCH=83184
IDSOLVER=1654
FILE ID=node105/3249768-1304065877
RUNJOBID= node105-1304064673-11620
PBS_JOBID= 13170404
Free space on /tmp= 71924 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-21x39.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249768-1304065877/watcher-3249768-1304065877 -o /tmp/evaluation-result-3249768-1304065877/solver-3249768-1304065877 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249768-1304065877.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e7679ca49bb432c9b90305d907365c4e
RANDOM SEED=303031272

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15541604 kB
Buffers:       2301288 kB
Cached:       14446644 kB
SwapCached:          0 kB
Active:        8986128 kB
Inactive:      7853016 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15541604 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           15156 kB
Writeback:           0 kB
AnonPages:       90384 kB
Mapped:          15160 kB
Slab:           507432 kB
PageTables:       5024 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   282444 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71924 MiB
End job on node105 at 2011-04-29 10:51:35