Trace number 3264774

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.45 1230.34

General information on the benchmark

NameSAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/
mod2c-rand3bip-sat-240-3.shuffled-as.sat05-2520.cnf
MD5SUMdfc7796289f8cacbd4db5eb947a1ba09
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark194.024
Satisfiable
(Un)Satisfiability was proved
Number of variables339
Number of clauses2416
Sum of the clauses size10720
Maximum clause length6
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 41496
Number of clauses of size 5784
Number of clauses of size over 5136

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264774-1304100313.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 339
0.00/0.00	c Clauses = 2416
0.00/0.00	c Literals = 10720
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 339
0.00/0.00	c Clauses = 2416
0.00/0.00	c Literals = 10720
0.00/0.00	c a2
0.00/0.00	c a1
0.00/0.07	c Independent sub-problems = 1
0.00/0.07	c Sub-problem with 339 variables and 2416 clauses
0.00/0.07	c using permutation method 4
0.00/0.07	c count = 2
0.00/0.07	c Ordering finished 0.070 seconds
0.00/0.07	c mapped 0.070 seconds
0.00/0.07	c permuted 0.070 seconds
0.00/0.07	c about to sort
0.00/0.08	c sorted
0.00/0.08	c clauses now 2416
0.00/0.08	c init finished 0.080 seconds
0.00/0.08	c tree built 0.080 seconds
0.00/0.08	c 
0.00/0.08	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.00/0.08	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.00/0.08	c ................................ | ..........                    |                                  0.1         0.0             
0.00/0.08	c ................................ | ..........                  1 |              1                   0.1        10.0             
0.00/0.08	c ................................ | ..........                  2 |              2                   0.1        20.0             
0.00/0.08	c ................................ | ..........                  3 |              2                   0.1        30.0             
0.00/0.08	c ................................ | ..........                  4 |              2                   0.1        40.0             
0.00/0.08	c ................................ | ..........                  5 |              2                   0.1        50.0             
0.00/0.10	c ................................ | ..........               1000 | 116          490                 0.1     10000.0         9278
95.74/24.29	c ................................ | ..........            1302000 | 349090       670619             96.1     13548.4     28861441
191.69/48.59	c ................................ | ..........            2044000 | 554302       1049868           192.1     10642.5     39837345
287.87/72.74	c ................................ | ..........            2668000 | 731900       1353917           288.1      9262.3     54154396
384.06/97.00	c ................................ | ..........            3218000 | 887631       1624988           384.1      8378.2     64730982
479.87/121.24	c ................................ | *.........            3696000 | 1023898      1862331           480.0      7699.4     73982936
575.86/145.56	c ................................ | *.........            4148000 | 1153690      2085991           576.1      7199.6     81967978
671.88/169.93	c ................................ | *.........            4585000 | 1281346      2299433           672.0      6822.9     84673813
767.80/194.18	c ................................ | *.........            5004000 | 1404514      2495610           768.1      6514.6     87160378
864.12/218.62	c ................................ | *.........            5412000 | 1525033      2694954           864.2      6262.4     91091755
959.64/243.19	c ................................ | **........            5953000 | 1690390      2924504           960.0      6200.9     94580439
1055.89/267.74	c ................................ | **........            6469000 | 1845805      3163208          1056.1      6125.5     97259087
1151.82/292.16	c ................................ | **........            6968000 | 1997022      3382626          1152.1      6048.2     99421418
1247.82/316.65	c ................................ | **........            7410000 | 2131675      3586690          1248.0      5937.4    104433198
1344.12/341.21	c ................................ | **........            7841000 | 2263623      3773455          1344.2      5833.3    111252289
1439.66/365.78	c ................................ | ***.......            8252000 | 2388028      3967092          1440.0      5730.5    116189662
1535.76/390.26	c ................................ | ***.......            8638000 | 2504501      4150422          1536.0      5623.7    118964127
1632.03/414.72	c ................................ | ***.......            9034000 | 2623371      4339542          1632.1      5535.1    119497172
1727.68/439.58	c ................................ | ***.......            9429000 | 2742881      4518553          1728.0      5456.6    120132294
1823.94/464.05	c ................................ | ***.......            9816000 | 2861245      4692817          1824.1      5381.2    121825847
1920.03/488.64	c ................................ | ****......           10205000 | 2980351      4869966          1920.2      5314.6    123091912
2015.86/513.13	c ................................ | ****......           10638000 | 3109720      5082730          2016.0      5276.8    123945990
2111.73/537.97	c ................................ | ****......           11059000 | 3235944      5277961          2112.0      5236.2    123945990
2207.66/562.59	c ................................ | ****......           11474000 | 3360001      5474248          2208.0      5196.5    123945990
2303.90/587.14	c ................................ | ****......           11871000 | 3479131      5664608          2304.1      5152.2    124143654
2399.84/611.94	c ................................ | *****.....           12252000 | 3593699      5845280          2400.0      5105.0    128982085
2495.92/636.57	c ................................ | *****.....           12630000 | 3705838      6027799          2496.2      5059.7    130597826
2592.10/661.13	c ................................ | *****.....           13002000 | 3817270      6202096          2592.2      5015.8    133130655
2687.93/685.86	c ................................ | *****.....           13359000 | 3923420      6377864          2688.2      4969.5    135114334
2784.18/710.41	c ................................ | *****.....           13721000 | 4029744      6549955          2784.2      4928.1    137945564
2880.01/735.24	c ................................ | ******....           14084000 | 4135890      6728974          2880.2      4890.0    137945564
2976.08/759.90	c ................................ | ******....           14498000 | 4259536      6924448          2976.1      4871.5    139554286
3071.91/784.67	c ................................ | ******....           14927000 | 4388888      7116183          3072.2      4858.7    143310774
3168.12/809.22	c ................................ | ******....           15320000 | 4506500      7297419          3168.2      4835.6    145976701
3264.22/833.88	c ................................ | ******....           15680000 | 4615982      7459261          3264.6      4803.1    148007538
3359.94/858.63	c ................................ | *******...           16088000 | 4738654      7650753          3360.0      4788.0    148007538
3456.15/883.50	c ................................ | *******...           16475000 | 4854649      7830644          3456.2      4766.8    148007538
3552.56/908.25	c ................................ | *******...           16840000 | 4964763      8002151          3552.7      4740.0    148007538
3647.78/932.89	c ................................ | *******...           17183000 | 5067261      8163745          3648.2      4710.0    148125795
3743.88/957.48	c ................................ | *******...           17543000 | 5175439      8330874          3744.2      4685.4    148125795
3840.12/982.12	c ................................ | ********..           17918000 | 5287152      8506474          3840.2      4665.9    149464537
3935.75/1006.78	c ................................ | ********..           18365000 | 5419601      8735111          3936.1      4665.8    152244346
4032.22/1031.92	c ................................ | ********..           18910000 | 5579350      8972352          4032.3      4689.6    153027347
4127.82/1056.77	c ................................ | ********..           19379000 | 5716358      9193404          4128.1      4694.4    154270069
4224.21/1081.71	c ................................ | ********..           19821000 | 5844460      9391546          4224.2      4692.2    155976808
4319.87/1106.67	c ................................ | *********.           20219000 | 5959472      9577341          4320.2      4680.1    157544218
4416.03/1131.34	c ................................ | *********.           20604000 | 6071707      9762560          4416.2      4665.5    157601364
4512.51/1156.10	c ................................ | *********.           20991000 | 6186117      9941644          4512.5      4651.7    160644692
4608.02/1180.96	c ................................ | *********.           21346000 | 6292878      10103632         4608.2      4632.1    162619018
4703.76/1205.57	c ................................ | *********.           21697000 | 6397582      10264432         4704.1      4612.4    163601414

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264774-1304100313/watcher-3264774-1304100313 -o /tmp/evaluation-result-3264774-1304100313/solver-3264774-1304100313 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264774-1304100313.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.99 8.12 8.04 2/181 10883
/proc/meminfo: memFree=25015312/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=9640 CPUtime=0 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) R 10881 10883 10639 0 -1 4202496 492 0 0 0 0 0 0 0 25 0 1 0 521629471 9871360 433 33554432000 4194304 4338100 140735044390320 18446744073709551615 246516877447 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10883/statm: 2410 433 152 36 0 315 0

[startup+0.129196 s]
/proc/loadavg: 7.99 8.12 8.04 2/181 10883
/proc/meminfo: memFree=25015312/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=211296 CPUtime=0.77 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 7703 0 0 0 76 1 0 0 25 0 5 0 521629471 216367104 7606 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 52824 7607 175 36 0 50729 0
Current children cumulated CPU time (s) 0.77
Current children cumulated vsize (KiB) 211296

[startup+0.420536 s]
/proc/loadavg: 7.99 8.12 8.04 2/181 10883
/proc/meminfo: memFree=25015312/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=236448 CPUtime=1.41 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 11906 0 0 0 140 1 0 0 25 0 5 0 521629471 242122752 11809 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 67778 21421 175 36 0 65683 0
Current children cumulated CPU time (s) 1.41
Current children cumulated vsize (KiB) 236448

[startup+0.99088 s]
/proc/loadavg: 7.99 8.12 8.04 2/181 10883
/proc/meminfo: memFree=25015312/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=274368 CPUtime=3.69 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 23268 0 0 0 365 4 0 0 25 0 5 0 521629471 280952832 23171 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 68592 23171 175 36 0 66497 0
Current children cumulated CPU time (s) 3.69
Current children cumulated vsize (KiB) 274368

[startup+1.04276 s]
/proc/loadavg: 7.99 8.12 8.04 2/181 10883
/proc/meminfo: memFree=25015312/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=286880 CPUtime=4.09 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 24925 0 0 0 405 4 0 0 25 0 5 0 521629471 293765120 24828 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 71720 24828 175 36 0 69625 0
Current children cumulated CPU time (s) 4.09
Current children cumulated vsize (KiB) 286880

[startup+1.50251 s]
/proc/loadavg: 7.99 8.12 8.04 2/181 10883
/proc/meminfo: memFree=25015312/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=312160 CPUtime=5.74 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 31996 0 0 0 568 6 0 0 25 0 5 0 521629471 319651840 31899 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 78040 31899 175 36 0 75945 0
Current children cumulated CPU time (s) 5.74
Current children cumulated vsize (KiB) 312160

[startup+3.1005 s]
/proc/loadavg: 7.83 8.09 8.03 6/186 10888
/proc/meminfo: memFree=24814696/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=466556 CPUtime=12.09 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 54662 0 0 0 1197 12 0 0 25 0 5 0 521629471 477753344 54565 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 116639 54565 175 36 0 114544 0
[pid=10883/tid=10885] ppid=10881 vsize=466556 CPUtime=2.99 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 12880 0 0 0 296 3 0 0 25 0 5 0 521629479 477753344 54566 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=466556 CPUtime=2.98 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 13268 0 0 0 296 2 0 0 19 0 5 0 521629479 477753344 54566 33554432000 4194304 4338100 140735044390320 18446744073709551615 246516492097 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=466556 CPUtime=3 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 13419 0 0 0 298 2 0 0 20 0 5 0 521629479 477753344 54566 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=466556 CPUtime=2.99 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 13620 0 0 0 297 2 0 0 25 0 5 0 521629479 477753344 54566 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.09
Current children cumulated vsize (KiB) 466556

[startup+6.30091 s]
/proc/loadavg: 7.83 8.09 8.03 9/190 10892
/proc/meminfo: memFree=24420360/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=613328 CPUtime=24.83 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 91699 0 0 0 2462 21 0 0 25 0 5 0 521629471 628047872 91602 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 153332 91603 176 36 0 151237 0
[pid=10883/tid=10885] ppid=10881 vsize=613328 CPUtime=6.18 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 22252 0 0 0 612 6 0 0 25 0 5 0 521629479 628047872 91603 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=613328 CPUtime=6.16 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 22859 0 0 0 611 5 0 0 21 0 5 0 521629479 628047872 91603 33554432000 4194304 4338100 140735044390320 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=613328 CPUtime=6.19 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) S 10881 10883 10639 0 -1 4202560 22104 0 0 0 615 4 0 0 19 0 5 0 521629479 628047872 91603 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528646918 0 0 4096 0 18446604458852449280 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=613328 CPUtime=6.19 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 23009 0 0 0 615 4 0 0 19 0 5 0 521629479 628047872 91603 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.83
Current children cumulated vsize (KiB) 613328

[startup+12.7007 s]
/proc/loadavg: 7.93 8.10 8.03 9/190 10892
/proc/meminfo: memFree=23574476/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=856172 CPUtime=50.28 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 153696 0 0 0 4993 35 0 0 25 0 5 0 521629471 876720128 153599 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 214043 153599 176 36 0 211948 0
[pid=10883/tid=10885] ppid=10881 vsize=856172 CPUtime=12.52 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 38631 0 0 0 1242 10 0 0 25 0 5 0 521629479 876720128 153603 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=856172 CPUtime=12.49 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 37485 0 0 0 1241 8 0 0 25 0 5 0 521629479 876720128 153605 33554432000 4194304 4338100 140735044390320 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=856172 CPUtime=12.56 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 39115 0 0 0 1249 7 0 0 25 0 5 0 521629479 876720128 153606 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=856172 CPUtime=12.58 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 36995 0 0 0 1251 7 0 0 25 0 5 0 521629479 876720128 153607 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.28
Current children cumulated vsize (KiB) 856172

[startup+25.5004 s]
/proc/loadavg: 7.78 8.06 8.02 9/190 10892
/proc/meminfo: memFree=22506572/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=1205136 CPUtime=100.86 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 241147 0 0 0 10031 55 0 0 25 0 5 0 521629471 1234059264 241050 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 301284 241051 176 36 0 299189 0
[pid=10883/tid=10885] ppid=10881 vsize=1205136 CPUtime=25.12 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 61546 0 0 0 2496 16 0 0 25 0 5 0 521629479 1234059264 241051 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=1205136 CPUtime=25.15 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 61311 0 0 0 2501 14 0 0 25 0 5 0 521629479 1234059264 241051 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=1205136 CPUtime=25.26 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 59177 0 0 0 2516 10 0 0 25 0 5 0 521629479 1234059264 241051 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254259 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=1205136 CPUtime=25.2 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 57638 0 0 0 2508 12 0 0 25 0 5 0 521629479 1234059264 241051 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.86
Current children cumulated vsize (KiB) 1205136

[startup+51.1007 s]
/proc/loadavg: 7.85 8.06 8.02 9/190 10893
/proc/meminfo: memFree=21479096/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=1584552 CPUtime=202.03 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 337456 0 0 0 20122 81 0 0 25 0 5 0 521629471 1622581248 337359 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 396138 337359 176 36 0 394043 0
[pid=10883/tid=10885] ppid=10881 vsize=1584552 CPUtime=50.25 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 88880 0 0 0 5001 24 0 0 25 0 5 0 521629479 1622581248 337359 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=1584552 CPUtime=50.44 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 90439 0 0 0 5023 21 0 0 25 0 5 0 521629479 1622581248 337359 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=1584552 CPUtime=50.67 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 79588 0 0 0 5051 16 0 0 25 0 5 0 521629479 1622581248 337359 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=1584552 CPUtime=50.54 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 77073 0 0 0 5037 17 0 0 25 0 5 0 521629479 1622581248 337359 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 202.03
Current children cumulated vsize (KiB) 1584552

[startup+102.3 s]
/proc/loadavg: 7.99 8.05 8.01 9/190 10894
/proc/meminfo: memFree=20053448/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=2418084 CPUtime=405.02 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 549715 0 0 0 40364 138 0 0 25 0 5 0 521629471 2476118016 549618 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 604521 549618 176 36 0 602426 0
[pid=10883/tid=10885] ppid=10881 vsize=2418084 CPUtime=100.71 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 152704 0 0 0 10029 42 0 0 25 0 5 0 521629479 2476118016 549618 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=2418084 CPUtime=101.3 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 141421 0 0 0 10094 36 0 0 25 0 5 0 521629479 2476118016 549618 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=2418084 CPUtime=101.45 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 132075 0 0 0 10115 30 0 0 25 0 5 0 521629479 2476118016 549618 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=2418084 CPUtime=101.44 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 122039 0 0 0 10116 28 0 0 25 0 5 0 521629479 2476118016 549618 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 405.02
Current children cumulated vsize (KiB) 2418084

[startup+162.3 s]
/proc/loadavg: 8.08 8.05 8.01 8/190 10896
/proc/meminfo: memFree=18698956/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=3006532 CPUtime=641.91 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 696781 0 0 0 64007 184 0 0 25 0 5 0 521629471 3078688768 696684 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 751633 696684 176 36 0 749538 0
[pid=10883/tid=10885] ppid=10881 vsize=3006532 CPUtime=159.79 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 203256 0 0 0 15922 57 0 0 25 0 5 0 521629479 3078688768 696684 33554432000 4194304 4338100 140735044390320 18446744073709551615 4235684 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=3006532 CPUtime=160.71 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 173856 0 0 0 16023 48 0 0 25 0 5 0 521629479 3078688768 696684 33554432000 4194304 4338100 140735044390320 18446744073709551615 4235858 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=3006532 CPUtime=160.61 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 148643 0 0 0 16027 34 0 0 25 0 5 0 521629479 3078688768 696685 33554432000 4194304 4338100 140735044390320 18446744073709551615 4240533 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=3006532 CPUtime=160.69 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) S 10881 10883 10639 0 -1 4202560 169551 0 0 0 16026 43 0 0 25 0 5 0 521629479 3078688768 696685 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528646918 0 0 4096 0 18446604458852449280 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 641.91
Current children cumulated vsize (KiB) 3006532

[startup+222.3 s]
/proc/loadavg: 7.91 8.01 8.00 9/190 10898
/proc/meminfo: memFree=17589384/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=3226076 CPUtime=878.49 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 758093 0 0 0 87643 206 0 0 25 0 5 0 521629471 3303501824 757996 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 806519 757996 176 36 0 804424 0
[pid=10883/tid=10885] ppid=10881 vsize=3226076 CPUtime=218.81 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 224932 0 0 0 21816 65 0 0 25 0 5 0 521629479 3303501824 757996 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 1 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=10078576/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=4949492 CPUtime=2985.25 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1195062 0 0 0 298139 386 0 0 25 0 5 0 521629471 5068279808 1194965 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1237373 1194965 176 36 0 1235278 0
[pid=10883/tid=10885] ppid=10881 vsize=4949492 CPUtime=740.3 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 477799 0 0 0 73872 158 0 0 25 0 5 0 521629479 5068279808 1194965 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=4949492 CPUtime=749.45 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 227722 0 0 0 74865 80 0 0 25 0 5 0 521629479 5068279808 1194965 33554432000 4194304 4338100 140735044390320 18446744073709551615 246516229369 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=4949492 CPUtime=747.64 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 228673 0 0 0 74698 66 0 0 25 0 5 0 521629479 5068279808 1194965 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=4949492 CPUtime=747.74 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 259392 0 0 0 74694 80 0 0 25 0 5 0 521629479 5068279808 1194965 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2985.25
Current children cumulated vsize (KiB) 4949492

[startup+822.301 s]
/proc/loadavg: 7.64 7.75 7.86 8/190 10914
/proc/meminfo: memFree=9105272/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=5200060 CPUtime=3219.49 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1261183 0 0 0 321537 412 0 0 25 0 5 0 521629471 5324861440 1261086 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1300015 1261086 176 36 0 1297920 0
[pid=10883/tid=10885] ppid=10881 vsize=5200060 CPUtime=798.14 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 536614 0 0 0 79634 180 0 0 25 0 5 0 521629479 5324861440 1261086 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=5200060 CPUtime=808.41 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 230479 0 0 0 80760 81 0 0 25 0 5 0 521629479 5324861440 1261086 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=5200060 CPUtime=806.39 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 232438 0 0 0 80571 68 0 0 25 0 5 0 521629479 5324861440 1261086 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=5200060 CPUtime=806.45 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 260176 0 0 0 80563 82 0 0 25 0 5 0 521629479 5324861440 1261086 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3219.49
Current children cumulated vsize (KiB) 5200060

[startup+882.301 s]
/proc/loadavg: 7.35 7.65 7.82 9/190 10915
/proc/meminfo: memFree=8305400/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=5367100 CPUtime=3451.47 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1291479 0 0 0 344719 428 0 0 25 0 5 0 521629471 5495910400 1291382 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1341775 1291382 176 36 0 1339680 0
[pid=10883/tid=10885] ppid=10881 vsize=5367100 CPUtime=855.29 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 546090 0 0 0 85344 185 0 0 25 0 5 0 521629479 5495910400 1291382 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=5367100 CPUtime=867.02 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 237003 0 0 0 86618 84 0 0 25 0 5 0 521629479 5495910400 1291382 33554432000 4194304 4338100 140735044390320 18446744073709551615 246516229698 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=5367100 CPUtime=864.28 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 235707 0 0 0 86358 70 0 0 25 0 5 0 521629479 5495910400 1291382 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=5367100 CPUtime=864.76 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 271203 0 0 0 86389 87 0 0 25 0 5 0 521629479 5495910400 1291382 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254231 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3451.47
Current children cumulated vsize (KiB) 5367100

[startup+942.307 s]
/proc/loadavg: 7.60 7.69 7.82 9/190 10917
/proc/meminfo: memFree=7779336/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=5373172 CPUtime=3684.52 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1295455 0 0 0 368018 434 0 0 25 0 5 0 521629471 5502128128 1295358 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1343293 1295358 176 36 0 1341198 0
[pid=10883/tid=10885] ppid=10881 vsize=5373172 CPUtime=913.01 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 547826 0 0 0 91112 189 0 0 25 0 5 0 521629479 5502128128 1295358 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=5373172 CPUtime=925.34 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 237802 0 0 0 92448 86 0 0 25 0 5 0 521629479 5502128128 1295358 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=5373172 CPUtime=922.87 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 236420 0 0 0 92216 71 0 0 25 0 5 0 521629479 5502128128 1295358 33554432000 4194304 4338100 140735044390320 18446744073709551615 4239191 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=5373172 CPUtime=923.19 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 271931 0 0 0 92232 87 0 0 25 0 5 0 521629479 5502128128 1295358 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3684.52
Current children cumulated vsize (KiB) 5373172

[startup+1002.3 s]
/proc/loadavg: 7.52 7.65 7.80 6/190 10918
/proc/meminfo: memFree=6739752/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=5536260 CPUtime=3919.27 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1322534 0 0 0 391478 449 0 0 25 0 5 0 521629471 5669130240 1322437 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1384065 1322438 176 36 0 1381970 0
[pid=10883/tid=10885] ppid=10881 vsize=5536260 CPUtime=971.37 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 572277 0 0 0 96938 199 0 0 25 0 5 0 521629479 5669130240 1322438 33554432000 4194304 4338100 140735044390320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=5536260 CPUtime=984.25 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 238626 0 0 0 98338 87 0 0 25 0 5 0 521629479 5669130240 1322438 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=5536260 CPUtime=981.5 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 237318 0 0 0 98077 73 0 0 25 0 5 0 521629479 5669130240 1322438 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=5536260 CPUtime=982.02 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 272838 0 0 0 98114 88 0 0 25 0 5 0 521629479 5669130240 1322438 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3919.27
Current children cumulated vsize (KiB) 5536260

[startup+1062.3 s]
/proc/loadavg: 7.20 7.57 7.76 9/190 10920
/proc/meminfo: memFree=5992464/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=5775420 CPUtime=4149.76 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1352651 0 0 0 414510 466 0 0 25 0 5 0 521629471 5914030080 1352554 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1443855 1352554 176 36 0 1441760 0
[pid=10883/tid=10885] ppid=10881 vsize=5775420 CPUtime=1028.32 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 599442 0 0 0 102622 210 0 0 25 0 5 0 521629479 5914030080 1352554 33554432000 4194304 4338100 140735044390320 18446744073709551615 246516530633 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=5775420 CPUtime=1042.26 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 239702 0 0 0 104138 88 0 0 25 0 5 0 521629479 5914030080 1352554 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=5775420 CPUtime=1039.22 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 238249 0 0 0 103846 76 0 0 25 0 5 0 521629479 5914030080 1352554 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=5775420 CPUtime=1039.84 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 273782 0 0 0 103894 90 0 0 25 0 5 0 521629479 5914030080 1352554 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4149.76
Current children cumulated vsize (KiB) 5775420

[startup+1122.3 s]
/proc/loadavg: 7.25 7.52 7.73 9/190 10922
/proc/meminfo: memFree=5243032/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=5876100 CPUtime=4381.07 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1380280 0 0 0 437626 481 0 0 25 0 5 0 521629471 6017126400 1380183 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1469025 1380183 176 36 0 1466930 0
[pid=10883/tid=10885] ppid=10881 vsize=5876100 CPUtime=1085.41 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 624550 0 0 0 108321 220 0 0 25 0 5 0 521629479 6017126400 1380183 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=5876100 CPUtime=1100.27 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 240630 0 0 0 109937 90 0 0 25 0 5 0 521629479 6017126400 1380183 33554432000 4194304 4338100 140735044390320 18446744073709551615 246516229545 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=5876100 CPUtime=1097.12 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 239044 0 0 0 109634 78 0 0 25 0 5 0 521629479 6017126400 1380183 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=5876100 CPUtime=1098.16 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 274580 0 0 0 109724 92 0 0 25 0 5 0 521629479 6017126400 1380183 33554432000 4194304 4338100 140735044390320 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4381.07
Current children cumulated vsize (KiB) 5876100

[startup+1182.3 s]
/proc/loadavg: 7.17 7.45 7.69 9/190 10923
/proc/meminfo: memFree=4675216/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=6038828 CPUtime=4613.57 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1423113 0 0 0 460858 499 0 0 25 0 5 0 521629471 6183759872 1423016 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1509707 1423016 176 36 0 1507612 0
[pid=10883/tid=10885] ppid=10881 vsize=6038828 CPUtime=1142.77 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 659119 0 0 0 114045 232 0 0 25 0 5 0 521629479 6183759872 1423016 33554432000 4194304 4338100 140735044390320 18446744073709551615 4253901 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=6038828 CPUtime=1158.8 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 241435 0 0 0 115790 90 0 0 25 0 5 0 521629479 6183759872 1423016 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=6038828 CPUtime=1155.38 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 245765 0 0 0 115457 81 0 0 25 0 5 0 521629479 6183759872 1423016 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=6038828 CPUtime=1156.49 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 275318 0 0 0 115555 94 0 0 25 0 5 0 521629479 6183759872 1423016 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4613.57
Current children cumulated vsize (KiB) 6038828



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1230.1 s]
/proc/loadavg: 6.31 7.21 7.60 5/180 10943
/proc/meminfo: memFree=19272208/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=6072668 CPUtime=4800.19 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) S 10881 10883 10639 0 -1 4202496 1433241 0 0 0 479510 509 0 0 25 0 5 0 521629471 6218412032 1433144 33554432000 4194304 4338100 140735044390320 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 1518167 1433144 176 36 0 1516072 0
[pid=10883/tid=10885] ppid=10881 vsize=6072668 CPUtime=1188.89 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 666473 0 0 0 118653 236 0 0 25 0 5 0 521629479 6218412032 1433144 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=10883/tid=10886] ppid=10881 vsize=6072668 CPUtime=1205.71 cores=1,3,5,7
/proc/10883/task/10886/stat : 10886 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 242062 0 0 0 120479 92 0 0 25 0 5 0 521629479 6218412032 1433144 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=10883/tid=10887] ppid=10881 vsize=6072668 CPUtime=1202.37 cores=1,3,5,7
/proc/10883/task/10887/stat : 10887 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 247331 0 0 0 120154 83 0 0 25 0 5 0 521629479 6218412032 1433144 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=10883/tid=10888] ppid=10881 vsize=6072668 CPUtime=1203.1 cores=1,3,5,7
/proc/10883/task/10888/stat : 10888 (ttsth-5-4) R 10881 10883 10639 0 -1 4202560 275899 0 0 0 120214 96 0 0 25 0 5 0 521629479 6218412032 1433144 33554432000 4194304 4338100 140735044390320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 6072668

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1230.3 s]
/proc/loadavg: 6.31 7.21 7.60 5/180 10943
/proc/meminfo: memFree=19272208/32951124 swapFree=67111352/67111528
[pid=10883] ppid=10881 vsize=0 CPUtime=4800.4 cores=1,3,5,7
/proc/10883/stat : 10883 (ttsth-5-4) Z 10881 10883 10639 0 -1 4203548 1433241 0 0 0 479512 528 0 0 18 0 2 0 521629471 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/10883/statm: 0 0 0 0 0 0 0
[pid=10883/tid=10885] ppid=10881 vsize=0 CPUtime=1189.08 cores=1,3,5,7
/proc/10883/task/10885/stat : 10885 (ttsth-5-4) R 10881 10883 10639 0 -1 4203588 666473 0 0 0 118653 255 0 0 25 0 2 0 521629479 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1230.34
CPU time (s): 4800.45
CPU user time (s): 4795.13
CPU system time (s): 5.32319
CPU usage (%): 390.173
Max. virtual memory (cumulated for all children) (KiB): 6072668

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.13
system time used= 5.32319
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1433241
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 23093
involuntary context switches= 31782

runsolver used 2.05269 second user time and 4.66929 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 20:05:13
IDJOB=3264774
IDBENCH=85343
IDSOLVER=1654
FILE ID=node134/3264774-1304100313
RUNJOBID= node134-1304097856-10658
PBS_JOBID= 13170616
Free space on /tmp= 71584 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/jarvisalo05/mod2c-rand3bip-sat/mod2c-rand3bip-sat-240-3.shuffled-as.sat05-2520.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264774-1304100313/watcher-3264774-1304100313 -o /tmp/evaluation-result-3264774-1304100313/solver-3264774-1304100313 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264774-1304100313.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= dfc7796289f8cacbd4db5eb947a1ba09
RANDOM SEED=200147760

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25015576 kB
Buffers:       1492348 kB
Cached:        5855992 kB
SwapCached:         64 kB
Active:        1467448 kB
Inactive:      5946148 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25015576 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:             228 kB
Writeback:           0 kB
AnonPages:       65340 kB
Mapped:          15340 kB
Slab:           459512 kB
PageTables:       4448 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   210200 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71584 MiB
End job on node134 at 2011-04-29 20:25:45