Trace number 3260990

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.45 1202.14

General information on the benchmark

Namecrafted/skvortsov/battleship/
battleship-11-11-unsat.cnf
MD5SUMf5821fd80ce137a7b8cae2070a9b28d7
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark4.97324
Satisfiable
(Un)Satisfiability was proved
Number of variables121
Number of clauses726
Sum of the clauses size2541
Maximum clause length11
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2605
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5121

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260990-1304086609.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 121
0.00/0.00	c Clauses = 726
0.00/0.00	c Literals = 2541
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 121
0.00/0.00	c Clauses = 726
0.00/0.00	c Literals = 2541
0.00/0.00	c a2
0.00/0.00	c a1
0.00/0.02	c Independent sub-problems = 1
0.00/0.02	c Sub-problem with 121 variables and 726 clauses
0.00/0.02	c using permutation method 4
0.00/0.02	c count = 2
0.00/0.02	c Ordering finished 0.020 seconds
0.00/0.02	c mapped 0.020 seconds
0.00/0.02	c permuted 0.020 seconds
0.00/0.02	c about to sort
0.00/0.02	c sorted
0.00/0.02	c clauses now 726
0.00/0.02	c init finished 0.020 seconds
0.00/0.02	c tree built 0.020 seconds
0.00/0.02	c 
0.00/0.02	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.00/0.02	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.00/0.02	c ................................ | ..........                    |                                  0.1         0.0             
0.00/0.02	c ................................ | ..........                  1 |                                  0.1        10.0             
0.00/0.02	c ................................ | ..........                  2 |                                  0.1        20.0             
0.00/0.02	c ................................ | ..........                  3 |                                  0.1        30.0             
0.00/0.02	c ................................ | ..........                  4 |                                  0.1        40.0             
0.00/0.02	c ................................ | ..........                  5 |                                  0.1        50.0             
0.00/0.03	c ................................ | ..........               1000 | 224          136                 0.1     10000.0         7157
96.16/24.17	c ................................ | ..........             220000 | 72384        25914              96.4      2281.4     10444263
192.01/48.18	c ................................ | ..........             300000 | 99859        39156             192.3      1559.8     16549136
289.03/72.40	c ................................ | ..........             369000 | 122902       47549             289.1      1276.6     19648299
384.49/96.38	c ................................ | ..........             459000 | 152987       53968             384.8      1192.7     20624809
481.14/120.57	c ................................ | *.........             537000 | 179092       59641             481.4      1115.4     22619696
576.21/144.38	c ................................ | *.........             618000 | 205631       65012             576.5      1071.9     26448625
672.88/168.51	c ................................ | *.........             681000 | 226672       68517             672.9      1012.0     30094208
768.71/192.56	c ................................ | *.........             742000 | 247237       72868             769.0       965.0     30728625
864.08/216.43	c ................................ | *.........             807000 | 268118       78305             864.2       933.8     32677967
961.14/240.73	c ................................ | **........             865000 | 287644       81765             961.3       899.8     35176623
1056.60/264.64	c ................................ | **........             910000 | 302814       85315            1056.8       861.1     37897924
1152.07/288.53	c ................................ | **........             951000 | 316557       90342            1152.2       825.4     40628582
1248.73/312.72	c ................................ | **........             988000 | 329153       92481            1248.8       791.1     42987064
1346.20/337.18	c ................................ | **........            1019000 | 339873       95657            1346.5       756.7     45024087
1441.67/361.08	c ................................ | ***.......            1054000 | 351806       98497            1442.0       730.9     47387378
1537.94/385.15	c ................................ | ***.......            1094000 | 365392       101778           1538.1       711.2     49549655
1633.41/409.09	c ................................ | ***.......            1116000 | 373041       105385           1633.8       683.1     51656379
1732.08/433.79	c ................................ | ***.......            1139000 | 380997       106838           1732.5       657.4     53325373
1825.15/457.00	c ................................ | ***.......            1170000 | 391486       107473           1825.2       641.0     54865207
1921.01/481.07	c ................................ | ****......            1205000 | 403234       108341           1921.3       627.2     57040615
2016.51/504.95	c ................................ | ****......            1231000 | 411982       108968           2016.7       610.4     59401549
2116.29/529.99	c ................................ | ****......            1259000 | 421430       109548           2116.7       594.8     61014565
2210.15/553.45	c ................................ | ****......            1282000 | 429372       110271           2210.4       580.0     62217716
2304.41/577.00	c ................................ | ****......            1321000 | 442462       112286           2304.4       573.2     63863058
2401.46/601.39	c ................................ | *****.....            1363000 | 456279       113048           2401.8       567.5     65316589
2496.92/625.25	c ................................ | *****.....            1400000 | 468244       113949           2497.1       560.6     66600048
2592.39/649.19	c ................................ | *****.....            1437000 | 480665       114952           2592.8       554.2     68004622
2690.21/673.61	c ................................ | *****.....            1481000 | 495409       116000           2690.3       550.5     69102304
2784.09/697.17	c ................................ | *****.....            1524000 | 509616       117554           2784.4       547.3     70338400
2880.75/721.35	c ................................ | ******....            1554000 | 520027       118651           2881.0       539.4     71883380
2977.83/745.61	c ................................ | ******....            1580000 | 528743       119023           2977.9       530.6     73287459
3072.90/769.42	c ................................ | ******....            1615000 | 540114       120606           3073.0       525.5     74459027
3168.37/793.37	c ................................ | ******....            1648000 | 550972       121531           3168.7       520.1     76134933
3264.64/817.46	c ................................ | ******....            1694000 | 566051       123349           3264.9       518.9     77911370
3360.11/841.30	c ................................ | *******...            1719000 | 574578       125350           3360.1       511.6     79620677
3456.75/865.59	c ................................ | *******...            1738000 | 580850       126238           3457.1       502.7     80795222
3553.84/889.88	c ................................ | *******...            1763000 | 589341       127807           3554.1       496.0     82350566
3652.09/914.44	c ................................ | *******...            1787000 | 597433       129463           3652.3       489.3     84184961
3744.37/937.59	c ................................ | *******...            1813000 | 606424       133183           3744.7       484.1     86433008
3843.04/962.29	c ................................ | ********..            1837000 | 614754       135738           3843.4       478.0     88463708
3938.02/986.00	c ................................ | ********..            1867000 | 624807       138436           3938.1       474.1     90341619
4032.92/1009.86	c ................................ | ********..            1897000 | 635010       141432           4033.2       470.4     91046900
4130.36/1034.22	c ................................ | ********..            1913000 | 640733       145922           4130.4       463.1     91831277
4225.00/1057.91	c ................................ | ********..            1928000 | 645979       149688           4225.1       456.3     93392459
4323.04/1082.56	c ................................ | *********.            1950000 | 653476       154722           4323.2       451.0     93685774
4416.40/1105.93	c ................................ | *********.            1984000 | 664952       158126           4416.5       449.2     93685774
4512.67/1130.05	c ................................ | *********.            2009000 | 673299       160875           4512.9       445.2     93685774
4607.75/1153.86	c ................................ | *********.            2036000 | 682370       162657           4608.0       441.8     93685774
4704.42/1178.06	c ................................ | *********.            2061000 | 691010       163932           4704.6       438.1     93924286

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3260990-1304086609/watcher-3260990-1304086609 -o /tmp/evaluation-result-3260990-1304086609/solver-3260990-1304086609 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260990-1304086609.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.01 2.16 5.28 1/172 25076
/proc/meminfo: memFree=17385568/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=8972 CPUtime=0 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) R 25074 25076 25048 0 -1 4202496 334 0 0 0 0 0 0 0 22 0 1 0 520261854 9187328 277 33554432000 4194304 4338100 140735273567392 18446744073709551615 4295924 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/25076/statm: 2243 277 152 36 0 148 0

[startup+0.0708469 s]
/proc/loadavg: 0.01 2.16 5.28 1/172 25076
/proc/meminfo: memFree=17385568/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=195512 CPUtime=0.18 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 1859 0 0 0 18 0 0 0 22 0 5 0 520261854 200204288 1802 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 48878 1804 173 36 0 46783 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 195512

[startup+0.100848 s]
/proc/loadavg: 0.01 2.16 5.28 1/172 25076
/proc/meminfo: memFree=17385568/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=195640 CPUtime=0.3 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 2509 0 0 0 30 0 0 0 22 0 5 0 520261854 200335360 2452 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 48910 2452 173 36 0 46815 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 195640

[startup+0.30079 s]
/proc/loadavg: 0.01 2.16 5.28 1/172 25076
/proc/meminfo: memFree=17385568/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=208152 CPUtime=1.1 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 5322 0 0 0 109 1 0 0 22 0 5 0 520261854 213147648 5265 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 52038 5265 173 36 0 49943 0
Current children cumulated CPU time (s) 1.1
Current children cumulated vsize (KiB) 208152

[startup+0.700722 s]
/proc/loadavg: 0.01 2.16 5.28 1/172 25076
/proc/meminfo: memFree=17385568/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=220664 CPUtime=2.7 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 9100 0 0 0 267 3 0 0 22 0 5 0 520261854 225959936 9043 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 55166 9043 173 36 0 53071 0
Current children cumulated CPU time (s) 2.7
Current children cumulated vsize (KiB) 220664

[startup+1.50058 s]
/proc/loadavg: 0.01 2.16 5.28 6/182 25091
/proc/meminfo: memFree=17339592/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=245816 CPUtime=5.89 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 14500 0 0 0 583 6 0 0 22 0 5 0 520261854 251715584 14443 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 61454 14443 173 36 0 59359 0
[pid=25076/tid=25078] ppid=25074 vsize=245816 CPUtime=1.46 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 3233 0 0 0 146 0 0 0 20 0 5 0 520261857 251715584 14443 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=245816 CPUtime=1.46 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 2480 0 0 0 144 2 0 0 25 0 5 0 520261857 251715584 14443 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=245816 CPUtime=1.46 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 5147 0 0 0 146 0 0 0 20 0 5 0 520261857 251715584 14443 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=245816 CPUtime=1.46 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 2868 0 0 0 143 3 0 0 25 0 5 0 520261857 251715584 14443 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.89
Current children cumulated vsize (KiB) 245816

[startup+3.1003 s]
/proc/loadavg: 0.01 2.16 5.28 5/177 25112
/proc/meminfo: memFree=17310212/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=283352 CPUtime=12.29 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 24370 0 0 0 1216 13 0 0 22 0 5 0 520261854 290152448 24313 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 70838 24314 173 36 0 68743 0
[pid=25076/tid=25078] ppid=25074 vsize=283352 CPUtime=3.06 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 5164 0 0 0 306 0 0 0 25 0 5 0 520261857 290152448 24314 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=283352 CPUtime=3.05 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 4449 0 0 0 300 5 0 0 25 0 5 0 520261857 290152448 24314 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=283352 CPUtime=3.06 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 9101 0 0 0 305 1 0 0 25 0 5 0 520261857 290152448 24314 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=283352 CPUtime=3.06 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 4885 0 0 0 301 5 0 0 25 0 5 0 520261857 290152448 24314 33554432000 4194304 4338100 140735273567392 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.29
Current children cumulated vsize (KiB) 283352

[startup+6.30073 s]
/proc/loadavg: 0.33 2.19 5.27 5/177 25112
/proc/meminfo: memFree=17247980/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=336660 CPUtime=25.06 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 37995 0 0 0 2481 25 0 0 22 0 5 0 520261854 344739840 37938 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 84165 37938 173 36 0 82070 0
[pid=25076/tid=25078] ppid=25074 vsize=336660 CPUtime=6.26 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 7537 0 0 0 625 1 0 0 25 0 5 0 520261857 344739840 37938 33554432000 4194304 4338100 140735273567392 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=336660 CPUtime=6.23 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 7170 0 0 0 613 10 0 0 25 0 5 0 520261857 344739840 37938 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=336660 CPUtime=6.26 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 14747 0 0 0 624 2 0 0 25 0 5 0 520261857 344739840 37938 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254347 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=336660 CPUtime=6.26 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 7769 0 0 0 616 10 0 0 25 0 5 0 520261857 344739840 37938 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 25.06
Current children cumulated vsize (KiB) 336660

[startup+12.7006 s]
/proc/loadavg: 0.78 2.26 5.27 7/182 25123
/proc/meminfo: memFree=17146372/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=408604 CPUtime=50.61 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 55179 0 0 0 5018 43 0 0 22 0 5 0 520261854 418410496 55122 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 102151 55122 173 36 0 100056 0
[pid=25076/tid=25078] ppid=25074 vsize=408604 CPUtime=12.66 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 11191 0 0 0 1264 2 0 0 25 0 5 0 520261857 418410496 55122 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=408604 CPUtime=12.59 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 10592 0 0 0 1242 17 0 0 25 0 5 0 520261857 418410496 55122 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=408604 CPUtime=12.66 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 21371 0 0 0 1262 4 0 0 25 0 5 0 520261857 418410496 55122 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=408604 CPUtime=12.65 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 11253 0 0 0 1247 18 0 0 25 0 5 0 520261857 418410496 55122 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.61
Current children cumulated vsize (KiB) 408604

[startup+25.5003 s]
/proc/loadavg: 1.71 2.39 5.27 7/182 25123
/proc/meminfo: memFree=17014028/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=535508 CPUtime=101.73 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 87564 0 0 0 10102 71 0 0 22 0 5 0 520261854 548360192 87504 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 133877 87504 176 36 0 131782 0
[pid=25076/tid=25078] ppid=25074 vsize=535508 CPUtime=25.45 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 17377 0 0 0 2542 3 0 0 25 0 5 0 520261857 548360192 87504 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=535508 CPUtime=25.31 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 16532 0 0 0 2502 29 0 0 25 0 5 0 520261857 548360192 87504 33554432000 4194304 4338100 140735273567392 18446744073709551615 4256770 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=535508 CPUtime=25.46 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 35247 0 0 0 2538 8 0 0 25 0 5 0 520261857 548360192 87504 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=535508 CPUtime=25.44 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 17636 0 0 0 2515 29 0 0 25 0 5 0 520261857 548360192 87504 33554432000 4194304 4338100 140735273567392 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.73
Current children cumulated vsize (KiB) 535508

[startup+51.1008 s]
/proc/loadavg: 3.37 2.71 5.30 10/186 25128
/proc/meminfo: memFree=16658704/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=729572 CPUtime=203.99 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 135948 0 0 0 20286 113 0 0 22 0 5 0 520261854 747081728 135888 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 182393 135888 176 36 0 180298 0
[pid=25076/tid=25078] ppid=25074 vsize=729572 CPUtime=51.05 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 24833 0 0 0 5100 5 0 0 25 0 5 0 520261857 747081728 135888 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=729572 CPUtime=50.77 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 26255 0 0 0 5031 46 0 0 25 0 5 0 520261857 747081728 135888 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=729572 CPUtime=51.06 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 56235 0 0 0 5092 14 0 0 25 0 5 0 520261857 747081728 135888 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=729572 CPUtime=51.04 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 27853 0 0 0 5059 45 0 0 25 0 5 0 520261857 747081728 135888 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.99
Current children cumulated vsize (KiB) 729572

[startup+102.307 s]
/proc/loadavg: 6.08 3.56 5.45 10/186 25129
/proc/meminfo: memFree=16094100/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=899392 CPUtime=408.47 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 176682 0 0 0 40680 167 0 0 22 0 5 0 520261854 920977408 176622 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 224848 176622 176 36 0 222753 0
[pid=25076/tid=25078] ppid=25074 vsize=899392 CPUtime=102.26 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 32191 0 0 0 10219 7 0 0 25 0 5 0 520261857 920977408 176622 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=899392 CPUtime=101.67 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 35416 0 0 0 10096 71 0 0 25 0 5 0 520261857 920977408 176622 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=899392 CPUtime=102.26 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 74104 0 0 0 10207 19 0 0 25 0 5 0 520261857 920977408 176622 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=899392 CPUtime=102.22 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 34199 0 0 0 10154 68 0 0 25 0 5 0 520261857 920977408 176622 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 408.47
Current children cumulated vsize (KiB) 899392

[startup+162.301 s]
/proc/loadavg: 7.33 4.38 5.61 10/186 25131
/proc/meminfo: memFree=15659596/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=1221552 CPUtime=648.12 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 240090 0 0 0 64576 236 0 0 22 0 5 0 520261854 1250869248 240030 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 305388 240030 176 36 0 303293 0
[pid=25076/tid=25078] ppid=25074 vsize=1221552 CPUtime=162.25 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 40632 0 0 0 16216 9 0 0 25 0 5 0 520261857 1250869248 240030 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=1221552 CPUtime=161.33 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 48991 0 0 0 16034 99 0 0 25 0 5 0 520261857 1250869248 240030 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=1221552 CPUtime=162.25 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 102169 0 0 0 16197 28 0 0 25 0 5 0 520261857 1250869248 240030 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=1221552 CPUtime=162.22 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 47526 0 0 0 16125 97 0 0 25 0 5 0 520261857 1250869248 240030 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 648.12
Current children cumulated vsize (KiB) 1221552


################
# More data... #
################

/proc/meminfo: memFree=13496008/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=2628404 CPUtime=3044.56 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 592644 0 0 0 303816 640 0 0 22 0 5 0 520261854 2691485696 592584 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 657101 592584 176 36 0 655006 0
[pid=25076/tid=25078] ppid=25074 vsize=2628404 CPUtime=761.32 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 116112 0 0 0 76095 37 0 0 25 0 5 0 520261857 2691485696 592584 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=2628404 CPUtime=761.03 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 123022 0 0 0 75837 266 0 0 25 0 5 0 520261857 2691485696 592584 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=2628404 CPUtime=762.24 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 233912 0 0 0 76152 72 0 0 25 0 5 0 520261857 2691485696 592584 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=2628404 CPUtime=759.92 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 118826 0 0 0 75728 264 0 0 25 0 5 0 520261857 2691485696 592584 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3044.56
Current children cumulated vsize (KiB) 2628404

[startup+822.3 s]
/proc/loadavg: 8.23 7.79 6.90 10/186 25148
/proc/meminfo: memFree=13313056/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=2756652 CPUtime=3284.21 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 625032 0 0 0 327745 676 0 0 22 0 5 0 520261854 2822811648 624972 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 689163 624972 176 36 0 687068 0
[pid=25076/tid=25078] ppid=25074 vsize=2756652 CPUtime=821.11 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 123622 0 0 0 82071 40 0 0 25 0 5 0 520261857 2822811648 624972 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=2756652 CPUtime=821.02 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 129474 0 0 0 81823 279 0 0 25 0 5 0 520261857 2822811648 624972 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=2756652 CPUtime=822.11 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 245791 0 0 0 82135 76 0 0 25 0 5 0 520261857 2822811648 624972 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254317 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=2756652 CPUtime=819.91 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 125373 0 0 0 81713 278 0 0 25 0 5 0 520261857 2822811648 624972 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3284.21
Current children cumulated vsize (KiB) 2756652

[startup+882.301 s]
/proc/loadavg: 8.08 7.82 6.97 10/186 25150
/proc/meminfo: memFree=13177348/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=2875516 CPUtime=3523.86 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 653817 0 0 0 351673 713 0 0 22 0 5 0 520261854 2944528384 653757 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 718879 653757 176 36 0 716784 0
[pid=25076/tid=25078] ppid=25074 vsize=2875516 CPUtime=881.09 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 128498 0 0 0 88066 43 0 0 25 0 5 0 520261857 2944528384 653757 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=2875516 CPUtime=881.02 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 135281 0 0 0 87809 293 0 0 25 0 5 0 520261857 2944528384 653757 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=2875516 CPUtime=881.78 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 257709 0 0 0 88096 82 0 0 25 0 5 0 520261857 2944528384 653757 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=2875516 CPUtime=879.91 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 131557 0 0 0 87698 293 0 0 25 0 5 0 520261857 2944528384 653757 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3523.86
Current children cumulated vsize (KiB) 2875516

[startup+942.301 s]
/proc/loadavg: 8.07 7.87 7.03 10/186 25151
/proc/meminfo: memFree=12980876/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=3025660 CPUtime=3763.55 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 692194 0 0 0 375600 755 0 0 22 0 5 0 520261854 3098275840 692134 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 756415 692134 176 36 0 754320 0
[pid=25076/tid=25078] ppid=25074 vsize=3025660 CPUtime=941.09 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 134085 0 0 0 94064 45 0 0 25 0 5 0 520261857 3098275840 692134 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=3025660 CPUtime=941.02 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 143583 0 0 0 93792 310 0 0 25 0 5 0 520261857 3098275840 692134 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=3025660 CPUtime=941.46 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 273920 0 0 0 94059 87 0 0 25 0 5 0 520261857 3098275840 692134 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=3025660 CPUtime=939.91 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 139834 0 0 0 93680 311 0 0 25 0 5 0 520261857 3098275840 692134 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3763.55
Current children cumulated vsize (KiB) 3025660

[startup+1002.3 s]
/proc/loadavg: 8.09 7.91 7.10 10/186 25153
/proc/meminfo: memFree=12782292/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=3180772 CPUtime=4002.93 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 727242 0 0 0 399492 801 0 0 22 0 5 0 520261854 3257110528 727182 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 795193 727182 176 36 0 793098 0
[pid=25076/tid=25078] ppid=25074 vsize=3180772 CPUtime=1001.06 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 139840 0 0 0 100059 47 0 0 25 0 5 0 520261857 3257110528 727182 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=3180772 CPUtime=1000.96 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 149662 0 0 0 99767 329 0 0 25 0 5 0 520261857 3257110528 727182 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=3180772 CPUtime=1000.99 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 289136 0 0 0 100006 93 0 0 25 0 5 0 520261857 3257110528 727182 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=3180772 CPUtime=999.86 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 147832 0 0 0 99657 329 0 0 25 0 5 0 520261857 3257110528 727182 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4002.93
Current children cumulated vsize (KiB) 3180772

[startup+1062.3 s]
/proc/loadavg: 8.11 7.95 7.16 10/186 25155
/proc/meminfo: memFree=12681556/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=3262100 CPUtime=4242.58 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 748339 0 0 0 423419 839 0 0 22 0 5 0 520261854 3340390400 748279 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 815525 748282 176 36 0 813430 0
[pid=25076/tid=25078] ppid=25074 vsize=3262100 CPUtime=1061.06 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 139841 0 0 0 106059 47 0 0 25 0 5 0 520261857 3340390400 748284 33554432000 4194304 4338100 140735273567392 18446744073709551615 4235858 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=3262100 CPUtime=1060.94 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 155806 0 0 0 105747 347 0 0 25 0 5 0 520261857 3340390400 748287 33554432000 4194304 4338100 140735273567392 18446744073709551615 4240533 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=3262100 CPUtime=1060.67 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 303988 0 0 0 105968 99 0 0 25 0 5 0 520261857 3340390400 748290 33554432000 4194304 4338100 140735273567392 18446744073709551615 4235756 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=3262100 CPUtime=1059.85 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 147943 0 0 0 105641 344 0 0 25 0 5 0 520261857 3340390400 748292 33554432000 4194304 4338100 140735273567392 18446744073709551615 4240533 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4242.58
Current children cumulated vsize (KiB) 3262100

[startup+1122.3 s]
/proc/loadavg: 8.08 7.97 7.22 10/186 25156
/proc/meminfo: memFree=12511748/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=3439308 CPUtime=4481.91 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 782855 0 0 0 447311 880 0 0 22 0 5 0 520261854 3521851392 782795 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 859827 782795 176 36 0 857732 0
[pid=25076/tid=25078] ppid=25074 vsize=3439308 CPUtime=1121.03 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 154283 0 0 0 112051 52 0 0 25 0 5 0 520261857 3521851392 782795 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=3439308 CPUtime=1120.61 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 162803 0 0 0 111696 365 0 0 25 0 5 0 520261857 3521851392 782795 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=3439308 CPUtime=1120.52 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 307710 0 0 0 111952 100 0 0 25 0 5 0 520261857 3521851392 782795 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=3439308 CPUtime=1119.71 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 157287 0 0 0 111609 362 0 0 25 0 5 0 520261857 3521851392 782795 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4481.91
Current children cumulated vsize (KiB) 3439308

[startup+1182.3 s]
/proc/loadavg: 8.06 7.99 7.27 10/186 25158
/proc/meminfo: memFree=12420040/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=3445564 CPUtime=4721.58 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 787099 0 0 0 471258 900 0 0 22 0 5 0 520261854 3528257536 787039 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 861391 787039 176 36 0 859296 0
[pid=25076/tid=25078] ppid=25074 vsize=3445564 CPUtime=1181.03 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 154286 0 0 0 118051 52 0 0 25 0 5 0 520261857 3528257536 787039 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=3445564 CPUtime=1180.28 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 162812 0 0 0 117654 374 0 0 25 0 5 0 520261857 3528257536 787039 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=3445564 CPUtime=1180.51 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 309583 0 0 0 117951 100 0 0 25 0 5 0 520261857 3528257536 787039 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=3445564 CPUtime=1179.71 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 159646 0 0 0 117598 373 0 0 25 0 5 0 520261857 3528257536 787039 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4721.58
Current children cumulated vsize (KiB) 3445564



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202 s]
/proc/loadavg: 8.11 8.00 7.29 9/186 25158
/proc/meminfo: memFree=12396092/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=3445564 CPUtime=4800.29 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) S 25074 25076 25048 0 -1 4202496 787289 0 0 0 479121 908 0 0 22 0 5 0 520261854 3528257536 787229 33554432000 4194304 4338100 140735273567392 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 861391 787229 176 36 0 859296 0
[pid=25076/tid=25078] ppid=25074 vsize=3445564 CPUtime=1200.74 cores=1,3,5,7
/proc/25076/task/25078/stat : 25078 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 154288 0 0 0 120022 52 0 0 25 0 5 0 520261857 3528257536 787229 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=3445564 CPUtime=1199.86 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 162816 0 0 0 119609 377 0 0 25 0 5 0 520261857 3528257536 787229 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25076/tid=25080] ppid=25074 vsize=3445564 CPUtime=1200.22 cores=1,3,5,7
/proc/25076/task/25080/stat : 25080 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 309587 0 0 0 119922 100 0 0 25 0 5 0 520261857 3528257536 787229 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25076/tid=25081] ppid=25074 vsize=3445564 CPUtime=1199.41 cores=1,3,5,7
/proc/25076/task/25081/stat : 25081 (ttsth-5-4) R 25074 25076 25048 0 -1 4202560 159826 0 0 0 119564 377 0 0 25 0 5 0 520261857 3528257536 787229 33554432000 4194304 4338100 140735273567392 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 3445564

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.11 s]
/proc/loadavg: 8.11 8.00 7.29 9/186 25158
/proc/meminfo: memFree=12396092/32951124 swapFree=67111528/67111528
[pid=25076] ppid=25074 vsize=0 CPUtime=4800.41 cores=1,3,5,7
/proc/25076/stat : 25076 (ttsth-5-4) Z 25074 25076 25048 0 -1 4203548 787289 0 0 0 479123 918 0 0 18 0 2 0 520261854 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25076/statm: 0 0 0 0 0 0 0
[pid=25076/tid=25079] ppid=25074 vsize=0 CPUtime=1199.97 cores=1,3,5,7
/proc/25076/task/25079/stat : 25079 (ttsth-5-4) R 25074 25076 25048 0 -1 4203588 162816 0 0 0 119610 387 0 0 25 0 2 0 520261857 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.41
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.14
CPU time (s): 4800.45
CPU user time (s): 4791.24
CPU system time (s): 9.2146
CPU usage (%): 399.326
Max. virtual memory (cumulated for all children) (KiB): 3445564

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4791.24
system time used= 9.2146
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 787289
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2607
involuntary context switches= 30477

runsolver used 2.04769 second user time and 4.50031 second system time

The end

Launcher Data 

Begin job on node108 at 2011-04-29 16:16:49
IDJOB=3260990
IDBENCH=83203
IDSOLVER=1654
FILE ID=node108/3260990-1304086609
RUNJOBID= node108-1304086608-25067
PBS_JOBID= 13170551
Free space on /tmp= 72120 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/skvortsov/battleship/battleship-11-11-unsat.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3260990-1304086609/watcher-3260990-1304086609 -o /tmp/evaluation-result-3260990-1304086609/solver-3260990-1304086609 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260990-1304086609.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f5821fd80ce137a7b8cae2070a9b28d7
RANDOM SEED=866884112

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17385848 kB
Buffers:       2164508 kB
Cached:       12755556 kB
SwapCached:          0 kB
Active:        4200324 kB
Inactive:     10783240 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17385848 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:              28 kB
Writeback:           0 kB
AnonPages:       62996 kB
Mapped:          14400 kB
Slab:           517756 kB
PageTables:       4592 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184752 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72120 MiB
End job on node108 at 2011-04-29 16:36:53