Trace number 3261562

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.33 1203.62

General information on the benchmark

Namecrafted/skvortsov/battleship/
battleship-17-33-sat.cnf
MD5SUMff3a31481d1bf95f014a91f5192a55fa
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.038993
Satisfiable
(Un)Satisfiability was proved
Number of variables561
Number of clauses4777
Sum of the clauses size18513
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 24488
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5289

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3261562-1304088793.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 561
0.00/0.00	c Clauses = 4777
0.00/0.00	c Literals = 18513
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 561
0.00/0.00	c Clauses = 4777
0.00/0.00	c Literals = 18513
0.00/0.00	c a2
0.00/0.03	c a1
1.89/1.90	c Independent sub-problems = 1
1.89/1.90	c Sub-problem with 561 variables and 4777 clauses
1.89/1.90	c using permutation method 4
1.89/1.90	c count = 2
1.89/1.90	c Ordering finished 1.900 seconds
1.89/1.90	c mapped 1.900 seconds
1.89/1.90	c permuted 1.900 seconds
1.89/1.90	c about to sort
1.89/1.90	c sorted
1.89/1.92	c clauses now 4777
1.89/1.92	c init finished 1.910 seconds
1.89/1.92	c tree built 1.910 seconds
1.89/1.92	c 
1.89/1.92	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.89/1.92	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.89/1.92	c ................................ | ..........                    |                                  1.9         0.0             
1.89/1.92	c ................................ | ..........                  1 |                                  1.9         0.5             
1.89/1.92	c ................................ | ..........                  2 |                                  1.9         1.0             
1.89/1.92	c ................................ | ..........                  3 |                                  1.9         1.6             
1.89/1.92	c ................................ | ..........                  4 |                                  1.9         2.1             
1.89/1.92	c ................................ | ..........                  5 |                                  1.9         2.6             
1.89/1.92	c ................................ | ..........               1000 | 94           93                  1.9       520.8         2749
96.05/25.50	c ................................ | ..........             464000 | 145097       2226               96.0      4830.8      9207880
191.90/49.57	c ................................ | ..........             763000 | 238532       2448              192.2      3970.2     12981547
288.91/73.80	c ................................ | ..........             897000 | 281246       2490              288.9      3104.7     16522621
383.97/97.66	c ................................ | ..........            1073000 | 337272       2505              384.2      2792.5     20821628
480.60/121.84	c ................................ | *.........            1255000 | 393315       2796              480.8      2610.3     22123534
576.43/145.82	c ................................ | *.........            1442000 | 451947       2805              576.5      2501.1     22888273
671.86/169.77	c ................................ | *.........            1563000 | 490506       2905              672.1      2325.4     24080301
768.08/193.85	c ................................ | *.........            1709000 | 537048       2917              768.3      2224.4     26005525
863.98/217.87	c ................................ | *.........            1847000 | 581111       2923              864.3      2137.1     28630837
961.37/242.29	c ................................ | **........            1916000 | 603355       2975              961.8      1992.2     31107785
1056.85/266.14	c ................................ | **........            1989000 | 626796       2978             1057.0      1881.7     32587853
1152.65/290.16	c ................................ | **........            2078000 | 655370       3086             1152.9      1802.4     32840378
1247.71/313.99	c ................................ | **........            2187000 | 689855       3086             1248.1      1752.3     34270280
1344.36/338.16	c ................................ | **........            2302000 | 725906       3116             1344.6      1712.0     37242352
1439.76/362.07	c ................................ | ***.......            2418000 | 762114       3125             1440.1      1679.1     37242352
1536.38/386.29	c ................................ | ***.......            2505000 | 789747       3128             1536.7      1630.1     37601500
1633.44/410.59	c ................................ | ***.......            2641000 | 832716       3291             1633.8      1616.5     39521308
1728.88/434.45	c ................................ | ***.......            2781000 | 876857       3396             1729.1      1608.4     42067388
1826.33/458.87	c ................................ | ***.......            2847000 | 898039       3399             1826.6      1558.6     43303712
1921.23/482.66	c ................................ | ****......            2891000 | 912070       3459             1921.5      1504.6     45258275
2016.70/506.54	c ................................ | ****......            2974000 | 938706       3477             2016.9      1474.6     45258275
2113.32/530.73	c ................................ | ****......            3051000 | 963056       3486             2113.4      1443.6     47502750
2208.00/554.48	c ................................ | ****......            3137000 | 990164       3489             2208.3      1420.6     50535673
2305.05/578.70	c ................................ | ****......            3202000 | 1010543      3504             2305.1      1389.1     52445745
2401.15/602.81	c ................................ | *****.....            3349000 | 1055475      3561             2401.2      1394.7     54198254
2496.97/626.86	c ................................ | *****.....            3397000 | 1070318      3576             2497.2      1360.3     54198254
2592.44/650.72	c ................................ | *****.....            3454000 | 1088094      3576             2592.6      1332.3     55143962
2687.91/674.69	c ................................ | *****.....            3522000 | 1109076      3576             2688.3      1310.1     56780381
2784.57/698.80	c ................................ | *****.....            3646000 | 1146962      3615             2784.6      1309.3     58379359
2880.06/722.73	c ................................ | ******....            3761000 | 1181373      3651             2880.2      1305.8     59794156
2977.50/747.13	c ................................ | ******....            3862000 | 1212692      3651             2977.6      1297.0     60932919
3072.57/770.91	c ................................ | ******....            3895000 | 1222941      3651             3072.6      1267.6     61784172
3168.04/794.86	c ................................ | ******....            3950000 | 1239722      3660             3168.3      1246.7     62990690
3263.89/818.87	c ................................ | ******....            4014000 | 1259605      3672             3264.2      1229.7     64814530
3360.13/842.95	c ................................ | *******...            4090000 | 1283164      3675             3360.4      1217.1     67096823
3456.39/867.05	c ................................ | *******...            4164000 | 1306343      3740             3456.6      1204.7     69231049
3552.28/891.05	c ................................ | *******...            4246000 | 1330258      3812             3552.5      1195.2     69520959
3648.09/915.08	c ................................ | *******...            4334000 | 1357152      3812             3648.4      1187.9     70141178
3746.35/939.65	c ................................ | *******...            4469000 | 1396836      3872             3746.6      1192.8     71213071
3843.42/963.96	c ................................ | ********..            4514000 | 1410583      3887             3843.7      1174.4     72897838
3937.28/987.46	c ................................ | ********..            4566000 | 1426544      3935             3937.6      1159.6     73605272
4032.35/1011.29	c ................................ | ********..            4602000 | 1437696      3953             4032.7      1141.2     75029507
4129.82/1035.62	c ................................ | ********..            4665000 | 1456837      3980             4129.9      1129.6     76962721
4224.44/1059.31	c ................................ | ********..            4729000 | 1476100      4068             4224.5      1119.4     77866687
4319.82/1083.24	c ................................ | *********.            4828000 | 1506509      4285             4320.0      1117.6     79316498
4416.08/1107.39	c ................................ | *********.            4927000 | 1536404      4333             4416.5      1115.6     80743306
4512.24/1131.45	c ................................ | *********.            5008000 | 1561645      4339             4512.4      1109.8     81842385
4608.51/1155.55	c ................................ | *********.            5059000 | 1577761      4393             4608.7      1097.7     82840659
4703.96/1179.43	c ................................ | *********.            5097000 | 1589499      4459             4704.1      1083.5     84162159

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3261562-1304088793/watcher-3261562-1304088793 -o /tmp/evaluation-result-3261562-1304088793/solver-3261562-1304088793 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3261562-1304088793.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.04 8.06 8.00 2/174 24326
/proc/meminfo: memFree=25321660/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=9376 CPUtime=0 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) R 24324 24326 24182 0 -1 4202496 404 0 0 0 0 0 0 0 25 0 1 0 520482079 9601024 346 33554432000 4194304 4338100 140736653658128 18446744073709551615 4209920 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/24326/statm: 2344 346 149 36 0 249 0

[startup+0.0887149 s]
/proc/loadavg: 8.04 8.06 8.00 2/174 24326
/proc/meminfo: memFree=25321660/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=14352 CPUtime=0.08 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) R 24324 24326 24182 0 -1 4202496 1662 0 0 0 8 0 0 0 25 0 1 0 520482079 14696448 1603 33554432000 4194304 4338100 140736653658128 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/24326/statm: 3588 1603 154 36 0 1493 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 14352

[startup+0.100701 s]
/proc/loadavg: 8.04 8.06 8.00 2/174 24326
/proc/meminfo: memFree=25321660/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=14352 CPUtime=0.09 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) R 24324 24326 24182 0 -1 4202496 1662 0 0 0 9 0 0 0 25 0 1 0 520482079 14696448 1603 33554432000 4194304 4338100 140736653658128 18446744073709551615 4245192 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/24326/statm: 3588 1603 154 36 0 1493 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 14352

[startup+0.300666 s]
/proc/loadavg: 8.04 8.06 8.00 2/174 24326
/proc/meminfo: memFree=25321660/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=14352 CPUtime=0.29 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) R 24324 24326 24182 0 -1 4202496 1663 0 0 0 29 0 0 0 25 0 1 0 520482079 14696448 1604 33554432000 4194304 4338100 140736653658128 18446744073709551615 4295924 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/24326/statm: 3588 1604 155 36 0 1493 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 14352

[startup+0.700593 s]
/proc/loadavg: 8.04 8.06 8.00 2/174 24326
/proc/meminfo: memFree=25321660/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=14352 CPUtime=0.69 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) R 24324 24326 24182 0 -1 4202496 1663 0 0 0 69 0 0 0 25 0 1 0 520482079 14696448 1604 33554432000 4194304 4338100 140736653658128 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/24326/statm: 3588 1604 155 36 0 1493 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 14352

[startup+1.50046 s]
/proc/loadavg: 8.04 8.06 8.00 3/178 24348
/proc/meminfo: memFree=25315436/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=14352 CPUtime=1.49 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) R 24324 24326 24182 0 -1 4202496 1663 0 0 0 149 0 0 0 25 0 1 0 520482079 14696448 1604 33554432000 4194304 4338100 140736653658128 18446744073709551615 4297987 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/24326/statm: 3588 1604 155 36 0 1493 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 14352

[startup+3.10114 s]
/proc/loadavg: 8.04 8.06 8.00 6/182 24352
/proc/meminfo: memFree=25299208/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=310216 CPUtime=6.61 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 15568 0 0 0 655 6 0 0 25 0 5 0 520482079 317661184 15112 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 77554 15112 175 36 0 75459 0
[pid=24326/tid=24349] ppid=24324 vsize=310216 CPUtime=1.16 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 3147 0 0 0 116 0 0 0 19 0 5 0 520482272 317661184 15112 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=310216 CPUtime=1.16 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 1959 0 0 0 115 1 0 0 19 0 5 0 520482272 317661184 15112 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=310216 CPUtime=1.16 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 4899 0 0 0 115 1 0 0 19 0 5 0 520482272 317661184 15112 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=310216 CPUtime=1.16 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 1613 0 0 0 115 1 0 0 25 0 5 0 520482272 317661184 15112 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 6.61
Current children cumulated vsize (KiB) 310216

[startup+6.30057 s]
/proc/loadavg: 7.79 8.01 7.99 6/182 24352
/proc/meminfo: memFree=25215012/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=372776 CPUtime=19.38 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 31191 0 0 0 1925 13 0 0 25 0 5 0 520482079 381722624 30735 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 93194 30735 175 36 0 91099 0
[pid=24326/tid=24349] ppid=24324 vsize=372776 CPUtime=4.37 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 7527 0 0 0 436 1 0 0 25 0 5 0 520482272 381722624 30735 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254226 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=372776 CPUtime=4.36 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 3743 0 0 0 432 4 0 0 25 0 5 0 520482272 381722624 30735 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=372776 CPUtime=4.34 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 12555 0 0 0 431 3 0 0 25 0 5 0 520482272 381722624 30736 33554432000 4194304 4338100 140736653658128 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=372776 CPUtime=4.37 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 3416 0 0 0 433 4 0 0 25 0 5 0 520482272 381722624 30737 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 19.38
Current children cumulated vsize (KiB) 372776

[startup+12.7004 s]
/proc/loadavg: 7.57 7.96 7.97 9/186 24418
/proc/meminfo: memFree=25096060/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=470324 CPUtime=44.93 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 55208 0 0 0 4465 28 0 0 25 0 5 0 520482079 481611776 54744 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 117581 54744 176 36 0 115486 0
[pid=24326/tid=24349] ppid=24324 vsize=470324 CPUtime=10.76 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 11173 0 0 0 1074 2 0 0 25 0 5 0 520482272 481611776 54744 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=470324 CPUtime=10.76 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 7112 0 0 0 1067 9 0 0 25 0 5 0 520482272 481611776 54744 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=470324 CPUtime=10.7 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 26885 0 0 0 1064 6 0 0 25 0 5 0 520482272 481611776 54744 33554432000 4194304 4338100 140736653658128 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=470324 CPUtime=10.76 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 6088 0 0 0 1067 9 0 0 25 0 5 0 520482272 481611776 54744 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 44.93
Current children cumulated vsize (KiB) 470324

[startup+25.5011 s]
/proc/loadavg: 7.66 7.96 7.97 9/186 24419
/proc/meminfo: memFree=24788684/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=579916 CPUtime=96.05 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 80619 0 0 0 9560 45 0 0 25 0 5 0 520482079 593833984 80155 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 144979 80155 176 36 0 142884 0
[pid=24326/tid=24349] ppid=24324 vsize=579916 CPUtime=23.56 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 18804 0 0 0 2352 4 0 0 25 0 5 0 520482272 593833984 80155 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=579916 CPUtime=23.56 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 9922 0 0 0 2340 16 0 0 25 0 5 0 520482272 593833984 80155 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=579916 CPUtime=23.43 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 39204 0 0 0 2335 8 0 0 25 0 5 0 520482272 593833984 80155 33554432000 4194304 4338100 140736653658128 18446744073709551615 218370352149 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=579916 CPUtime=23.55 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 8739 0 0 0 2340 15 0 0 25 0 5 0 520482272 593833984 80155 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 96.05
Current children cumulated vsize (KiB) 579916

[startup+51.1006 s]
/proc/loadavg: 7.78 7.96 7.97 9/186 24419
/proc/meminfo: memFree=24539532/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=703900 CPUtime=198.27 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 110914 0 0 0 19757 70 0 0 25 0 5 0 520482079 720793600 110450 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 175975 110450 176 36 0 173880 0
[pid=24326/tid=24349] ppid=24324 vsize=703900 CPUtime=49.15 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 32774 0 0 0 4908 7 0 0 25 0 5 0 520482272 720793600 110450 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=703900 CPUtime=49.14 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 10122 0 0 0 4889 25 0 0 25 0 5 0 520482272 720793600 110450 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=703900 CPUtime=48.88 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 54671 0 0 0 4877 11 0 0 25 0 5 0 520482272 720793600 110450 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=703900 CPUtime=49.13 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 9397 0 0 0 4889 24 0 0 25 0 5 0 520482272 720793600 110450 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 198.27
Current children cumulated vsize (KiB) 703900

[startup+102.306 s]
/proc/loadavg: 7.95 7.98 7.98 9/186 24421
/proc/meminfo: memFree=24160584/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=956264 CPUtime=402.75 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 173814 0 0 0 40150 125 0 0 25 0 5 0 520482079 979214336 173350 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 239066 173350 176 36 0 236971 0
[pid=24326/tid=24349] ppid=24324 vsize=956264 CPUtime=100.35 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 45171 0 0 0 10024 11 0 0 25 0 5 0 520482272 979214336 173350 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=956264 CPUtime=100.33 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 14455 0 0 0 9989 44 0 0 25 0 5 0 520482272 979214336 173350 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=956264 CPUtime=99.8 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 95040 0 0 0 9956 24 0 0 25 0 5 0 520482272 979214336 173350 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=956264 CPUtime=100.32 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 15198 0 0 0 9987 45 0 0 25 0 5 0 520482272 979214336 173350 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 402.75
Current children cumulated vsize (KiB) 956264

[startup+162.301 s]
/proc/loadavg: 8.09 8.01 7.99 9/186 24422
/proc/meminfo: memFree=23868184/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=1059340 CPUtime=642.3 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 198188 0 0 0 64058 172 0 0 25 0 5 0 520482079 1084764160 197724 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 264835 197724 176 36 0 262740 0
[pid=24326/tid=24349] ppid=24324 vsize=1059340 CPUtime=160.34 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 56912 0 0 0 16019 15 0 0 25 0 5 0 520482272 1084764160 197724 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=1059340 CPUtime=160.3 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 15689 0 0 0 15967 63 0 0 25 0 5 0 520482272 1084764160 197724 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254213 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=1059340 CPUtime=159.43 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 106237 0 0 0 15914 29 0 0 25 0 5 0 520482272 1084764160 197724 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=1059340 CPUtime=160.28 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 15400 0 0 0 15964 64 0 0 25 0 5 0 520482272 1084764160 197724 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382783157 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 642.3
Current children cumulated vsize (KiB) 1059340

[startup+222.301 s]
/proc/loadavg: 8.09 8.02 7.99 9/186 24424
/proc/meminfo: memFree=23517620/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=1234508 CPUtime=881.93 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 242272 0 0 0 87974 219 0 0 25 0 5 0 520482079 1264136192 241808 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 308627 241808 176 36 0 306532 0
[pid=24326/tid=24349] ppid=24324 vsize=1234508 CPUtime=220.31 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 75410 0 0 0 22010 21 0 0 25 0 5 0 520482272 1264136192 241808 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=22066304/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2296052 CPUtime=3038.24 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 503585 0 0 0 303307 517 0 0 25 0 5 0 520482079 2351157248 503121 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 574013 503121 176 36 0 571918 0
[pid=24326/tid=24349] ppid=24324 vsize=2296052 CPUtime=759.79 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 146193 0 0 0 75931 48 0 0 25 0 5 0 520482272 2351157248 503121 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254294 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2296052 CPUtime=759.8 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 20502 0 0 0 75795 185 0 0 25 0 5 0 520482272 2351157248 503121 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2296052 CPUtime=758.05 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 301266 0 0 0 75710 95 0 0 25 0 5 0 520482272 2351157248 503121 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2296052 CPUtime=758.65 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 31674 0 0 0 75678 187 0 0 25 0 5 0 520482272 2351157248 503121 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3038.24
Current children cumulated vsize (KiB) 2296052

[startup+822.301 s]
/proc/loadavg: 8.19 8.11 8.02 9/186 24452
/proc/meminfo: memFree=21926552/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2405532 CPUtime=3277.86 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 530974 0 0 0 327241 545 0 0 25 0 5 0 520482079 2463264768 530510 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 601383 530510 176 36 0 599288 0
[pid=24326/tid=24349] ppid=24324 vsize=2405532 CPUtime=819.77 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 153176 0 0 0 81926 51 0 0 25 0 5 0 520482272 2463264768 530510 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2405532 CPUtime=819.79 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 20502 0 0 0 81785 194 0 0 25 0 5 0 520482272 2463264768 530510 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2405532 CPUtime=817.72 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 321671 0 0 0 81670 102 0 0 25 0 5 0 520482272 2463264768 530510 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2405532 CPUtime=818.64 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 31675 0 0 0 81667 197 0 0 25 0 5 0 520482272 2463264768 530510 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3277.86
Current children cumulated vsize (KiB) 2405532

[startup+882.3 s]
/proc/loadavg: 8.17 8.12 8.03 9/186 24454
/proc/meminfo: memFree=21766572/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2557388 CPUtime=3517.5 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 568013 0 0 0 351164 586 0 0 25 0 5 0 520482079 2618765312 567549 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 639347 567549 176 36 0 637252 0
[pid=24326/tid=24349] ppid=24324 vsize=2557388 CPUtime=879.75 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 161628 0 0 0 87921 54 0 0 25 0 5 0 520482272 2618765312 567549 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2557388 CPUtime=879.59 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 21734 0 0 0 87750 209 0 0 25 0 5 0 520482272 2618765312 567549 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2557388 CPUtime=877.57 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 346495 0 0 0 87648 109 0 0 25 0 5 0 520482272 2618765312 567549 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2557388 CPUtime=878.64 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 34206 0 0 0 87652 212 0 0 25 0 5 0 520482272 2618765312 567549 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3517.5
Current children cumulated vsize (KiB) 2557388

[startup+942.301 s]
/proc/loadavg: 8.18 8.12 8.03 9/186 24455
/proc/meminfo: memFree=21631520/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2623208 CPUtime=3757.14 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 584570 0 0 0 375099 615 0 0 25 0 5 0 520482079 2686164992 584106 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 655802 584106 176 36 0 653707 0
[pid=24326/tid=24349] ppid=24324 vsize=2623208 CPUtime=939.72 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 161629 0 0 0 93918 54 0 0 25 0 5 0 520482272 2686164992 584106 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2623208 CPUtime=939.26 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 21735 0 0 0 93705 221 0 0 25 0 5 0 520482272 2686164992 584106 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2623208 CPUtime=937.56 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 363050 0 0 0 93641 115 0 0 25 0 5 0 520482272 2686164992 584106 33554432000 4194304 4338100 140736653658128 18446744073709551615 4256025 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2623208 CPUtime=938.64 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 34206 0 0 0 93642 222 0 0 25 0 5 0 520482272 2686164992 584106 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3757.14
Current children cumulated vsize (KiB) 2623208

[startup+1002.3 s]
/proc/loadavg: 8.10 8.11 8.03 9/186 24457
/proc/meminfo: memFree=21475388/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2726432 CPUtime=3996.81 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 615888 0 0 0 399026 655 0 0 25 0 5 0 520482079 2791866368 615424 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 681608 615424 176 36 0 679513 0
[pid=24326/tid=24349] ppid=24324 vsize=2726432 CPUtime=999.73 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 162899 0 0 0 99918 55 0 0 25 0 5 0 520482272 2791866368 615424 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2726432 CPUtime=998.93 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 24071 0 0 0 99657 236 0 0 25 0 5 0 520482272 2791866368 615424 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2726432 CPUtime=997.56 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 388156 0 0 0 99632 124 0 0 25 0 5 0 520482272 2791866368 615424 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2726432 CPUtime=998.63 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 36812 0 0 0 99626 237 0 0 25 0 5 0 520482272 2791866368 615424 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3996.81
Current children cumulated vsize (KiB) 2726432

[startup+1062.3 s]
/proc/loadavg: 8.14 8.12 8.04 9/186 24458
/proc/meminfo: memFree=21277828/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2832784 CPUtime=4236.33 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 641612 0 0 0 422947 686 0 0 25 0 5 0 520482079 2900770816 641148 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 708196 641148 176 36 0 706101 0
[pid=24326/tid=24349] ppid=24324 vsize=2832784 CPUtime=1059.68 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 163055 0 0 0 105913 55 0 0 25 0 5 0 520482272 2900770816 641148 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2832784 CPUtime=1058.58 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 24071 0 0 0 105612 246 0 0 25 0 5 0 520482272 2900770816 641148 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2832784 CPUtime=1057.53 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 413723 0 0 0 105619 134 0 0 25 0 5 0 520482272 2900770816 641148 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2832784 CPUtime=1058.59 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 36813 0 0 0 105610 249 0 0 25 0 5 0 520482272 2900770816 641148 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4236.33
Current children cumulated vsize (KiB) 2832784

[startup+1122.3 s]
/proc/loadavg: 8.08 8.11 8.04 9/186 24460
/proc/meminfo: memFree=21106080/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=2948520 CPUtime=4476.01 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 671487 0 0 0 446885 716 0 0 25 0 5 0 520482079 3019284480 671023 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 737130 671023 176 36 0 735035 0
[pid=24326/tid=24349] ppid=24324 vsize=2948520 CPUtime=1119.68 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 176314 0 0 0 111908 60 0 0 25 0 5 0 520482272 3019284480 671023 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=2948520 CPUtime=1118.25 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 24071 0 0 0 111569 256 0 0 25 0 5 0 520482272 3019284480 671023 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=2948520 CPUtime=1117.53 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 430338 0 0 0 111614 139 0 0 25 0 5 0 520482272 3019284480 671023 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=2948520 CPUtime=1118.58 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 36814 0 0 0 111600 258 0 0 25 0 5 0 520482272 3019284480 671023 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4476.01
Current children cumulated vsize (KiB) 2948520

[startup+1182.3 s]
/proc/loadavg: 8.06 8.10 8.04 9/186 24462
/proc/meminfo: memFree=20956908/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=3029848 CPUtime=4715.54 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 691560 0 0 0 470806 748 0 0 25 0 5 0 520482079 3102564352 691096 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 757462 691096 176 36 0 755367 0
[pid=24326/tid=24349] ppid=24324 vsize=3029848 CPUtime=1179.66 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 182592 0 0 0 117903 63 0 0 25 0 5 0 520482272 3102564352 691096 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=3029848 CPUtime=1178.1 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 24071 0 0 0 117543 267 0 0 25 0 5 0 520482272 3102564352 691096 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=3029848 CPUtime=1177.5 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 444133 0 0 0 117606 144 0 0 25 0 5 0 520482272 3102564352 691096 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=3029848 CPUtime=1178.32 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 36814 0 0 0 117560 272 0 0 25 0 5 0 520482272 3102564352 691096 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254407 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4715.54
Current children cumulated vsize (KiB) 3029848



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.5 s]
/proc/loadavg: 8.04 8.09 8.03 9/186 24462
/proc/meminfo: memFree=20893532/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=3067384 CPUtime=4800.19 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) S 24324 24326 24182 0 -1 4202496 701228 0 0 0 479259 760 0 0 25 0 5 0 520482079 3141001216 700764 33554432000 4194304 4338100 140736653658128 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 766846 700764 176 36 0 764751 0
[pid=24326/tid=24349] ppid=24324 vsize=3067384 CPUtime=1200.86 cores=1,3,5,7
/proc/24326/task/24349/stat : 24349 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 183978 0 0 0 120023 63 0 0 25 0 5 0 520482272 3141001216 700764 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=24326/tid=24350] ppid=24324 vsize=3067384 CPUtime=1199.29 cores=1,3,5,7
/proc/24326/task/24350/stat : 24350 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 24071 0 0 0 119657 272 0 0 25 0 5 0 520482272 3141001216 700764 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=24326/tid=24351] ppid=24324 vsize=3067384 CPUtime=1198.71 cores=1,3,5,7
/proc/24326/task/24351/stat : 24351 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 452415 0 0 0 119723 148 0 0 25 0 5 0 520482272 3141001216 700764 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=3067384 CPUtime=1199.37 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4202560 36814 0 0 0 119662 275 0 0 25 0 5 0 520482272 3141001216 700764 33554432000 4194304 4338100 140736653658128 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 3067384

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.61 s]
/proc/loadavg: 8.04 8.09 8.03 9/186 24462
/proc/meminfo: memFree=20893532/32951124 swapFree=67051468/67111528
[pid=24326] ppid=24324 vsize=0 CPUtime=4800.31 cores=1,3,5,7
/proc/24326/stat : 24326 (ttsth-5-4) Z 24324 24326 24182 0 -1 4203548 701230 0 0 0 479261 770 0 0 18 0 2 0 520482079 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/24326/statm: 0 0 0 0 0 0 0
[pid=24326/tid=24352] ppid=24324 vsize=0 CPUtime=1199.48 cores=1,3,5,7
/proc/24326/task/24352/stat : 24352 (ttsth-5-4) R 24324 24326 24182 0 -1 4203588 36814 0 0 0 119663 285 0 0 25 0 2 0 520482272 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.62
CPU time (s): 4800.33
CPU user time (s): 4792.61
CPU system time (s): 7.71483
CPU usage (%): 398.825
Max. virtual memory (cumulated for all children) (KiB): 3067384

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4792.61
system time used= 7.71483
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 701230
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5373
involuntary context switches= 33692

runsolver used 2.08668 second user time and 4.48632 second system time

The end

Launcher Data 

Begin job on node101 at 2011-04-29 16:53:13
IDJOB=3261562
IDBENCH=83234
IDSOLVER=1654
FILE ID=node101/3261562-1304088793
RUNJOBID= node101-1304087588-24201
PBS_JOBID= 13170559
Free space on /tmp= 72024 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/skvortsov/battleship/battleship-17-33-sat.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3261562-1304088793/watcher-3261562-1304088793 -o /tmp/evaluation-result-3261562-1304088793/solver-3261562-1304088793 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3261562-1304088793.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ff3a31481d1bf95f014a91f5192a55fa
RANDOM SEED=1086102447

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25321940 kB
Buffers:       2025232 kB
Cached:        5126000 kB
SwapCached:      30452 kB
Active:        3059244 kB
Inactive:      4126592 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25321940 kB
SwapTotal:    67111528 kB
SwapFree:     67051468 kB
Dirty:             212 kB
Writeback:           0 kB
AnonPages:       25772 kB
Mapped:           8932 kB
Slab:           380360 kB
PageTables:       4588 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   192808 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72024 MiB
End job on node101 at 2011-04-29 17:13:19