Trace number 3265082

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.38 1204.1

General information on the benchmark

NameSAT05/crafted/sabharwal05/planning/logistics/unsat/
logistics-rotate-07t5.shuffled-as.sat05-1137.cnf
MD5SUM89957f6ab4a6aa83303efbf463f54b2c
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.413936
Satisfiable
(Un)Satisfiability was proved
Number of variables2721
Number of clauses88373
Sum of the clauses size178033
Maximum clause length9
Minimum clause length1
Number of clauses of size 171
Number of clauses of size 287954
Number of clauses of size 3168
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5180

Solver Data 

0.09/0.10	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265082-1304101602.cnf 
0.09/0.10	c Using SATTIMEOUT of 4800
0.09/0.10	c Using SATRAM of 15500
0.09/0.10	c Size of subset node = 32
0.09/0.10	c Variables = 2721
0.09/0.10	c Clauses = 88373
0.09/0.10	c Literals = 178033
0.09/0.10	c Reduced size:
0.09/0.10	c Variables = 2721
0.09/0.10	c Clauses = 80292
0.09/0.10	c Literals = 161942
0.09/0.10	c a2
0.09/0.15	c a1
0.19/0.27	c Independent sub-problems = 1
0.19/0.27	c Sub-problem with 2346 variables and 80292 clauses
0.19/0.27	c using permutation method 4
0.19/0.27	c count = 1
0.19/0.27	c Ordering finished 0.250 seconds
0.19/0.27	c mapped 0.260 seconds
0.19/0.27	c permuted 0.260 seconds
0.19/0.27	c about to sort
0.19/0.28	c sorted
0.39/0.43	c clauses now 80292
0.39/0.43	c init finished 0.300 seconds
0.39/0.43	c tree built 0.430 seconds
0.39/0.43	c 
0.39/0.43	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.39/0.43	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.39/0.43	c ................................ | ..........                    |                                  0.4         0.0             
0.39/0.43	c ................................ | ..........                  1 |                                  0.4         2.3             
0.39/0.44	c ................................ | ..........                  2 |                                  0.4         4.7             
0.39/0.44	c ................................ | ..........                  3 |                                  0.4         7.0             
0.39/0.44	c ................................ | ..........                  4 |                                  0.4         9.3             
0.39/0.44	c ................................ | ..........                  5 |                                  0.4        11.6             
0.39/0.44	c ................................ | ..........               1000 |              34                  0.4      2272.7             
95.90/24.42	c ................................ | ..........            2436000 | 42146        2032233            96.0     25369.7      2274423
191.84/48.54	c ................................ | ..........            4169000 | 72488        3293182           192.0     21711.3      5171590
287.94/72.64	c ................................ | ..........            5937000 | 86958        3828428           288.1     20608.1      7523406
383.62/96.69	c ................................ | ..........            7118000 | 107155       4499651           384.0     18536.5      9827658
479.77/120.76	c ................................ | *.........            9086000 | 119932       5064422           480.0     18928.4     10923018
575.86/144.84	c ................................ | *.........           11150000 | 131187       5645846           576.0     19357.0     11962446
671.99/168.91	c ................................ | *.........           13066000 | 140977       6176705           672.1     19441.7     13003159
768.01/193.00	c ................................ | *.........           15173000 | 155722       6743333           768.0     19756.0     14614640
863.69/217.08	c ................................ | *.........           17204000 | 169283       7360937           864.0     19911.6     15884216
959.83/241.14	c ................................ | **........           19400000 | 182095       7999998           960.0     20207.9     16687937
1055.98/265.20	c ................................ | **........           21243000 | 190780       8446832          1056.0     20116.3     16955074
1151.69/289.29	c ................................ | **........           23126000 | 205947       9259543          1152.0     20074.1     18205966
1247.69/313.40	c ................................ | **........           24431000 | 225129       10006084         1248.1     19574.4     21064367
1343.87/337.53	c ................................ | **........           27408000 | 230462       11062188         1344.0     20392.7     22858988
1439.92/361.64	c ................................ | ***.......           28334000 | 245440       11639613         1440.1     19675.0     25228572
1535.99/385.71	c ................................ | ***.......           29317000 | 259622       12279027         1536.0     19086.2     27234078
1631.66/409.79	c ................................ | ***.......           30173000 | 275119       12846654         1632.0     18488.0     29442907
1727.76/433.87	c ................................ | ***.......           30904000 | 288435       13309770         1728.0     17883.8     31243196
1823.91/457.96	c ................................ | ***.......           31399000 | 301002       13715081         1824.2     17212.9     32700713
1920.06/482.02	c ................................ | ****......           31959000 | 314300       14122608         1920.2     16644.0     33309800
2015.80/506.08	c ................................ | ****......           32656000 | 329007       14731785         2016.0     16198.1     34234730
2111.91/530.13	c ................................ | ****......           33300000 | 344648       15310192         2112.0     15766.9     35198266
2207.66/554.18	c ................................ | ****......           33728000 | 356969       15746741         2208.0     15275.4     36443010
2303.81/578.27	c ................................ | ****......           34135000 | 369172       16198410         2304.1     14814.8     37512615
2399.94/602.38	c ................................ | *****.....           34541000 | 380652       16593767         2400.3     14390.4     39132409
2495.74/626.38	c ................................ | *****.....           34875000 | 390969       16933608         2496.1     13972.0     40547197
2591.89/650.49	c ................................ | *****.....           35293000 | 401984       17312354         2592.3     13614.8     42070346
2687.66/674.49	c ................................ | *****.....           35647000 | 412450       17675658         2688.0     13261.4     43413451
2783.85/698.54	c ................................ | *****.....           36001000 | 422092       18009305         2784.0     12931.3     43965418
2879.99/722.62	c ................................ | ******....           36416000 | 434092       18431445         2880.1     12644.1     43976270
2976.15/746.70	c ................................ | ******....           36830000 | 444691       18809585         2976.2     12374.9     44778212
3071.93/770.75	c ................................ | ******....           37151000 | 454655       19133201         3072.1     12092.9     45796173
3167.68/794.79	c ................................ | ******....           37497000 | 464799       19494438         3168.1     11836.0     46696815
3263.87/818.84	c ................................ | ******....           37830000 | 475231       19823526         3264.0     11589.9     47009085
3360.00/842.90	c ................................ | *******...           38252000 | 487864       20237098         3360.0     11384.4     47647668
3456.18/867.03	c ................................ | *******...           38578000 | 498540       20531592         3456.3     11161.6     48256946
3551.98/891.02	c ................................ | *******...           39040000 | 509315       20853669         3552.1     10990.8     48986253
3648.15/915.13	c ................................ | *******...           39408000 | 518353       21180124         3648.3     10801.8     49906437
3743.82/939.15	c ................................ | *******...           40621000 | 528278       21567204         3744.0     10849.5     51992096
3839.89/963.23	c ................................ | ********..           41917000 | 537414       21897905         3840.0     10915.8     54216332
3935.93/987.31	c ................................ | ********..           43308000 | 546281       22177823         3936.0     11003.0     55154740
4032.00/1011.41	c ................................ | ********..           44705000 | 554451       22494497         4032.0     11087.4     56932407
4127.64/1035.49	c ................................ | ********..           46077000 | 563113       22821892         4128.0     11162.0     58697021
4223.73/1059.57	c ................................ | ********..           46970000 | 575584       23255917         4224.0     11119.8     60368035
4319.90/1083.64	c ................................ | *********.           47360000 | 588089       23676166         4320.1     10962.7     61314667
4416.10/1107.70	c ................................ | *********.           47715000 | 599996       24033292         4416.1     10804.7     61314667
4511.87/1131.77	c ................................ | *********.           48076000 | 612734       24398431         4512.1     10654.9     61669010
4608.03/1155.85	c ................................ | *********.           48504000 | 626476       24802415         4608.2     10525.5     62043614
4704.21/1179.90	c ................................ | *********.           48873000 | 637952       25142509         4704.2     10389.1     62943002
4799.99/1203.96	c CPU limit (4800) reached
4799.99/1203.96	c Time = 4800.23 seconds
4799.99/1203.96	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265082-1304101602/watcher-3265082-1304101602 -o /tmp/evaluation-result-3265082-1304101602/solver-3265082-1304101602 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265082-1304101602.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.18 7.29 7.67 2/177 2158
/proc/meminfo: memFree=25621356/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=9112 CPUtime=0 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) R 2156 2158 1454 0 -1 4202496 353 0 0 0 0 0 0 0 18 0 1 0 502575650 9330688 296 33554432000 4194304 4338100 140736352770080 18446744073709551615 228095111877 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/2158/statm: 2278 297 147 36 0 183 0

[startup+0.0422341 s]
/proc/loadavg: 5.18 7.29 7.67 2/177 2158
/proc/meminfo: memFree=25621356/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=16552 CPUtime=0.03 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) R 2156 2158 1454 0 -1 4202496 2159 0 0 0 3 0 0 0 18 0 1 0 502575650 16949248 2102 33554432000 4194304 4338100 140736352770080 18446744073709551615 228095111877 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/2158/statm: 4138 2103 148 36 0 2043 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 16552

[startup+0.100262 s]
/proc/loadavg: 5.18 7.29 7.67 2/177 2158
/proc/meminfo: memFree=25621356/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=19192 CPUtime=0.09 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) R 2156 2158 1454 0 -1 4202496 2849 0 0 0 9 0 0 0 18 0 1 0 502575650 19652608 2792 33554432000 4194304 4338100 140736352770080 18446744073709551615 4221800 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/2158/statm: 4798 2792 150 36 0 2703 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 19192

[startup+0.301231 s]
/proc/loadavg: 5.18 7.29 7.67 2/177 2158
/proc/meminfo: memFree=25621356/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=37128 CPUtime=0.29 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) R 2156 2158 1454 0 -1 4202496 7850 0 0 0 28 1 0 0 18 0 1 0 502575650 38019072 7305 33554432000 4194304 4338100 140736352770080 18446744073709551615 228095111877 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/2158/statm: 9348 7338 163 36 0 7253 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 37128

[startup+0.755072 s]
/proc/loadavg: 5.18 7.29 7.67 2/177 2158
/proc/meminfo: memFree=25621356/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=251980 CPUtime=1.67 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 16327 0 0 0 164 3 0 0 18 0 5 0 502575650 258027520 15781 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 62995 15781 176 36 0 60900 0
Current children cumulated CPU time (s) 1.67
Current children cumulated vsize (KiB) 251980

[startup+1.5009 s]
/proc/loadavg: 5.18 7.29 7.67 9/186 2167
/proc/meminfo: memFree=25533004/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=319100 CPUtime=4.65 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 17883 0 0 0 461 4 0 0 18 0 5 0 502575650 326758400 17337 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 79775 17337 176 36 0 77680 0
[pid=2158/tid=2160] ppid=2156 vsize=319100 CPUtime=1.05 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 848 0 0 0 105 0 0 0 19 0 5 0 502575694 326758400 17337 33554432000 4194304 4338100 140736352770080 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=319100 CPUtime=1.05 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 1038 0 0 0 105 0 0 0 19 0 5 0 502575694 326758400 17337 33554432000 4194304 4338100 140736352770080 18446744073709551615 4229565 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=319100 CPUtime=1.04 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 874 0 0 0 104 0 0 0 19 0 5 0 502575694 326758400 17337 33554432000 4194304 4338100 140736352770080 18446744073709551615 4255947 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=319100 CPUtime=1.04 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 832 0 0 0 104 0 0 0 19 0 5 0 502575694 326758400 17337 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846377 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4.65
Current children cumulated vsize (KiB) 319100

[startup+3.10061 s]
/proc/loadavg: 5.18 7.29 7.67 10/186 2167
/proc/meminfo: memFree=25514528/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=325792 CPUtime=11.04 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 20632 0 0 0 1096 8 0 0 18 0 5 0 502575650 333611008 20086 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 81448 20086 176 36 0 79353 0
[pid=2158/tid=2160] ppid=2156 vsize=325792 CPUtime=2.65 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 1388 0 0 0 264 1 0 0 19 0 5 0 502575694 333611008 20086 33554432000 4194304 4338100 140736352770080 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=325792 CPUtime=2.65 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 2054 0 0 0 264 1 0 0 19 0 5 0 502575694 333611008 20086 33554432000 4194304 4338100 140736352770080 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=325792 CPUtime=2.63 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 1481 0 0 0 262 1 0 0 19 0 5 0 502575694 333611008 20086 33554432000 4194304 4338100 140736352770080 18446744073709551615 4230663 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=325792 CPUtime=2.65 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 1419 0 0 0 263 2 0 0 22 0 5 0 502575694 333611008 20087 33554432000 4194304 4338100 140736352770080 18446744073709551615 228095112225 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.04
Current children cumulated vsize (KiB) 325792

[startup+6.30102 s]
/proc/loadavg: 5.40 7.30 7.67 9/186 2167
/proc/meminfo: memFree=25467908/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=333936 CPUtime=23.8 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 26500 0 0 0 2363 17 0 0 18 0 5 0 502575650 341950464 25954 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 83484 25954 176 36 0 81389 0
[pid=2158/tid=2160] ppid=2156 vsize=333936 CPUtime=5.85 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 2538 0 0 0 582 3 0 0 25 0 5 0 502575694 341950464 25954 33554432000 4194304 4338100 140736352770080 18446744073709551615 4245175 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=333936 CPUtime=5.84 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 4375 0 0 0 581 3 0 0 25 0 5 0 502575694 341950464 25954 33554432000 4194304 4338100 140736352770080 18446744073709551615 4240550 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=333936 CPUtime=5.81 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 2653 0 0 0 578 3 0 0 25 0 5 0 502575694 341950464 25954 33554432000 4194304 4338100 140736352770080 18446744073709551615 4256241 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=333936 CPUtime=5.84 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 2643 0 0 0 580 4 0 0 25 0 5 0 502575694 341950464 25954 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 23.8
Current children cumulated vsize (KiB) 333936

[startup+12.7008 s]
/proc/loadavg: 5.61 7.31 7.67 10/186 2168
/proc/meminfo: memFree=25398688/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=363844 CPUtime=49.3 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 36438 0 0 0 4879 51 0 0 18 0 5 0 502575650 372576256 35892 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 90961 35892 176 36 0 88866 0
[pid=2158/tid=2160] ppid=2156 vsize=363844 CPUtime=12.23 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 4355 0 0 0 1214 9 0 0 19 0 5 0 502575694 372576256 35892 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=363844 CPUtime=12.22 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 7818 0 0 0 1210 12 0 0 25 0 5 0 502575694 372576256 35892 33554432000 4194304 4338100 140736352770080 18446744073709551615 4256753 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=363844 CPUtime=12.15 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 4995 0 0 0 1203 12 0 0 24 0 5 0 502575694 372576256 35892 33554432000 4194304 4338100 140736352770080 18446744073709551615 4255963 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=363844 CPUtime=12.22 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 4979 0 0 0 1209 13 0 0 25 0 5 0 502575694 372576256 35892 33554432000 4194304 4338100 140736352770080 18446744073709551615 4256007 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.3
Current children cumulated vsize (KiB) 363844

[startup+25.5005 s]
/proc/loadavg: 6.14 7.35 7.67 10/186 2168
/proc/meminfo: memFree=25296276/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=410976 CPUtime=100.29 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 51712 0 0 0 9911 118 0 0 18 0 5 0 502575650 420839424 51166 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 102744 51166 176 36 0 100649 0
[pid=2158/tid=2160] ppid=2156 vsize=410976 CPUtime=25.03 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 7272 0 0 0 2481 22 0 0 18 0 5 0 502575694 420839424 51166 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846177 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=410976 CPUtime=24.98 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 12469 0 0 0 2468 30 0 0 21 0 5 0 502575694 420839424 51166 33554432000 4194304 4338100 140736352770080 18446744073709551615 228095147707 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=410976 CPUtime=24.84 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 8738 0 0 0 2454 30 0 0 18 0 5 0 502575694 420839424 51166 33554432000 4194304 4338100 140736352770080 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=410976 CPUtime=24.97 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 8942 0 0 0 2465 32 0 0 18 0 5 0 502575694 420839424 51166 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094845968 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.29
Current children cumulated vsize (KiB) 410976

[startup+51.1007 s]
/proc/loadavg: 6.77 7.40 7.68 9/186 2169
/proc/meminfo: memFree=25062252/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=515508 CPUtime=202.21 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 86073 0 0 0 19950 271 0 0 18 0 5 0 502575650 527880192 85527 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 128877 85527 176 36 0 126782 0
[pid=2158/tid=2160] ppid=2156 vsize=515508 CPUtime=50.51 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 14344 0 0 0 4994 57 0 0 19 0 5 0 502575694 527880192 85527 33554432000 4194304 4338100 140736352770080 18446744073709551615 4256028 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=515508 CPUtime=50.45 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 22757 0 0 0 4974 71 0 0 18 0 5 0 502575694 527880192 85527 33554432000 4194304 4338100 140736352770080 18446744073709551615 4241570 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=515508 CPUtime=50.31 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 17006 0 0 0 4960 71 0 0 25 0 5 0 502575694 527880192 85527 33554432000 4194304 4338100 140736352770080 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=515508 CPUtime=50.46 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 17675 0 0 0 4979 67 0 0 22 0 5 0 502575694 527880192 85527 33554432000 4194304 4338100 140736352770080 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 202.21
Current children cumulated vsize (KiB) 515508

[startup+102.306 s]
/proc/loadavg: 7.43 7.47 7.69 10/186 2170
/proc/meminfo: memFree=24776788/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=673216 CPUtime=406.38 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 132517 0 0 0 40153 485 0 0 18 0 5 0 502575650 689373184 131971 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 168304 131971 176 36 0 166209 0
[pid=2158/tid=2160] ppid=2156 vsize=673216 CPUtime=101.36 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 23786 0 0 0 10020 116 0 0 25 0 5 0 502575694 689373184 131971 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846333 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=673216 CPUtime=101.52 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 37542 0 0 0 10021 131 0 0 25 0 5 0 502575694 689373184 131971 33554432000 4194304 4338100 140736352770080 18446744073709551615 4286738 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=673216 CPUtime=101.37 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 29083 0 0 0 10004 133 0 0 19 0 5 0 502575694 689373184 131971 33554432000 4194304 4338100 140736352770080 18446744073709551615 4241503 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=673216 CPUtime=101.64 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 27815 0 0 0 10063 101 0 0 25 0 5 0 502575694 689373184 131971 33554432000 4194304 4338100 140736352770080 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 406.38
Current children cumulated vsize (KiB) 673216

[startup+162.301 s]
/proc/loadavg: 7.90 7.60 7.72 9/186 2172
/proc/meminfo: memFree=24608748/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=764756 CPUtime=645.67 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 160856 0 0 0 63894 673 0 0 18 0 5 0 502575650 783110144 160310 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 191189 160310 176 36 0 189094 0
[pid=2158/tid=2160] ppid=2156 vsize=764756 CPUtime=161.09 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 29951 0 0 0 15940 169 0 0 21 0 5 0 502575694 783110144 160310 33554432000 4194304 4338100 140736352770080 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=764756 CPUtime=161.39 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 46844 0 0 0 15955 184 0 0 18 0 5 0 502575694 783110144 160310 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846177 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=764756 CPUtime=161.09 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 36032 0 0 0 15924 185 0 0 18 0 5 0 502575694 783110144 160310 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846398 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=764756 CPUtime=161.61 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 33738 0 0 0 16031 130 0 0 18 0 5 0 502575694 783110144 160310 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846099 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 645.67
Current children cumulated vsize (KiB) 764756


################
# More data... #
################

[startup+702.3 s]
/proc/loadavg: 8.05 7.94 7.83 10/186 2186
/proc/meminfo: memFree=22810808/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2286160 CPUtime=2798.99 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 511701 0 0 0 277167 2732 0 0 18 0 5 0 502575650 2341027840 510547 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 571540 510547 176 36 0 569445 0
[pid=2158/tid=2160] ppid=2156 vsize=2286160 CPUtime=700.43 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 99963 0 0 0 69476 567 0 0 25 0 5 0 502575694 2341027840 510547 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2286160 CPUtime=700.09 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 152548 0 0 0 69253 756 0 0 20 0 5 0 502575694 2341027840 510547 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2286160 CPUtime=699.03 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 124606 0 0 0 69153 750 0 0 25 0 5 0 502575694 2341027840 510547 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2286160 CPUtime=698.97 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 120293 0 0 0 69242 655 0 0 18 0 5 0 502575694 2341027840 510547 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2798.99
Current children cumulated vsize (KiB) 2286160

[startup+762.306 s]
/proc/loadavg: 8.06 7.97 7.85 10/186 2188
/proc/meminfo: memFree=22704404/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2338004 CPUtime=3038.43 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 530723 0 0 0 300955 2888 0 0 18 0 5 0 502575650 2394116096 529569 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 584501 529569 176 36 0 582406 0
[pid=2158/tid=2160] ppid=2156 vsize=2338004 CPUtime=760.41 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 101984 0 0 0 75444 597 0 0 25 0 5 0 502575694 2394116096 529569 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2338004 CPUtime=760 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 159813 0 0 0 75203 797 0 0 23 0 5 0 502575694 2394116096 529569 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2338004 CPUtime=758.93 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 130102 0 0 0 75100 793 0 0 18 0 5 0 502575694 2394116096 529569 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2338004 CPUtime=758.6 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 124533 0 0 0 75164 696 0 0 21 0 5 0 502575694 2394116096 529569 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3038.43
Current children cumulated vsize (KiB) 2338004

[startup+822.301 s]
/proc/loadavg: 8.02 7.97 7.85 10/186 2189
/proc/meminfo: memFree=22607036/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2408944 CPUtime=3277.85 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 555132 0 0 0 324742 3043 0 0 18 0 5 0 502575650 2466758656 553978 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 602236 553978 176 36 0 600141 0
[pid=2158/tid=2160] ppid=2156 vsize=2408944 CPUtime=820.39 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 103877 0 0 0 81417 622 0 0 25 0 5 0 502575694 2466758656 553978 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2408944 CPUtime=819.91 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 169021 0 0 0 81151 840 0 0 19 0 5 0 502575694 2466758656 553978 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2408944 CPUtime=818.84 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 136934 0 0 0 81047 837 0 0 25 0 5 0 502575694 2466758656 553978 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2408944 CPUtime=818.23 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 131009 0 0 0 81084 739 0 0 21 0 5 0 502575694 2466758656 553978 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3277.85
Current children cumulated vsize (KiB) 2408944

[startup+882.301 s]
/proc/loadavg: 8.04 7.99 7.86 9/186 2191
/proc/meminfo: memFree=22516008/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2466384 CPUtime=3517.24 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 574710 0 0 0 348519 3205 0 0 18 0 5 0 502575650 2525577216 573556 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 616596 573556 176 36 0 614501 0
[pid=2158/tid=2160] ppid=2156 vsize=2466384 CPUtime=880.37 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 105710 0 0 0 87388 649 0 0 25 0 5 0 502575694 2525577216 573556 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2466384 CPUtime=879.81 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 170855 0 0 0 87094 887 0 0 25 0 5 0 502575694 2525577216 573556 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2466384 CPUtime=878.75 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 144834 0 0 0 86994 881 0 0 19 0 5 0 502575694 2525577216 573556 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2466384 CPUtime=877.84 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 139020 0 0 0 87000 784 0 0 25 0 5 0 502575694 2525577216 573556 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3517.24
Current children cumulated vsize (KiB) 2466384

[startup+942.301 s]
/proc/loadavg: 8.01 7.99 7.87 10/186 2192
/proc/meminfo: memFree=22367064/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2586252 CPUtime=3756.58 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 609891 0 0 0 372256 3402 0 0 18 0 5 0 502575650 2648322048 608737 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 646563 608737 176 36 0 644468 0
[pid=2158/tid=2160] ppid=2156 vsize=2586252 CPUtime=940.34 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 115210 0 0 0 93351 683 0 0 25 0 5 0 502575694 2648322048 608737 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2586252 CPUtime=939.7 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 176151 0 0 0 93028 942 0 0 25 0 5 0 502575694 2648322048 608737 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2586252 CPUtime=938.4 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 154700 0 0 0 92906 934 0 0 19 0 5 0 502575694 2648322048 608737 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2586252 CPUtime=937.68 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 149539 0 0 0 92928 840 0 0 25 0 5 0 502575694 2648322048 608737 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3756.58
Current children cumulated vsize (KiB) 2586252

[startup+1002.3 s]
/proc/loadavg: 7.93 7.96 7.86 10/186 2194
/proc/meminfo: memFree=22184876/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2717180 CPUtime=3995.72 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 646526 0 0 0 395871 3701 0 0 18 0 5 0 502575650 2782392320 645372 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 679295 645372 176 36 0 677200 0
[pid=2158/tid=2160] ppid=2156 vsize=2717180 CPUtime=1000.3 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 119069 0 0 0 99297 733 0 0 23 0 5 0 502575694 2782392320 645372 33554432000 4194304 4338100 140736352770080 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2717180 CPUtime=999.51 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 188987 0 0 0 98924 1027 0 0 20 0 5 0 502575694 2782392320 645372 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846094 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2717180 CPUtime=997.94 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 164775 0 0 0 98780 1014 0 0 25 0 5 0 502575694 2782392320 645372 33554432000 4194304 4338100 140736352770080 18446744073709551615 228094846702 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2717180 CPUtime=997.49 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 159404 0 0 0 98826 923 0 0 25 0 5 0 502575694 2782392320 645372 33554432000 4194304 4338100 140736352770080 18446744073709551615 228095147505 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3995.72
Current children cumulated vsize (KiB) 2717180

[startup+1062.3 s]
/proc/loadavg: 7.94 7.95 7.86 10/186 2196
/proc/meminfo: memFree=21963652/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2859604 CPUtime=4234.89 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 688142 0 0 0 419510 3979 0 0 18 0 5 0 502575650 2928234496 686988 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 714901 686988 176 36 0 712806 0
[pid=2158/tid=2160] ppid=2156 vsize=2859604 CPUtime=1060.27 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 122339 0 0 0 105247 780 0 0 18 0 5 0 502575694 2928234496 686988 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2859604 CPUtime=1059.35 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 204023 0 0 0 104829 1106 0 0 18 0 5 0 502575694 2928234496 686988 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2859604 CPUtime=1057.49 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 176068 0 0 0 104661 1088 0 0 18 0 5 0 502575694 2928234496 686988 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2859604 CPUtime=1057.32 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 171421 0 0 0 104730 1002 0 0 18 0 5 0 502575694 2928234496 686988 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4234.89
Current children cumulated vsize (KiB) 2859604

[startup+1122.3 s]
/proc/loadavg: 8.12 7.99 7.88 10/186 2197
/proc/meminfo: memFree=21875584/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=2892680 CPUtime=4474.36 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 703734 0 0 0 443307 4129 0 0 18 0 5 0 502575650 2962104320 702580 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 723170 702580 176 36 0 721075 0
[pid=2158/tid=2160] ppid=2156 vsize=2892680 CPUtime=1120.25 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 125805 0 0 0 111218 807 0 0 25 0 5 0 502575694 2962104320 702580 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=2892680 CPUtime=1119.27 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 208777 0 0 0 110778 1149 0 0 25 0 5 0 502575694 2962104320 702580 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=2892680 CPUtime=1117.18 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 179462 0 0 0 110589 1129 0 0 18 0 5 0 502575694 2962104320 702580 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=2892680 CPUtime=1117.19 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 175399 0 0 0 110679 1040 0 0 18 0 5 0 502575694 2962104320 702580 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4474.36
Current children cumulated vsize (KiB) 2892680

[startup+1182.3 s]
/proc/loadavg: 8.18 8.02 7.90 10/186 2199
/proc/meminfo: memFree=21786416/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=3153644 CPUtime=4713.79 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) S 2156 2158 1454 0 -1 4202496 724447 0 0 0 467077 4302 0 0 18 0 5 0 502575650 3229331456 723293 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 788411 723293 176 36 0 786316 0
[pid=2158/tid=2160] ppid=2156 vsize=3153644 CPUtime=1180.08 cores=0,2,4,6
/proc/2158/task/2160/stat : 2160 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 128511 0 0 0 117162 846 0 0 25 0 5 0 502575694 3229331456 723293 33554432000 4194304 4338100 140736352770080 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=2158/tid=2161] ppid=2156 vsize=3153644 CPUtime=1179.18 cores=0,2,4,6
/proc/2158/task/2161/stat : 2161 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 216528 0 0 0 116719 1199 0 0 25 0 5 0 502575694 3229331456 723293 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=2158/tid=2162] ppid=2156 vsize=3153644 CPUtime=1177.01 cores=0,2,4,6
/proc/2158/task/2162/stat : 2162 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 183527 0 0 0 116534 1167 0 0 21 0 5 0 502575694 3229331456 723293 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=2158/tid=2163] ppid=2156 vsize=3153644 CPUtime=1177.06 cores=0,2,4,6
/proc/2158/task/2163/stat : 2163 (ttsth-5-4) R 2156 2158 1454 0 -1 4202560 181590 0 0 0 116619 1087 0 0 19 0 5 0 502575694 3229331456 723293 33554432000 4194304 4338100 140736352770080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4713.79
Current children cumulated vsize (KiB) 3153644



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204 s]
/proc/loadavg: 8.18 8.03 7.90 2/176 2200
/proc/meminfo: memFree=23482900/32951124 swapFree=67111524/67111528
[pid=2158] ppid=2156 vsize=0 CPUtime=4800.28 cores=0,2,4,6
/proc/2158/stat : 2158 (ttsth-5-4) R 2156 2158 1454 0 -1 4203524 735662 0 0 0 475665 4363 0 0 18 0 1 0 502575650 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2158/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.28
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1204.1
CPU time (s): 4800.38
CPU user time (s): 4756.65
CPU system time (s): 43.7274
CPU usage (%): 398.67
Max. virtual memory (cumulated for all children) (KiB): 3187960

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4756.65
system time used= 43.7274
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 735662
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 470457
involuntary context switches= 19145

runsolver used 1.61075 second user time and 3.88641 second system time

The end

Launcher Data 

Begin job on node142 at 2011-04-29 20:26:42
IDJOB=3265082
IDBENCH=85421
IDSOLVER=1654
FILE ID=node142/3265082-1304101602
RUNJOBID= node142-1304101602-2138
PBS_JOBID= 13170602
Free space on /tmp= 71312 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/sabharwal05/planning/logistics/unsat/logistics-rotate-07t5.shuffled-as.sat05-1137.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265082-1304101602/watcher-3265082-1304101602 -o /tmp/evaluation-result-3265082-1304101602/solver-3265082-1304101602 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265082-1304101602.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 89957f6ab4a6aa83303efbf463f54b2c
RANDOM SEED=2042016810

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25622008 kB
Buffers:       1681184 kB
Cached:        5039296 kB
SwapCached:          4 kB
Active:         876144 kB
Inactive:      5911788 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25622008 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1556 kB
Writeback:           0 kB
AnonPages:       66028 kB
Mapped:          14928 kB
Slab:           478220 kB
PageTables:       4724 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   208904 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71312 MiB
End job on node142 at 2011-04-29 20:46:48