Trace number 3265126

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.15 1203.93

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/php/unsat/
easier/php-045-035.shuffled-as.sat05-1181.cnf
MD5SUM057b78ff24cc0773c9c67e770a7b9de2
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark44.4072
Satisfiable
(Un)Satisfiability was proved
Number of variables1575
Number of clauses34695
Sum of the clauses size70875
Maximum clause length35
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 234650
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.02	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265126-1304101602.cnf 
0.00/0.02	c Using SATTIMEOUT of 4800
0.00/0.02	c Using SATRAM of 15500
0.00/0.02	c Size of subset node = 32
0.00/0.02	c Variables = 1575
0.00/0.02	c Clauses = 34695
0.00/0.02	c Literals = 70875
0.00/0.02	c Reduced size:
0.00/0.02	c Variables = 1575
0.00/0.02	c Clauses = 34695
0.00/0.02	c Literals = 70875
0.00/0.02	c a2
0.02/0.05	c a1
0.89/0.95	c Independent sub-problems = 1
0.89/0.95	c Sub-problem with 1575 variables and 34695 clauses
0.89/0.95	c using permutation method 4
0.89/0.95	c count = 2
0.89/0.95	c Ordering finished 0.940 seconds
0.89/0.95	c mapped 0.940 seconds
0.89/0.95	c permuted 0.940 seconds
0.89/0.95	c about to sort
0.89/0.95	c sorted
0.99/1.00	c clauses now 34695
0.99/1.00	c init finished 0.960 seconds
0.99/1.00	c tree built 0.990 seconds
0.99/1.00	c 
0.99/1.00	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.99/1.00	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.99/1.00	c ................................ | ..........                    |                                  1.0         0.0             
0.99/1.00	c ................................ | ..........                  1 |                                  1.0         1.0             
0.99/1.00	c ................................ | ..........                  2 |                                  1.0         2.0             
0.99/1.00	c ................................ | ..........                  3 |                                  1.0         3.0             
0.99/1.00	c ................................ | ..........                  4 |                                  1.0         4.0             
0.99/1.00	c ................................ | ..........                  5 |                                  1.0         5.1             
0.99/1.00	c ................................ | ..........               1000 |                                  1.0      1000.0             
95.99/24.81	c ................................ | ..........            3176000 | 524253       22942              96.0     33073.0      3062859
191.81/48.85	c ................................ | ..........            5901000 | 1023636      24785             192.0     30732.8      5942424
288.02/72.90	c ................................ | ..........            8062000 | 1405229      25920             288.0     27990.1      7856541
383.84/96.94	c ................................ | ..........            9871000 | 1741052      27020             384.0     25705.1      8924447
480.06/121.00	c ................................ | *.........           11346000 | 2021218      29057             480.1     23634.1     10045207
575.85/145.04	c ................................ | *.........           12570000 | 2256353      32667             576.0     21822.5     10405064
671.67/169.09	c ................................ | *.........           13652000 | 2464583      36709             672.0     20314.6     10884487
767.88/193.14	c ................................ | *.........           14627000 | 2655714      38379             768.1     19043.8     10997712
863.72/217.17	c ................................ | *.........           15405000 | 2814671      38583             864.0     17829.4     11242943
959.96/241.25	c ................................ | **........           15905000 | 2920381      38643             960.2     16564.3     11651293
1055.70/265.28	c ................................ | **........           17655000 | 3235557      38964            1056.0     16718.4     12307638
1151.91/289.34	c ................................ | **........           19711000 | 3611597      40151            1152.1     17109.4     13566752
1247.64/313.39	c ................................ | **........           21378000 | 3924434      41602            1248.0     17129.7     14353015
1343.82/337.45	c ................................ | **........           22666000 | 4174852      43390            1344.0     16864.1     14623570
1440.03/361.51	c ................................ | ***.......           23742000 | 4392390      44441            1440.1     16486.7     14935683
1535.84/385.54	c ................................ | ***.......           24640000 | 4574266      45896            1536.0     16041.6     14935683
1632.05/409.68	c ................................ | ***.......           25236000 | 4700264      46156            1632.4     15459.6     14935683
1727.72/433.66	c ................................ | ***.......           27455000 | 5106145      46397            1728.0     15888.3     15914946
1823.88/457.73	c ................................ | ***.......           29024000 | 5408799      47078            1824.0     15912.3     16759133
1920.07/481.80	c ................................ | ****......           30257000 | 5660214      47380            1920.1     15758.3     17085550
2015.85/505.84	c ................................ | ****......           31148000 | 5847264      47562            2016.0     15450.0     17341174
2111.96/529.91	c ................................ | ****......           32564000 | 6119486      47693            2112.0     15418.4     17937150
2207.68/553.98	c ................................ | ****......           34430000 | 6480482      47801            2208.0     15593.2     18934304
2303.83/578.04	c ................................ | ****......           35506000 | 6697415      47801            2304.0     15410.6     19328115
2399.92/602.12	c ................................ | *****.....           37330000 | 7049327      47844            2400.0     15554.2     20554376
2496.00/626.20	c ................................ | *****.....           38890000 | 7359790      47868            2496.0     15580.9     21820201
2592.04/650.30	c ................................ | *****.....           40563000 | 7704010      47868            2592.1     15648.9     22336992
2687.77/674.37	c ................................ | *****.....           42224000 | 8018020      48167            2688.0     15708.2     23013296
2783.88/698.44	c ................................ | *****.....           44022000 | 8337204      48326            2784.0     15812.4     23045701
2880.00/722.50	c ................................ | ******....           45482000 | 8609752      48416            2880.0     15792.3     23381044
2975.74/746.57	c ................................ | ******....           46690000 | 8843132      48416            2976.0     15688.6     24034235
3071.88/770.68	c ................................ | ******....           47748000 | 9051298      48416            3072.2     15541.8     24064265
3167.66/794.69	c ................................ | ******....           48546000 | 9213435      48416            3168.0     15323.7     24064265
3263.85/818.73	c ................................ | ******....           49038000 | 9317245      48468            3264.0     15023.9     24064265
3359.85/842.84	c ................................ | *******...           50896000 | 9643268      48928            3360.0     15147.5     24064265
3455.89/866.93	c ................................ | *******...           52424000 | 9921942      49033            3456.0     15168.9     24067397
3551.69/890.98	c ................................ | *******...           53682000 | 10162781     49033            3552.0     15113.1     24067397
3648.23/915.12	c ................................ | *******...           54604000 | 10345842     49033            3648.3     14966.9     24142372
3743.95/939.11	c ................................ | *******...           55933000 | 10588765     49296            3744.0     14939.3     24348485
3839.64/963.20	c ................................ | ********..           57456000 | 10877666     49355            3840.0     14962.4     24898996
3935.81/987.28	c ................................ | ********..           58465000 | 11081734     49355            3936.2     14853.3     25191524
4031.95/1011.31	c ................................ | ********..           60187000 | 11399542     49486            4032.0     14927.3     25880958
4127.61/1035.39	c ................................ | ********..           61674000 | 11684129     49532            4128.0     14940.4     26528775
4223.68/1059.49	c ................................ | ********..           63381000 | 12017239     49532            4224.1     15004.8     26889965
4319.82/1083.56	c ................................ | *********.           64552000 | 12244860     50026            4320.1     14942.4     27305686
4415.98/1107.61	c ................................ | *********.           65283000 | 12389137     50776            4416.0     14783.3     27415474
4512.05/1131.70	c ................................ | *********.           66445000 | 12589443     54132            4512.1     14726.1     27804421
4607.82/1155.77	c ................................ | *********.           67488000 | 12787732     54486            4608.1     14645.5     27846367
4703.98/1179.81	c ................................ | *********.           68299000 | 12946180     54486            4704.0     14519.3     27906543
4799.77/1203.88	c CPU limit (4800) reached
4799.77/1203.88	c Time = 4800.10 seconds
4799.77/1203.88	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265126-1304101602/watcher-3265126-1304101602 -o /tmp/evaluation-result-3265126-1304101602/solver-3265126-1304101602 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265126-1304101602.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.18 7.29 7.67 2/174 2152
/proc/meminfo: memFree=25621752/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=9112 CPUtime=0 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) R 2150 2152 1454 0 -1 4202496 361 0 0 0 0 0 0 0 23 0 1 0 502575649 9330688 303 33554432000 4194304 4338100 140735860966400 18446744073709551615 228095111877 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/2152/statm: 2278 304 147 36 0 183 0

[startup+0.026714 s]
/proc/loadavg: 5.18 7.29 7.67 2/174 2152
/proc/meminfo: memFree=25621752/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=13252 CPUtime=0.02 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) R 2150 2152 1454 0 -1 4202496 1390 0 0 0 2 0 0 0 23 0 1 0 502575649 13570048 1332 33554432000 4194304 4338100 140735860966400 18446744073709551615 228095111877 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/2152/statm: 3346 1340 151 36 0 1251 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 13252

[startup+0.100699 s]
/proc/loadavg: 5.18 7.29 7.67 2/174 2152
/proc/meminfo: memFree=25621752/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=17292 CPUtime=0.09 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) R 2150 2152 1454 0 -1 4202496 2541 0 0 0 9 0 0 0 24 0 1 0 502575649 17707008 2347 33554432000 4194304 4338100 140735860966400 18446744073709551615 4295806 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/2152/statm: 4323 2347 154 36 0 2228 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 17292

[startup+0.300656 s]
/proc/loadavg: 5.18 7.29 7.67 2/174 2152
/proc/meminfo: memFree=25621752/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=17424 CPUtime=0.29 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) R 2150 2152 1454 0 -1 4202496 2573 0 0 0 29 0 0 0 25 0 1 0 502575649 17842176 2379 33554432000 4194304 4338100 140735860966400 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/2152/statm: 4356 2379 154 36 0 2261 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 17424

[startup+0.700583 s]
/proc/loadavg: 5.18 7.29 7.67 2/174 2152
/proc/meminfo: memFree=25621752/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=17896 CPUtime=0.69 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) R 2150 2152 1454 0 -1 4202496 2666 0 0 0 69 0 0 0 25 0 1 0 502575649 18325504 2472 33554432000 4194304 4338100 140735860966400 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/2152/statm: 4474 2472 154 36 0 2379 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 17896

[startup+1.5004 s]
/proc/loadavg: 5.18 7.29 7.67 6/182 2163
/proc/meminfo: memFree=25535020/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=283752 CPUtime=2.95 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 8553 0 0 0 293 2 0 0 25 0 5 0 502575649 290562048 8291 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 70938 8291 175 36 0 68843 0
Current children cumulated CPU time (s) 2.95
Current children cumulated vsize (KiB) 283752

[startup+3.10111 s]
/proc/loadavg: 5.18 7.29 7.67 9/186 2167
/proc/meminfo: memFree=25515024/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=293136 CPUtime=9.36 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 11032 0 0 0 926 10 0 0 25 0 5 0 502575649 300171264 10770 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 73284 10770 175 36 0 71189 0
[pid=2152/tid=2164] ppid=2150 vsize=293136 CPUtime=2.08 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 1102 0 0 0 206 2 0 0 25 0 5 0 502575749 300171264 10770 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=293136 CPUtime=2.08 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 2125 0 0 0 206 2 0 0 21 0 5 0 502575749 300171264 10770 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=293136 CPUtime=2.08 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 726 0 0 0 206 2 0 0 25 0 5 0 502575749 300171264 10770 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=293136 CPUtime=2.08 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 390 0 0 0 206 2 0 0 21 0 5 0 502575749 300171264 10770 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 9.36
Current children cumulated vsize (KiB) 293136

[startup+6.30052 s]
/proc/loadavg: 5.40 7.30 7.67 9/186 2167
/proc/meminfo: memFree=25468280/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=311904 CPUtime=22.12 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 15748 0 0 0 2187 25 0 0 25 0 5 0 502575649 319389696 15486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 77976 15486 175 36 0 75881 0
[pid=2152/tid=2164] ppid=2150 vsize=311904 CPUtime=5.28 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 2221 0 0 0 522 6 0 0 25 0 5 0 502575749 319389696 15486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=311904 CPUtime=5.28 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 4035 0 0 0 523 5 0 0 25 0 5 0 502575749 319389696 15486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=311904 CPUtime=5.27 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 1963 0 0 0 521 6 0 0 25 0 5 0 502575749 319389696 15486 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=311904 CPUtime=5.26 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 840 0 0 0 520 6 0 0 25 0 5 0 502575749 319389696 15486 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 22.12
Current children cumulated vsize (KiB) 311904

[startup+12.7003 s]
/proc/loadavg: 5.61 7.31 7.67 9/186 2168
/proc/meminfo: memFree=25398812/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=335012 CPUtime=47.67 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 22055 0 0 0 4704 63 0 0 25 0 5 0 502575649 343052288 21793 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 83753 21793 175 36 0 81658 0
[pid=2152/tid=2164] ppid=2150 vsize=335012 CPUtime=11.67 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 3163 0 0 0 1153 14 0 0 25 0 5 0 502575749 343052288 21793 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=335012 CPUtime=11.68 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 7639 0 0 0 1153 15 0 0 25 0 5 0 502575749 343052288 21793 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=335012 CPUtime=11.67 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 3089 0 0 0 1151 16 0 0 25 0 5 0 502575749 343052288 21793 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276660 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=335012 CPUtime=11.62 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 1475 0 0 0 1146 16 0 0 25 0 5 0 502575749 343052288 21793 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 47.67
Current children cumulated vsize (KiB) 335012

[startup+25.501 s]
/proc/loadavg: 6.14 7.35 7.67 9/186 2168
/proc/meminfo: memFree=25296648/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=375996 CPUtime=98.78 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 31748 0 0 0 9721 157 0 0 25 0 5 0 502575649 385019904 31486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 93999 31486 175 36 0 91904 0
[pid=2152/tid=2164] ppid=2150 vsize=375996 CPUtime=24.47 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 5325 0 0 0 2409 38 0 0 25 0 5 0 502575749 385019904 31486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=375996 CPUtime=24.48 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 11805 0 0 0 2409 39 0 0 25 0 5 0 502575749 385019904 31486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=375996 CPUtime=24.46 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 5396 0 0 0 2405 41 0 0 25 0 5 0 502575749 385019904 31486 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=375996 CPUtime=24.33 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 2533 0 0 0 2397 36 0 0 25 0 5 0 502575749 385019904 31486 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 98.78
Current children cumulated vsize (KiB) 375996

[startup+51.1002 s]
/proc/loadavg: 6.77 7.40 7.68 9/186 2169
/proc/meminfo: memFree=25062500/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=471124 CPUtime=200.98 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 56620 0 0 0 19740 358 0 0 25 0 5 0 502575649 482430976 56358 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 117781 56358 175 36 0 115686 0
[pid=2152/tid=2164] ppid=2150 vsize=471124 CPUtime=50.05 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 10296 0 0 0 4915 90 0 0 25 0 5 0 502575749 482430976 56358 33554432000 4194304 4338100 140735860966400 18446744073709551615 228094845968 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=471124 CPUtime=50.05 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 23475 0 0 0 4914 91 0 0 20 0 5 0 502575749 482430976 56358 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=471124 CPUtime=50.05 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 9667 0 0 0 4916 89 0 0 25 0 5 0 502575749 482430976 56358 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=471124 CPUtime=49.79 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 6493 0 0 0 4894 85 0 0 25 0 5 0 502575749 482430976 56358 33554432000 4194304 4338100 140735860966400 18446744073709551615 228094846433 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 200.98
Current children cumulated vsize (KiB) 471124

[startup+102.307 s]
/proc/loadavg: 7.43 7.47 7.69 9/186 2170
/proc/meminfo: memFree=24776788/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=570016 CPUtime=405.42 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 80069 0 0 0 39843 699 0 0 25 0 5 0 502575649 583696384 79807 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 142504 79807 175 36 0 140409 0
[pid=2152/tid=2164] ppid=2150 vsize=570016 CPUtime=101.07 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 15844 0 0 0 9930 177 0 0 25 0 5 0 502575749 583696384 79807 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=570016 CPUtime=101.22 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 31176 0 0 0 9943 179 0 0 25 0 5 0 502575749 583696384 79807 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=570016 CPUtime=101.24 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 16397 0 0 0 9951 173 0 0 25 0 5 0 502575749 583696384 79807 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=570016 CPUtime=100.86 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 9963 0 0 0 9918 168 0 0 25 0 5 0 502575749 583696384 79807 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 405.42
Current children cumulated vsize (KiB) 570016

[startup+162.301 s]
/proc/loadavg: 7.90 7.60 7.72 9/186 2172
/proc/meminfo: memFree=24608748/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=627676 CPUtime=644.93 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 93792 0 0 0 63456 1037 0 0 25 0 5 0 502575649 642740224 93530 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 156919 93530 175 36 0 154824 0
[pid=2152/tid=2164] ppid=2150 vsize=627676 CPUtime=160.84 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 21621 0 0 0 15817 267 0 0 25 0 5 0 502575749 642740224 93530 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=627676 CPUtime=161.17 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 35979 0 0 0 15854 263 0 0 25 0 5 0 502575749 642740224 93530 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=627676 CPUtime=161.19 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 18463 0 0 0 15865 254 0 0 25 0 5 0 502575749 642740224 93530 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=627676 CPUtime=160.68 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 11040 0 0 0 15818 250 0 0 25 0 5 0 502575749 642740224 93530 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 644.93
Current children cumulated vsize (KiB) 627676

[startup+222.301 s]
/proc/loadavg: 8.04 7.68 7.74 9/186 2173
/proc/meminfo: memFree=24433020/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=645876 CPUtime=884.49 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 98318 0 0 0 87103 1346 0 0 25 0 5 0 502575649 661377024 98056 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 161469 98056 175 36 0 159374 0
[pid=2152/tid=2164] ppid=2150 vsize=645876 CPUtime=220.78 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 24340 0 0 0 21734 344 0 0 25 0 5 0 502575749 661377024 98056 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0

################
# More data... #
################

/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 18645 0 0 0 68836 1075 0 0 25 0 5 0 502575749 1046622208 191621 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2799.42
Current children cumulated vsize (KiB) 1022092

[startup+762.307 s]
/proc/loadavg: 8.06 7.97 7.85 9/186 2188
/proc/meminfo: memFree=22704528/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1053372 CPUtime=3038.79 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 199305 0 0 0 299170 4709 0 0 25 0 5 0 502575649 1078652928 199043 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 263343 199043 175 36 0 261248 0
[pid=2152/tid=2164] ppid=2150 vsize=1053372 CPUtime=758.75 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 66771 0 0 0 74685 1190 0 0 25 0 5 0 502575749 1078652928 199043 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1053372 CPUtime=759.81 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 55727 0 0 0 74813 1168 0 0 25 0 5 0 502575749 1078652928 199043 33554432000 4194304 4338100 140735860966400 18446744073709551615 228095118338 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1053372 CPUtime=760.15 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 51473 0 0 0 74840 1175 0 0 25 0 5 0 502575749 1078652928 199043 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1053372 CPUtime=759.05 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 18645 0 0 0 74731 1174 0 0 25 0 5 0 502575749 1078652928 199043 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3038.79
Current children cumulated vsize (KiB) 1053372

[startup+822.301 s]
/proc/loadavg: 8.02 7.97 7.85 9/186 2189
/proc/meminfo: memFree=22607036/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1053372 CPUtime=3278.19 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 199550 0 0 0 322814 5005 0 0 25 0 5 0 502575649 1078652928 199288 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 263343 199288 175 36 0 261248 0
[pid=2152/tid=2164] ppid=2150 vsize=1053372 CPUtime=818.65 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 66772 0 0 0 80598 1267 0 0 25 0 5 0 502575749 1078652928 199288 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1053372 CPUtime=819.61 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 55886 0 0 0 80720 1241 0 0 25 0 5 0 502575749 1078652928 199288 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1053372 CPUtime=819.89 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 51474 0 0 0 80742 1247 0 0 25 0 5 0 502575749 1078652928 199288 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1053372 CPUtime=819 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 18729 0 0 0 80652 1248 0 0 25 0 5 0 502575749 1078652928 199288 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3278.19
Current children cumulated vsize (KiB) 1053372

[startup+882.301 s]
/proc/loadavg: 8.04 7.99 7.86 9/186 2191
/proc/meminfo: memFree=22516132/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1063612 CPUtime=3517.37 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 202310 0 0 0 346296 5441 0 0 25 0 5 0 502575649 1089138688 202048 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 265903 202048 175 36 0 263808 0
[pid=2152/tid=2164] ppid=2150 vsize=1063612 CPUtime=878.27 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 67184 0 0 0 86447 1380 0 0 25 0 5 0 502575749 1089138688 202048 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1063612 CPUtime=879.5 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 57153 0 0 0 86603 1347 0 0 18 0 5 0 502575749 1089138688 202048 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1063612 CPUtime=879.72 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 51788 0 0 0 86616 1356 0 0 25 0 5 0 502575749 1089138688 202048 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1063612 CPUtime=878.85 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 19496 0 0 0 86529 1356 0 0 25 0 5 0 502575749 1089138688 202048 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3517.37
Current children cumulated vsize (KiB) 1063612

[startup+942.3 s]
/proc/loadavg: 8.01 7.99 7.87 9/186 2192
/proc/meminfo: memFree=22367064/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1069868 CPUtime=3756.7 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 204998 0 0 0 369870 5800 0 0 25 0 5 0 502575649 1095544832 204736 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 267467 204736 175 36 0 265372 0
[pid=2152/tid=2164] ppid=2150 vsize=1069868 CPUtime=937.86 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 68333 0 0 0 92320 1466 0 0 25 0 5 0 502575749 1095544832 204736 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1069868 CPUtime=939.41 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 57321 0 0 0 92502 1439 0 0 25 0 5 0 502575749 1095544832 204736 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1069868 CPUtime=939.61 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 53158 0 0 0 92515 1446 0 0 25 0 5 0 502575749 1095544832 204736 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1069868 CPUtime=938.78 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 19497 0 0 0 92432 1446 0 0 19 0 5 0 502575749 1095544832 204736 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3756.7
Current children cumulated vsize (KiB) 1069868

[startup+1002.3 s]
/proc/loadavg: 7.93 7.96 7.86 9/186 2194
/proc/meminfo: memFree=22185000/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1111388 CPUtime=3996.05 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 213452 0 0 0 393451 6154 0 0 25 0 5 0 502575649 1138061312 213190 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 277847 213190 175 36 0 275752 0
[pid=2152/tid=2164] ppid=2150 vsize=1111388 CPUtime=997.74 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 72998 0 0 0 98215 1559 0 0 25 0 5 0 502575749 1138061312 213190 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1111388 CPUtime=999.32 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 58044 0 0 0 98406 1526 0 0 22 0 5 0 502575749 1138061312 213190 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1111388 CPUtime=999.25 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 56224 0 0 0 98392 1533 0 0 25 0 5 0 502575749 1138061312 213190 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254030 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1111388 CPUtime=998.69 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 19497 0 0 0 98336 1533 0 0 25 0 5 0 502575749 1138061312 213190 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3996.05
Current children cumulated vsize (KiB) 1111388

[startup+1062.3 s]
/proc/loadavg: 7.94 7.95 7.86 9/186 2196
/proc/meminfo: memFree=21963652/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1166556 CPUtime=4235.25 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 227212 0 0 0 416971 6554 0 0 25 0 5 0 502575649 1194553344 226950 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 291639 226950 175 36 0 289544 0
[pid=2152/tid=2164] ppid=2150 vsize=1166556 CPUtime=1057.48 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 80411 0 0 0 104085 1663 0 0 25 0 5 0 502575749 1194553344 226950 33554432000 4194304 4338100 140735860966400 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1166556 CPUtime=1059.19 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 59374 0 0 0 104297 1622 0 0 25 0 5 0 502575749 1194553344 226950 33554432000 4194304 4338100 140735860966400 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1166556 CPUtime=1058.95 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 61119 0 0 0 104262 1633 0 0 21 0 5 0 502575749 1194553344 226950 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1166556 CPUtime=1058.6 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 19619 0 0 0 104226 1634 0 0 25 0 5 0 502575749 1194553344 226950 33554432000 4194304 4338100 140735860966400 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4235.25
Current children cumulated vsize (KiB) 1166556

[startup+1122.3 s]
/proc/loadavg: 8.12 7.99 7.88 9/186 2197
/proc/meminfo: memFree=21875584/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1188452 CPUtime=4474.58 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 233429 0 0 0 440579 6879 0 0 25 0 5 0 502575649 1216974848 233167 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 297113 233167 175 36 0 295018 0
[pid=2152/tid=2164] ppid=2150 vsize=1188452 CPUtime=1117.16 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 82214 0 0 0 109970 1746 0 0 25 0 5 0 502575749 1216974848 233167 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1188452 CPUtime=1119.09 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 59376 0 0 0 110209 1700 0 0 25 0 5 0 502575749 1216974848 233167 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1188452 CPUtime=1118.79 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 65528 0 0 0 110163 1716 0 0 25 0 5 0 502575749 1216974848 233167 33554432000 4194304 4338100 140735860966400 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1188452 CPUtime=1118.51 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 19622 0 0 0 110136 1715 0 0 25 0 5 0 502575749 1216974848 233167 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107276981 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4474.58
Current children cumulated vsize (KiB) 1188452

[startup+1182.3 s]
/proc/loadavg: 8.18 8.02 7.90 9/186 2199
/proc/meminfo: memFree=21786416/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=1197836 CPUtime=4713.97 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) S 2150 2152 1454 0 -1 4202496 235183 0 0 0 464192 7205 0 0 25 0 5 0 502575649 1226584064 234921 33554432000 4194304 4338100 140735860966400 18446744073709551615 228107253733 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 299459 234921 175 36 0 297364 0
[pid=2152/tid=2164] ppid=2150 vsize=1197836 CPUtime=1176.94 cores=1,3,5,7
/proc/2152/task/2164/stat : 2164 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 83359 0 0 0 115866 1828 0 0 25 0 5 0 502575749 1226584064 234921 33554432000 4194304 4338100 140735860966400 18446744073709551615 4235770 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=2152/tid=2165] ppid=2150 vsize=1197836 CPUtime=1178.85 cores=1,3,5,7
/proc/2152/task/2165/stat : 2165 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 59378 0 0 0 116104 1781 0 0 25 0 5 0 502575749 1226584064 234921 33554432000 4194304 4338100 140735860966400 18446744073709551615 4235674 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=2152/tid=2166] ppid=2150 vsize=1197836 CPUtime=1178.71 cores=1,3,5,7
/proc/2152/task/2166/stat : 2166 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 66133 0 0 0 116073 1798 0 0 25 0 5 0 502575749 1226584064 234921 33554432000 4194304 4338100 140735860966400 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=1197836 CPUtime=1178.43 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4202560 19624 0 0 0 116048 1795 0 0 22 0 5 0 502575749 1226584064 234921 33554432000 4194304 4338100 140735860966400 18446744073709551615 4235674 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4713.97
Current children cumulated vsize (KiB) 1197836



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.9 s]
/proc/loadavg: 8.18 8.03 7.90 6/183 2199
/proc/meminfo: memFree=22102128/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=0 CPUtime=4800.12 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) Z 2150 2152 1454 0 -1 4203548 235184 0 0 0 472705 7307 0 0 18 0 2 0 502575649 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 0 0 0 0 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=0 CPUtime=1200.02 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4203588 19624 0 0 0 118181 1821 0 0 25 0 2 0 502575749 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.12
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.9 s]
/proc/loadavg: 8.18 8.03 7.90 6/183 2199
/proc/meminfo: memFree=22102128/32951124 swapFree=67111524/67111528
[pid=2152] ppid=2150 vsize=0 CPUtime=4800.12 cores=1,3,5,7
/proc/2152/stat : 2152 (ttsth-5-4) Z 2150 2152 1454 0 -1 4203548 235184 0 0 0 472705 7307 0 0 18 0 2 0 502575649 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/2152/statm: 0 0 0 0 0 0 0
[pid=2152/tid=2167] ppid=2150 vsize=0 CPUtime=1200.02 cores=1,3,5,7
/proc/2152/task/2167/stat : 2167 (ttsth-5-4) R 2150 2152 1454 0 -1 4203588 19624 0 0 0 118181 1821 0 0 25 0 2 0 502575749 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.12
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.93
CPU time (s): 4800.15
CPU user time (s): 4727.06
CPU system time (s): 73.0959
CPU usage (%): 398.708
Max. virtual memory (cumulated for all children) (KiB): 1197836

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4727.06
system time used= 73.0959
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 235184
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 362222
involuntary context switches= 33740

runsolver used 1.84272 second user time and 4.33434 second system time

The end

Launcher Data 

Begin job on node142 at 2011-04-29 20:26:42
IDJOB=3265126
IDBENCH=85438
IDSOLVER=1654
FILE ID=node142/3265126-1304101602
RUNJOBID= node142-1304101602-2139
PBS_JOBID= 13170602
Free space on /tmp= 71312 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/sabharwal05/counting/php/unsat/easier/php-045-035.shuffled-as.sat05-1181.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265126-1304101602/watcher-3265126-1304101602 -o /tmp/evaluation-result-3265126-1304101602/solver-3265126-1304101602 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265126-1304101602.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 057b78ff24cc0773c9c67e770a7b9de2
RANDOM SEED=2042016810

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25622032 kB
Buffers:       1681180 kB
Cached:        5038912 kB
SwapCached:          4 kB
Active:         875576 kB
Inactive:      5912620 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25622032 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1168 kB
Writeback:           0 kB
AnonPages:       67580 kB
Mapped:          14540 kB
Slab:           478220 kB
PageTables:       4724 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   191800 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71312 MiB
End job on node142 at 2011-04-29 20:46:48