Trace number 3259538

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.24 1203.96

General information on the benchmark

NameCRAFTED/rbsat/crafted/forced/
rbsat-v945c61409gyes1.cnf
MD5SUM26b9098de258b0434de90e744962df46
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark3.17352
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61409
Sum of the clauses size123673
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261364
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.04	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259538-1304055782.cnf 
0.00/0.04	c Using SATTIMEOUT of 4800
0.00/0.04	c Using SATRAM of 15500
0.00/0.04	c Size of subset node = 32
0.00/0.04	c Variables = 945
0.00/0.04	c Clauses = 61409
0.00/0.04	c Literals = 123673
0.00/0.04	c Reduced size:
0.00/0.04	c Variables = 945
0.00/0.04	c Clauses = 61409
0.00/0.04	c Literals = 123673
0.00/0.04	c a2
0.00/0.07	c a1
0.09/0.14	c Independent sub-problems = 1
0.09/0.14	c Sub-problem with 945 variables and 61409 clauses
0.09/0.14	c using permutation method 4
0.09/0.14	c count = 1
0.09/0.14	c Ordering finished 0.130 seconds
0.09/0.14	c mapped 0.140 seconds
0.09/0.14	c permuted 0.140 seconds
0.09/0.14	c about to sort
0.09/0.15	c sorted
0.19/0.27	c clauses now 61409
0.19/0.27	c init finished 0.150 seconds
0.19/0.27	c tree built 0.270 seconds
0.19/0.27	c 
0.19/0.27	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.19/0.27	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.19/0.27	c ................................ | ..........                    |                                  0.3         0.0             
0.19/0.27	c ................................ | ..........                  1 |                                  0.3         3.7             
0.19/0.27	c ................................ | ..........                  2 |                                  0.3         7.4             
0.19/0.27	c ................................ | ..........                  3 |                                  0.3        11.1             
0.19/0.27	c ................................ | ..........                  4 |                                  0.3        14.8             
0.19/0.27	c ................................ | ..........                  5 |                                  0.3        18.5             
0.19/0.28	c ................................ | ..........               1000 | 7            70                  0.3      3703.7         3390
95.91/24.32	c ................................ | ..........            3607000 | 85392        533101             96.0     37572.9     17845725
191.89/48.43	c ................................ | ..........            6636000 | 153114       934365            192.0     34560.7     30564296
287.91/72.53	c ................................ | ..........            9157000 | 210805       1280913           288.0     31791.8     37385055
383.94/96.66	c ................................ | ..........           11512000 | 267408       1594995           384.0     29976.8     43409626
479.99/120.71	c ................................ | *.........           13664000 | 322042       1878554           480.0     28464.3     45217333
576.05/144.80	c ................................ | *.........           15602000 | 370740       2133660           576.1     27084.0     51532471
671.67/168.89	c ................................ | *.........           17838000 | 436677       2437365           672.0     26543.5     56754559
767.67/192.98	c ................................ | *.........           19924000 | 492903       2712407           768.0     25942.7     61510114
863.67/217.09	c ................................ | *.........           21678000 | 544027       2969991           864.1     25088.5     65058611
959.70/241.18	c ................................ | **........           23251000 | 592105       3207461           960.0     24219.3     67836815
1055.74/265.26	c ................................ | **........           24797000 | 640643       3424496          1056.0     23481.8     72660959
1151.71/289.37	c ................................ | **........           26149000 | 683045       3664905          1152.0     22698.4     73430134
1247.72/313.48	c ................................ | **........           27626000 | 728963       3875519          1248.0     22135.3     73993654
1343.79/337.56	c ................................ | **........           29029000 | 773032       4070155          1344.0     21598.5     78365101
1439.86/361.64	c ................................ | ***.......           30514000 | 818489       4263073          1440.0     21190.3     82568336
1535.95/385.73	c ................................ | ***.......           31769000 | 859372       4444470          1536.0     20682.3     84938621
1631.61/409.79	c ................................ | ***.......           32947000 | 897127       4619961          1632.0     20188.1     89266038
1727.71/433.88	c ................................ | ***.......           34173000 | 938126       4780293          1728.0     19775.6     89608671
1823.79/457.97	c ................................ | ***.......           35160000 | 973451       4927823          1824.1     19275.6     89894931
1919.91/482.02	c ................................ | ****......           36350000 | 1011218      5081392          1920.0     18932.2     92943191
2015.93/506.13	c ................................ | ****......           37541000 | 1050279      5298305          2016.1     18620.9     96743237
2111.78/530.17	c ................................ | ****......           38532000 | 1083996      5482328          2112.1     18243.8    100679886
2207.99/554.20	c ................................ | ****......           39517000 | 1118601      5669148          2208.0     17897.1    103788837
2303.86/578.25	c ................................ | ****......           40656000 | 1154195      5843782          2304.0     17645.6    104042752
2399.98/602.30	c ................................ | *****.....           42407000 | 1196375      6040489          2400.0     17669.5    109143613
2495.81/626.35	c ................................ | *****.....           43891000 | 1232541      6229252          2496.0     17584.4    110575333
2592.00/650.40	c ................................ | *****.....           45295000 | 1270484      6401266          2592.0     17474.7    113934274
2687.71/674.47	c ................................ | *****.....           46664000 | 1308046      6585731          2688.0     17360.0    117020126
2783.92/698.53	c ................................ | *****.....           47977000 | 1344324      6778146          2784.1     17232.7    122795308
2879.74/722.58	c ................................ | ******....           49268000 | 1378074      6951973          2880.1     17106.5    127593246
2975.95/746.63	c ................................ | ******....           50674000 | 1415281      7113579          2976.0     17027.3    131126776
3071.71/770.68	c ................................ | ******....           52029000 | 1450107      7268366          3072.1     16936.2    132852547
3167.90/794.72	c ................................ | ******....           53328000 | 1485907      7418352          3168.0     16833.3    135233627
3264.01/818.80	c ................................ | ******....           54585000 | 1523589      7573550          3264.0     16723.3    138200452
3359.80/842.86	c ................................ | *******...           55786000 | 1557779      7730657          3360.1     16602.7    140210171
3456.02/866.90	c ................................ | *******...           56952000 | 1586961      7883858          3456.1     16478.9    141902150
3551.87/890.94	c ................................ | *******...           58060000 | 1615542      8025753          3552.1     16345.4    143542682
3648.00/915.01	c ................................ | *******...           59092000 | 1646271      8161377          3648.1     16198.2    147472165
3743.88/939.04	c ................................ | *******...           60192000 | 1676180      8295147          3744.0     16076.8    150776449
3839.67/963.09	c ................................ | ********..           61258000 | 1706829      8431014          3840.0     15952.5    153062165
3935.82/987.14	c ................................ | ********..           62216000 | 1735569      8589217          3936.0     15806.9    153062165
4031.98/1011.20	c ................................ | ********..           63105000 | 1763408      8746141          4032.0     15651.0    153062165
4127.71/1035.27	c ................................ | ********..           64049000 | 1790795      8903424          4128.0     15515.7    153062165
4223.96/1059.31	c ................................ | ********..           64925000 | 1819390      9052032          4224.0     15370.4    153891815
4319.67/1083.38	c ................................ | *********.           65870000 | 1848394      9202320          4320.0     15247.6    157411911
4415.81/1107.46	c ................................ | *********.           67040000 | 1880932      9347849          4416.1     15181.0    161073119
4511.90/1131.53	c ................................ | *********.           68145000 | 1913359      9489215          4512.0     15103.0    164928330
4607.62/1155.59	c ................................ | *********.           69210000 | 1945584      9622269          4608.0     15019.5    168880900
4703.72/1179.67	c ................................ | *********.           70292000 | 1976779      9748895          4704.0     14943.0    171485747
4799.83/1203.74	c CPU limit (4800) reached
4799.83/1203.74	c Time = 4800.02 seconds
4799.83/1203.74	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259538-1304055782/watcher-3259538-1304055782 -o /tmp/evaluation-result-3259538-1304055782/solver-3259538-1304055782 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259538-1304055782.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.17 7.90 7.96 2/177 30906
/proc/meminfo: memFree=15162308/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=9760 CPUtime=0 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) R 30904 30906 30368 0 -1 4202496 489 0 0 0 0 0 0 0 23 0 1 0 517177848 9994240 431 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473370992 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30906/statm: 2440 432 147 36 0 345 0

[startup+0.100795 s]
/proc/loadavg: 7.17 7.90 7.96 2/177 30906
/proc/meminfo: memFree=15162308/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=20760 CPUtime=0.09 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) R 30904 30906 30368 0 -1 4202496 3834 0 0 0 9 0 0 0 23 0 1 0 517177848 21258240 3208 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30906/statm: 5190 3209 155 36 0 3095 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 20760

[startup+0.20078 s]
/proc/loadavg: 7.17 7.90 7.96 2/177 30906
/proc/meminfo: memFree=15162308/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=36548 CPUtime=0.19 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) R 30904 30906 30368 0 -1 4202496 7869 0 0 0 19 0 0 0 24 0 1 0 517177848 37425152 7123 33554432000 4194304 4338100 140734923881504 18446744073709551615 4278345 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30906/statm: 9137 7123 164 36 0 7042 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 36548

[startup+0.300759 s]
/proc/loadavg: 7.17 7.90 7.96 2/177 30906
/proc/meminfo: memFree=15162308/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=237056 CPUtime=0.35 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 12481 0 0 0 34 1 0 0 25 0 5 0 517177848 242745344 11615 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 59264 11615 176 36 0 57169 0
Current children cumulated CPU time (s) 0.35
Current children cumulated vsize (KiB) 237056

[startup+1.38157 s]
/proc/loadavg: 7.17 7.90 7.96 2/177 30906
/proc/meminfo: memFree=15162308/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=278844 CPUtime=4.67 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 23909 0 0 0 462 5 0 0 25 0 5 0 517177848 285536256 23043 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 69711 23044 176 36 0 67616 0
Current children cumulated CPU time (s) 4.67
Current children cumulated vsize (KiB) 278844

[startup+1.50055 s]
/proc/loadavg: 7.17 7.90 7.96 2/177 30906
/proc/meminfo: memFree=15162308/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=281972 CPUtime=5.14 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 24853 0 0 0 509 5 0 0 25 0 5 0 517177848 288739328 23987 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 70493 23987 176 36 0 68398 0
Current children cumulated CPU time (s) 5.14
Current children cumulated vsize (KiB) 281972

[startup+3.10027 s]
/proc/loadavg: 7.31 7.92 7.96 9/186 30915
/proc/meminfo: memFree=14860676/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=338848 CPUtime=11.51 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 39125 0 0 0 1140 11 0 0 25 0 5 0 517177848 346980352 38259 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 84712 38259 176 36 0 82617 0
[pid=30906/tid=30912] ppid=30904 vsize=338848 CPUtime=2.81 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 9624 0 0 0 279 2 0 0 24 0 5 0 517177876 346980352 38259 33554432000 4194304 4338100 140734923881504 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=338848 CPUtime=2.76 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 4416 0 0 0 274 2 0 0 18 0 5 0 517177876 346980352 38259 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473362828 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=338848 CPUtime=2.81 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 9245 0 0 0 280 1 0 0 23 0 5 0 517177876 346980352 38259 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473362823 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=338848 CPUtime=2.81 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 3958 0 0 0 278 3 0 0 23 0 5 0 517177876 346980352 38260 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473362662 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.51
Current children cumulated vsize (KiB) 338848

[startup+6.3067 s]
/proc/loadavg: 7.37 7.92 7.96 9/186 30915
/proc/meminfo: memFree=14663940/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=408376 CPUtime=24.28 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 56733 0 0 0 2409 19 0 0 25 0 5 0 517177848 418177024 55139 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 102094 55139 176 36 0 99999 0
[pid=30906/tid=30912] ppid=30904 vsize=408376 CPUtime=6.02 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 16257 0 0 0 598 4 0 0 25 0 5 0 517177876 418177024 55139 33554432000 4194304 4338100 140734923881504 18446744073709551615 4257944 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=408376 CPUtime=5.92 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 6606 0 0 0 588 4 0 0 18 0 5 0 517177876 418177024 55139 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=408376 CPUtime=6.02 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 16098 0 0 0 599 3 0 0 25 0 5 0 517177876 418177024 55139 33554432000 4194304 4338100 140734923881504 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=408376 CPUtime=6.02 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 5890 0 0 0 596 6 0 0 18 0 5 0 517177876 418177024 55139 33554432000 4194304 4338100 140734923881504 18446744073709551615 4240545 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.28
Current children cumulated vsize (KiB) 408376

[startup+12.7006 s]
/proc/loadavg: 7.50 7.94 7.97 9/186 30915
/proc/meminfo: memFree=14423968/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=621588 CPUtime=49.74 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 94115 0 0 0 4936 38 0 0 25 0 5 0 517177848 636506112 92521 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 155397 92521 176 36 0 153302 0
[pid=30906/tid=30912] ppid=30904 vsize=621588 CPUtime=12.41 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 32479 0 0 0 1234 7 0 0 25 0 5 0 517177876 636506112 92521 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473363182 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=621588 CPUtime=12.2 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 9981 0 0 0 1213 7 0 0 18 0 5 0 517177876 636506112 92521 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=621588 CPUtime=12.41 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 30472 0 0 0 1236 5 0 0 25 0 5 0 517177876 636506112 92522 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485781763 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=621588 CPUtime=12.41 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 9301 0 0 0 1225 16 0 0 25 0 5 0 517177876 636506112 92522 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 49.74
Current children cumulated vsize (KiB) 621588

[startup+25.5003 s]
/proc/loadavg: 7.58 7.94 7.97 9/186 30915
/proc/meminfo: memFree=13808940/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=894648 CPUtime=100.68 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 162640 0 0 0 9994 74 0 0 25 0 5 0 517177848 916119552 161046 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 223662 161046 176 36 0 221567 0
[pid=30906/tid=30912] ppid=30904 vsize=894648 CPUtime=25.21 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 60505 0 0 0 2507 14 0 0 25 0 5 0 517177876 916119552 161047 33554432000 4194304 4338100 140734923881504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=894648 CPUtime=24.85 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 20499 0 0 0 2466 19 0 0 25 0 5 0 517177876 916119552 161048 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473362657 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=894648 CPUtime=25.21 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 50822 0 0 0 2512 9 0 0 25 0 5 0 517177876 916119552 161048 33554432000 4194304 4338100 140734923881504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=894648 CPUtime=25.12 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 18934 0 0 0 2482 30 0 0 18 0 5 0 517177876 916119552 161048 33554432000 4194304 4338100 140734923881504 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.68
Current children cumulated vsize (KiB) 894648

[startup+51.1008 s]
/proc/loadavg: 7.80 7.96 7.97 9/186 30916
/proc/meminfo: memFree=12889840/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=1325804 CPUtime=202.66 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 266670 0 0 0 20130 136 0 0 25 0 5 0 517177848 1357623296 265076 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 331451 265076 176 36 0 329356 0
[pid=30906/tid=30912] ppid=30904 vsize=1325804 CPUtime=50.8 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 110688 0 0 0 5056 24 0 0 25 0 5 0 517177876 1357623296 265076 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=1325804 CPUtime=50.45 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 32944 0 0 0 4989 56 0 0 25 0 5 0 517177876 1357623296 265076 33554432000 4194304 4338100 140734923881504 18446744073709551615 4256294 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=1325804 CPUtime=50.8 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 80180 0 0 0 5065 15 0 0 25 0 5 0 517177876 1357623296 265076 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=1325804 CPUtime=50.29 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 30976 0 0 0 4991 38 0 0 18 0 5 0 517177876 1357623296 265076 33554432000 4194304 4338100 140734923881504 18446744073709551615 4256753 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.66
Current children cumulated vsize (KiB) 1325804

[startup+102.301 s]
/proc/loadavg: 8.08 8.01 7.99 9/186 30917
/proc/meminfo: memFree=11992688/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=1757224 CPUtime=406.65 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 373640 0 0 0 40436 229 0 0 25 0 5 0 517177848 1799397376 372046 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 439306 372046 176 36 0 437211 0
[pid=30906/tid=30912] ppid=30904 vsize=1757224 CPUtime=101.94 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 177269 0 0 0 10154 40 0 0 25 0 5 0 517177876 1799397376 372046 33554432000 4194304 4338100 140734923881504 18446744073709551615 4235914 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=1757224 CPUtime=101.2 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 37226 0 0 0 10035 85 0 0 18 0 5 0 517177876 1799397376 372046 33554432000 4194304 4338100 140734923881504 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=1757224 CPUtime=101.99 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 114700 0 0 0 10177 22 0 0 25 0 5 0 517177876 1799397376 372046 33554432000 4194304 4338100 140734923881504 18446744073709551615 4235847 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=1757224 CPUtime=101.2 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 32563 0 0 0 10042 78 0 0 25 0 5 0 517177876 1799397376 372046 33554432000 4194304 4338100 140734923881504 18446744073709551615 4235674 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 406.65
Current children cumulated vsize (KiB) 1757224

[startup+162.3 s]
/proc/loadavg: 8.03 8.01 7.99 9/186 30919
/proc/meminfo: memFree=11033412/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=2186916 CPUtime=645.75 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 477206 0 0 0 64242 333 0 0 25 0 5 0 517177848 2239401984 475612 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 546729 475612 176 36 0 544634 0
[pid=30906/tid=30912] ppid=30904 vsize=2186916 CPUtime=161.61 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 232446 0 0 0 16104 57 0 0 25 0 5 0 517177876 2239401984 475612 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=2186916 CPUtime=160.66 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 38734 0 0 0 15969 97 0 0 18 0 5 0 517177876 2239401984 475612 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=2186916 CPUtime=161.98 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 160321 0 0 0 16168 30 0 0 25 0 5 0 517177876 2239401984 475612 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=2186916 CPUtime=161.19 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 33823 0 0 0 15973 146 0 0 25 0 5 0 517177876 2239401984 475612 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 645.75
Current children cumulated vsize (KiB) 2186916

[startup+222.301 s]
/proc/loadavg: 8.01 8.00 7.99 9/186 30921
/proc/meminfo: memFree=10245752/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=2501596 CPUtime=884.79 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 553355 0 0 0 88051 428 0 0 25 0 5 0 517177848 2561634304 551761 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 625399 551761 176 36 0 623304 0
[pid=30906/tid=30912] ppid=30904 vsize=2501596 CPUtime=221.28 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 273169 0 0 0 22059 69 0 0 25 0 5 0 517177876 2561634304 551761 33554432000 4194304 4338100 140734923881504 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 41927 0 0 0 69059 958 0 0 25 0 5 0 517177876 4478738432 1020451 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2799.08
Current children cumulated vsize (KiB) 4373768

[startup+762.301 s]
/proc/loadavg: 8.09 8.09 8.02 9/186 30935
/proc/meminfo: memFree=5184912/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=4655948 CPUtime=3038.55 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1093262 0 0 0 301780 2075 0 0 25 0 5 0 517177848 4767690752 1091668 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1163987 1091668 176 36 0 1161892 0
[pid=30906/tid=30912] ppid=30904 vsize=4655948 CPUtime=760.51 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 583744 0 0 0 75874 177 0 0 25 0 5 0 517177876 4767690752 1091668 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=4655948 CPUtime=757.13 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 47827 0 0 0 74982 731 0 0 25 0 5 0 517177876 4767690752 1091668 33554432000 4194304 4338100 140734923881504 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=4655948 CPUtime=760.67 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 407723 0 0 0 75978 89 0 0 25 0 5 0 517177876 4767690752 1091668 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=4655948 CPUtime=759.94 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 42086 0 0 0 74919 1075 0 0 25 0 5 0 517177876 4767690752 1091668 33554432000 4194304 4338100 140734923881504 18446744073709551615 4240510 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3038.55
Current children cumulated vsize (KiB) 4655948

[startup+822.301 s]
/proc/loadavg: 8.07 8.09 8.02 9/186 30937
/proc/meminfo: memFree=4794548/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=4872440 CPUtime=3278 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1148966 0 0 0 325484 2316 0 0 25 0 5 0 517177848 4989378560 1147372 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1218110 1147372 176 36 0 1216015 0
[pid=30906/tid=30912] ppid=30904 vsize=4872440 CPUtime=820.42 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 619059 0 0 0 81853 189 0 0 25 0 5 0 517177876 4989378560 1147372 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254041 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=4872440 CPUtime=816.76 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 47929 0 0 0 80834 842 0 0 25 0 5 0 517177876 4989378560 1147372 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=4872440 CPUtime=820.62 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 426397 0 0 0 81968 94 0 0 25 0 5 0 517177876 4989378560 1147372 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=4872440 CPUtime=819.89 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 43699 0 0 0 80801 1188 0 0 25 0 5 0 517177876 4989378560 1147372 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3278
Current children cumulated vsize (KiB) 4872440

[startup+882.301 s]
/proc/loadavg: 8.02 8.07 8.02 9/186 30938
/proc/meminfo: memFree=4500892/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=4985576 CPUtime=3517.52 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1184512 0 0 0 349206 2546 0 0 25 0 5 0 517177848 5105229824 1182918 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1246394 1182918 176 36 0 1244299 0
[pid=30906/tid=30912] ppid=30904 vsize=4985576 CPUtime=880.39 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 643412 0 0 0 87840 199 0 0 25 0 5 0 517177876 5105229824 1182919 33554432000 4194304 4338100 140734923881504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=4985576 CPUtime=876.42 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 50876 0 0 0 86688 954 0 0 25 0 5 0 517177876 5105229824 1182919 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=4985576 CPUtime=880.56 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 430406 0 0 0 87960 96 0 0 25 0 5 0 517177876 5105229824 1182919 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=4985576 CPUtime=879.85 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 47937 0 0 0 86691 1294 0 0 25 0 5 0 517177876 5105229824 1182919 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3517.52
Current children cumulated vsize (KiB) 4985576

[startup+942.301 s]
/proc/loadavg: 8.01 8.05 8.01 7/186 30940
/proc/meminfo: memFree=4086212/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=5258108 CPUtime=3757.03 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1252651 0 0 0 372923 2780 0 0 25 0 5 0 517177848 5384302592 1251057 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1314527 1251057 176 36 0 1312432 0
[pid=30906/tid=30912] ppid=30904 vsize=5258108 CPUtime=940.34 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 690546 0 0 0 93821 213 0 0 25 0 5 0 517177876 5384302592 1251057 33554432000 4194304 4338100 140734923881504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=5258108 CPUtime=936.17 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 50970 0 0 0 92554 1063 0 0 25 0 5 0 517177876 5384302592 1251057 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=5258108 CPUtime=940.53 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 451227 0 0 0 93950 103 0 0 25 0 5 0 517177876 5384302592 1251057 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=5258108 CPUtime=939.69 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 48026 0 0 0 92570 1399 0 0 25 0 5 0 517177876 5384302592 1251057 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3757.03
Current children cumulated vsize (KiB) 5258108

[startup+1002.3 s]
/proc/loadavg: 8.07 8.06 8.01 9/186 30941
/proc/meminfo: memFree=3810908/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=5502520 CPUtime=3996.46 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1275235 0 0 0 396614 3032 0 0 25 0 5 0 517177848 5634580480 1273641 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1375630 1273641 176 36 0 1373535 0
[pid=30906/tid=30912] ppid=30904 vsize=5502520 CPUtime=1000.25 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 703814 0 0 0 99805 220 0 0 25 0 5 0 517177876 5634580480 1273641 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=5502520 CPUtime=996.1 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 53459 0 0 0 98426 1184 0 0 25 0 5 0 517177876 5634580480 1273641 33554432000 4194304 4338100 140734923881504 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=5502520 CPUtime=1000.49 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 454558 0 0 0 99946 103 0 0 25 0 5 0 517177876 5634580480 1273641 33554432000 4194304 4338100 140734923881504 18446744073709551615 274473362526 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=5502520 CPUtime=999.29 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 51522 0 0 0 98408 1521 0 0 25 0 5 0 517177876 5634580480 1273641 33554432000 4194304 4338100 140734923881504 18446744073709551615 4256753 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3996.46
Current children cumulated vsize (KiB) 5502520

[startup+1062.31 s]
/proc/loadavg: 8.13 8.08 8.02 8/186 30943
/proc/meminfo: memFree=3462204/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=5543712 CPUtime=4235.96 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1292327 0 0 0 420284 3312 0 0 25 0 5 0 517177848 5676761088 1290733 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1385928 1290733 176 36 0 1383833 0
[pid=30906/tid=30912] ppid=30904 vsize=5543712 CPUtime=1060.22 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 708456 0 0 0 105800 222 0 0 25 0 5 0 517177876 5676761088 1290733 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=5543712 CPUtime=1056.01 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 56384 0 0 0 104281 1320 0 0 18 0 5 0 517177876 5676761088 1290733 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=5543712 CPUtime=1060.46 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 461171 0 0 0 105941 105 0 0 25 0 5 0 517177876 5676761088 1290733 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=5543712 CPUtime=1058.96 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 54434 0 0 0 104234 1662 0 0 25 0 5 0 517177876 5676761088 1290733 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4235.96
Current children cumulated vsize (KiB) 5543712

[startup+1122.3 s]
/proc/loadavg: 8.14 8.09 8.02 9/186 30954
/proc/meminfo: memFree=3006212/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=5822632 CPUtime=4475.2 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1363002 0 0 0 444130 3390 0 0 25 0 5 0 517177848 5962375168 1361408 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1455658 1361408 176 36 0 1453563 0
[pid=30906/tid=30912] ppid=30904 vsize=5822632 CPUtime=1120.22 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 752771 0 0 0 111785 237 0 0 25 0 5 0 517177876 5962375168 1361409 33554432000 4194304 4338100 140734923881504 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=5822632 CPUtime=1115.6 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 56482 0 0 0 110235 1325 0 0 18 0 5 0 517177876 5962375168 1361409 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=5822632 CPUtime=1120.14 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 487337 0 0 0 111902 112 0 0 25 0 5 0 517177876 5962375168 1361410 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=5822632 CPUtime=1118.93 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 54531 0 0 0 110180 1713 0 0 25 0 5 0 517177876 5962375168 1361410 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4475.2
Current children cumulated vsize (KiB) 5822632

[startup+1182.3 s]
/proc/loadavg: 8.14 8.10 8.03 9/186 30955
/proc/meminfo: memFree=2704636/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=6076396 CPUtime=4714.49 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) S 30904 30906 30368 0 -1 4202496 1427048 0 0 0 467985 3464 0 0 25 0 5 0 517177848 6222229504 1425454 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 1519099 1425454 176 36 0 1517004 0
[pid=30906/tid=30912] ppid=30904 vsize=6076396 CPUtime=1180.19 cores=0,2,4,6
/proc/30906/task/30912/stat : 30912 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 795463 0 0 0 117769 250 0 0 25 0 5 0 517177876 6222229504 1425454 33554432000 4194304 4338100 140734923881504 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30906/tid=30913] ppid=30904 vsize=6076396 CPUtime=1175.27 cores=0,2,4,6
/proc/30906/task/30913/stat : 30913 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 56565 0 0 0 116195 1332 0 0 18 0 5 0 517177876 6222229504 1425454 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=6076396 CPUtime=1179.81 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 508517 0 0 0 117862 119 0 0 25 0 5 0 517177876 6222229504 1425454 33554432000 4194304 4338100 140734923881504 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30906/tid=30915] ppid=30904 vsize=6076396 CPUtime=1178.91 cores=0,2,4,6
/proc/30906/task/30915/stat : 30915 (ttsth-5-4) R 30904 30906 30368 0 -1 4202560 54621 0 0 0 116131 1760 0 0 25 0 5 0 517177876 6222229504 1425454 33554432000 4194304 4338100 140734923881504 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4714.49
Current children cumulated vsize (KiB) 6076396



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.8 s]
/proc/loadavg: 8.16 8.11 8.03 2/177 30957
/proc/meminfo: memFree=10688056/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=0 CPUtime=4800.08 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) Z 30904 30906 30368 0 -1 4203548 1427178 0 0 0 476508 3500 0 0 18 0 2 0 517177848 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 0 0 0 0 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=0 CPUtime=1201.2 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4203588 508549 0 0 0 119996 124 0 0 25 0 2 0 517177876 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.91 s]
/proc/loadavg: 8.16 8.11 8.03 2/177 30957
/proc/meminfo: memFree=10688056/32950928 swapFree=67111528/67111528
[pid=30906] ppid=30904 vsize=0 CPUtime=4800.18 cores=0,2,4,6
/proc/30906/stat : 30906 (ttsth-5-4) Z 30904 30906 30368 0 -1 4203548 1427178 0 0 0 476508 3510 0 0 18 0 2 0 517177848 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30906/statm: 0 0 0 0 0 0 0
[pid=30906/tid=30914] ppid=30904 vsize=0 CPUtime=1201.31 cores=0,2,4,6
/proc/30906/task/30914/stat : 30914 (ttsth-5-4) R 30904 30906 30368 0 -1 4203588 508549 0 0 0 119996 135 0 0 25 0 2 0 517177876 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.18
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.96
CPU time (s): 4800.24
CPU user time (s): 4765.08
CPU system time (s): 35.1557
CPU usage (%): 398.705
Max. virtual memory (cumulated for all children) (KiB): 6076660

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4765.08
system time used= 35.1557
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1427178
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 10504
involuntary context switches= 21878

runsolver used 2.01869 second user time and 4.33134 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-29 07:43:02
IDJOB=3259538
IDBENCH=71583
IDSOLVER=1654
FILE ID=node125/3259538-1304055782
RUNJOBID= node125-1304055782-30886
PBS_JOBID= 13170412
Free space on /tmp= 72764 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v945c61409gyes1.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259538-1304055782/watcher-3259538-1304055782 -o /tmp/evaluation-result-3259538-1304055782/solver-3259538-1304055782 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3259538-1304055782.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 26b9098de258b0434de90e744962df46
RANDOM SEED=1063350183

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15162820 kB
Buffers:       2218316 kB
Cached:       14972268 kB
SwapCached:          0 kB
Active:        9043040 kB
Inactive:      8213720 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15162820 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1340 kB
Writeback:           0 kB
AnonPages:       64960 kB
Mapped:          14876 kB
Slab:           468732 kB
PageTables:       5092 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   208892 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72764 MiB
End job on node125 at 2011-04-29 08:03:08