Trace number 3258394

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.25 1204.65

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2400c266431gyes7.cnf
MD5SUM92e61742efd2e322c59b4442206bebf1
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data 

0.09/0.18	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258394-1304050944.cnf 
0.09/0.18	c Using SATTIMEOUT of 4800
0.09/0.18	c Using SATRAM of 15500
0.09/0.18	c Size of subset node = 32
0.09/0.18	c Variables = 2400
0.09/0.18	c Clauses = 266431
0.09/0.18	c Literals = 535112
0.09/0.18	c Reduced size:
0.09/0.18	c Variables = 2400
0.09/0.18	c Clauses = 266431
0.09/0.18	c Literals = 535112
0.09/0.18	c a2
0.29/0.33	c a1
0.69/0.70	c Independent sub-problems = 1
0.69/0.70	c Sub-problem with 2400 variables and 266431 clauses
0.69/0.70	c using permutation method 4
0.69/0.70	c count = 1
0.69/0.70	c Ordering finished 0.660 seconds
0.69/0.70	c mapped 0.690 seconds
0.69/0.70	c permuted 0.690 seconds
0.69/0.70	c about to sort
0.69/0.72	c sorted
1.29/1.38	c clauses now 266431
1.29/1.38	c init finished 0.770 seconds
1.29/1.38	c tree built 1.370 seconds
1.29/1.38	c 
1.29/1.38	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.29/1.38	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.29/1.38	c ................................ | ..........                    |                                  1.4         0.0             
1.29/1.39	c ................................ | ..........                  1 |                                  1.4         0.7             
1.29/1.39	c ................................ | ..........                  2 |                                  1.4         1.5             
1.29/1.39	c ................................ | ..........                  3 |                                  1.4         2.2             
1.29/1.39	c ................................ | ..........                  4 |                                  1.4         2.9             
1.29/1.39	c ................................ | ..........                  5 |                                  1.4         3.6             
1.29/1.39	c ................................ | ..........               1000 |              113                 1.4       714.3         4363
95.72/25.17	c ................................ | ..........            3534000 | 98403        197591             96.0     36812.5     12939785
191.67/49.28	c ................................ | ..........            6422000 | 186919       336143            192.0     33447.9     19528542
287.66/73.38	c ................................ | ..........            8825000 | 269541       457258            288.0     30642.4     29121622
383.64/97.49	c ................................ | ..........           11018000 | 345328       569589            384.0     28691.2     32946047
479.96/121.61	c ................................ | *.........           13531000 | 424049       683659            480.0     28189.6     37642715
575.94/145.71	c ................................ | *.........           15744000 | 500443       777478            576.0     27333.3     40706677
671.95/169.81	c ................................ | *.........           17943000 | 570386       872776            672.0     26700.1     45315226
767.93/193.92	c ................................ | *.........           20285000 | 642391       970448            768.0     26412.1     49566209
863.91/218.02	c ................................ | *.........           22371000 | 708592       1062088           864.0     25891.8     51145092
959.95/242.12	c ................................ | **........           25099000 | 770631       1176697           960.0     26144.2     52188801
1055.78/266.16	c ................................ | **........           27735000 | 834458       1265610          1056.0     26264.0     55657440
1151.78/290.25	c ................................ | **........           30101000 | 891656       1347297          1152.0     26128.9     56713826
1247.79/314.35	c ................................ | **........           32800000 | 952808       1444110          1248.0     26281.6     58985668
1343.77/338.45	c ................................ | **........           35207000 | 1007821      1540372          1344.0     26195.7     63572584
1439.81/362.55	c ................................ | ***.......           37674000 | 1065433      1658175          1440.0     26162.3     66787614
1535.95/386.61	c ................................ | ***.......           40035000 | 1126860      1762001          1536.0     26064.3     71772251
1631.72/410.67	c ................................ | ***.......           42421000 | 1179934      1857533          1632.0     25992.8     74592673
1727.97/434.71	c ................................ | ***.......           44672000 | 1228211      1952508          1728.0     25851.4     76346654
1823.81/458.76	c ................................ | ***.......           46820000 | 1280756      2031777          1824.0     25668.3     80257957
1919.98/482.81	c ................................ | ****......           48782000 | 1326229      2106065          1920.0     25406.9     80527159
2015.76/506.86	c ................................ | ****......           51078000 | 1380335      2182251          2016.0     25336.3     80527159
2111.99/530.90	c ................................ | ****......           53901000 | 1447321      2266277          2112.0     25521.3     82309876
2207.70/554.97	c ................................ | ****......           56102000 | 1506245      2359600          2208.0     25408.4     83207287
2303.96/579.01	c ................................ | ****......           58592000 | 1570850      2451537          2304.0     25430.3     87852913
2399.76/603.06	c ................................ | *****.....           60746000 | 1626155      2543048          2400.0     25310.8     91441062
2496.01/627.10	c ................................ | *****.....           62674000 | 1674484      2626325          2496.0     25109.6     94645043
2591.71/651.18	c ................................ | *****.....           64456000 | 1718920      2712358          2592.0     24867.1     98152647
2687.73/675.28	c ................................ | *****.....           66173000 | 1770532      2793669          2688.1     24617.5    102145286
2783.79/699.35	c ................................ | *****.....           67852000 | 1820917      2877721          2784.0     24372.1    106253474
2879.79/723.45	c ................................ | ******....           69591000 | 1869024      2960913          2880.0     24163.5    110160582
2975.89/747.53	c ................................ | ******....           71660000 | 1923144      3058239          2976.0     24079.1    112244876
3071.73/771.57	c ................................ | ******....           73512000 | 1972991      3145810          3072.0     23929.4    112684957
3167.96/795.62	c ................................ | ******....           75289000 | 2021507      3228363          3168.1     23765.0    113349722
3263.80/819.65	c ................................ | ******....           76972000 | 2070947      3302271          3264.0     23582.0    114530116
3359.62/843.69	c ................................ | *******...           78481000 | 2117677      3382450          3360.0     23357.4    115528428
3455.74/867.77	c ................................ | *******...           79974000 | 2158240      3457436          3456.0     23140.4    117129572
3551.94/891.82	c ................................ | *******...           81437000 | 2205443      3532100          3552.0     22926.8    120065127
3647.73/915.88	c ................................ | *******...           82896000 | 2242558      3604507          3648.1     22723.4    124019652
3743.93/939.92	c ................................ | *******...           84401000 | 2285476      3673060          3744.0     22542.8    127484972
3839.71/963.97	c ................................ | ********..           85808000 | 2323718      3784365          3840.0     22345.8    131244490
3935.95/988.02	c ................................ | ********..           87263000 | 2367467      3885166          3936.0     22170.3    133411940
4031.77/1012.07	c ................................ | ********..           88609000 | 2413119      3972858          4032.0     21976.2    136871925
4127.99/1036.12	c ................................ | ********..           89989000 | 2458232      4057397          4128.1     21799.4    139541790
4223.76/1060.17	c ................................ | ********..           91368000 | 2499188      4141297          4224.0     21630.5    142369259
4319.98/1084.20	c ................................ | *********.           92780000 | 2544801      4238980          4320.0     21476.8    143352012
4415.80/1108.26	c ................................ | *********.           94129000 | 2584578      4334691          4416.1     21315.2    145636251
4511.97/1132.32	c ................................ | *********.           95445000 | 2626309      4417684          4512.1     21153.4    145970011
4607.77/1156.35	c ................................ | *********.           96874000 | 2666483      4507223          4608.0     21023.0    147344662
4704.03/1180.41	c ................................ | *********.           98156000 | 2705733      4604225          4704.1     20866.2    148283765
4799.86/1204.45	c CPU limit (4800) reached
4799.86/1204.45	c Time = 4800.05 seconds
4799.86/1204.45	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3258394-1304050944/watcher-3258394-1304050944 -o /tmp/evaluation-result-3258394-1304050944/solver-3258394-1304050944 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258394-1304050944.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.57 8.00 7.72 2/177 30505
/proc/meminfo: memFree=15090604/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=9760 CPUtime=0 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) R 30503 30505 30368 0 -1 4202496 492 0 0 0 0 0 0 0 25 0 1 0 516694092 9994240 434 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30505/statm: 2440 435 147 36 0 345 0

[startup+0.0369221 s]
/proc/loadavg: 7.57 8.00 7.72 2/177 30505
/proc/meminfo: memFree=15090604/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=15788 CPUtime=0.02 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) R 30503 30505 30368 0 -1 4202496 2026 0 0 0 2 0 0 0 25 0 1 0 516694092 16166912 1968 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30505/statm: 3947 1970 148 36 0 1852 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 15788

[startup+0.10088 s]
/proc/loadavg: 7.57 8.00 7.72 2/177 30505
/proc/meminfo: memFree=15090604/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=28808 CPUtime=0.09 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) R 30503 30505 30368 0 -1 4202496 5191 0 0 0 8 1 0 0 25 0 1 0 516694092 29499392 5133 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473628357 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30505/statm: 7202 5135 148 36 0 5107 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28808

[startup+0.300904 s]
/proc/loadavg: 7.57 8.00 7.72 2/177 30505
/proc/meminfo: memFree=15090604/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=60944 CPUtime=0.29 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) R 30503 30505 30368 0 -1 4202496 13198 0 0 0 27 2 0 0 25 0 1 0 516694092 62406656 13139 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362574 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30505/statm: 15236 13139 153 36 0 13141 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 60944

[startup+0.700848 s]
/proc/loadavg: 7.57 8.00 7.72 2/177 30505
/proc/meminfo: memFree=15090604/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=68420 CPUtime=0.69 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) R 30503 30505 30368 0 -1 4202496 16210 0 0 0 67 2 0 0 25 0 1 0 516694092 70062080 15109 33554432000 4194304 4338100 140736750543168 18446744073709551615 4228283 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30505/statm: 17105 15109 161 36 0 15010 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 68420

[startup+1.50064 s]
/proc/loadavg: 7.57 8.00 7.72 6/182 30510
/proc/meminfo: memFree=14861048/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=358840 CPUtime=1.81 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 45240 0 0 0 174 7 0 0 18 0 5 0 516694092 367452160 43097 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 89710 43097 176 36 0 87615 0
Current children cumulated CPU time (s) 1.81
Current children cumulated vsize (KiB) 358840

[startup+3.10035 s]
/proc/loadavg: 7.57 8.00 7.72 9/186 30514
/proc/meminfo: memFree=14740508/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=397300 CPUtime=8.16 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 55233 0 0 0 804 12 0 0 18 0 5 0 516694092 406835200 53090 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 99325 53090 176 36 0 97230 0
[pid=30505/tid=30511] ppid=30503 vsize=397300 CPUtime=1.69 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 3939 0 0 0 169 0 0 0 25 0 5 0 516694230 406835200 53090 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362913 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=397300 CPUtime=1.66 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 1923 0 0 0 165 1 0 0 18 0 5 0 516694230 406835200 53090 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362662 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=397300 CPUtime=1.7 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 4076 0 0 0 170 0 0 0 20 0 5 0 516694230 406835200 53090 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=397300 CPUtime=1.69 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 1957 0 0 0 167 2 0 0 19 0 5 0 516694231 406835200 53091 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473363182 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.16
Current children cumulated vsize (KiB) 397300

[startup+6.30079 s]
/proc/loadavg: 7.69 8.02 7.73 10/186 30514
/proc/meminfo: memFree=14586376/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=463560 CPUtime=20.89 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 72182 0 0 0 2069 20 0 0 18 0 5 0 516694092 474685440 70039 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 115890 70039 176 36 0 113795 0
[pid=30505/tid=30511] ppid=30503 vsize=463560 CPUtime=4.87 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 9497 0 0 0 486 1 0 0 25 0 5 0 516694230 474685440 70039 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=463560 CPUtime=4.8 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 4658 0 0 0 477 3 0 0 18 0 5 0 516694230 474685440 70039 33554432000 4194304 4338100 140736750543168 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=463560 CPUtime=4.9 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 9981 0 0 0 489 1 0 0 25 0 5 0 516694230 474685440 70039 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=463560 CPUtime=4.89 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 4708 0 0 0 483 6 0 0 25 0 5 0 516694231 474685440 70040 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362860 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 20.89
Current children cumulated vsize (KiB) 463560

[startup+12.7007 s]
/proc/loadavg: 7.71 8.02 7.73 10/186 30514
/proc/meminfo: memFree=14327400/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=574176 CPUtime=46.36 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 99780 0 0 0 4600 36 0 0 18 0 5 0 516694092 587956224 97637 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 143544 97637 176 36 0 141449 0
[pid=30505/tid=30511] ppid=30503 vsize=574176 CPUtime=11.24 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 19292 0 0 0 1120 4 0 0 25 0 5 0 516694230 587956224 97637 33554432000 4194304 4338100 140736750543168 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=574176 CPUtime=11.11 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 8813 0 0 0 1105 6 0 0 19 0 5 0 516694230 587956224 97637 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=574176 CPUtime=11.3 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 18866 0 0 0 1127 3 0 0 25 0 5 0 516694230 587956224 97637 33554432000 4194304 4338100 140736750543168 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=574176 CPUtime=11.29 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 9471 0 0 0 1115 14 0 0 18 0 5 0 516694231 587956224 97637 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 46.36
Current children cumulated vsize (KiB) 574176

[startup+25.5004 s]
/proc/loadavg: 7.78 8.02 7.74 9/186 30515
/proc/meminfo: memFree=13907568/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=763840 CPUtime=97.31 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 147823 0 0 0 9665 66 0 0 18 0 5 0 516694092 782172160 145680 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 190960 145680 176 36 0 188865 0
[pid=30505/tid=30511] ppid=30503 vsize=763840 CPUtime=23.97 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 38421 0 0 0 2388 9 0 0 25 0 5 0 516694230 782172160 145680 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362833 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=763840 CPUtime=23.73 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 16095 0 0 0 2363 10 0 0 18 0 5 0 516694230 782172160 145680 33554432000 4194304 4338100 140736750543168 18446744073709551615 4230468 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=763840 CPUtime=24.1 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 33849 0 0 0 2404 6 0 0 25 0 5 0 516694230 782172160 145680 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=763840 CPUtime=24.1 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 16120 0 0 0 2377 33 0 0 25 0 5 0 516694231 782172160 145680 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 97.31
Current children cumulated vsize (KiB) 763840

[startup+51.1009 s]
/proc/loadavg: 7.99 8.04 7.75 9/186 30515
/proc/meminfo: memFree=13409188/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=1018360 CPUtime=199.24 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 209187 0 0 0 19804 120 0 0 18 0 5 0 516694092 1042800640 207044 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 254590 207044 176 36 0 252495 0
[pid=30505/tid=30511] ppid=30503 vsize=1018360 CPUtime=49.43 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 66242 0 0 0 4924 19 0 0 25 0 5 0 516694230 1042800640 207044 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362670 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=1018360 CPUtime=48.99 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 21751 0 0 0 4883 16 0 0 18 0 5 0 516694230 1042800640 207044 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=1018360 CPUtime=49.7 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 58969 0 0 0 4959 11 0 0 25 0 5 0 516694230 1042800640 207044 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=1018360 CPUtime=49.69 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 18887 0 0 0 4904 65 0 0 25 0 5 0 516694231 1042800640 207044 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 199.24
Current children cumulated vsize (KiB) 1018360

[startup+102.307 s]
/proc/loadavg: 8.16 8.08 7.78 9/186 30517
/proc/meminfo: memFree=12532252/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=1463396 CPUtime=403.17 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 320458 0 0 0 40100 217 0 0 18 0 5 0 516694092 1498517504 318315 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 365849 318315 176 36 0 363754 0
[pid=30505/tid=30511] ppid=30503 vsize=1463396 CPUtime=100.37 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 108266 0 0 0 10004 33 0 0 25 0 5 0 516694230 1498517504 318315 33554432000 4194304 4338100 140736750543168 18446744073709551615 4240540 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=1463396 CPUtime=99.59 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 37744 0 0 0 9932 27 0 0 18 0 5 0 516694230 1498517504 318315 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=1463396 CPUtime=100.9 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 96886 0 0 0 10073 17 0 0 25 0 5 0 516694230 1498517504 318315 33554432000 4194304 4338100 140736750543168 18446744073709551615 4198872 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=1463396 CPUtime=100.89 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 34224 0 0 0 9958 131 0 0 25 0 5 0 516694231 1498517504 318315 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 403.17
Current children cumulated vsize (KiB) 1463396

[startup+162.3 s]
/proc/loadavg: 8.13 8.08 7.79 9/186 30518
/proc/meminfo: memFree=11761080/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=1856100 CPUtime=642.06 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 399606 0 0 0 63889 317 0 0 18 0 5 0 516694092 1900646400 396036 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 464025 396036 176 36 0 461930 0
[pid=30505/tid=30511] ppid=30503 vsize=1856100 CPUtime=160 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 154531 0 0 0 15950 50 0 0 25 0 5 0 516694230 1900646400 396036 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=1856100 CPUtime=158.9 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 38343 0 0 0 15854 36 0 0 18 0 5 0 516694230 1900646400 396036 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362818 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=1856100 CPUtime=160.87 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 127516 0 0 0 16062 25 0 0 25 0 5 0 516694230 1900646400 396036 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=1856100 CPUtime=160.86 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 35878 0 0 0 15889 197 0 0 25 0 5 0 516694231 1900646400 396036 33554432000 4194304 4338100 140736750543168 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 642.06
Current children cumulated vsize (KiB) 1856100

[startup+222.301 s]
/proc/loadavg: 8.08 8.08 7.81 9/186 30520
/proc/meminfo: memFree=11089852/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=2094684 CPUtime=881.05 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 462565 0 0 0 87693 412 0 0 18 0 5 0 516694092 2144956416 458995 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 523671 458995 176 36 0 521576 0
[pid=30505/tid=30511] ppid=30503 vsize=2094684 CPUtime=219.67 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 190496 0 0 0 21903 64 0 0 25 0 5 0 516694230 2144956416 458995 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 54189 0 0 0 68935 995 0 0 25 0 5 0 516694231 4233154560 929544 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2795.69
Current children cumulated vsize (KiB) 4133940

[startup+762.307 s]
/proc/loadavg: 8.13 8.10 7.92 9/186 30534
/proc/meminfo: memFree=7243988/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=4305980 CPUtime=3035.01 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 976671 0 0 0 301305 2196 0 0 18 0 5 0 516694092 4409323520 973101 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1076495 973101 176 36 0 1074400 0
[pid=30505/tid=30511] ppid=30503 vsize=4305980 CPUtime=758.48 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 508050 0 0 0 75655 193 0 0 25 0 5 0 516694230 4409323520 973101 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=4305980 CPUtime=755.8 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 56460 0 0 0 74730 850 0 0 25 0 5 0 516694230 4409323520 973101 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=4305980 CPUtime=760.17 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 314634 0 0 0 75948 69 0 0 25 0 5 0 516694230 4409323520 973101 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=4305980 CPUtime=759.14 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 54189 0 0 0 74838 1076 0 0 25 0 5 0 516694231 4409323520 973101 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3035.01
Current children cumulated vsize (KiB) 4305980

[startup+822.301 s]
/proc/loadavg: 8.08 8.09 7.92 9/186 30536
/proc/meminfo: memFree=7065296/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=4381052 CPUtime=3274.59 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 998786 0 0 0 325008 2451 0 0 18 0 5 0 516694092 4486197248 995216 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1095263 995216 176 36 0 1093168 0
[pid=30505/tid=30511] ppid=30503 vsize=4381052 CPUtime=818.46 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 524623 0 0 0 81645 201 0 0 25 0 5 0 516694230 4486197248 995216 33554432000 4194304 4338100 140736750543168 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=4381052 CPUtime=815.79 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 58695 0 0 0 80610 969 0 0 25 0 5 0 516694230 4486197248 995216 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=4381052 CPUtime=820.15 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 317178 0 0 0 81945 70 0 0 25 0 5 0 516694230 4486197248 995216 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=4381052 CPUtime=818.77 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 54952 0 0 0 80674 1203 0 0 25 0 5 0 516694231 4486197248 995216 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3274.59
Current children cumulated vsize (KiB) 4381052

[startup+882.301 s]
/proc/loadavg: 8.11 8.09 7.93 9/186 30537
/proc/meminfo: memFree=6705564/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=4515556 CPUtime=3514.03 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 1034849 0 0 0 348692 2711 0 0 18 0 5 0 516694092 4623929344 1031279 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1128889 1031279 176 36 0 1126794 0
[pid=30505/tid=30511] ppid=30503 vsize=4515556 CPUtime=878.4 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 544510 0 0 0 87630 210 0 0 25 0 5 0 516694230 4623929344 1031279 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362681 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=4515556 CPUtime=875.69 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 59970 0 0 0 86477 1092 0 0 25 0 5 0 516694230 4623929344 1031280 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=4515556 CPUtime=880.13 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 329827 0 0 0 87941 72 0 0 25 0 5 0 516694230 4623929344 1031280 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=4515556 CPUtime=878.38 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 57204 0 0 0 86510 1328 0 0 25 0 5 0 516694231 4623929344 1031280 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254244 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3514.03
Current children cumulated vsize (KiB) 4515556

[startup+942.301 s]
/proc/loadavg: 8.07 8.08 7.93 9/186 30539
/proc/meminfo: memFree=6357244/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=4794080 CPUtime=3753.52 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 1104816 0 0 0 372384 2968 0 0 18 0 5 0 516694092 4909137920 1101246 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1198520 1101246 176 36 0 1196425 0
[pid=30505/tid=30511] ppid=30503 vsize=4794080 CPUtime=938.33 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 580082 0 0 0 93609 224 0 0 25 0 5 0 516694230 4909137920 1101246 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362484 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=4794080 CPUtime=935.66 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 59972 0 0 0 92356 1210 0 0 25 0 5 0 516694230 4909137920 1101246 33554432000 4194304 4338100 140736750543168 18446744073709551615 4288397 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=4794080 CPUtime=940.11 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 364218 0 0 0 93931 80 0 0 25 0 5 0 516694230 4909137920 1101246 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=4794080 CPUtime=938.01 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 57206 0 0 0 92354 1447 0 0 25 0 5 0 516694231 4909137920 1101246 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362857 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3753.52
Current children cumulated vsize (KiB) 4794080

[startup+1002.3 s]
/proc/loadavg: 8.10 8.08 7.94 9/186 30541
/proc/meminfo: memFree=5871876/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=5035068 CPUtime=3993 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 1165588 0 0 0 396003 3297 0 0 18 0 5 0 516694092 5155909632 1162018 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1258767 1162018 176 36 0 1256672 0
[pid=30505/tid=30511] ppid=30503 vsize=5035068 CPUtime=998.25 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 612829 0 0 0 99587 238 0 0 25 0 5 0 516694230 5155909632 1162018 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=5035068 CPUtime=995.62 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 59993 0 0 0 98198 1364 0 0 25 0 5 0 516694230 5155909632 1162018 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=5035068 CPUtime=1000.06 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 392207 0 0 0 99921 85 0 0 25 0 5 0 516694230 5155909632 1162018 33554432000 4194304 4338100 140736750543168 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=5035068 CPUtime=997.66 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 57221 0 0 0 98164 1602 0 0 25 0 5 0 516694231 5155909632 1162018 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3993
Current children cumulated vsize (KiB) 5035068

[startup+1062.3 s]
/proc/loadavg: 8.07 8.08 7.94 9/186 30542
/proc/meminfo: memFree=5592480/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=5247772 CPUtime=4232.54 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 1218587 0 0 0 419667 3587 0 0 18 0 5 0 516694092 5373718528 1215017 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1311943 1215017 176 36 0 1309848 0
[pid=30505/tid=30511] ppid=30503 vsize=5247772 CPUtime=1058.21 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 642805 0 0 0 105571 250 0 0 25 0 5 0 516694230 5373718528 1215017 33554432000 4194304 4338100 140736750543168 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=5247772 CPUtime=1055.43 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 59993 0 0 0 104045 1498 0 0 25 0 5 0 516694230 5373718528 1215017 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=5247772 CPUtime=1059.86 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 415228 0 0 0 105895 91 0 0 25 0 5 0 516694230 5373718528 1215017 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=5247772 CPUtime=1057.63 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 57223 0 0 0 104023 1740 0 0 25 0 5 0 516694231 5373718528 1215017 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485793461 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4232.54
Current children cumulated vsize (KiB) 5247772

[startup+1122.3 s]
/proc/loadavg: 8.03 8.06 7.94 9/186 30544
/proc/meminfo: memFree=5422596/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=5357252 CPUtime=4472.04 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 1246231 0 0 0 443316 3888 0 0 18 0 5 0 516694092 5485826048 1242661 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1339313 1242661 176 36 0 1337218 0
[pid=30505/tid=30511] ppid=30503 vsize=5357252 CPUtime=1118.15 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 664794 0 0 0 111555 260 0 0 25 0 5 0 516694230 5485826048 1242661 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473363182 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=5357252 CPUtime=1115.37 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 59994 0 0 0 109894 1643 0 0 25 0 5 0 516694230 5485826048 1242661 33554432000 4194304 4338100 140736750543168 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=5357252 CPUtime=1119.78 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 420882 0 0 0 111886 92 0 0 25 0 5 0 516694230 5485826048 1242661 33554432000 4194304 4338100 140736750543168 18446744073709551615 4263773 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=5357252 CPUtime=1117.32 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 57223 0 0 0 109847 1885 0 0 25 0 5 0 516694231 5485826048 1242661 33554432000 4194304 4338100 140736750543168 18446744073709551615 274473362579 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4472.04
Current children cumulated vsize (KiB) 5357252

[startup+1182.3 s]
/proc/loadavg: 8.07 8.06 7.95 9/186 30545
/proc/meminfo: memFree=5120020/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=5441840 CPUtime=4711.62 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) S 30503 30505 30368 0 -1 4202496 1266740 0 0 0 467000 4162 0 0 18 0 5 0 516694092 5572444160 1263170 33554432000 4194304 4338100 140736750543168 18446744073709551615 274485770213 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 1360460 1263170 176 36 0 1358365 0
[pid=30505/tid=30511] ppid=30503 vsize=5441840 CPUtime=1178.12 cores=0,2,4,6
/proc/30505/task/30511/stat : 30511 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 673153 0 0 0 117547 265 0 0 25 0 5 0 516694230 5572444160 1263170 33554432000 4194304 4338100 140736750543168 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=5441840 CPUtime=1175.34 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 59994 0 0 0 115758 1776 0 0 25 0 5 0 516694230 5572444160 1263170 33554432000 4194304 4338100 140736750543168 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=30505/tid=30513] ppid=30503 vsize=5441840 CPUtime=1179.76 cores=0,2,4,6
/proc/30505/task/30513/stat : 30513 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 433032 0 0 0 117882 94 0 0 25 0 5 0 516694230 5572444160 1263170 33554432000 4194304 4338100 140736750543168 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=30505/tid=30514] ppid=30503 vsize=5441840 CPUtime=1176.97 cores=0,2,4,6
/proc/30505/task/30514/stat : 30514 (ttsth-5-4) R 30503 30505 30368 0 -1 4202560 57223 0 0 0 115680 2017 0 0 25 0 5 0 516694231 5572444160 1263170 33554432000 4194304 4338100 140736750543168 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4711.62
Current children cumulated vsize (KiB) 5441840



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.51 s]
/proc/loadavg: 7.72 7.99 7.93 3/179 30574
/proc/meminfo: memFree=11235228/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=0 CPUtime=4800.11 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) Z 30503 30505 30368 0 -1 4203548 1291304 0 0 0 475744 4267 0 0 18 0 2 0 516694092 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 0 0 0 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=0 CPUtime=1197.54 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4203588 59994 0 0 0 117927 1827 0 0 25 0 2 0 516694230 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.61 s]
/proc/loadavg: 7.72 7.99 7.93 3/179 30574
/proc/meminfo: memFree=11235228/32950928 swapFree=67111528/67111528
[pid=30505] ppid=30503 vsize=0 CPUtime=4800.21 cores=0,2,4,6
/proc/30505/stat : 30505 (ttsth-5-4) Z 30503 30505 30368 0 -1 4203548 1291304 0 0 0 475744 4277 0 0 18 0 2 0 516694092 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/30505/statm: 0 0 0 0 0 0 0
[pid=30505/tid=30512] ppid=30503 vsize=0 CPUtime=1197.64 cores=0,2,4,6
/proc/30505/task/30512/stat : 30512 (ttsth-5-4) R 30503 30505 30368 0 -1 4203588 59994 0 0 0 117927 1837 0 0 25 0 2 0 516694230 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1204.65
CPU time (s): 4800.25
CPU user time (s): 4757.45
CPU system time (s): 42.8055
CPU usage (%): 398.478
Max. virtual memory (cumulated for all children) (KiB): 5538808

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4757.45
system time used= 42.8055
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1291304
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9819
involuntary context switches= 37140

runsolver used 1.9827 second user time and 4.46032 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-29 06:22:24
IDJOB=3258394
IDBENCH=71353
IDSOLVER=1654
FILE ID=node125/3258394-1304050944
RUNJOBID= node125-1304049739-30386
PBS_JOBID= 13170412
Free space on /tmp= 72756 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2400c266431gyes7.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3258394-1304050944/watcher-3258394-1304050944 -o /tmp/evaluation-result-3258394-1304050944/solver-3258394-1304050944 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258394-1304050944.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 92e61742efd2e322c59b4442206bebf1
RANDOM SEED=550892240

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15090884 kB
Buffers:       2218172 kB
Cached:       14978196 kB
SwapCached:          0 kB
Active:        9115972 kB
Inactive:      8213260 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15090884 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            7632 kB
Writeback:          64 kB
AnonPages:      132116 kB
Mapped:          15228 kB
Slab:           468824 kB
PageTables:       4996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   277192 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72760 MiB
End job on node125 at 2011-04-29 06:42:31